Current mechanism does not work well due to need to upload version.
Rebuilding a container from an alder revision also leads to overriding
existing container version.
The current default version in Chocolatey, 5.28, fails to install.
This was previously worked around by pinning a newer upcoming
version, 5.28.0.20210106. However this version never got approved
into Chocolatey (for unknown reasons), so it's no longer available.
Therefore instead request the previous older version, 5.24.3.2404001,
which installs correctly.
The current default version in Chocolatey (5.28) fails to install,
with the following error:
ERROR: The response content cannot be parsed because the Internet
Explorer engine is not available, or Internet Explorer's first-launch
configuration is not complete. Specify the UseBasicParsing parameter
and try again.
This is fixed in a newer version of packaging of ActivePerl, thus request
this particular version.
The libcxx tests currently require bash to be available (although
that might change), and the bash provided with Git is enough for
fulfilling this need. (Bash can also be useful for running other
scripts in the libcxx CI chain outside of the tests themselves.)
For the main LLVM testing, this works once
https://reviews.llvm.org/D98858 is merged.
This also allows getting rid of GnuWin altogether.
The GnuWin32 tools installed by Chocolatey reside in C:\GnuWin, and
are added automatically to the path.
The CMake tools aren't added automatically to the path though, so
that entry still is useful.
(However, the Visual Studio installation also brings along
a different version of CMake that ends up found if the one we
installed via Chocolatey isn't available.)
The main LLVM build runs fine with MSVC itself, but building libcxx
isn't supported with cl.exe, only with clang-cl.
This shouldn't matter for the main LLVM build, as it still is
configured to use CXX=cl (and even then, CMake defaults to cl
if nothing is specified).
Now we run a pre-commit hook that tries to kill any process that locked
a file in target directory.
Updated timeout interval on windows from 120 to 90 minutes.
Fixes#243
- moved linux agents to a new node pool (basically renamed old one)
- removed some out of date scripts or moved them to playlists
- removed shell scrips invoked from jenkins
- minor docs updates
Some python classes methods are not used anymore and will be cleaned up
separately.
https://ccache.dev/manual/3.7.11.html#_configuration uses CCACHE_DIR to set
cache directory. CCACHE_PATH has a different meaning.
+ removed LLVM_CCACHE_DIR ccache argument to use CCACHE_DIR from env.
+ print additional info about ccache config and machine readable stats.
apt.llvm.org signature has changed, we also should use debian:stable
(=buster) as a base image to correctly install packages.
Removed jdk from the base image as it was used only by Jenkins.
Log packages version for every build.
Related issues: #112
- configured sccache on buildkite windows machines
- final result is sent to phabricator from "summary.py" that waits for both builds to complete
- extracted "add_url_artifact" to a runnable script
- reorganized code and fixed some of TODOs
I have tried to install new windows machine for buildkite and updated
some scrips and docs on along the way.
- Updated base image for k8s agent installation as it gave a warning that
previous version was old.
- Now buildkite secret is configured in a powershell script along with
tags (and possible other parameters).
- Split "windows_agent_start.ps1" to "..jenkins" and "..buildkite" as
some parameters are different.
- Created a "windows-development" machine in GCP stat can be resumed
to build docker images / expriment.
Harbormaster can send a request in url encoded form while buildkite
expects a POST request with json. Instead of modifying harbormastar or
buildkite this adds a simple proxy that accepts url encoded form and
creates a request that buildkite expects. To avoid potential abuse,
nginx asks for simple http auth credentials stored in harbormaster.
All build parameters passed by proxy are put into build metadata and as
ph_* env variables available during the build.
Secrets involved:
- harbormastert knows http-auth to proxy (stored in privatly and in k8
buildkite/http-auth as auth file);
- proxy knows buildkite api token (mine atm);
- build agent knows conduit API token (mine atm),
and SSH key of llvm-premerge-tests-bot (in k8 buildkite/github-ssh).
Sample build: https://reviews.llvm.org/harbormaster/build/64828/8/https://buildkite.com/llvm-project/premerge/builds/48
- added simple script to define pipeline, it will check out from this
repo later
- refactored docker container, e.g. merged most of the RUN steps into
one and moved steps that are likely to change closer to the end. That
should improve rebuild / upload speed.
Sample run: https://buildkite.com/llvm-project/premerge/builds/14
maybe we will later use same agents for everything but for now I don't
want to modify working build much. But I have updated deployment of
release agent to use correct docker image (there were renamed a while
ago) and pass tags and agent token via env variables.
Have not deployed new deployment for release yet.
* added cmake parameters for sccache
* added sccache
* removed stray debug output
* starting sccache with VS environment
* added container for Visual Studio 2019
* fixed comments
* considering WIndows version
* updated to vs2019
* using MS install method
* snapshot of VS2019 experiments
* using --installRecommended
* cleanup of dockerfile
* updated path
* fixed dockerfile
* dumped version number
* exclude for virus scan
* added testing option
this does not start the agent
* write results on failures
* added timeouts for pipelines
* moving master pipelines to python scripts
* added flang to automatic project selection
based on #159, this will enable flang for beta testers
* added persistent workspace for testing containers
* added secure delete function
* added better log message
* deleting read-only files
* checking existence before setting flags
* using unlink
* deleting recursively
* using pathlib for chmod
* using custom workspace
* fixed drive
* separate handling of single files
* simplified read-only handling again
* removed vsdevcmd calls
as it's already set in the docker Entrypoint
* renamed container folders
* windows version not needed any more
* bumped version number
* bumped version number
* script cleanup
* added cmake parameters for sccache
* added sccache
* starting sccache with VS environment
* added container for Visual Studio 2019
* fixed comments
* considering WIndows version
* updated to vs2019
* using MS install method
* snapshot of VS2019 experiments
* using --installRecommended
* cleanup of dockerfile
* updated path
* fixed dockerfile
* dumped version number
* removed vsdevcmd calls
as it's already set in the docker Entrypoint
* renamed container folders
* windows version not needed any more
* bumped version number
* bumped version number
* script cleanup
* removed sccache from vs2017
* making windows image configurable
* added versioning
* created windows BETA pipeline
* added Jenkins label for vs2019
E.g.: https://reviews.llvm.org/D71197?id=233029
Will add a link to file and code to apply the patch in the next PR.
+ don't create TARGET_DIR in scripts;
+ updated section about local build;
+ partially specified inputs / outputs of scripts so it's more transparent what are the results;
+ added symlink to compile_command.json (clang-tidy will need it);
+ add IDEA files to .gitignore.
* first steps towards windows builds #25
* added build scripts for windows
* moved from powershell to CMD
* script cleanup
* set up jenkins swarm plugin
* added deployment script for windows
* first draft of jenkins file for windows
* first shot at windows pods
* first draft for buildkite agent
* added call to run_cmake.bat
* added call to run_ninja.bat
* added new buildkite script
* added Dockerfile for buildkite agent
* fixed userprofile path
* added /IncludeOptional for VS
I hope that includes all required libraries
* moved work folder to W:
* added git config for long file names
* added more CMake flags
* setting x64 bit arch in a different way
* and yet another way to set 64bit
* merged buildkite changes to jenkins scripts
* hiding license in batch script runs
* running release build
* added test xml log
* compilation is working, but not all tests
* cmake is in patch now
* updated the dependencies from buildkite config
* dockerfile cleanup
* change how git handles line endings
* updated comment
* performance improvment for volume
* changed workdir folder
* using gnuwin package
split installation into separate layers to speed up incremental changes
