Previous approach had drawbacks:
- every step had to implement exporting of results in fixed format
- if step failed then failure will not be detected
Now report step will fetch results directly from Buildkite.
Agents have to be updated to have BUILDKITE_API_TOKEN env.
- moved linux agents to a new node pool (basically renamed old one)
- removed some out of date scripts or moved them to playlists
- removed shell scrips invoked from jenkins
- minor docs updates
Some python classes methods are not used anymore and will be cleaned up
separately.
- configured sccache on buildkite windows machines
- final result is sent to phabricator from "summary.py" that waits for both builds to complete
- extracted "add_url_artifact" to a runnable script
- reorganized code and fixed some of TODOs
Harbormaster can send a request in url encoded form while buildkite
expects a POST request with json. Instead of modifying harbormastar or
buildkite this adds a simple proxy that accepts url encoded form and
creates a request that buildkite expects. To avoid potential abuse,
nginx asks for simple http auth credentials stored in harbormaster.
All build parameters passed by proxy are put into build metadata and as
ph_* env variables available during the build.
Secrets involved:
- harbormastert knows http-auth to proxy (stored in privatly and in k8
buildkite/http-auth as auth file);
- proxy knows buildkite api token (mine atm);
- build agent knows conduit API token (mine atm),
and SSH key of llvm-premerge-tests-bot (in k8 buildkite/github-ssh).
Sample build: https://reviews.llvm.org/harbormaster/build/64828/8/https://buildkite.com/llvm-project/premerge/builds/48
- added simple script to define pipeline, it will check out from this
repo later
- refactored docker container, e.g. merged most of the RUN steps into
one and moved steps that are likely to change closer to the end. That
should improve rebuild / upload speed.
Sample run: https://buildkite.com/llvm-project/premerge/builds/14
maybe we will later use same agents for everything but for now I don't
want to modify working build much. But I have updated deployment of
release agent to use correct docker image (there were renamed a while
ago) and pass tags and agent token via env variables.
Have not deployed new deployment for release yet.
