1
0
Fork 0
Commit graph

45 commits

Author SHA1 Message Date
Mikhail Goncharov
55177548b0 sccache on linux + deployment sizes 2023-09-06 14:39:09 +02:00
Mikhail Goncharov
d57be337f9 deployment configs 2023-09-04 18:20:10 +02:00
Mikhail Goncharov
e1a4fda524 linux builders 2023-09-01 16:54:16 +02:00
Mikhail Goncharov
96e3f866ae linux agents and trigger new pipeline only 2023-08-31 18:18:13 +02:00
Mikhail Goncharov
ff87c03ba6 windows agents config 2023-08-31 14:12:38 +02:00
Mikhail Goncharov
ecf81eb2b6 restore cmake and other bins in path 2023-08-30 16:49:28 +02:00
Mikhail Goncharov
9ee7fac437 rename win deployment 2023-08-29 12:07:41 +02:00
Mikhail Goncharov
6883776b75 renames, new linux image (for test) 2023-08-29 12:07:41 +02:00
Mikhail Goncharov
91536475c7 windows-cluster config 2023-08-28 15:26:02 +02:00
Mikhail Goncharov
80978127ee update containers for linux + container for win 2023-08-28 12:05:01 +02:00
Mikhail Goncharov
e5aee86500 sample win deployment and github actions runner container 2023-08-25 10:15:58 +02:00
Mikhail Goncharov
b3ba37eb05 enabled linux queue from google gcp
--pipeline parameter for sample build script
2023-07-17 15:38:21 +02:00
Mikhail Goncharov
858ed55646 resurrect linux-agents 2023-07-17 13:20:05 +02:00
Mikhail Goncharov
7034885209 scale down test agents 2023-05-26 18:31:46 +02:00
Mikhail Goncharov
b595d573bc scale down linux agents as we run them on LLVM cluster 2023-05-26 18:09:45 +02:00
Mikhail Goncharov
a271c42e6f update cmake to 3.23 and llvm to 16 in linux image 2023-04-21 10:15:01 +02:00
Mikhail Goncharov
76d6f29154 use one dockerfile, upgrade to clang 15
successfully run
https://buildkite.com/llvm-project/llvm-main/builds/6204 based on this
image
2022-12-25 15:01:13 +01:00
Mikhail Goncharov
4f5a16bbdb update test deployment 2022-01-19 17:46:01 +01:00
Christian Kühnel
6394ae39b4 Adding an emptyDir to kubernetes deployment
This should resolve the out-of-disk problems reported in #379
2022-01-19 17:44:42 +01:00
Mikhail Goncharov
a6c2eb7f23 set BUILDKITE_BUILD_PATH for service agents 2021-06-18 12:32:07 +02:00
Mikhail Goncharov
d11188e407 use same db for all tables 2021-06-18 12:21:41 +02:00
Mikhail Goncharov
10da060717 increase number of linux agents 2021-05-20 12:18:17 +02:00
Mikhail Goncharov
f0d1cc6ab3 update linux node pool (machines w/ no external ssd)
updated llvm version to 12 with relevant fixes in scrips
2021-05-17 10:06:54 +02:00
Mikhail Goncharov
1b603e884a migrage linux-agents to a new pool 2021-05-15 10:58:23 +02:00
Mikhail Goncharov
e703a856cb Add graceful termination period for linux agents
Now builds will not be dropped on cluster upgrades. That requires
container updates.
2021-05-14 14:56:18 +02:00
Mikhail Goncharov
08bb6492fa service queue 2021-04-26 19:56:20 +00:00
Mikhail Goncharov
ab1b8d5ae2 Use buildkite API instead of passing stat via files
Previous approach had drawbacks:

- every step had to implement exporting of results in fixed format

- if step failed then failure will not be detected

Now report step will fetch results directly from Buildkite.
Agents have to be updated to have BUILDKITE_API_TOKEN env.
2020-10-08 14:40:57 +02:00
Mikhail Goncharov
8b36ff5888 remove jenkins configs and scrips
- moved linux agents to a new node pool (basically renamed old one)

- removed some out of date scripts or moved them to playlists

- removed shell scrips invoked from jenkins

- minor docs updates

Some python classes methods are not used anymore and will be cleaned up
separately.
2020-09-01 09:51:49 +02:00
Mikhail Goncharov
c384553e0b Increase linux agent memory #215 2020-07-21 18:40:36 +02:00
Mikhail Goncharov
79537bfc38 updated deployment 2020-07-21 17:42:02 +02:00
Mikhail Goncharov
00e8188f47 deployment definition 2020-07-21 15:21:06 +02:00
Mikhail Goncharov
d4e5126090 Update docs on ssd disk initialization
+ change number of linux agents

+ removed outputs in job analysis

+ updated windows image
2020-07-21 10:40:26 +02:00
Mikhail Goncharov
ffaba986c1 Deployment changes
do not restart win docker as it makes it impossible to stop agent from
buildkite UI for maintenance
2020-06-17 09:46:42 +02:00
Mikhail Goncharov
af7bd86875 disable clang format / tidy for master and release builds 2020-06-16 17:29:04 +02:00
Mikhail Goncharov
4c3cd1b6f5 Use os in queue name
That is useful for metrics collection as buildkite metrics only reports
per queue metrics
2020-06-16 10:30:24 +02:00
Mikhail Goncharov
397f7f38e9 removed 'release' buildkite queue 2020-06-15 17:07:36 +02:00
Mikhail Goncharov
de0d5afd78 Move agents to default queue
- fix windows defender exclusion path
2020-06-05 10:50:42 +02:00
Mikhail Goncharov
017ca44a75 Add windows build
- configured sccache on buildkite windows machines
- final result is sent to phabricator from "summary.py" that waits for both builds to complete
- extracted "add_url_artifact" to a runnable script
- reorganized code and fixed some of TODOs
2020-06-03 13:40:22 +02:00
Mikhail Goncharov
1505cfb635 build 2020-05-14 12:57:38 +02:00
Mikhail Goncharov
a152d97e3c Simple service to integrate harbormaster w/ buildkite
Harbormaster can send a request in url encoded form while buildkite
expects a POST request with json. Instead of modifying harbormastar or
buildkite this adds a simple proxy that accepts url encoded form and
creates a request that buildkite expects. To avoid potential abuse,
nginx asks for simple http auth credentials stored in harbormaster.

All build parameters passed by proxy are put into build metadata and as
ph_* env variables available during the build.

Secrets involved:
- harbormastert knows http-auth to proxy (stored in privatly and in k8
  buildkite/http-auth as auth file);
- proxy knows buildkite api token (mine atm);
- build agent knows conduit API token (mine atm),
  and SSH key of llvm-premerge-tests-bot (in k8 buildkite/github-ssh).

Sample build: https://reviews.llvm.org/harbormaster/build/64828/8/
https://buildkite.com/llvm-project/premerge/builds/48
2020-05-13 13:13:33 +02:00
Mikhail Goncharov
27ab5abd0a Forked docker image for premerge
- added simple script to define pipeline, it will check out from this
  repo later

- refactored docker container, e.g. merged most of the RUN steps into
  one and moved steps that are likely to change closer to the end. That
  should improve rebuild / upload speed.

Sample run: https://buildkite.com/llvm-project/premerge/builds/14
2020-04-24 12:29:44 +02:00
Mikhail Goncharov
51734a7175 Remove port mapping and rename buildkite release deployment 2020-04-22 12:26:52 +02:00
Mikhail Goncharov
eca78eb366 Add new deployment for agent that will do premerge checks
maybe we will later use same agents for everything but for now I don't
want to modify working build much. But I have updated deployment of
release agent to use correct docker image (there were renamed a while
ago) and pass tags and agent token via env variables.

Have not deployed new deployment for release yet.
2020-04-22 12:11:23 +02:00
Christian Kühnel
5752b6939c fixed namespaces 2020-01-17 11:42:16 +01:00
Christian Kühnel
53dfed10c0 added deployment for buildkite 2020-01-17 10:53:21 +01:00