1
0
Fork 0
Commit graph

106 commits

Author SHA1 Message Date
Mikhail Goncharov
46c9fefb77 rename CCACHE_PATH to CCACHE_DIR
https://ccache.dev/manual/3.7.11.html#_configuration uses CCACHE_DIR to set
cache directory. CCACHE_PATH has a different meaning.

+ removed LLVM_CCACHE_DIR ccache argument to use CCACHE_DIR from env.
+ print additional info about ccache config and machine readable stats.
2020-08-26 14:22:37 +02:00
Mikhail Goncharov
06db27454c add ATL component, build windows image without cache 2020-08-12 22:31:07 +02:00
Christian Kühnel
5da900df30 added DIA support for windows image 2020-08-12 22:31:07 +02:00
Mikhail Goncharov
5e9a7be295 add perl to windows image 2020-07-08 10:25:05 +02:00
Christian Kühnel
f84641775a removed buildbot container 2020-06-26 10:36:52 +02:00
Mikhail Goncharov
77b08754c6 use unstable debian release 2020-06-18 16:16:07 +02:00
Mikhail Goncharov
fb830daebe Update clang version
apt.llvm.org signature has changed, we also should use debian:stable
(=buster) as a base image to correctly install packages.

Removed jdk from the base image as it was used only by Jenkins.

Log packages version for every build.

Related issues: #112
2020-06-18 10:55:15 +02:00
Mikhail Goncharov
dd4e6210bb pre-checkout hook
That will fix #112 by checking out into 'llvm-project' directory
(derived from repository name).
2020-06-17 14:41:00 +02:00
Mikhail Goncharov
e08efbcf2a Script updates:
- option not to shut down sccache after 10 minutes
- zero sccache stats before running the build
- decrease number of jenkins agents
2020-06-04 12:36:46 +02:00
Mikhail Goncharov
017ca44a75 Add windows build
- configured sccache on buildkite windows machines
- final result is sent to phabricator from "summary.py" that waits for both builds to complete
- extracted "add_url_artifact" to a runnable script
- reorganized code and fixed some of TODOs
2020-06-03 13:40:22 +02:00
Mikhail Goncharov
afed3a7898 Update docs and script for installing win machines
I have tried to install new windows machine for buildkite and updated
some scrips and docs on along the way.

- Updated base image for k8s agent installation as it gave a warning that
previous version was old.

- Now buildkite secret is configured in a powershell script along with
tags (and possible other parameters).

- Split "windows_agent_start.ps1" to "..jenkins" and "..buildkite" as
  some parameters are different.

- Created a "windows-development" machine in GCP stat can be resumed
  to build docker images / expriment.
2020-05-27 15:41:49 +02:00
Mikhail Goncharov
1505cfb635 build 2020-05-14 12:57:38 +02:00
Mikhail Goncharov
a152d97e3c Simple service to integrate harbormaster w/ buildkite
Harbormaster can send a request in url encoded form while buildkite
expects a POST request with json. Instead of modifying harbormastar or
buildkite this adds a simple proxy that accepts url encoded form and
creates a request that buildkite expects. To avoid potential abuse,
nginx asks for simple http auth credentials stored in harbormaster.

All build parameters passed by proxy are put into build metadata and as
ph_* env variables available during the build.

Secrets involved:
- harbormastert knows http-auth to proxy (stored in privatly and in k8
  buildkite/http-auth as auth file);
- proxy knows buildkite api token (mine atm);
- build agent knows conduit API token (mine atm),
  and SSH key of llvm-premerge-tests-bot (in k8 buildkite/github-ssh).

Sample build: https://reviews.llvm.org/harbormaster/build/64828/8/
https://buildkite.com/llvm-project/premerge/builds/48
2020-05-13 13:13:33 +02:00
Christian Kühnel
d6e9b1f6c4 manual commands for vs config 2020-05-04 14:23:02 +00:00
Mikhail Goncharov
39667898a7 use env variables 2020-04-24 15:43:09 +02:00
Mikhail Goncharov
e860c29a47 Checkout scripts and define pipeline in python 2020-04-24 14:47:00 +02:00
Mikhail Goncharov
8f2cc6f5b4
Merge pull request #166 from google/update-image-pipeline-bootstrap
Forked docker image for premerge
2020-04-24 12:30:36 +02:00
Mikhail Goncharov
27ab5abd0a Forked docker image for premerge
- added simple script to define pipeline, it will check out from this
  repo later

- refactored docker container, e.g. merged most of the RUN steps into
  one and moved steps that are likely to change closer to the end. That
  should improve rebuild / upload speed.

Sample run: https://buildkite.com/llvm-project/premerge/builds/14
2020-04-24 12:29:44 +02:00
Christian Kühnel
8039b8a6f7 changed the way sccache is started 2020-04-24 09:54:36 +00:00
Christian Kühnel
7a87f7d453 fixed version file path 2020-04-24 09:54:36 +00:00
Christian Kühnel
d8a19b36e4 fixed vs code target architecture 2020-04-24 09:54:36 +00:00
Christian Kühnel
b9f8ce9f20 bumped version numbers 2020-04-23 09:47:47 +00:00
Mikhail Goncharov
eca78eb366 Add new deployment for agent that will do premerge checks
maybe we will later use same agents for everything but for now I don't
want to modify working build much. But I have updated deployment of
release agent to use correct docker image (there were renamed a while
ago) and pass tags and agent token via env variables.

Have not deployed new deployment for release yet.
2020-04-22 12:11:23 +02:00
ChristianKuehnel
a47751ebe8
Upgrade Windows agents to Visual Studio 2019 and sccache (#162)
* added cmake parameters for sccache

* added sccache

* removed stray debug output

* starting sccache with VS environment

* added container for Visual Studio 2019

* fixed comments

* considering WIndows version

* updated to vs2019

* using MS install method

* snapshot of VS2019 experiments

* using --installRecommended

* cleanup of dockerfile

* updated path

* fixed dockerfile

* dumped version number

* exclude for virus scan

* added testing option

this does not start the agent

* write results on failures

* added timeouts for pipelines

* moving master pipelines to python scripts

* added flang to automatic project selection

based on #159, this will enable flang for beta testers

* added persistent workspace for testing containers

* added secure delete function

* added better log message

* deleting read-only files

* checking existence before setting flags

* using unlink

* deleting recursively

* using pathlib for chmod

* using custom workspace

* fixed drive

* separate handling of single files

* simplified read-only handling again

* removed vsdevcmd calls

as it's already set in the docker Entrypoint

* renamed container folders

* windows version not needed any more

* bumped version number

* bumped version number

* script cleanup

* added cmake parameters for sccache

* added sccache

* starting sccache with VS environment

* added container for Visual Studio 2019

* fixed comments

* considering WIndows version

* updated to vs2019

* using MS install method

* snapshot of VS2019 experiments

* using --installRecommended

* cleanup of dockerfile

* updated path

* fixed dockerfile

* dumped version number

* removed vsdevcmd calls

as it's already set in the docker Entrypoint

* renamed container folders

* windows version not needed any more

* bumped version number

* bumped version number

* script cleanup

* removed sccache from vs2017

* making windows image configurable

* added versioning

* created windows BETA pipeline

* added Jenkins label for vs2019
2020-04-21 11:46:41 +02:00
Christian Kühnel
b36a7733de fixed jeninkins version
To avoid upgrade to later version with mandatory CSRF enabled.
This did break the Phabricator integration in #163.
2020-04-20 21:52:07 +02:00
Christian Kühnel
651bf69a89 added persistent workspace for testing containers 2020-04-15 18:00:53 +02:00
Christian Kühnel
2899b93c16 added gdb
fixed python bdist_wheel warning
2020-03-27 13:51:54 +01:00
Christian Kühnel
24e751f6ec updated container and deployment to store ssh keys for github uploads 2020-03-23 15:34:23 +01:00
Christian Kühnel
9b2e225461 addd buildbot installation 2020-03-19 14:53:19 +01:00
Christian Kühnel
f182e84f1a Docker build toolchain working 2020-03-19 14:21:32 +01:00
Christian Kühnel
e18bd6fca7 incremented version number 2020-02-24 15:28:59 +00:00
Christian Kühnel
d00cd053e1 tagging and pushing "latest" 2020-02-20 16:45:04 +00:00
Christian Kühnel
548e07d5d4 added versioning to windows docker containers 2020-02-20 15:18:26 +00:00
Christian Kühnel
a0f676dfba set fixed windows version number 2020-02-20 09:15:11 +00:00
Christian Kühnel
dd375f71db using upstream jenkins image 2020-02-17 10:50:37 +01:00
Christian Kühnel
a4e320ac04 Merge branch 'master' of ssh://github.com/google/llvm-premerge-checks 2020-02-06 14:08:26 +00:00
Christian Kühnel
66a653578b changed way of authenticating gsutil 2020-02-06 14:07:51 +00:00
Christian Kühnel
5e0b5873a9 changing authentication for gsutil 2020-02-06 14:26:46 +01:00
Christian Kühnel
39319ab89d named the buildkite windows agents 2020-02-06 14:05:46 +01:00
Christian Kühnel
9e431d4ab9 added comment on host names 2020-02-06 13:32:26 +01:00
Christian Kühnel
e8a9d7bf17 removing perl
messes up the entire build toolchain
2020-02-04 15:46:47 +00:00
Christian Kühnel
1f7eff3c41 Merge branch 'master' of ssh://github.com/google/llvm-premerge-checks 2020-02-04 10:14:36 +00:00
Christian Kühnel
7cceab06d9 using push/pop-location 2020-02-04 11:14:22 +01:00
Christian Kühnel
dc132819d6 removed perl from the path 2020-02-04 08:58:35 +00:00
Christian Kühnel
69ebbaa3f7 moved temp to SSD 2020-02-03 10:46:29 +00:00
Christian Kühnel
87c591d6b2 added perl for OpenMP 2020-01-31 13:19:07 +00:00
Christian Kühnel
64e18e7f85 changed path to requirements.txt 2020-01-28 16:40:08 +01:00
Mikhail Goncharov
ed0fb93e67 updated buildkite to llvm-10 2020-01-24 10:06:42 +01:00
Mikhail Goncharov
caef6c8238 fix build 2020-01-23 17:55:56 +01:00
Mikhail Goncharov
e13822f0f5 Use llvm-10 on phabricator agent
- removed 'clang8' from the image name;

- added --no-cache to ./build_deploy.sh to make sure that image will use
  most recent packages.

Tested locally
2020-01-23 16:17:19 +01:00