1
0
Fork 0
Commit graph

120 commits

Author SHA1 Message Date
Mikhail Goncharov
d29b27ff10 published new images for win 2021-04-23 13:55:50 +00:00
Martin Storsjö
ffd904922d Install an older version of ActivePerl
The current default version in Chocolatey, 5.28, fails to install.
This was previously worked around by pinning a newer upcoming
version, 5.28.0.20210106. However this version never got approved
into Chocolatey (for unknown reasons), so it's no longer available.
Therefore instead request the previous older version, 5.24.3.2404001,
which installs correctly.
2021-04-23 14:49:31 +02:00
Martin Storsjö
18ff65644e Remove GnuWin, add Git Bash to the path
Both the main and the release/12.x branches of llvm-project should
work when built in a Windows Container with Git Bash in path now.
2021-04-23 14:48:24 +02:00
Mikhail Goncharov
609f614040 put back gnuwin 2021-03-19 14:35:02 +00:00
Mikhail Goncharov
feb44f59ad update build_run for win 2021-03-19 10:20:17 +00:00
Martin Storsjö
59993105bd windows: Stop installing GnuWin
These tools aren't actually needed; the same tools from Git
from Windows fill the same need.
2021-03-19 10:17:43 +01:00
Martin Storsjö
515acd1736 windows: Install a specific version of ActivePerl
The current default version in Chocolatey (5.28) fails to install,
with the following error:

    ERROR: The response content cannot be parsed because the Internet
    Explorer engine is not available, or Internet Explorer's first-launch
    configuration is not complete. Specify the UseBasicParsing parameter
    and try again.

This is fixed in a newer version of packaging of ActivePerl, thus request
this particular version.
2021-03-19 09:57:18 +01:00
Martin Storsjö
e7cc8d1de8 windows: Add Git Bash to the path
The libcxx tests currently require bash to be available (although
that might change), and the bash provided with Git is enough for
fulfilling this need. (Bash can also be useful for running other
scripts in the libcxx CI chain outside of the tests themselves.)

For the main LLVM testing, this works once
https://reviews.llvm.org/D98858 is merged.

This also allows getting rid of GnuWin altogether.
2021-03-19 09:56:58 +01:00
Martin Storsjö
b11b61d04f windows: Don't manually add c:\Program Files (x86)\GnuWin32\bin to the path
The GnuWin32 tools installed by Chocolatey reside in C:\GnuWin, and
are added automatically to the path.

The CMake tools aren't added automatically to the path though, so
that entry still is useful.

(However, the Visual Studio installation also brings along
a different version of CMake that ends up found if the one we
installed via Chocolatey isn't available.)
2021-03-19 09:56:58 +01:00
Martin Storsjö
dee182ad1c windows: Install LLVM/Clang in the docker image
The main LLVM build runs fine with MSVC itself, but building libcxx
isn't supported with cl.exe, only with clang-cl.

This shouldn't matter for the main LLVM build, as it still is
configured to use CXX=cl (and even then, CMake defaults to cl
if nothing is specified).
2021-03-19 09:56:44 +01:00
Mikhail Goncharov
99e43a8d9a rename master to main branch following LLVM 2020-12-10 09:29:24 +01:00
Mikhail Goncharov
ea943424f3 Enable windows builds back
Now we run a pre-commit hook that tries to kill any process that locked
a file in target directory.

Updated timeout interval on windows from 120 to 90 minutes.

Fixes #243
2020-10-07 16:59:37 +02:00
Mikhail Goncharov
8b36ff5888 remove jenkins configs and scrips
- moved linux agents to a new node pool (basically renamed old one)

- removed some out of date scripts or moved them to playlists

- removed shell scrips invoked from jenkins

- minor docs updates

Some python classes methods are not used anymore and will be cleaned up
separately.
2020-09-01 09:51:49 +02:00
Mikhail Goncharov
5380ea63b4 docs about machine config and ingress 2020-08-28 13:08:40 +02:00
Mikhail Goncharov
46c9fefb77 rename CCACHE_PATH to CCACHE_DIR
https://ccache.dev/manual/3.7.11.html#_configuration uses CCACHE_DIR to set
cache directory. CCACHE_PATH has a different meaning.

+ removed LLVM_CCACHE_DIR ccache argument to use CCACHE_DIR from env.
+ print additional info about ccache config and machine readable stats.
2020-08-26 14:22:37 +02:00
Mikhail Goncharov
06db27454c add ATL component, build windows image without cache 2020-08-12 22:31:07 +02:00
Christian Kühnel
5da900df30 added DIA support for windows image 2020-08-12 22:31:07 +02:00
Mikhail Goncharov
5e9a7be295 add perl to windows image 2020-07-08 10:25:05 +02:00
Christian Kühnel
f84641775a removed buildbot container 2020-06-26 10:36:52 +02:00
Mikhail Goncharov
77b08754c6 use unstable debian release 2020-06-18 16:16:07 +02:00
Mikhail Goncharov
fb830daebe Update clang version
apt.llvm.org signature has changed, we also should use debian:stable
(=buster) as a base image to correctly install packages.

Removed jdk from the base image as it was used only by Jenkins.

Log packages version for every build.

Related issues: #112
2020-06-18 10:55:15 +02:00
Mikhail Goncharov
dd4e6210bb pre-checkout hook
That will fix #112 by checking out into 'llvm-project' directory
(derived from repository name).
2020-06-17 14:41:00 +02:00
Mikhail Goncharov
e08efbcf2a Script updates:
- option not to shut down sccache after 10 minutes
- zero sccache stats before running the build
- decrease number of jenkins agents
2020-06-04 12:36:46 +02:00
Mikhail Goncharov
017ca44a75 Add windows build
- configured sccache on buildkite windows machines
- final result is sent to phabricator from "summary.py" that waits for both builds to complete
- extracted "add_url_artifact" to a runnable script
- reorganized code and fixed some of TODOs
2020-06-03 13:40:22 +02:00
Mikhail Goncharov
afed3a7898 Update docs and script for installing win machines
I have tried to install new windows machine for buildkite and updated
some scrips and docs on along the way.

- Updated base image for k8s agent installation as it gave a warning that
previous version was old.

- Now buildkite secret is configured in a powershell script along with
tags (and possible other parameters).

- Split "windows_agent_start.ps1" to "..jenkins" and "..buildkite" as
  some parameters are different.

- Created a "windows-development" machine in GCP stat can be resumed
  to build docker images / expriment.
2020-05-27 15:41:49 +02:00
Mikhail Goncharov
1505cfb635 build 2020-05-14 12:57:38 +02:00
Mikhail Goncharov
a152d97e3c Simple service to integrate harbormaster w/ buildkite
Harbormaster can send a request in url encoded form while buildkite
expects a POST request with json. Instead of modifying harbormastar or
buildkite this adds a simple proxy that accepts url encoded form and
creates a request that buildkite expects. To avoid potential abuse,
nginx asks for simple http auth credentials stored in harbormaster.

All build parameters passed by proxy are put into build metadata and as
ph_* env variables available during the build.

Secrets involved:
- harbormastert knows http-auth to proxy (stored in privatly and in k8
  buildkite/http-auth as auth file);
- proxy knows buildkite api token (mine atm);
- build agent knows conduit API token (mine atm),
  and SSH key of llvm-premerge-tests-bot (in k8 buildkite/github-ssh).

Sample build: https://reviews.llvm.org/harbormaster/build/64828/8/
https://buildkite.com/llvm-project/premerge/builds/48
2020-05-13 13:13:33 +02:00
Christian Kühnel
d6e9b1f6c4 manual commands for vs config 2020-05-04 14:23:02 +00:00
Mikhail Goncharov
39667898a7 use env variables 2020-04-24 15:43:09 +02:00
Mikhail Goncharov
e860c29a47 Checkout scripts and define pipeline in python 2020-04-24 14:47:00 +02:00
Mikhail Goncharov
8f2cc6f5b4
Merge pull request #166 from google/update-image-pipeline-bootstrap
Forked docker image for premerge
2020-04-24 12:30:36 +02:00
Mikhail Goncharov
27ab5abd0a Forked docker image for premerge
- added simple script to define pipeline, it will check out from this
  repo later

- refactored docker container, e.g. merged most of the RUN steps into
  one and moved steps that are likely to change closer to the end. That
  should improve rebuild / upload speed.

Sample run: https://buildkite.com/llvm-project/premerge/builds/14
2020-04-24 12:29:44 +02:00
Christian Kühnel
8039b8a6f7 changed the way sccache is started 2020-04-24 09:54:36 +00:00
Christian Kühnel
7a87f7d453 fixed version file path 2020-04-24 09:54:36 +00:00
Christian Kühnel
d8a19b36e4 fixed vs code target architecture 2020-04-24 09:54:36 +00:00
Christian Kühnel
b9f8ce9f20 bumped version numbers 2020-04-23 09:47:47 +00:00
Mikhail Goncharov
eca78eb366 Add new deployment for agent that will do premerge checks
maybe we will later use same agents for everything but for now I don't
want to modify working build much. But I have updated deployment of
release agent to use correct docker image (there were renamed a while
ago) and pass tags and agent token via env variables.

Have not deployed new deployment for release yet.
2020-04-22 12:11:23 +02:00
ChristianKuehnel
a47751ebe8
Upgrade Windows agents to Visual Studio 2019 and sccache (#162)
* added cmake parameters for sccache

* added sccache

* removed stray debug output

* starting sccache with VS environment

* added container for Visual Studio 2019

* fixed comments

* considering WIndows version

* updated to vs2019

* using MS install method

* snapshot of VS2019 experiments

* using --installRecommended

* cleanup of dockerfile

* updated path

* fixed dockerfile

* dumped version number

* exclude for virus scan

* added testing option

this does not start the agent

* write results on failures

* added timeouts for pipelines

* moving master pipelines to python scripts

* added flang to automatic project selection

based on #159, this will enable flang for beta testers

* added persistent workspace for testing containers

* added secure delete function

* added better log message

* deleting read-only files

* checking existence before setting flags

* using unlink

* deleting recursively

* using pathlib for chmod

* using custom workspace

* fixed drive

* separate handling of single files

* simplified read-only handling again

* removed vsdevcmd calls

as it's already set in the docker Entrypoint

* renamed container folders

* windows version not needed any more

* bumped version number

* bumped version number

* script cleanup

* added cmake parameters for sccache

* added sccache

* starting sccache with VS environment

* added container for Visual Studio 2019

* fixed comments

* considering WIndows version

* updated to vs2019

* using MS install method

* snapshot of VS2019 experiments

* using --installRecommended

* cleanup of dockerfile

* updated path

* fixed dockerfile

* dumped version number

* removed vsdevcmd calls

as it's already set in the docker Entrypoint

* renamed container folders

* windows version not needed any more

* bumped version number

* bumped version number

* script cleanup

* removed sccache from vs2017

* making windows image configurable

* added versioning

* created windows BETA pipeline

* added Jenkins label for vs2019
2020-04-21 11:46:41 +02:00
Christian Kühnel
b36a7733de fixed jeninkins version
To avoid upgrade to later version with mandatory CSRF enabled.
This did break the Phabricator integration in #163.
2020-04-20 21:52:07 +02:00
Christian Kühnel
651bf69a89 added persistent workspace for testing containers 2020-04-15 18:00:53 +02:00
Christian Kühnel
2899b93c16 added gdb
fixed python bdist_wheel warning
2020-03-27 13:51:54 +01:00
Christian Kühnel
24e751f6ec updated container and deployment to store ssh keys for github uploads 2020-03-23 15:34:23 +01:00
Christian Kühnel
9b2e225461 addd buildbot installation 2020-03-19 14:53:19 +01:00
Christian Kühnel
f182e84f1a Docker build toolchain working 2020-03-19 14:21:32 +01:00
Christian Kühnel
e18bd6fca7 incremented version number 2020-02-24 15:28:59 +00:00
Christian Kühnel
d00cd053e1 tagging and pushing "latest" 2020-02-20 16:45:04 +00:00
Christian Kühnel
548e07d5d4 added versioning to windows docker containers 2020-02-20 15:18:26 +00:00
Christian Kühnel
a0f676dfba set fixed windows version number 2020-02-20 09:15:11 +00:00
Christian Kühnel
dd375f71db using upstream jenkins image 2020-02-17 10:50:37 +01:00
Christian Kühnel
a4e320ac04 Merge branch 'master' of ssh://github.com/google/llvm-premerge-checks 2020-02-06 14:08:26 +00:00