Christian Kühnel
c75649476d
simple report on master failues
2020-08-07 10:11:31 +02:00
Christian Sigg
36808fc0d0
No clang-tidy under mlir-{cuda,rocm}-runner. ( #226 )
...
All files include CUDA/ROCm headers, which trips up clang-tidy without --cuda-path/--rocm-path.
2020-08-03 17:15:58 +02:00
Christian Sigg
e5be8d45ca
clang-tidy ignore {cuda,rocm}-runtime-wrappers.cpp
...
Both files include GPU header files for which clang implicitly adds include directories. However, clang-tidy does not and therefore we get compile errors (i.e. not diagnostics that we could disable with NOLINT).
2020-07-30 18:14:07 +02:00
Mikhail Goncharov
cb41bb38dd
reset before checking out master
2020-07-30 09:32:50 +02:00
Mikhail Goncharov
d75dcdd2fb
cleanup before checking out master
2020-07-30 09:27:05 +02:00
Mikhail Goncharov
9cfdbbb8fe
fix git clenup
2020-07-30 09:20:19 +02:00
Mikhail Goncharov
e41afba72d
update docs
2020-07-27 19:16:33 +02:00
Mikhail Goncharov
3c28129968
custom agent constraints
...
+ updated image for win machines
2020-07-24 15:59:00 +02:00
Mikhail Goncharov
fb5e91fdeb
fix auto revision usage
2020-07-24 10:10:48 +02:00
Mikhail Goncharov
5b07569602
fix auto revision usage
2020-07-24 10:06:04 +02:00
Mikhail Goncharov
57702114b0
fix fallback to master branch
2020-07-24 08:58:51 +02:00
Mikhail Goncharov
2932ea50e3
handle broader range or git exceptions
2020-07-23 21:49:44 +02:00
Mikhail Goncharov
f99e98e434
use logging in apply_patch2
...
+ possible fix for base commit
+ ph_ options to control commit behavior
2020-07-23 20:24:02 +02:00
Mikhail Goncharov
ac3cdef23d
Add clang-tidy and clang-format command to "reproduce locally"
2020-07-23 09:50:06 +02:00
Mikhail Goncharov
00a2c76d0b
Merge pull request #219 from google/ignore-tests-format
...
ignore **/test for clang-format
2020-07-22 19:34:34 +02:00
Mikhail Goncharov
3d1d7af150
Merge pull request #218 from google/notify
...
Notify
2020-07-22 19:22:52 +02:00
Mikhail Goncharov
5362bc52a3
ignore **/test for clang-format
2020-07-22 18:59:47 +02:00
Mikhail Goncharov
82d0ae033b
fix script branch commit message
2020-07-22 17:04:02 +02:00
Mikhail Goncharov
3ababb3b88
notify-email
2020-07-22 16:08:52 +02:00
Mikhail Goncharov
605f18bad1
Add link to ninja log instead of XML #216
...
XML tooks too long to load and not that helpful
+ link tell how many tests have failed
+ link is added only if something is off
2020-07-22 15:45:53 +02:00
Mikhail Goncharov
fa11647670
retry build step if agent is gone or shut down
2020-07-21 10:42:46 +02:00
Mikhail Goncharov
d4e5126090
Update docs on ssd disk initialization
...
+ change number of linux agents
+ removed outputs in job analysis
+ updated windows image
2020-07-21 10:40:26 +02:00
Mikhail Goncharov
42d7237491
Add step to schedule agent restart on windows
2020-07-17 18:37:29 +02:00
Mikhail Goncharov
95cdccb68f
ipython notebook to analyze build speed per agent
...
+ add ssd as an option to windows agent bootstrap
2020-07-16 18:04:09 +02:00
Mikhail Goncharov
304f2e88d5
Additional logging for project selection
...
+ option to skip win / linux build from diff checks
2020-07-14 18:26:53 +02:00
Mikhail Goncharov
89d294ef42
return exit status of ninja command
...
- Fix for #207 . Previously grep was exiting with 1 and fail build if all
lines from ninja are filtered. Also piping might hide exit code of the
initial command. Now "tee" and "grep" is implemented on python level
and we get exit code of the original command.
- Fixed default value of "scripts_branch", print scripts commit;
- Added option to filter ninja output (on by default as before).
- Copied cache usege reporting and error handling from "master" branch
build to diff checks (useful for debugging).
2020-07-10 19:35:29 +02:00
Mikhail Goncharov
6bf3c85cc6
Print commands to reproduce build locally
...
- additional arguments to set enabled projects, log level and reset
cache.
2020-07-09 15:31:34 +02:00
Mikhail Goncharov
7667fc8004
Allow "building" in ninja all log
...
That is a temporary workaround for #207 that caused by grep exiting with
1 as whole `ninja all` output is filtered out.
2020-07-09 15:09:58 +02:00
Mikhail Goncharov
0e36e0cf7c
Disable openmp build on Windows until it fixed
2020-07-09 11:00:47 +02:00
Mikhail Goncharov
3f52fb7969
Show stats on linux / windows
2020-07-08 14:32:59 +02:00
Mikhail Goncharov
f125728908
remove compiler-rt and flang temporarily
2020-07-08 12:01:17 +02:00
Mikhail Goncharov
1fa24a55ec
Add option to disable cache
2020-07-08 09:45:22 +02:00
Mikhail Goncharov
1421dc1184
Merge pull request #199 from joker-eph/build_examples
...
Enable -DLLVM_BUILD_EXAMPLES=ON during pre-merge testing
2020-07-08 09:09:53 +02:00
Mikhail Goncharov
517f0309dd
Merge pull request #202 from fooishbar/clang-tidy-ignore-libclc
...
clang-tidy: Ignore all of libclc source
2020-07-07 15:48:34 +02:00
Mikhail Goncharov
efaa8b73e6
fix build command on windows
2020-07-07 15:13:43 +02:00
Mikhail Goncharov
23fecd6765
additional projects for master branch builds
2020-07-01 09:31:13 +02:00
Mikhail Goncharov
ffa1882445
Specify timeout for every step
2020-06-29 12:17:03 +02:00
Mikhail Goncharov
af7204418f
error handling
2020-06-29 11:41:48 +02:00
Mikhail Goncharov
9a2a236dd0
log text updates
2020-06-29 11:14:47 +02:00
Mikhail Goncharov
6fb2c53be8
cleanup script logging and failure count fix
2020-06-29 11:07:05 +02:00
Mikhail Goncharov
ac3c287766
chmod +x cleanup_branches
2020-06-29 10:56:21 +02:00
Mikhail Goncharov
4241d6b53e
Doc update
2020-06-29 10:56:21 +02:00
Mikhail Goncharov
5205b53fbf
add directory
2020-06-18 11:54:28 +02:00
Mikhail Goncharov
fb830daebe
Update clang version
...
apt.llvm.org signature has changed, we also should use debian:stable
(=buster) as a base image to correctly install packages.
Removed jdk from the base image as it was used only by Jenkins.
Log packages version for every build.
Related issues: #112
2020-06-18 10:55:15 +02:00
Mikhail Goncharov
89e3f73689
check out fork into different directory
2020-06-17 16:09:10 +02:00
Mikhail Goncharov
24da609a10
Flush stdout to fix log update
...
for #201
2020-06-17 15:07:19 +02:00
Mikhail Goncharov
9e3995bbe0
get queue prefix from URL params
2020-06-17 13:52:33 +02:00
Daniel Stone
d26f5ef00e
clang-tidy: Ignore all of libclc source
...
libclc implements OpenCL C, which is not actual C/C++, and implements an
external API with different naming conventions.
As clang-tidy cannot parse the source[0], just blacklist all of libclc's
sources for now.
[0]: https://buildkite.com/llvm-project/premerge-checks/builds/695
2020-06-17 12:00:26 +01:00
Mikhail Goncharov
ffaba986c1
Deployment changes
...
do not restart win docker as it makes it impossible to stop agent from
buildkite UI for maintenance
2020-06-17 09:46:42 +02:00
Mikhail Goncharov
af7bd86875
disable clang format / tidy for master and release builds
2020-06-16 17:29:04 +02:00
Mikhail Goncharov
4c3cd1b6f5
Use os in queue name
...
That is useful for metrics collection as buildkite metrics only reports
per queue metrics
2020-06-16 10:30:24 +02:00
Mehdi Amini
655f87d98c
Enable -DLLVM_BUILD_EXAMPLES=ON during pre-merge testing
2020-06-15 19:16:39 +00:00
Mikhail Goncharov
6812950908
Report build error if failed to apply patch
2020-06-05 12:32:09 +02:00
Mikhail Goncharov
de0d5afd78
Move agents to default queue
...
- fix windows defender exclusion path
2020-06-05 10:50:42 +02:00
Mikhail Goncharov
82daa321a2
Don't use all projects for master / release builds
2020-06-04 14:10:08 +02:00
Mikhail Goncharov
e08efbcf2a
Script updates:
...
- option not to shut down sccache after 10 minutes
- zero sccache stats before running the build
- decrease number of jenkins agents
2020-06-04 12:36:46 +02:00
Mikhail Goncharov
3170572590
don't report back to pahabricator in master build
2020-06-04 08:47:46 +02:00
Mikhail Goncharov
770f6726bc
Add new pipeline to build a master branch
2020-06-04 08:40:04 +02:00
Mikhail Goncharov
ac3e35d965
fix string format
2020-06-03 17:21:49 +02:00
Mikhail Goncharov
017ca44a75
Add windows build
...
- configured sccache on buildkite windows machines
- final result is sent to phabricator from "summary.py" that waits for both builds to complete
- extracted "add_url_artifact" to a runnable script
- reorganized code and fixed some of TODOs
2020-06-03 13:40:22 +02:00
Christian Kühnel
654b6e5da8
ignoring mlir/examples/standalone in clang-tidy
2020-05-29 08:00:23 +02:00
Mikhail Goncharov
afed3a7898
Update docs and script for installing win machines
...
I have tried to install new windows machine for buildkite and updated
some scrips and docs on along the way.
- Updated base image for k8s agent installation as it gave a warning that
previous version was old.
- Now buildkite secret is configured in a powershell script along with
tags (and possible other parameters).
- Split "windows_agent_start.ps1" to "..jenkins" and "..buildkite" as
some parameters are different.
- Created a "windows-development" machine in GCP stat can be resumed
to build docker images / expriment.
2020-05-27 15:41:49 +02:00
Mikhail Goncharov
d796cfec6d
fix typo
2020-05-26 12:54:31 +02:00
Mikhail Goncharov
72156ea180
Merge pull request #188 from google/bk-build-2
...
building with buildkite
2020-05-26 10:42:54 +02:00
Mikhail Goncharov
8eee5355ea
building with buildkite
2020-05-25 16:42:40 +02:00
Christian Kühnel
856ce4b148
added missing dependencies to libc
2020-05-20 06:10:28 +02:00
Mikhail Goncharov
1505cfb635
build
2020-05-14 12:57:38 +02:00
Mikhail Goncharov
a152d97e3c
Simple service to integrate harbormaster w/ buildkite
...
Harbormaster can send a request in url encoded form while buildkite
expects a POST request with json. Instead of modifying harbormastar or
buildkite this adds a simple proxy that accepts url encoded form and
creates a request that buildkite expects. To avoid potential abuse,
nginx asks for simple http auth credentials stored in harbormaster.
All build parameters passed by proxy are put into build metadata and as
ph_* env variables available during the build.
Secrets involved:
- harbormastert knows http-auth to proxy (stored in privatly and in k8
buildkite/http-auth as auth file);
- proxy knows buildkite api token (mine atm);
- build agent knows conduit API token (mine atm),
and SSH key of llvm-premerge-tests-bot (in k8 buildkite/github-ssh).
Sample build: https://reviews.llvm.org/harbormaster/build/64828/8/
https://buildkite.com/llvm-project/premerge/builds/48
2020-05-13 13:13:33 +02:00
Christian Kühnel
169744c617
extended log scan preiod to 90 days
...
fixes #181
2020-05-12 08:46:24 +02:00
Christian Kühnel
514a87aa4f
added missing dependency
2020-05-11 16:55:20 +02:00
Christian Kühnel
2264ff6c41
fix issue as 'all' can't be built
2020-05-11 16:09:27 +02:00
Christian Kühnel
07404db9a2
disabling lldb again
2020-05-11 09:18:57 +02:00
Christian Kühnel
da722d3e40
continuing after failures
2020-05-08 16:24:16 +02:00
Christian Kühnel
510ed904f8
adding flush after every print
...
so it appears in the Jenkins logs
2020-05-08 15:36:13 +02:00
Christian Kühnel
8ea2a3b846
using different way for deleting remote branches
2020-05-08 15:33:45 +02:00
Christian Kühnel
d48ff97d0d
only push if we deleted something
2020-05-08 15:27:09 +02:00
Christian Kühnel
45b28dd0df
first version of repo cleanup script
2020-05-08 15:23:10 +02:00
Christian Kühnel
4c48379707
improved windows agent creation
2020-05-08 13:19:04 +02:00
Christian Kühnel
f9a9fb2be0
lldb tests were fixed, re-enabling it
2020-05-07 17:27:07 +02:00
Christian Kühnel
795d8270f7
fixed report writing
2020-05-06 17:24:26 +02:00
Christian Kühnel
10454dfba8
improved error handling for agent identification
2020-05-06 13:11:28 +02:00
Christian Kühnel
0e12d53def
added agent detection
2020-05-06 12:43:35 +02:00
Christian Kühnel
971d4b8b74
fixes #175
2020-05-06 12:41:58 +02:00
Christian Kühnel
d55096bb7c
getting workflow stages
2020-05-06 12:06:40 +02:00
Christian Kühnel
27da151340
creating tmp dir as needed
2020-05-05 16:54:08 +02:00
Christian Kühnel
9fabda9800
wiping sccache on timeouts
2020-05-04 14:22:49 +00:00
Christian Kühnel
b819a22205
added --dryrun
2020-05-04 14:16:50 +00:00
Christian Kühnel
5fd70b614d
disabled lldb because of failing tests
...
see https://results.llvm-merge-guard.org/master_debian_clang8-2152/console-log.txt
2020-04-29 12:17:52 +02:00
Christian Kühnel
9841a13bbf
re-enabled lldb
...
after failing test was fixed
2020-04-29 11:46:26 +02:00
Mikhail Goncharov
39667898a7
use env variables
2020-04-24 15:43:09 +02:00
Mikhail Goncharov
1a3952afc4
Merge pull request #171 from google/run-bootstrap-from-scripts
...
Checkout scripts and define pipeline in python
2020-04-24 14:48:09 +02:00
Mikhail Goncharov
e860c29a47
Checkout scripts and define pipeline in python
2020-04-24 14:47:00 +02:00
Christian Kühnel
9db1fd9654
disabled compiler-rt
2020-04-24 13:56:28 +02:00
Christian Kühnel
cd7f06c58d
removed lldb
2020-04-24 11:59:37 +02:00
Christian Kühnel
6e2d4d612a
disabled lldb
...
test keeps failing, problem with lack of IPv6 networking on Kubernetes:
lldb-unit :: Host/./HostTests/ConnectionFileDescriptorTest.TCPGetURIv6
2020-04-24 09:29:56 +02:00
Christian Kühnel
184d216b24
Flush stdout
...
So that the log output appears in the Jenkins logs before the cmake output.
2020-04-23 15:07:54 +02:00
Christian Kühnel
58ce45a053
Revert "added -DCMAKE_BUILD_TYPE=RelWithDebInfo for Windows builds"
...
This reverts commit 963a5abc7f
.
2020-04-23 11:45:52 +02:00
Christian Kühnel
d1a1896933
excluding flang on Windows
2020-04-23 11:40:40 +02:00
Christian Kühnel
963a5abc7f
added -DCMAKE_BUILD_TYPE=RelWithDebInfo for Windows builds
2020-04-23 11:31:38 +02:00
Christian Kühnel
71144d1922
By default: building all enabled projects.
...
This is the fallback solution in case the project detection fails for
whatever reason. This is also used for building the master branches.
2020-04-23 11:22:31 +02:00
ChristianKuehnel
a47751ebe8
Upgrade Windows agents to Visual Studio 2019 and sccache ( #162 )
...
* added cmake parameters for sccache
* added sccache
* removed stray debug output
* starting sccache with VS environment
* added container for Visual Studio 2019
* fixed comments
* considering WIndows version
* updated to vs2019
* using MS install method
* snapshot of VS2019 experiments
* using --installRecommended
* cleanup of dockerfile
* updated path
* fixed dockerfile
* dumped version number
* exclude for virus scan
* added testing option
this does not start the agent
* write results on failures
* added timeouts for pipelines
* moving master pipelines to python scripts
* added flang to automatic project selection
based on #159 , this will enable flang for beta testers
* added persistent workspace for testing containers
* added secure delete function
* added better log message
* deleting read-only files
* checking existence before setting flags
* using unlink
* deleting recursively
* using pathlib for chmod
* using custom workspace
* fixed drive
* separate handling of single files
* simplified read-only handling again
* removed vsdevcmd calls
as it's already set in the docker Entrypoint
* renamed container folders
* windows version not needed any more
* bumped version number
* bumped version number
* script cleanup
* added cmake parameters for sccache
* added sccache
* starting sccache with VS environment
* added container for Visual Studio 2019
* fixed comments
* considering WIndows version
* updated to vs2019
* using MS install method
* snapshot of VS2019 experiments
* using --installRecommended
* cleanup of dockerfile
* updated path
* fixed dockerfile
* dumped version number
* removed vsdevcmd calls
as it's already set in the docker Entrypoint
* renamed container folders
* windows version not needed any more
* bumped version number
* bumped version number
* script cleanup
* removed sccache from vs2017
* making windows image configurable
* added versioning
* created windows BETA pipeline
* added Jenkins label for vs2019
2020-04-21 11:46:41 +02:00
Christian Kühnel
ba3e9c29da
added lldb
2020-04-21 09:27:59 +02:00
Christian Kühnel
f4f6f5881c
removed redundant backslashes
2020-04-17 16:33:51 +02:00
Christian Kühnel
7b90806189
removing --ignore-whitespace --whitespace=fix from git apply
...
Now patches are applies as they are with whatever white space changes
they contain.
fixes #147
2020-04-17 16:33:26 +02:00
Christian Kühnel
140d9f6f59
changed Error to Info
...
fixes #155
2020-04-17 16:25:27 +02:00
Mehdi Amini
aea569d4c8
Add flang to the list of project to test ( #159 )
...
Co-authored-by: ChristianKuehnel <kuhnel@google.com>
2020-04-17 14:30:18 +02:00
Christian Kühnel
bf0fb3fc68
simplified read-only handling again
2020-04-16 13:54:50 +02:00
Christian Kühnel
620945da3c
separate handling of single files
2020-04-16 13:52:08 +02:00
Christian Kühnel
8e45e65375
using pathlib for chmod
2020-04-16 10:25:24 +00:00
Christian Kühnel
e3b384dc7f
deleting recursively
2020-04-16 11:23:02 +02:00
Christian Kühnel
2850e18a47
using unlink
2020-04-16 11:15:40 +02:00
Christian Kühnel
66444f4952
checking existence before setting flags
2020-04-16 11:06:26 +02:00
Christian Kühnel
7521aeeda1
deleting read-only files
2020-04-16 10:30:39 +02:00
Christian Kühnel
b2c68fa9be
added better log message
2020-04-16 10:12:57 +02:00
Christian Kühnel
eaa931da8c
added secure delete function
2020-04-15 18:16:41 +02:00
Christian Kühnel
b8dcbcb273
added flang to automatic project selection
...
based on #159 , this will enable flang for beta testers
2020-04-15 17:59:02 +02:00
Christian Kühnel
e1a8db6108
removed stray print statement
2020-04-15 17:54:18 +02:00
Christian Kühnel
e77f46914c
write results on failures
2020-04-02 09:44:13 +02:00
Christian Kühnel
ace846991e
added testing option
...
this does not start the agent
2020-04-02 08:42:08 +02:00
Christian Kühnel
2f25035eb7
exclude for virus scan
2020-04-01 17:36:03 +02:00
Christian Kühnel
6d9b38c62a
wait for folder to be deleted
...
should fix issues with existing folder on windows
2020-04-01 16:29:19 +02:00
Christian Kühnel
0a7b6daabf
added dryrun flag
2020-04-01 14:17:49 +02:00
Christian Kühnel
0e84fd14bb
gettig stats from buildbots
2020-03-31 16:56:35 +02:00
Christian Kühnel
51e01caa5d
excluding debuginfo-tests as tests are failing
2020-03-27 14:19:52 +01:00
Christian Kühnel
4465b0fc52
changed output paths of lint.sh
2020-03-26 10:19:20 +01:00
Christian Kühnel
580d4ba63c
fixed linter script
...
now support staging area and HEAD~1
2020-03-26 09:47:45 +01:00
Christian Kühnel
3f81b7fb2a
changed startup type for docker
2020-03-25 16:12:31 +01:00
Christian Kühnel
8fd5aa6550
SSD seems to have a new name...
2020-03-25 16:00:57 +01:00
Christian Kühnel
248a27b2c2
fixed ordering of commands to test git pull
2020-03-25 12:42:03 +00:00
Christian Kühnel
1df9edbd78
logging more data
2020-03-25 12:34:22 +00:00
Christian Kühnel
9a1988fece
simplified benchmark steps
2020-03-25 11:40:51 +01:00
Christian Kühnel
188cb427f9
using default cmake targets
2020-03-25 11:21:45 +01:00
Christian Kühnel
9d17569b80
Merge branch 'master' of ssh://github.com/google/llvm-premerge-checks
2020-03-25 09:51:02 +00:00
Christian Kühnel
b212480d76
added missing cmake steps
2020-03-25 09:50:58 +00:00
Christian Kühnel
ff730bd3a4
removed unused benchmark.ps1
2020-03-25 10:41:39 +01:00
Christian Kühnel
ad685d68d5
using own Remove command
...
Shutil failed on windows
2020-03-25 09:41:19 +00:00
Christian Kühnel
cc3a1ffb57
added remove steps
2020-03-25 09:25:50 +00:00
Christian Kühnel
c0879efea1
first draft of benchmark script
...
to be used for performance tuning Windows builds
2020-03-25 09:07:28 +01:00
Christian Kühnel
06ca090e8c
fixes #149
2020-03-25 07:37:25 +01:00
Christian Kühnel
67a9698557
printing CMake arguments before execution.
...
Should help analyzing problems.
2020-03-25 07:35:03 +01:00
Christian Kühnel
9f41b8768b
added log output
...
To get more data to analyze #149
2020-03-25 07:26:55 +01:00
Christian Kühnel
2b3d0e127b
Handle problems when unable to push to origin/master
2020-03-25 07:09:31 +01:00
Christian Kühnel
7023688d1b
fixxed choose_project for legagacy usage
2020-03-24 08:27:43 +00:00
Christian Kühnel
4492534331
creating symlink for compile_commands.json
2020-03-23 17:46:21 +01:00
Christian Kühnel
fe92469ae0
create branch for every diff
...
This should be compatible with the current pipelines.
2020-03-23 10:39:49 +01:00
ChristianKuehnel
a638dd5342
migrate scripts to python ( #148 )
...
* run_cmake.py working locally on Linux
* added os detection of choose_projects
* cmake works on windows
* run_ninja works on windows
* fixed newline characters
2020-03-23 09:03:24 +01:00
Christian Kühnel
5f2b26d44c
added raw data export
2020-03-19 09:44:41 +01:00
Christian Kühnel
bfdf97e10a
added reviews per path
2020-03-18 14:18:07 +01:00
Christian Kühnel
02493d440d
fine tuning of metrics
2020-03-17 11:22:55 +01:00
Christian Kühnel
75d432e913
added documentation, fixed constructor
2020-03-13 16:58:12 +01:00