revi-archive/llvm-premerge-checks
1
0
Fork 0
Code Activity
838 commits 20 branches 1 tag 1.7 MiB
Commit graph

421 commits

Author SHA1 Message Date
Mikhail Goncharov
82daa321a2 Don't use all projects for master / release builds 2020-06-04 14:10:08 +02:00
Mikhail Goncharov
e08efbcf2a Script updates: 
- option not to shut down sccache after 10 minutes
- zero sccache stats before running the build
- decrease number of jenkins agents
 2020-06-04 12:36:46 +02:00
Mikhail Goncharov
3170572590 don't report back to pahabricator in master build 2020-06-04 08:47:46 +02:00
Mikhail Goncharov
770f6726bc Add new pipeline to build a master branch 2020-06-04 08:40:04 +02:00
Mikhail Goncharov
ac3e35d965 fix string format 2020-06-03 17:21:49 +02:00
Mikhail Goncharov
017ca44a75 Add windows build 
- configured sccache on buildkite windows machines
- final result is sent to phabricator from "summary.py" that waits for both builds to complete
- extracted "add_url_artifact" to a runnable script
- reorganized code and fixed some of TODOs
 2020-06-03 13:40:22 +02:00
Christian Kühnel
654b6e5da8 ignoring mlir/examples/standalone in clang-tidy 2020-05-29 08:00:23 +02:00
Mikhail Goncharov
afed3a7898 Update docs and script for installing win machines 
I have tried to install new windows machine for buildkite and updated
some scrips and docs on along the way.

- Updated base image for k8s agent installation as it gave a warning that
previous version was old.

- Now buildkite secret is configured in a powershell script along with
tags (and possible other parameters).

- Split "windows_agent_start.ps1" to "..jenkins" and "..buildkite" as
  some parameters are different.

- Created a "windows-development" machine in GCP stat can be resumed
  to build docker images / expriment.
 2020-05-27 15:41:49 +02:00
Mikhail Goncharov
d796cfec6d fix typo 2020-05-26 12:54:31 +02:00
Mikhail Goncharov
72156ea180
Merge pull request #188 from google/bk-build-2 
building with buildkite
 2020-05-26 10:42:54 +02:00
Mikhail Goncharov
8eee5355ea building with buildkite 2020-05-25 16:42:40 +02:00
Christian Kühnel
856ce4b148 added missing dependencies to libc 2020-05-20 06:10:28 +02:00
Mikhail Goncharov
1505cfb635 build 2020-05-14 12:57:38 +02:00
Mikhail Goncharov
a152d97e3c Simple service to integrate harbormaster w/ buildkite 
Harbormaster can send a request in url encoded form while buildkite
expects a POST request with json. Instead of modifying harbormastar or
buildkite this adds a simple proxy that accepts url encoded form and
creates a request that buildkite expects. To avoid potential abuse,
nginx asks for simple http auth credentials stored in harbormaster.

All build parameters passed by proxy are put into build metadata and as
ph_* env variables available during the build.

Secrets involved:
- harbormastert knows http-auth to proxy (stored in privatly and in k8
  buildkite/http-auth as auth file);
- proxy knows buildkite api token (mine atm);
- build agent knows conduit API token (mine atm),
  and SSH key of llvm-premerge-tests-bot (in k8 buildkite/github-ssh).

Sample build: https://reviews.llvm.org/harbormaster/build/64828/8/
https://buildkite.com/llvm-project/premerge/builds/48
 2020-05-13 13:13:33 +02:00
Christian Kühnel
169744c617 extended log scan preiod to 90 days 
fixes #181
 2020-05-12 08:46:24 +02:00
Christian Kühnel
514a87aa4f added missing dependency 2020-05-11 16:55:20 +02:00
Christian Kühnel
2264ff6c41 fix issue as 'all' can't be built 2020-05-11 16:09:27 +02:00
Christian Kühnel
07404db9a2 disabling lldb again 2020-05-11 09:18:57 +02:00
Christian Kühnel
da722d3e40 continuing after failures 2020-05-08 16:24:16 +02:00
Christian Kühnel
510ed904f8 adding flush after every print 
so it appears in the Jenkins logs
 2020-05-08 15:36:13 +02:00
Christian Kühnel
8ea2a3b846 using different way for deleting remote branches 2020-05-08 15:33:45 +02:00
Christian Kühnel
d48ff97d0d only push if we deleted something 2020-05-08 15:27:09 +02:00
Christian Kühnel
45b28dd0df first version of repo cleanup script 2020-05-08 15:23:10 +02:00
Christian Kühnel
4c48379707 improved windows agent creation 2020-05-08 13:19:04 +02:00
Christian Kühnel
f9a9fb2be0 lldb tests were fixed, re-enabling it 2020-05-07 17:27:07 +02:00
Christian Kühnel
795d8270f7 fixed report writing 2020-05-06 17:24:26 +02:00
Christian Kühnel
10454dfba8 improved error handling for agent identification 2020-05-06 13:11:28 +02:00
Christian Kühnel
0e12d53def added agent detection 2020-05-06 12:43:35 +02:00
Christian Kühnel
971d4b8b74 fixes #175 2020-05-06 12:41:58 +02:00
Christian Kühnel
d55096bb7c getting workflow stages 2020-05-06 12:06:40 +02:00
Christian Kühnel
27da151340 creating tmp dir as needed 2020-05-05 16:54:08 +02:00
Christian Kühnel
9fabda9800 wiping sccache on timeouts 2020-05-04 14:22:49 +00:00
Christian Kühnel
b819a22205 added --dryrun 2020-05-04 14:16:50 +00:00
Christian Kühnel
5fd70b614d disabled lldb because of failing tests 
see https://results.llvm-merge-guard.org/master_debian_clang8-2152/console-log.txt
 2020-04-29 12:17:52 +02:00
Christian Kühnel
9841a13bbf re-enabled lldb 
after failing test was fixed
 2020-04-29 11:46:26 +02:00
Mikhail Goncharov
39667898a7 use env variables 2020-04-24 15:43:09 +02:00
Mikhail Goncharov
1a3952afc4
Merge pull request #171 from google/run-bootstrap-from-scripts 
Checkout scripts and define pipeline in python
 2020-04-24 14:48:09 +02:00
Mikhail Goncharov
e860c29a47 Checkout scripts and define pipeline in python 2020-04-24 14:47:00 +02:00
Christian Kühnel
9db1fd9654 disabled compiler-rt 2020-04-24 13:56:28 +02:00
Christian Kühnel
cd7f06c58d removed lldb 2020-04-24 11:59:37 +02:00
Christian Kühnel
6e2d4d612a disabled lldb 
test keeps failing, problem with lack of IPv6 networking on Kubernetes:
lldb-unit :: Host/./HostTests/ConnectionFileDescriptorTest.TCPGetURIv6
 2020-04-24 09:29:56 +02:00
Christian Kühnel
184d216b24 Flush stdout 
So that the log output appears in the Jenkins logs before the cmake output.
 2020-04-23 15:07:54 +02:00
Christian Kühnel
58ce45a053 Revert "added -DCMAKE_BUILD_TYPE=RelWithDebInfo for Windows builds" 
This reverts commit 963a5abc7f.
 2020-04-23 11:45:52 +02:00
Christian Kühnel
d1a1896933 excluding flang on Windows 2020-04-23 11:40:40 +02:00
Christian Kühnel
963a5abc7f added -DCMAKE_BUILD_TYPE=RelWithDebInfo for Windows builds 2020-04-23 11:31:38 +02:00
Christian Kühnel
71144d1922 By default: building all enabled projects. 
This is the fallback solution in case the project detection fails for
whatever reason. This is also used for building the master branches.
 2020-04-23 11:22:31 +02:00
ChristianKuehnel
a47751ebe8
Upgrade Windows agents to Visual Studio 2019 and sccache (#162) 
* added cmake parameters for sccache

* added sccache

* removed stray debug output

* starting sccache with VS environment

* added container for Visual Studio 2019

* fixed comments

* considering WIndows version

* updated to vs2019

* using MS install method

* snapshot of VS2019 experiments

* using --installRecommended

* cleanup of dockerfile

* updated path

* fixed dockerfile

* dumped version number

* exclude for virus scan

* added testing option

this does not start the agent

* write results on failures

* added timeouts for pipelines

* moving master pipelines to python scripts

* added flang to automatic project selection

based on #159, this will enable flang for beta testers

* added persistent workspace for testing containers

* added secure delete function

* added better log message

* deleting read-only files

* checking existence before setting flags

* using unlink

* deleting recursively

* using pathlib for chmod

* using custom workspace

* fixed drive

* separate handling of single files

* simplified read-only handling again

* removed vsdevcmd calls

as it's already set in the docker Entrypoint

* renamed container folders

* windows version not needed any more

* bumped version number

* bumped version number

* script cleanup

* added cmake parameters for sccache

* added sccache

* starting sccache with VS environment

* added container for Visual Studio 2019

* fixed comments

* considering WIndows version

* updated to vs2019

* using MS install method

* snapshot of VS2019 experiments

* using --installRecommended

* cleanup of dockerfile

* updated path

* fixed dockerfile

* dumped version number

* removed vsdevcmd calls

as it's already set in the docker Entrypoint

* renamed container folders

* windows version not needed any more

* bumped version number

* bumped version number

* script cleanup

* removed sccache from vs2017

* making windows image configurable

* added versioning

* created windows BETA pipeline

* added Jenkins label for vs2019
 2020-04-21 11:46:41 +02:00
Christian Kühnel
ba3e9c29da added lldb 2020-04-21 09:27:59 +02:00
Christian Kühnel
f4f6f5881c removed redundant backslashes 2020-04-17 16:33:51 +02:00
Christian Kühnel
7b90806189 removing --ignore-whitespace --whitespace=fix from git apply 
Now patches are applies as they are with whatever white space changes
they contain.

fixes #147
 2020-04-17 16:33:26 +02:00
Christian Kühnel
140d9f6f59 changed Error to Info 
fixes #155
 2020-04-17 16:25:27 +02:00
Mehdi Amini
aea569d4c8
Add flang to the list of project to test (#159) 
Co-authored-by: ChristianKuehnel <kuhnel@google.com>
 2020-04-17 14:30:18 +02:00
Christian Kühnel
bf0fb3fc68 simplified read-only handling again 2020-04-16 13:54:50 +02:00
Christian Kühnel
620945da3c separate handling of single files 2020-04-16 13:52:08 +02:00
Christian Kühnel
8e45e65375 using pathlib for chmod 2020-04-16 10:25:24 +00:00
Christian Kühnel
e3b384dc7f deleting recursively 2020-04-16 11:23:02 +02:00
Christian Kühnel
2850e18a47 using unlink 2020-04-16 11:15:40 +02:00
Christian Kühnel
66444f4952 checking existence before setting flags 2020-04-16 11:06:26 +02:00
Christian Kühnel
7521aeeda1 deleting read-only files 2020-04-16 10:30:39 +02:00
Christian Kühnel
b2c68fa9be added better log message 2020-04-16 10:12:57 +02:00
Christian Kühnel
eaa931da8c added secure delete function 2020-04-15 18:16:41 +02:00
Christian Kühnel
b8dcbcb273 added flang to automatic project selection 
based on #159, this will enable flang for beta testers
 2020-04-15 17:59:02 +02:00
Christian Kühnel
e1a8db6108 removed stray print statement 2020-04-15 17:54:18 +02:00
Christian Kühnel
e77f46914c write results on failures 2020-04-02 09:44:13 +02:00
Christian Kühnel
ace846991e added testing option 
this does not start the agent
 2020-04-02 08:42:08 +02:00
Christian Kühnel
2f25035eb7 exclude for virus scan 2020-04-01 17:36:03 +02:00
Christian Kühnel
6d9b38c62a wait for folder to be deleted 
should fix issues with existing folder on windows
 2020-04-01 16:29:19 +02:00
Christian Kühnel
0a7b6daabf added dryrun flag 2020-04-01 14:17:49 +02:00
Christian Kühnel
0e84fd14bb gettig stats from buildbots 2020-03-31 16:56:35 +02:00
Christian Kühnel
51e01caa5d excluding debuginfo-tests as tests are failing 2020-03-27 14:19:52 +01:00
Christian Kühnel
4465b0fc52 changed output paths of lint.sh 2020-03-26 10:19:20 +01:00
Christian Kühnel
580d4ba63c fixed linter script 
now support staging area and HEAD~1
 2020-03-26 09:47:45 +01:00
Christian Kühnel
3f81b7fb2a changed startup type for docker 2020-03-25 16:12:31 +01:00
Christian Kühnel
8fd5aa6550 SSD seems to have a new name... 2020-03-25 16:00:57 +01:00
Christian Kühnel
248a27b2c2 fixed ordering of commands to test git pull 2020-03-25 12:42:03 +00:00
Christian Kühnel
1df9edbd78 logging more data 2020-03-25 12:34:22 +00:00
Christian Kühnel
9a1988fece simplified benchmark steps 2020-03-25 11:40:51 +01:00
Christian Kühnel
188cb427f9 using default cmake targets 2020-03-25 11:21:45 +01:00
Christian Kühnel
9d17569b80 Merge branch 'master' of ssh://github.com/google/llvm-premerge-checks 2020-03-25 09:51:02 +00:00
Christian Kühnel
b212480d76 added missing cmake steps 2020-03-25 09:50:58 +00:00
Christian Kühnel
ff730bd3a4 removed unused benchmark.ps1 2020-03-25 10:41:39 +01:00
Christian Kühnel
ad685d68d5 using own Remove command 
Shutil failed on windows
 2020-03-25 09:41:19 +00:00
Christian Kühnel
cc3a1ffb57 added remove steps 2020-03-25 09:25:50 +00:00
Christian Kühnel
c0879efea1 first draft of benchmark script 
to be used for performance tuning Windows builds
 2020-03-25 09:07:28 +01:00
Christian Kühnel
06ca090e8c fixes #149 2020-03-25 07:37:25 +01:00
Christian Kühnel
67a9698557 printing CMake arguments before execution. 
Should help analyzing problems.
 2020-03-25 07:35:03 +01:00
Christian Kühnel
9f41b8768b added log output 
To get more data to analyze #149
 2020-03-25 07:26:55 +01:00
Christian Kühnel
2b3d0e127b Handle problems when unable to push to origin/master 2020-03-25 07:09:31 +01:00
Christian Kühnel
7023688d1b fixxed choose_project for legagacy usage 2020-03-24 08:27:43 +00:00
Christian Kühnel
4492534331 creating symlink for compile_commands.json 2020-03-23 17:46:21 +01:00
Christian Kühnel
fe92469ae0 create branch for every diff 
This should be compatible with the current pipelines.
 2020-03-23 10:39:49 +01:00
ChristianKuehnel
a638dd5342
migrate scripts to python (#148) 
* run_cmake.py working locally on Linux

* added os detection of choose_projects

* cmake works on windows

* run_ninja works on windows

* fixed newline characters
 2020-03-23 09:03:24 +01:00
Christian Kühnel
5f2b26d44c added raw data export 2020-03-19 09:44:41 +01:00
Christian Kühnel
bfdf97e10a added reviews per path 2020-03-18 14:18:07 +01:00
Christian Kühnel
02493d440d fine tuning of metrics 2020-03-17 11:22:55 +01:00
Christian Kühnel
75d432e913 added documentation, fixed constructor 2020-03-13 16:58:12 +01:00
Christian Kühnel
5f4983f824 Extended documentation of status quo. 2020-03-13 16:57:37 +01:00
Christian Kühnel
cc0058c17e Improved sync script to consider individual diffs. 2020-03-13 13:50:30 +01:00
Christian Kühnel
1d171c0252 creating pull requests works 2020-03-13 13:50:30 +01:00
Christian Kühnel
4aed7decc3 creating branches working 2020-03-13 13:50:30 +01:00
Christian Kühnel
bed16617e7 cloning repo works 2020-03-13 13:50:30 +01:00
Christian Kühnel
0861e0ed25 added outline of the idea 2020-03-13 13:50:30 +01:00
Mikhail Goncharov
814517c89e add link to a full diff 2020-03-10 14:28:56 +01:00
Mikhail Goncharov
3ee08f18bc Ignore 'No relevant changes found.' text 2020-03-10 13:46:49 +01:00
Mikhail Goncharov
2fc3174a5c fix empty line removal for clang-tidy #136 2020-03-10 13:21:12 +01:00
Mikhail Goncharov
3bf868ec83 add link to build log 2020-03-10 13:07:26 +01:00
Mikhail Goncharov
dee5385784 Truncate clang-format comments 
If clang format diff is longer that 11 lines only show first 10 and add
note about omitted ones.

for #144
 2020-03-10 11:59:25 +01:00
Christian Kühnel
7131ef705c ignoring case in emails 2020-03-09 09:34:52 +01:00
Christian Kühnel
2975323644 added statistic on committer domains 2020-03-09 09:31:25 +01:00
Christian Kühnel
2eedd55b2b first jenkins statistics 2020-02-28 11:31:14 +01:00
Christian Kühnel
d1029befcc added first statistics for Jenkins 2020-02-27 23:54:54 +01:00
Alex Brachet
b8a3b31b48
Add libc to Pre-merge checks (#140) 2020-02-27 00:52:06 +01:00
Christian Kühnel
289d1c1205 added diffs 2020-02-24 17:54:11 +01:00
Christian Kühnel
c3c8f039cf try fixing whitespaces 2020-02-24 17:25:47 +01:00
Christian Kühnel
56d9ec34ab implemented Phabricator metrics 2020-02-24 16:38:29 +01:00
Christian Kühnel
14aaab86b1 upgrade gitpython to 3.0.6 
fixed #137
 2020-02-24 15:04:46 +00:00
Christian Kühnel
655f151bb8 first shot at getting build data from Phabricator 2020-02-21 13:20:33 +01:00
Christian Kühnel
814cd775bc script to get statistics from git repo 2020-02-21 13:19:11 +01:00
Christian Kühnel
3d3de17eb0 first version of git history evaluation 2020-02-20 19:04:34 +01:00
Christian Kühnel
3debb3046c code cleanup 2020-02-20 19:04:34 +01:00
Christian Kühnel
7fe1682620 start in background 2020-02-20 16:44:47 +00:00
Christian Kühnel
ee2d99ebd2 made master allow version numbers 2020-02-20 15:25:23 +00:00
Christian Kühnel
d91c7b653a flag to pick master 2020-02-20 15:24:09 +00:00
Mikhail Goncharov
70df407e0e Add name of build agent to summary 2020-02-19 17:48:51 +01:00
Mikhail Goncharov
c1396546e5 Remove empty lines at the end of clang-tidy report 2020-02-19 16:17:50 +01:00
Christian Kühnel
63ad2d1be6 retrying on phab.update_interfaces() 2020-02-19 12:08:18 +01:00
Christian Kühnel
b17bd93438 fixed division by zero problem 2020-02-18 13:37:13 +01:00
Christian Kühnel
22ce8e9632 handling broken XML files 2020-02-18 09:55:30 +01:00
Christian Kühnel
71240e7473 fixed types 2020-02-17 16:42:41 +00:00
Christian Kühnel
3b593e36cf no error message when no clang-tidy/format files given 2020-02-17 16:34:00 +00:00
Christian Kühnel
1c53d6ee86 added number of reviewed commits 2020-02-17 16:18:58 +01:00
Christian Kühnel
a2973f0f79 gathering data for reverts 2020-02-17 15:30:10 +01:00
Christian Kühnel
29c4333033 fixed initialisation 2020-02-17 14:55:10 +01:00
Christian Kühnel
ad0ad7ccc5 counting failed builds on buildbots 
storing results on Stackdriver
 2020-02-17 14:27:59 +01:00
ChristianKuehnel
16952faa46
Proposal for metrics (#127) 
* First draft for metrics
 2020-02-17 10:39:53 +01:00
ChristianKuehnel
f6ad5451a1
Improving phabtalk (#128) 
* enabled dryrun for add_artifact
 2020-02-17 10:23:47 +01:00
Christian Kühnel
2c36c12b93 remove llgo 2020-02-17 09:19:55 +00:00
Christian Kühnel
2da32a0cdf using git apply 
fixes problems for failing windows builds
 2020-02-17 09:08:34 +00:00
Christian Kühnel
a5f6516d71 re-enabled mlir 2020-02-11 11:54:22 +00:00
Christian Kühnel
64c68f1671 blacklisted more failing tests 2020-02-11 09:23:24 +01:00
Christian Kühnel
c7608ce4df blacklisting mlir 2020-02-10 18:14:51 +01:00
Christian Kühnel
f588cdf43d changing git diff after changes were committed 2020-02-10 16:28:41 +01:00
Mikhail Goncharov
fd5c753c69 small style fixes 2020-02-07 14:50:47 +01:00
Mikhail Goncharov
6b6558e22c Add reason of failure on top of summary 2020-02-07 12:30:39 +01:00
Mikhail Goncharov
41581c95be format failed tests results 2020-02-07 09:04:41 +01:00
Mikhail Goncharov
6012e873b5 ignore successfull 4 byte clang tidy report 2020-02-07 08:16:35 +01:00
Mikhail Goncharov
490939a4b7 also retry 'update_indterfaces' 2020-02-07 08:12:51 +01:00
Mikhail Goncharov
1859a7d4d3 trying to fix partial test XML 2020-02-06 17:48:36 +01:00
Mikhail Goncharov
144354bc04 Generate HTML summary instead of adding comment 
+ reworked phab-talk, now it's a class instead of free functions
 2020-02-06 16:49:42 +01:00
Mikhail Goncharov
0519b5a4e3 Attach artifacts to build 
Generate test summary HTML
 2020-02-06 14:24:31 +01:00