revi-archive/llvm-premerge-checks
1
0
Fork 0
Code Activity
1093 commits 20 branches 1 tag 1.7 MiB
Commit graph

560 commits

Author SHA1 Message Date
Mikhail Goncharov
b88d7a8dba always set buildkite branch and commit env 2022-03-05 17:35:41 +01:00
Mikhail Goncharov
5a16fb7b9a sync all branches for llvm-project fork 
There are multiple scenarios when script expects to see same branches
and commits. E.g. when upstream commits to release branch trigger
the build.

Reported by Louis Dionne.
 2022-03-05 17:35:41 +01:00
Mikhail Goncharov
dae1793781 add monitoring agent to windows init script 2022-02-11 12:38:12 +01:00
Mikhail Goncharov
c41ee66634 use ld.lld for bolt 2022-02-03 10:16:13 +01:00
Mikhail Goncharov
f3ac1b41dd enable bolt premerge checks 2022-02-01 18:37:17 +01:00
Mikhail Goncharov
36d951c46e Trigger git gc on windows only for ~5% of builds. 
It takes almost 5 minutes on windows and we don't need to run it that
often.

Fixes #375
 2021-12-23 10:47:18 +01:00
Christian Kühnel
8d10520ca3 Revert "Updated configuration for BOLT" 
This reverts commit 823df07a07.
 2021-12-16 15:52:05 +00:00
Christian Kühnel
823df07a07 Updated configuration for BOLT 2021-12-16 09:29:41 +01:00
Christian Kühnel
54fa9b1556 added more queries 2021-12-14 11:40:34 +01:00
Christian Kühnel
4ae48e963d added buildbot daily success rates 2021-12-14 11:40:34 +01:00
Christian Kühnel
c2526bc84c moved queries to subfolder 2021-12-14 11:40:34 +01:00
Christian Kühnel
0beb4a43ff added sql statemet for bulidbot view 
This statement adds a VIEW "buildbot_overview" containing all relevant
 information for a build. I plan to extend this file with additional
 views when needed.
 2021-12-14 11:40:34 +01:00
dependabot[bot]
96653261f3 Bump lxml from 4.6.3 to 4.6.5 in /scripts/metrics 
Bumps [lxml](https://github.com/lxml/lxml) from 4.6.3 to 4.6.5.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.6.3...lxml-4.6.5)

---
updated-dependencies:
- dependency-name: lxml
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-12-14 11:40:06 +01:00
Mikhail Goncharov
4df2c2116c enable flang on windows 2021-11-24 15:36:54 +01:00
Danila Malyutin
c608b6cb91 Don't attempt to build parallel-libs project as it was removed from LLVM 
db0486c46f
 2021-10-23 08:48:11 +02:00
Mikhail Goncharov
bd2fd477b4 cleanup deps 2021-10-04 19:19:38 +02:00
Mikhail Goncharov
77cbb15b64 fix excessive environment copying 2021-10-04 16:58:52 +02:00
Mikhail Goncharov
b69eb6f364 don't check clang-tidy 
it seems quite unreliable
 2021-10-04 16:44:40 +02:00
Mikhail Goncharov
fea73f5174 fix environment assignment and update check- definitions 2021-10-04 16:36:02 +02:00
Mikhail Goncharov
7907aeb6a0 fix usage of get_all_enabled_projects 2021-10-04 16:15:16 +02:00
Mikhail Goncharov
df318ccf4f fix project resolutions 
and some of the typing issues found by mypy
 2021-10-04 16:07:06 +02:00
Mikhail Goncharov
3b7d720763 add post checkout to run git gc 2021-09-30 12:46:36 +02:00
Mikhail Goncharov
fc8bf86cbb disable libcxx on windows 2021-09-29 17:06:15 +02:00
Mehdi Amini
30be93ac21 Fix fallback more in choose_projects 
When a path can't be mapped, we fallback to a "all" project.
This was broken by the recent changes to be more selective on the
testing.
 2021-09-29 15:58:56 +02:00
Mehdi Amini
4ec28b0bf7 Mark libc as unsupported on Windows 2021-09-29 15:58:56 +02:00
Mikhail Goncharov
53a2c6835b remove libc for win 
there is no such target for clang "check-libc"
 2021-09-29 14:43:48 +02:00
Mikhail Goncharov
6e2f418a2a remove libcxxabi from win builds 2021-09-29 13:41:42 +02:00
Mikhail Goncharov
1cc122d5f9 disable check-libc and check-cxxabi 2021-09-29 11:15:54 +02:00
Mikhail Goncharov
25f12dacbd disable libunwind on windows 2021-09-21 14:30:29 +02:00
Mehdi Amini
6e624c30f9 Change the premerge checks to only check the affected projects 
The current setup is configuring the "affected projects" as well
as their dependencies, and run `ninja all` followed by
`ninja check-all`.

This is quite a pessimization for leaf project which don't need to
build and run the tests for their dependencies.
For example a patch affecting only MLIR shouldn't need to build
and run LLVM and clang tests.
This patch changes this by running checks only for the affected
project. For example a patch touching `mlir` is affecting `mlir`
and `flang`. However `flang` depends on `clang`. So the list of
projects to configure is `mlir;flang;clang;llvm;`, but we want
to test only mlir and flang ; we'll run only `ninja check-mlir
check-flang`.

In practice in this example running `ninja all` builds 5658 targets
and `ninja check-all` after that adds 716 more targets. On the other
hands `ninja check-flang check-mlir` results in 3997 targets total.

Concretely the contract with premerge_checks.py is changed so that
the expected argument for the --projects flag is only the list of
affected project, dependencies are automatically added.
 2021-09-21 14:28:38 +02:00
dependabot[bot]
eb5f0871c9 Bump urllib3 from 1.26.4 to 1.26.5 in /scripts/metrics 
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.4 to 1.26.5.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.4...1.26.5)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-09-21 13:13:48 +02:00
Mehdi Amini
4cb0b28fb1 Remove llgo from the list of projects 
It has been deleted from the LLVM repo a while ago:
372bfc65de
 2021-09-21 13:13:07 +02:00
Mehdi Amini
65ab98ea2f Fix llvm-dependencies.yaml: the debuginfo-tests has been renamed into cross-project-tests 2021-09-17 15:51:13 +02:00
Mikhail Goncharov
3329564f50 Run bazel build on mainline 2021-09-03 15:54:53 +02:00
Mikhail Goncharov
307a6e71ee Only override 'head' alias 
For branch runs buildkite assigns a correct commit hash.
For #344.
 2021-08-30 18:26:46 +02:00
Kostya Serebryany
587255bfb4 Update clang-tidy.ignore 
Extend the wildcard to match more .inc files in sanitizer_common.
 2021-08-23 14:11:36 +02:00
Anders Waldenborg
1d9dbf5568 make clang_tidy_report handle diffs text files with invalid utf-8 
"git diff" handles text files encoded that is not valid UTF-8 (e.g
using ISO-8859-1) as text files and produces a diff of those (rather
saying "Binary files a/x and b/x differ").

This means that the diff output may contain such characters. Files
that did would cause clang_tidy_report.py do hit an UnicodeDecodeError
when reading the diff, including if it was on removed lines and
regardless if it was in the ignore file.

By specifying errors mode "replace" for decode() method the bytes
that are not a valid utf-8 encoding are replaced with the unicode
replacement question mark (U+FFFD). When parsing the diff
clang-tidy-diff is only looking at filenames and line numbers of the
diff, so this shouldn't be a problem if it doesn't get the exact same
byte sequence inside the actual change.
 2021-08-17 14:18:39 +02:00
Anders Waldenborg
d41a6769fc fix "clang-format/tidy" mixup in doc comment [NFC] 2021-08-17 14:18:39 +02:00
Mikhail Goncharov
49be688a27 script to sync fork llvm-premerge-tests/llvm-project 
- uniformly append env variables

- resolve current HEAD and pass it as BUILDKITE_COMMIT to script
generators. Add it as ph_commit_sha.
 2021-08-11 16:25:29 +02:00
Geoffrey Martin-Noble
8bc6bb7d5d Switch Bazel premerge checks to use llvm-bazel-premerge queue 
I created a new queue for the pre-merge checks so that they won't compete for resources
with the post-merge build already running. Obviously that creates some inefficiency, but is
a simple way to make sure neither build gets starved (in particular, we know that the
current setup of 4 agents running on a single 32-core VM is enough to handle the commit
traffic on the main branch whereas pre-merge testing volume is likely to be more
unpredictable.
 2021-07-29 17:57:11 +02:00
Geoffrey Martin-Noble
c47e555d0f Correctly skip project classification for utils/ 
`project` is a list here, not a string.
 2021-07-29 10:42:12 +02:00
Mikhail Goncharov
0522d2b402 Revert "Disable openmp tests" 
This reverts commit 6d1a03c400.

Offending https://reviews.llvm.org/D105719 was reverted upstream.
 2021-07-28 17:28:28 +02:00
Mikhail Goncharov
6c2410440e Run bazel builds in premerge 
if user is a member of "bazel_build" https://reviews.llvm.org/project/view/107/
or modified /utils/bazel/*

For #328
 2021-07-28 16:01:44 +02:00
Nikita Popov
6d1a03c400 Disable openmp tests 
libarcher tests (part of openmp) have been broken for the past few weeks, and nobody seems to care. As such, disable the openmp project. I hope this is the right place to do so.
 2021-07-26 09:55:29 +02:00
Mikhail Goncharov
c1b5c95071 Ignore changes in 'utils' 
Blaze build check should be done via separate pipeline

For #325
 2021-07-23 09:35:22 +02:00
Mikhail Goncharov
1b116862e6 Fix modified projects resolution when there are unmapped changes 
for #325
 2021-07-23 09:30:53 +02:00
Thomas Lively
75ca275b10 Ignore clang/lib/Headers/wasm_simd128.h 
See https://reviews.llvm.org/D106500 for an example of the spurious positives I would like to ignore.
 2021-07-22 16:23:11 +02:00
Mikhail Goncharov
4ec41ed8a8 enable flang 2021-06-30 15:44:56 +02:00
Mikhail Goncharov
ff4e998ef4 update buildkite monitoring 2021-06-28 10:16:36 +02:00
Mikhail Goncharov
19e290d0cb store builds in runnings state and check them later 
That removes a wrong assumption that everything interesting is
located within first few pages. Should also reduce API load a bit (as we
will not load additional 5+ pages just in case.
 2021-06-24 17:09:42 +02:00
Mikhail Goncharov
d11188e407 use same db for all tables 2021-06-18 12:21:41 +02:00
Mikhail Goncharov
8e156990fc dont pass 'testing' parameter from startup 2021-06-01 17:37:20 +02:00
Mikhail Goncharov
530c8bd77c debugging container start 2021-06-01 17:32:09 +02:00
Mikhail Goncharov
8960ff7dc9 Pre-checkout for windows 
Kill all running processes in BUILDKITE_BUILD_PATH and force unlock git
folder.
 2021-06-01 15:26:23 +02:00
Mikhail Goncharov
a44473098c cron jobs for buildbot and phab monitoring 
+ fix phabricator / buildbot uptime monitoring

+ data is stored in separate databases
 2021-05-20 21:44:19 +02:00
Christian Kühnel
46b7b8d8b7 fixed handling of empty responses 2021-05-20 17:35:19 +02:00
Christian Kühnel
915e8076d6 storing builder information 2021-05-20 17:35:19 +02:00
Christian Kühnel
3a8be70f79 speedup of build import 2021-05-20 17:35:19 +02:00
Christian Kühnel
b6d3986660 fixed import speed for buildsets 2021-05-20 17:35:19 +02:00
Christian Kühnel
9aefaf40cc profiling the slow database import 2021-05-20 17:35:19 +02:00
Christian Kühnel
37e38a6891 changed buildbot monitoring URL 2021-05-20 17:35:19 +02:00
Christian Kühnel
c0081cadb2 added TODOs 2021-05-20 17:35:19 +02:00
Christian Kühnel
ed838f6169 importing buildbot builds 2021-05-20 17:35:19 +02:00
Christian Kühnel
60abbd46ab storing worker information 2021-05-20 17:35:19 +02:00
Christian Kühnel
b79cedb325 Monitor buildbot status 2021-05-20 17:35:19 +02:00
Mikhail Goncharov
6b2003a664 Cron job to load BK data to DB 2021-05-20 17:30:43 +02:00
Mikhail Goncharov
26ffea31dd don't install nuget on new win agents 2021-05-20 12:18:18 +02:00
Mikhail Goncharov
f0d1cc6ab3 update linux node pool (machines w/ no external ssd) 
updated llvm version to 12 with relevant fixes in scrips
 2021-05-17 10:06:54 +02:00
Christian Kühnel
134ca4b801 repo_hist_db now using postgres DB 
Instead of a local database, this script now imports the data
into the shared postgres database. this way the data can be used
for other queries as well.

feel free to extend the data model if you
need additional columns.
 2021-05-11 10:09:51 +02:00
Louis Dionne
668e50298c Remove requirement added in 1f2c851 to try to fix diff-checks 
This is a wild attempt to fix the pre-commit CI which has been failing
for a few hours. I think the authors of 1f2c851 are not available right
now, so I'm taking the freedom to try this out. I hope I'm not overstepping
any boundary.
 2021-05-04 16:22:40 -04:00
Christian Kühnel
1f2c851799 script to log the status of phabricator to a DB 2021-05-04 20:09:14 +02:00
Mikhail Goncharov
1c04f70eb9 cancel previous builds for the same revisions 
using metadata tags set in #298 finds and cancels existing builds before
starting a new one.

One caveat is that no result is reported back to Phabricator for the
cancelled build. That should not be an issue in the normal usecase.

For #278
 2021-05-04 11:27:09 +02:00
Mikhail Goncharov
6b731dc4d5 set build info in metadata 2021-05-03 21:07:30 +02:00
Louis Dionne
980db3f06f Add a script to cancel previously started builds for the same review 
Fixes #278
 2021-05-03 19:55:58 +02:00
ChristianKuehnel
31cbc77e38
improved git metrics script (#295) 
* simplified database schema

* added generic CSV export

* scripts are generating first charts

* dumping entire DB into CSV file

* added more stats
 2021-04-27 16:42:38 +02:00
Mikhail Goncharov
ac4f2bcb82 update python packages 
mostly to bump phabricator plugin as 0.7 has issues with SSL
 2021-04-27 16:01:18 +02:00
Mikhail Goncharov
7aebf9d969 invoke phabtalk standalone 2021-04-27 11:59:00 +02:00
Christian Kühnel
06c97f1dc5 first query on DB working 2021-04-27 11:56:10 +02:00
Mikhail Goncharov
08bb6492fa service queue 2021-04-26 19:56:20 +00:00
Mikhail Goncharov
cb873efbbf update lxml 2021-04-24 19:00:01 +02:00
Mikhail Goncharov
58e13f22d1 inrease timeout for win, minor doc 2021-04-23 12:28:01 +02:00
Mikhail Goncharov
b9ae114deb add image name to agent tags 2021-03-22 11:01:02 +01:00
Mikhail Goncharov
ed35da102c add image digest to agent info 2021-03-22 09:59:23 +00:00
Mikhail Goncharov
f3367dc03e export command on win is 'set' 2021-03-19 15:36:08 +01:00
Mikhail Goncharov
f53c519843 updated docs on windows dev, new setting to skip generated steps 2021-03-19 09:22:21 +01:00
Mikhail Goncharov
67f8efa322 add manual instruction to clone instead 2021-03-12 15:53:15 +01:00
Mikhail Goncharov
637056379c update windows setup 2021-03-12 15:31:06 +01:00
Mikhail Goncharov
b28d130787 fix cloning of scripts directory 2021-03-12 13:48:49 +01:00
Mikhail Goncharov
65fa122f0e update windows setup 2021-03-12 12:10:43 +01:00
Yaxun (Sam) Liu
634cc2936e Exclude HIP headers from clang-tidy check 2021-03-10 10:14:04 +01:00
Vladislav Vinogradov
0f73b01874 Exclude MLIR bindings from clang-tidy checks 
Those bindings requires extra dependencies (like Python, pybind11)
and clang-tidy fails to analyze the source files if they are not
configured properly.
 2021-03-03 14:46:09 +01:00
Reid Kleckner
40e32358de Make clang-tidy ignore debuginfo-tests 
The only C/C++ source in this directory is for test cases. Example CL:

https://reviews.llvm.org/D97668
 2021-03-02 10:17:14 +01:00
ChristianKuehnel
859df4066f address security warning 2021-02-22 15:29:20 +01:00
Mikhail Goncharov
e83da03b17 add clang to flang deps 
related to #271: clang-tidy cannot find gtest/gtest
 2021-01-18 11:04:33 +01:00
Mikhail Goncharov
1d3a3049bb More accurate commands to reproduce build 
Added command to create a link to a compilation database and switch
directory to run tidy and format.
 2021-01-11 15:42:11 +01:00
Mikhail Goncharov
2e9a9563e1 enable periodic libcxx build 2020-12-15 09:24:49 +01:00
Mikhail Goncharov
268c4bb9b0 remove "pipeline_master" 
it was renamed to "pipeline_main"
 2020-12-10 11:03:22 +01:00
Mikhail Goncharov
99e43a8d9a rename master to main branch following LLVM 2020-12-10 09:29:24 +01:00
Mikhail Goncharov
12c4c43bb0 fix x86 debian step 2020-11-26 18:32:27 +01:00
Mikhail Goncharov
86226f3146 fix: builds were not marked as failed 2020-11-26 12:28:11 +01:00