1
0
Fork 0
Commit graph

823 commits

Author SHA1 Message Date
Mikhail Goncharov
1ce0e1ff35 remove bucket as we don't use it anymore 2020-10-15 13:19:36 +02:00
Mikhail Goncharov
43fae020f5 retry most of the requests with backoff
fixes #192

additionally:

- install python libs after scripts checkout, so we don't need to
rebuild images and restart agents only to add a new python dependency

- updated lib versions

- similar scripts checkout in steps
2020-10-12 17:08:34 +02:00
Mikhail Goncharov
322cfbda8c Add link to the build in "setup" step 2020-10-09 12:51:32 +02:00
Mikhail Goncharov
3df81c372f Do not run generic steps if project list is empty 2020-10-09 12:35:32 +02:00
Mikhail Goncharov
3925a2a844 get libcxx steps from sh output 2020-10-09 11:06:36 +02:00
Mikhail Goncharov
ab1b8d5ae2 Use buildkite API instead of passing stat via files
Previous approach had drawbacks:

- every step had to implement exporting of results in fixed format

- if step failed then failure will not be detected

Now report step will fetch results directly from Buildkite.
Agents have to be updated to have BUILDKITE_API_TOKEN env.
2020-10-08 14:40:57 +02:00
Mikhail Goncharov
563d6ba723 add backoff pip package 2020-10-08 13:48:44 +02:00
Mikhail Goncharov
2fec618227
Update README.md
Windows builds are enabled back, also issues with project exclusion is fixed
2020-10-08 10:11:04 +00:00
Mikhail Goncharov
1bbd71f712 Fix project exclusion rules
Previously list of projects was resolved when job run at target OS.
Now project resolution is moved to the premerge pipeline and logic
should be there.
2020-10-08 12:08:57 +02:00
Mikhail Goncharov
ea943424f3 Enable windows builds back
Now we run a pre-commit hook that tries to kill any process that locked
a file in target directory.

Updated timeout interval on windows from 120 to 90 minutes.

Fixes #243
2020-10-07 16:59:37 +02:00
Mikhail Goncharov
5cd16dde45 Don't fail if phabricator target is not specified
That is useful when testing pipelines
2020-10-05 13:33:19 +02:00
Mikhail Goncharov
c350101a23 Allow projects to define custom steps
Now it's possible to allow sub-projects to define own checks and skip
"generic" ones.

To properly accomodate affected projects that might not have special
treatment we:

1. extend the set of affected projecs with dependent (e.g. add 'libc' if
'clang' was modified)

2. add custom steps for projects that define own workflow. At the moment
it's only libcxx and it has a custom trigger pipeline so it's noop.

3. add dependent projects and run generic check on them.

To illustrate: imagine that we have a dependency graph:

llvm -> clang -> openmp

and only clang was modified in a diff; also clang defines own checks.
Thus list of affected projects will be [clang, openmp].
After adding custom checks and removing their projecst: [openmp].
After adding dependencies: [llvm, clang, openmp]. Generic linux /
windows checks will be run on thouse 3 projects.
So as you can see in some scenarios projects with custom checks will
still go through generic checks.

Note that clang-format and clang-tidy checks are run only for "generic"
checks at the moment.
2020-10-02 16:19:57 +02:00
Mikhail Goncharov
c082f1aa9b Organize scripts
Move all .py that are supposed to be run as __main__ under ./scripts
so there is no need to manipulate sys.path to import modules.

Runby cleanup
2020-10-02 14:18:22 +02:00
Mikhail Goncharov
9c383b2caf Fix refspec selection for the report step 2020-10-02 10:17:00 +02:00
Mikhail Goncharov
19974fd1cf Trigger libcxx pipeline if it's affected 2020-09-30 14:20:28 +02:00
Louis Dionne
fbdc3d9ecc Include the Phabricator review ID in commit messages
This allows jobs that trigger on these branches being created to
access this information.
2020-09-30 12:58:40 +02:00
Mikhail Goncharov
405643a089 Allow other refspecs to be used.
That gives us a way to test e.g. pull requests by specifying
ph_scripts_refspec="pull/123/head"

Also keep "ph_" prefix to remove confusion we had between
"scripts_branch" and "ph_scripts_branch" before.

Update docs and moved "testing" sections up.
2020-09-29 17:57:33 +02:00
Mikhail Goncharov
381b1bf1a6 Update nginx configuration to capture all traffic to the static IP
Right now we have only one service we need to serve.

I kept cert manager config for now in the case we want to get a hostname
in the near future.

Updated doc + minor doc / setup fixes.

fixes #242
2020-09-29 11:44:12 +02:00
Mikhail Goncharov
89a70c7b05
Update README.md 2020-09-28 12:22:08 +00:00
Mikhail Goncharov
8bd8cdbdd1 disable win build on master too 2020-09-28 12:19:03 +00:00
Mikhail Goncharov
7d2a6967f8
Update README.md 2020-09-28 12:10:53 +00:00
Mikhail Goncharov
e4e63d9947 temporary disable windows step 2020-09-28 08:38:19 +00:00
Mikhail Goncharov
24336ae2e4 don't fail if branch cannot be deleted 2020-09-25 13:59:03 +00:00
Mikhail Goncharov
33c5072489 pass phid to applyPatch 2020-09-25 13:49:16 +00:00
Louis Dionne
3a0eda3819 Add the Phabricator ID to the commit messages
This allows parsing the commit message to get the Phabricator ID when
triggering jobs from unrelated pipelines.
2020-09-25 12:13:16 +00:00
apink
606f209bf2 add some platform-specific asan files to clang-tidy.ignore list
some compiler-rt files contain architecture-specific assembly code or
complex macroses which clang-tidy can't process properly.

In case of assembly code: we can get "unknown register name 'a7' in asm"
In sanitizer_common_syscalls.inc we have issuels like:
  "clang-tidy: error: unknown type name '__sanitizer_iovec'"
2020-09-13 12:11:55 +02:00
Mikhail Goncharov
8b36ff5888 remove jenkins configs and scrips
- moved linux agents to a new node pool (basically renamed old one)

- removed some out of date scripts or moved them to playlists

- removed shell scrips invoked from jenkins

- minor docs updates

Some python classes methods are not used anymore and will be cleaned up
separately.
2020-09-01 09:51:49 +02:00
Mikhail Goncharov
5380ea63b4 docs about machine config and ingress 2020-08-28 13:08:40 +02:00
Mikhail Goncharov
5610d8d02f review fixes 2 2020-08-27 15:30:03 +02:00
Mikhail Goncharov
33947bdd1f review fixes, formatted text 2020-08-27 15:30:03 +02:00
Mikhail Goncharov
056e5cdbaa update docs
- move "user_doc" to "README" as it's publicly facing page visible when one opens repo
- move developer docs to "development"
- update docs to reflect current buildkite setup (not completely)
- grammar
2020-08-27 15:30:03 +02:00
Mikhail Goncharov
46c9fefb77 rename CCACHE_PATH to CCACHE_DIR
https://ccache.dev/manual/3.7.11.html#_configuration uses CCACHE_DIR to set
cache directory. CCACHE_PATH has a different meaning.

+ removed LLVM_CCACHE_DIR ccache argument to use CCACHE_DIR from env.
+ print additional info about ccache config and machine readable stats.
2020-08-26 14:22:37 +02:00
Mikhail Goncharov
dc1080fc4c
Update premerge_checks.py
use specific commit for reproducing the build
2020-08-14 15:54:11 +02:00
Mikhail Goncharov
a410c28d03
Update premerge_checks.py
add cd after clone
2020-08-14 15:17:21 +02:00
Christian Kühnel
afb8c84bde added more phabricator metrics 2020-08-13 13:14:54 +02:00
Mikhail Goncharov
06db27454c add ATL component, build windows image without cache 2020-08-12 22:31:07 +02:00
Christian Kühnel
5da900df30 added DIA support for windows image 2020-08-12 22:31:07 +02:00
Mikhail Goncharov
f3bf784fad fix no_cache flag on windows 2020-08-12 19:28:09 +02:00
Mikhail Goncharov
5e85ee2da8 typo 2020-08-12 18:58:05 +02:00
Christian Kühnel
0cb0fd670e extended metrics for phabricator 2020-08-12 13:42:14 +02:00
Christian Kühnel
e5a7ae4b36 improved stability of data download 2020-08-07 17:18:22 +02:00
Christian Kühnel
c75649476d simple report on master failues 2020-08-07 10:11:31 +02:00
Christian Sigg
36808fc0d0
No clang-tidy under mlir-{cuda,rocm}-runner. (#226)
All files include CUDA/ROCm headers, which trips up clang-tidy without --cuda-path/--rocm-path.
2020-08-03 17:15:58 +02:00
Christian Sigg
e5be8d45ca clang-tidy ignore {cuda,rocm}-runtime-wrappers.cpp
Both files include GPU header files for which clang implicitly adds include directories. However, clang-tidy does not and therefore we get compile errors (i.e. not diagnostics that we could disable with NOLINT).
2020-07-30 18:14:07 +02:00
Mikhail Goncharov
2372e9b399
Merge pull request #223 from google/playbook-test-branch-changes
Update playbooks.md
2020-07-30 09:40:56 +02:00
Mikhail Goncharov
3f7d29b49d
Merge branch 'master' into playbook-test-branch-changes 2020-07-30 09:40:47 +02:00
Mikhail Goncharov
cb41bb38dd reset before checking out master 2020-07-30 09:32:50 +02:00
Mikhail Goncharov
d75dcdd2fb cleanup before checking out master 2020-07-30 09:27:05 +02:00
Mikhail Goncharov
9cfdbbb8fe fix git clenup 2020-07-30 09:20:19 +02:00
Mikhail Goncharov
36612d77aa
Update playbooks.md
Described testing changes with branches
2020-07-28 15:26:43 +02:00