revi-archive/llvm-premerge-checks
1
0
Fork 0
Code Activity

review fixes, formatted text

Browse source
This commit is contained in:
Mikhail Goncharov 2020-08-27 15:24:16 +02:00
parent 056e5cdbaa
commit 33947bdd1f
3 changed files with 148 additions and 80 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
.editorconfig
View file

@ -6,6 +6,9 @@ ij_formatter_on_tag = @formatter:on
ij_formatter_tags_enabled = false
ij_smart_tabs = false
[*.md]
max_line_length = 80
ij_visual_guides = 80,100
[{*.pyw,*.py,*.pi}]
indent_style = space
@ -106,13 +109,6 @@ ij_html_space_around_equality_in_attribute = false
ij_html_space_inside_empty_tag = false
ij_html_text_wrap = normal
[{*.vsl,*.vm,*.ft}]
ij_vtl_keep_indents_on_empty_lines = false
[{*.xjsp,*.tag,*.jsp,*.jsf,*.jspf,*.tagf}]
ij_jsp_jsp_prefer_comma_separated_import_list = false
ij_jsp_keep_indents_on_empty_lines = false
[{*.yml,*.yaml}]
indent_size = 2
ij_yaml_keep_indents_on_empty_lines = false
@ -139,10 +135,3 @@ ij_json_space_before_comma = false
ij_json_spaces_within_braces = false
ij_json_spaces_within_brackets = false
ij_json_wrap_long_lines = false
[{BUILD,WORKSPACE,*.bzl}]
ij_continuation_indent_size = 4
ij_build_keep_indents_on_empty_lines = false
[{spring.schemas,spring.handlers,*.properties}]
ij_properties_align_group_field_declarations = false

95
README.md
View file

@ -1,66 +1,101 @@
# Overview
The *pre-merge checks* for the [LLVM project](http://llvm.org/) are a [continuous integration (CI)](https://en.wikipedia.org/wiki/Continuous_integration) workflow. The workflow checks the patches the developers upload to the [LLVM Phabricator](https://reviews.llvm.org) instance. *Phabricator* is the code review tool in the LLVM project. The workflow checks the patches before a user merges them the master branch - thus the term *pre-merge testing*. When a user uploads a patch to the LLVM Phabricator, Phabricator triggers the checks and then displays the results.
The *pre-merge checks* for the [LLVM project](http://llvm.org/) are a
[continuous integration
(CI)](https://en.wikipedia.org/wiki/Continuous_integration) workflow. The
workflow checks the patches the developers upload to the [LLVM
Phabricator](https://reviews.llvm.org) instance.
The CI system checks the patches **before** a user merges them to the master branch. This way bugs in a patch are contained during the code review stage and do not pollute the master branch. The more bugs the CI system can catch during the code review phase, the more stable and bug-free the master branch will become.
*Phabricator* (https://reviews.llvm.org) is the code review tool in the LLVM
project.
This repository contains the configurations and script to run pre-merge checks for the LLVM project.
The workflow checks the patches before a user merges them to the master branch -
thus the term *pre-merge testing**. When a user uploads a patch to the LLVM
Phabricator, Phabricator triggers the checks and then displays the results.
The CI system checks the patches **before** a user merges them to the master
branch. This way bugs in a patch are contained during the code review stage and
do not pollute the master branch. The more bugs the CI system can catch during
the code review phase, the more stable and bug-free the master branch will
become. <sup>[citation needed]()</sup>
This repository contains the configurations and script to run pre-merge checks
for the LLVM project.
## Feedback
If you notice issues or have an idea on how to improve pre-merge checks, please create a [new issue](https://github.com/google/llvm-premerge-checks/issues/new) or give a
:heart: to an existing one.
If you notice issues or have an idea on how to improve pre-merge checks, please
create a [new issue](https://github.com/google/llvm-premerge-checks/issues/new)
or give a :heart: to an existing one.
## Sign up for beta-test
To get the latest features and help us developing the project, sign up for the pre-merge beta testing by adding yourself to the ["pre-merge beta testing" project](https://reviews.llvm.org/project/members/78/) on Phabricator.
To get the latest features and help us developing the project, sign up for the
pre-merge beta testing by adding yourself to the ["pre-merge beta testing"
project](https://reviews.llvm.org/project/members/78/) on Phabricator.
## Opt-out
In case you want to opt-out entirely of pre-merge testing, add yourself to the [OPT OUT project](https://reviews.llvm.org/project/view/83/).
In case you want to opt-out entirely of pre-merge testing, add yourself to the
[OPT OUT project](https://reviews.llvm.org/project/view/83/).
If you decide to opt-out, please let us know why, so we might be able to improve in the future.
If you decide to opt-out, please let us know why, so we might be able to improve
in the future.
# Requirements
The builds are only triggered if the Revision in Phabricator is created/updated via `arc diff`.
If you update a Revision via the Web UI it will [not trigger](https://secure.phabricator.com/Q447) a build.
The builds are only triggered if the Revision in Phabricator is created/updated
via `arc diff`. If you update a Revision via the Web UI it will [not
trigger](https://secure.phabricator.com/Q447) a build.
To get a patch on Phabricator tested the build server must be able to apply the patch to the checked out git repository.
If you want to get your patch tested, please make sure that either:
To get a patch on Phabricator tested the build server must be able to apply the
patch to the checked out git repository. If you want to get your patch tested,
please make sure that either:
* You set a git hash as `sourceControlBaseRevision` in Phabricator which is available on the Github repository,
* **or** you define the dependencies of your patch in Phabricator,
* **or** your patch can be applied to the master branch.
* You set a git hash as `sourceControlBaseRevision` in Phabricator which is
* available on the Github repository, **or** you define the dependencies of your
* patch in Phabricator, **or** your patch can be applied to the master branch.
Only then can the build server apply the patch locally and run the builds and tests.
Only then can the build server apply the patch locally and run the builds and
tests.
# Accessing results on Phabricator
TODO: move closer to top and explain what different links mean currently. Update screenshots.
Phabricator will automatically trigger a build for every new patch you upload or
modify. Phabricator shows the build results at the top of the entry: ![build
status](docs/images/diff_detail.png)
Phabricator will automatically trigger a build for every new patch you upload or modify.
Phabricator shows the build results at the top of the entry:
![build status](docs/images/diff_detail.png)
The CI will compile and run tests, run clang-format and
[clang-tidy](docs/clang_tidy.md) on lines changed.
CI will compile and run tests, run clang-format and [clang-tidy](docs/clang_tidy.md) on lines changed.
If a unit test failed, this is shown below the build status. You can also expand the unit test to see the details:
![unit test results](docs/images/unit_tests.png).
If a unit test failed, this is shown below the build status. You can also expand
the unit test to see the details: ![unit test
results](docs/images/unit_tests.png).
# Contributing
We're happy to get help on improving the infrastructure and workflows!
Please check [contibuting](docs/contributing.md) first.
[Development](docs/development.md) gives an overview how different parts interact together.
[Playbooks](docs/playbooks.md) shows concrete examples how to, for example, build and run agents locally.
If you have any questions please contact [Christian Kühnel](mailto:kuhnel@google.com) or [Mikhail Goncharov](mailto:goncahrov@google.com).
[Development](docs/development.md) gives an overview how different parts
interact together.
[Playbooks](docs/playbooks.md) shows concrete examples how to, for example,
build and run agents locally.
If you have any questions please contact by [mail](mailto:goncahrov@google.com)
or find user "goncharov" on [LLVM Discord](https://discord.gg/VrcTUs).
# Additional Information
* [Playbooks](docs/playbooks.md) for installing/upgrading agents and testing changes.
* [Log of the service operations](https://github.com/google/llvm-premerge-checks/wiki/LLVM-pre-merge-tests-operations-blog)
- [Playbooks](docs/playbooks.md) for installing/upgrading agents and testing
changes.
- [Log of the service
operations](https://github.com/google/llvm-premerge-checks/wiki/LLVM-pre-merge-tests-operations-blog)
# License
This project is licensed under the "Apache 2.0 with LLVM Exception" license. See [LICENSE](LICENSE) for details.
This project is licensed under the "Apache 2.0 with LLVM Exception" license. See
[LICENSE](LICENSE) for details.

116
docs/development.md
View file

@ -11,27 +11,41 @@
# Overview
* [Buildkite](https://buildkite.com/llvm-project) orchestrates each build.
* multiple Linux and windows agents connected to Buildkite. Agents are run at Google Cloud Platform.
* [small proxy service](/phabricator-proxy) that takes build requests from [reviews.llvm.org](http://reviews.llvm.org) and converts them into Buildkite build request. Buildkite job sends build results directly to Phabricator.
* every review creates a new branch in [fork of llvm-project](https://github.com/llvm-premerge-tests/llvm-project).
- [Buildkite](https://buildkite.com/llvm-project) orchestrates each build.
- multiple Linux and windows agents connected to Buildkite. Agents are run at
Google Cloud Platform.
- [small proxy service](/phabricator-proxy) that takes build requests from
[reviews.llvm.org](http://reviews.llvm.org) and converts them into Buildkite
build request. Buildkite job sends build results directly to Phabricator.
- every review creates a new branch in [fork of
llvm-project](https://github.com/llvm-premerge-tests/llvm-project).
![deployment diagram](http://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/google/llvm-premerge-checks/master/docs/deployment.plantuml)
# Phabricator integration
* [Harbormaster build plan](https://reviews.llvm.org/harbormaster/plan/5On the Phabricator side these things were configured:/)
* Herald [rule for everyone](https://reviews.llvm.org/H576) and for [beta testers](https://reviews.llvm.org/H511).
Note that right now there is no difference between beta and "normal" builds.
* the [merge_guards_bot user](https://reviews.llvm.org/p/merge_guards_bot/) account for writing comments.
- [Harbormaster build plan](https://reviews.llvm.org/harbormaster/plan/5) the
Phabricator side these things were configured
- Herald [rule for everyone](https://reviews.llvm.org/H576) and for [beta
testers](https://reviews.llvm.org/H511). Note that right now there is no
difference between beta and "normal" builds.
- the [merge_guards_bot user](https://reviews.llvm.org/p/merge_guards_bot/)
account for writing comments.
# Buildkite pipelines
Buildkite allows [dynamically define pipelines as the output of a command](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines).
That gives us the flexibility to generate pipeline code using a script from a specific branch of pre-merge checks.
Buildkite allows [dynamically define pipelines as the output of a
command](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines).
That gives us the flexibility to generate pipeline code using a script from a
specific branch of pre-merge checks.
For example, "pre-merge" pipeline has a single fixed step, that checks out this repo and runs a python script to
generate further steps:
For example, "pre-merge" pipeline has a single fixed step, that checks out this
repo and runs a python script to generate further steps:
```shell script
export SRC=${BUILDKITE_BUILD_PATH}/llvm-premerge-checks
@ -41,60 +55,90 @@ export SCRIPT_DIR=${SRC}/scripts
${SCRIPT_DIR}/buildkite/build_branch_pipeline.py | tee /dev/tty | buildkite-agent pipeline upload
```
One typically edits corresponding script, not a pipeline definition in the Buildkite interface. [How to test changes](playbooks.md#testing-changes-before-merging).
One typically edits corresponding script, not a pipeline definition in the
Buildkite interface. [How to test
changes](playbooks.md#testing-changes-before-merging).
# Life of pre-merge check
When new diff arrives for review it triggers Herald rule ("everyone" or "beta testers").
When new diff arrives for review it triggers a Herald rule ("everyone" or "beta
testers").
That in sends an HTTP POST request to [**phab-proxy**](../phabricator-proxy) that submits a new buildkite job **diff-checks**.
All parameters from the original request are put in the build's environment with `ph_` prefix (to avoid shadowing any Buildkite environment variable).
"scripts_branch" parameter defines which branch of llvm-premerge-checks to use.
That in sends an HTTP POST request to [**phab-proxy**](../phabricator-proxy)
that submits a new buildkite job **diff-checks**. All parameters from the
original request are put in the build's environment with `ph_` prefix (to avoid
shadowing any Buildkite environment variable). "scripts_branch" parameter
defines which branch of llvm-premerge-checks to use.
**diff-checks** pipeline ([create_branch_pipeline.py](../scripts/buildkite/create_branch_pipeline.py)) downloads a patch (or series of patches) and applies it to a fork of the llvm-project repository.
Then it pushes a new state as a new branch (e.g. "phab-diff-288211") and triggers "premerge-checks" on it (all "ph_" env variables are passed to it).
This new branch can now be used to reproduce the build or by another tooling.
Periodical **cleanup-branches** pipeline deletes branches older than 30 days.
**diff-checks** pipeline
([create_branch_pipeline.py](../scripts/buildkite/create_branch_pipeline.py))
downloads a patch (or series of patches) and applies it to a fork of the
llvm-project repository. Then it pushes a new state as a new branch (e.g.
"phab-diff-288211") and triggers "premerge-checks" on it (all "ph_" env
variables are passed to it). This new branch can now be used to reproduce the
build or by another tooling. Periodical **cleanup-branches** pipeline deletes
branches older than 30 days.
**premerge-checks** pipeline ([build_branch_pipeline.py](../scripts/buildkite/build_branch_pipeline.py)) builds and tests
changes on Linux and Windows agents. Then it uploads a combined result to Phabricator.
**premerge-checks** pipeline
([build_branch_pipeline.py](../scripts/buildkite/build_branch_pipeline.py))
builds and tests changes on Linux and Windows agents. Then it uploads a
combined result to Phabricator.
# Enabled projects and project detection
To reduce build times and mask unrelated problems, we're only building and testing the projects that were modified by a patch.
[choose_projects.py](../scripts/choose_projects.py) uses manually maintained [config file](../scripts/llvm-dependencies.yaml) to define inter-project dependencies and exclude projects:
To reduce build times and mask unrelated problems, we're only building and
testing the projects that were modified by a patch.
[choose_projects.py](../scripts/choose_projects.py) uses manually maintained
[config file](../scripts/llvm-dependencies.yaml) to define inter-project
dependencies and exclude projects:
1. Get prefix (e.g. "llvm", "clang") of all paths modified by a patch.
1. Add all dependant projects.
1. Add all projects that this extended list depends on, completing the dependency subtree.
1. Add all projects that this extended list depends on, completing the
dependency subtree.
1. Remove all disabled projects.
# Agent machines
All build machines are running from Docker containers so that they can be debugged, updated, and scaled easily:
* [Linux](../containers/buildkite-premerge-debian/Dockerfile).
We use [Kubernetes deployemnt](../kubernetes/buildkite) to manage these agents.
* [Windows](../containers/agent-windows-buildkite/Dockerfile) based on [Windows vs2019](../containers/agent-windows-vs2019).
At the moment they are run as multiple individual VM instances.
All build machines are running from Docker containers so that they can be
debugged, updated, and scaled easily:
- [Linux](../containers/buildkite-premerge-debian/Dockerfile). We use
[Kubernetes deployment](../kubernetes/buildkite) to manage these agents.
- [Windows](../containers/agent-windows-buildkite/Dockerfile) based on [Windows
vs2019](../containers/agent-windows-vs2019). At the moment they are run as
multiple individual VM instances.
See [playbooks](playbooks.md) how to manage and set up machines.
# Compilation caching
Each build is performed on a clean copy of the git repository. To speed up the builds [ccache](https://ccache.dev/) is used on Linux and [sccache](https://github.com/mozilla/sccache) on Windows.
Each build is performed on a clean copy of the git repository. To speed up the
builds [ccache](https://ccache.dev/) is used on Linux and
[sccache](https://github.com/mozilla/sccache) on Windows.
# Buildkite monitoring
VM instance `buildkite-monitoring` exposes Buildkite metrics to GCP.
To set up a new instance:
1. Create as small Linux VM with full access to *Stackdriver Monitoring API*.
2. Follow instructions to [install monitoring agent](https://cloud.google.com/monitoring/agent/install-agent) and [enable statsd plugin](https://cloud.google.com/monitoring/agent/plugins/statsd).
3. Download recent release of [buildkite-agent-metrics](https://github.com/buildkite/buildkite-agent-metrics/releases).
4. Run in SSH session:
1. Follow instructions to [install monitoring
agent](https://cloud.google.com/monitoring/agent/install-agent) and [enable
statsd plugin](https://cloud.google.com/monitoring/agent/plugins/statsd).
1. Download recent release of
[buildkite-agent-metrics](https://github.com/buildkite/buildkite-agent-metrics/releases).
1. Run in SSH session:
```bash
chmod +x buildkite-agent-metrics-linux-amd64
nohup ./buildkite-agent-metrics-linux-amd64 -token XXXX -interval 30s -backend statsd &
```
.
TODO: update "Testing scripts locally" playbook on how to run Linux build locally with Docker.