1
0
Fork 0
llvm-premerge-checks/README.md

103 lines
3.6 KiB
Markdown
Raw Normal View History

2019-10-04 10:26:20 +02:00
# Overview
This repository contains the configuration files for the merge guards for the LLVM project. It configures a cluster of build machines that are used to check all incoming commits to the LLVM project.
# Merge guards
TODO(@christiankuehnel): describe objective of merge guards
# Cluster overview
2019-10-05 08:43:30 +02:00
The cluster consists of these services:
2019-10-10 09:27:31 +02:00
* Jenkins build server: http://jenkins.llvm-merge-guard.org
2019-10-05 08:43:30 +02:00
* a set of Jenkins agents running the builds
2019-10-10 11:01:17 +02:00
* an nginx server with the build results/logs http://results.llvm-merge-guard.org
2019-10-05 08:43:30 +02:00
2019-10-04 10:26:20 +02:00
# Jenkins-Phabricator integration
2019-10-10 09:27:31 +02:00
The Jenkins-Phabricator is based on the instructions provided with the [Phabricator-Jenkins Plugin](https://github.com/uber/phabricator-jenkins-plugin).
On the Phabricator side these things were configured:
* the Harbormaster [build plan](https://reviews.llvm.org/harbormaster/plan/3/)
* the Herald [rule](https://reviews.llvm.org/H511)
On the Jenkins side:
* in the Jenkins configuration page as explained in the instrucitons
* in the build [job](http://jenkins.llvm-merge-guard.org/job/Phabricator/)
There is no backup of the credentials. If you need to change it, generate a new one and update it in Jenkins and Phabricator.
2019-10-04 10:26:20 +02:00
# Playbooks
## deployment to a clean infrastructure
2019-10-10 09:27:31 +02:00
2019-10-04 10:26:20 +02:00
General remarks:
2019-10-10 09:27:31 +02:00
* GCP does not route any traffic to your services unless the service is "healthy". It might take a few minutes after startup before the services is classified as healthy. Until then you will only see some generic error message.
2019-10-04 10:26:20 +02:00
These are the steps to set up the build server on a clean infrastructure:
2019-10-10 09:27:31 +02:00
1. Configure the tools on your local machine:
2019-10-04 10:26:20 +02:00
```bash
2019-10-10 09:27:31 +02:00
./local_setup.sh
2019-10-04 10:26:20 +02:00
```
2019-10-10 09:27:31 +02:00
1. Delete the old cluster, if it still exists:
2019-10-04 10:26:20 +02:00
```bash
cd kubernetes/cluster
./cluster_delete.sh
```
1. Create the cluster:
```bash
cd kubernetes/cluster
./cluster_create.sh
```
1. Create the disk storage, if it does not yet exist:
```bash
cd kubernetes/cluster
./disk_create.sh
```
2019-10-10 09:27:31 +02:00
1. SSH into the VM instance mounting the volume, find the mount point and then set
2019-10-04 10:26:20 +02:00
```bash
# go to the mount point of the volume
cd /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/jenkins-home
# change the permissions
2019-10-10 09:27:31 +02:00
sudo chmod a+rwx
2019-10-04 10:26:20 +02:00
```
2019-10-10 09:27:31 +02:00
1. Push the docker images to gcr.io:
```bash
2019-10-04 10:26:20 +02:00
cd containers/debian-testing-clang8
./build_deploy.sh
2019-10-10 09:27:31 +02:00
2019-10-04 10:26:20 +02:00
cd ../jenkins-master
./build_deploy.sh
2019-10-10 09:27:31 +02:00
```
1. Deploy the stack:
2019-10-04 10:26:20 +02:00
```bash
cd kubernetes
./deploy.sh
```
2019-10-10 09:27:31 +02:00
1. Configure it
2019-10-04 10:26:20 +02:00
## handling SSH keys
The Jenkins server SSHs into the agents to start the agent application. Thus the master needs SSH access to the agent. To set this up:
1. Create an SSH key pair locally with `ssh-keygen`.
1. Copy the contents of `id_rsa` to the credentials section of the Jenkins UI.
1. Configure the agent in the Jenkins UI to use the new SSH keys you just uploaded.
1. Copy the contents of `id_rsa.pub` to `containers/<agent dir>/authorized keys`.
1. Rebuild and deploy the agents.
2019-10-10 09:27:31 +02:00
While this works, it does not fell like the perfect solution. I'm happy to get better ideas on this.
2019-10-04 10:26:20 +02:00
## creating basic authentication for reverse proxy
1. create auth file, based on [ingress-nginx documentation](https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/auth/basic)
```bash
cd kubernetes/reverse-proxy
htpasswd -c auth <username>
# enter password at prompt
# add more users as required
kubectl create secret generic proxy-auth --from-file=auth --namespace=jenkins
```
2019-10-04 14:31:57 +02:00
# License
2019-10-10 09:27:31 +02:00
This project is licensed unter the "Apache 2.0 with LLVM Exception" license. See [LICENSE](LICENSE) for details.