Merge branch 'master' into playbook-test-branch-changes

README.md

@@ -3,7 +3,7 @@
 This repository contains the configuration files for the pre-merge checks for the LLVM project. This github project contains the documentation and the server configuration cluster of build machines that are used to check all incoming commits to the LLVM project.
 
 # User documentation
-See [docs/user_doc.md](docs/user_doc.md)
+If you are interested in a high-level overview and how to best use the tool see [docs/user_doc.md](docs/user_doc.md)
 
 # Cluster overview
 
@@ -31,7 +31,7 @@ On the Jenkins side:
 There is no backup of the credentials. If you need to change it, generate a new one and update it in Jenkins and Phabricator.
 
 # Additional Information
-* [Playbooks](docs/playbooks.md) for installing/upgrading
+* [Playbooks](docs/playbooks.md) for installing/upgrading agents and testing changes.
 * [User documentation](docs/user_doc.md)
 * [Log of the service operations](https://github.com/google/llvm-premerge-checks/wiki/LLVM-pre-merge-tests-operations-blog)
 

docs/playbooks.md

@@ -139,7 +139,7 @@ To spawn a new windows agent:
 1. Add a task to start agent when machine restarts (make sure to pass correct parameters).
 ```
 git clone https://github.com/google/llvm-premerge-checks.git C:\llvm-premerge-checks
-schtasks.exe /create /tn "Start Buildkite agent" /ru SYSTEM /SC ONSTART /DELAY 0005:00 /tr "powershell -command 'C:\llvm-premerge-checks\scripts\windows_agent_start_buildkite.ps1'"
+schtasks.exe /create /tn "Start Buildkite agent" /ru SYSTEM /SC ONSTART /DELAY 0005:00 /tr "powershell -command 'C:\llvm-premerge-checks\scripts\windows_agent_start_buildkite.ps1 -workdir c:\ws'"
 ```
    
 ### Jenkins
@@ -227,5 +227,3 @@ You can *archive* a rule to disable it.
 We have these build plans in Harbormaster:
 * [Plan 4](https://reviews.llvm.org/harbormaster/plan/4/) Builds for everyone
 * [Plan 3](https://reviews.llvm.org/harbormaster/plan/3/) Builds for beta testers
-
-You can *disable* a build plan to stop it from building.

kubernetes/windows_agent_create.sh

@@ -28,8 +28,7 @@ NAME=$1
 gcloud beta compute instances create "${NAME}" \
   --project="${GCP_PROJECT}" \
   --zone="${GCP_ZONE}" \
-  --machine-type=n1-standard-32 \
+  --machine-type=n2d-standard-16 \
-  --local-ssd=device-name=local-ssd-0 \
   --image=windows-server-2019-dc-for-containers-v20200714 \
   --image-project=windows-cloud \
-  --boot-disk-size=100GB
+  --boot-disk-size=200GB --boot-disk-type=pd-ssd

scripts/buildkite/build_master_pipeline.py

@@ -23,7 +23,7 @@ if __name__ == '__main__':
     no_cache = os.getenv('ph_no_cache') is not None
     filter_output = '--filter-output' if os.getenv('ph_no_filter_output') is None else ''
     projects = os.getenv('ph_projects', 'clang;clang-tools-extra;libc;libcxx;libcxxabi;lld;libunwind;mlir;openmp;polly')
-    log_level = os.getenv('ph_log_level', 'WARNING')
+    log_level = os.getenv('C', 'WARNING')
     notify_emails = list(filter(None, os.getenv('ph_notify_emails', '').split(',')))
     steps = []
     linux_agents = {'queue': f'{queue_prefix}linux'}

scripts/phabtalk/apply_patch2.py

@@ -74,6 +74,7 @@ class ApplyPatch:
         else:
             logging.info('repository exist, will reuse')
             self.repo = Repo(path)  # type: Repo
+            # TODO: set origin url to fork
         os.chdir(path)
         logging.info(f'working dir {os.getcwd()}')
 
@@ -109,7 +110,7 @@ class ApplyPatch:
             self._create_branch(base_revision)
             logging.info('git reset, git cleanup...')
             self.repo.git.reset('--hard')
-            self.repo.git.clean('-fdx')
+            self.repo.git.clean('-ffxdq')
             logging.info('Analyzing {}'.format(diff_to_str(revision_id)))
             if len(dependencies) > 0:
                 logging.info('This diff depends on: {}'.format(diff_list_to_str(dependencies)))
@@ -144,10 +145,10 @@ class ApplyPatch:
             return
 
         logging.info('Syncing local, origin and upstream...')
+        self.repo.git.clean('-ffxdq')
+        self.repo.git.reset('--hard')
         self.repo.git.fetch('--all')
         self.repo.git.checkout('master')
-        self.repo.git.reset('--hard')
-        self.repo.git.clean('-fdx')
         if 'upstream' not in self.repo.remotes:
             self.repo.create_remote('upstream', url=LLVM_GITHUB_URL)
             self.repo.remotes.upstream.fetch()