reset before checking out master

README.md

@@ -3,7 +3,7 @@
 This repository contains the configuration files for the pre-merge checks for the LLVM project. This github project contains the documentation and the server configuration cluster of build machines that are used to check all incoming commits to the LLVM project.
 
 # User documentation
-See [docs/user_doc.md](docs/user_doc.md)
+If you are interested in a high-level overview and how to best use the tool see [docs/user_doc.md](docs/user_doc.md)
 
 # Cluster overview
 
@@ -31,7 +31,7 @@ On the Jenkins side:
 There is no backup of the credentials. If you need to change it, generate a new one and update it in Jenkins and Phabricator.
 
 # Additional Information
-* [Playbooks](docs/playbooks.md) for installing/upgrading
+* [Playbooks](docs/playbooks.md) for installing/upgrading agents and testing changes.
 * [User documentation](docs/user_doc.md)
 * [Log of the service operations](https://github.com/google/llvm-premerge-checks/wiki/LLVM-pre-merge-tests-operations-blog)
 

docs/playbooks.md

@@ -139,7 +139,7 @@ To spawn a new windows agent:
 1. Add a task to start agent when machine restarts (make sure to pass correct parameters).
 ```
 git clone https://github.com/google/llvm-premerge-checks.git C:\llvm-premerge-checks
-schtasks.exe /create /tn "Start Buildkite agent" /ru SYSTEM /SC ONSTART /DELAY 0005:00 /tr "powershell -command 'C:\llvm-premerge-checks\scripts\windows_agent_start_buildkite.ps1'"
+schtasks.exe /create /tn "Start Buildkite agent" /ru SYSTEM /SC ONSTART /DELAY 0005:00 /tr "powershell -command 'C:\llvm-premerge-checks\scripts\windows_agent_start_buildkite.ps1 -workdir c:\ws'"
 ```
    
 ### Jenkins
@@ -174,6 +174,18 @@ Within a container set environment variables similar to [pipeline](https://githu
 
 Additionally set `WORKSPACE`, `PHID` and `DIFF_ID` parameters. Set `CONDUIT_TOKEN` with your personal one from `https://reviews.llvm.org/settings/user/<USERNAME>/page/apitokens/`.
 
+## Custom environment variables for debugging
+
+Buildkite pipelines have a number of custom environment variables one can set to change their behavior to debug issues
+or test changes. The best way is to look on pipeline generators (e.g. build_master_pipeline) 
+
+## Testing changes before merging
+
+It's recommended to test even small changes before committing them to the `master` branch.
+
+1. create a branch with your changes, e.g. "my-feature" and push it to origin.
+1. manually create a buildkite build in the pipeline you are updating and specify environment variable
+   `scripts_branch="my-feature"` (see other )
 
 # Phabricator integration
 
@@ -203,14 +215,4 @@ We have these build plans in Harbormaster:
 * [Plan 4](https://reviews.llvm.org/harbormaster/plan/4/) Builds for everyone
 * [Plan 3](https://reviews.llvm.org/harbormaster/plan/3/) Builds for beta testers
 
-You can *disable* a build plan to stop it from building.
-
-## Per user Opt in/out
-
-You can also on a per-user bases opt in/out to premerge testing. 
-* To opt-in to pre-merge beta testing, add yourself to this project:
-https://reviews.llvm.org/project/view/78/
-* To opt-out of pre-merge testing entirely, add yourself to this project:
-https://reviews.llvm.org/project/view/83/
-
-These projects are checked in the Herald rules above.
+You can *disable* a build plan to stop it from building.

kubernetes/windows_agent_create.sh

@@ -28,8 +28,7 @@ NAME=$1
 gcloud beta compute instances create "${NAME}" \
   --project="${GCP_PROJECT}" \
   --zone="${GCP_ZONE}" \
-  --machine-type=n1-standard-32 \
-  --local-ssd=device-name=local-ssd-0 \
+  --machine-type=n2d-standard-16 \
   --image=windows-server-2019-dc-for-containers-v20200714 \
   --image-project=windows-cloud \
-  --boot-disk-size=100GB
+  --boot-disk-size=200GB --boot-disk-type=pd-ssd

scripts/buildkite/build_master_pipeline.py

@@ -23,7 +23,7 @@ if __name__ == '__main__':
     no_cache = os.getenv('ph_no_cache') is not None
     filter_output = '--filter-output' if os.getenv('ph_no_filter_output') is None else ''
     projects = os.getenv('ph_projects', 'clang;clang-tools-extra;libc;libcxx;libcxxabi;lld;libunwind;mlir;openmp;polly')
-    log_level = os.getenv('ph_log_level', 'WARNING')
+    log_level = os.getenv('C', 'WARNING')
     notify_emails = list(filter(None, os.getenv('ph_notify_emails', '').split(',')))
     steps = []
     linux_agents = {'queue': f'{queue_prefix}linux'}

scripts/phabtalk/apply_patch2.py

@@ -146,9 +146,9 @@ class ApplyPatch:
 
         logging.info('Syncing local, origin and upstream...')
         self.repo.git.clean('-ffxdq')
+        self.repo.git.reset('--hard')
         self.repo.git.fetch('--all')
         self.repo.git.checkout('master')
-        self.repo.git.reset('--hard')
         if 'upstream' not in self.repo.remotes:
             self.repo.create_remote('upstream', url=LLVM_GITHUB_URL)
             self.repo.remotes.upstream.fetch()