Add a global lock for the RepositoryPullDaemon

Summary: Allow multiple daemons to run without contention.

Test Plan: Ran multiple daemons simultaneously in "debug" mode, observed them acquiring (and sometimes failing to acquire) locks.

Reviewers: btrahan, jungejason, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1400

Differential Revision: https://secure.phabricator.com/D2877

src/applications/repository/daemon/PhabricatorRepositoryPullLocalDaemon.php

@@ -133,6 +133,13 @@ final class PhabricatorRepositoryPullLocalDaemon
         }
 
         try {
+          $callsign = $repository->getCallsign();
+          $this->log("Updating repository '{$callsign}'.");
+
+          $lock_name = get_class($this).':'.$callsign;
+          $lock = PhabricatorGlobalLock::newLock($lock_name);
+          $lock->lock();
+
           $this->pullRepository($repository);
 
           if (!$no_discovery) {
@@ -141,8 +148,13 @@ final class PhabricatorRepositoryPullLocalDaemon
             $this->discoverRepository($repository);
           }
 
+          $lock->unlock();
+
           $sleep_for = $repository->getDetail('pull-frequency', $min_sleep);
           $retry_after[$id] = time() + $sleep_for;
+        } catch (PhutilLockException $ex) {
+          $retry_after[$id] = time() + $min_sleep;
+          $this->log("Failed to acquire lock.");
         } catch (Exception $ex) {
           $retry_after[$id] = time() + $min_sleep;
           phlog($ex);

src/docs/userguide/diffusion.diviner

@@ -120,13 +120,30 @@ a large number of relatively inactive repositories, but might benefit from
 tuning in some cases. The daemon makes a rough effort to respect pull
 frequencies defined in repository configuration, but may not be able to import
 new commits very quickly if you have a large number of repositories (as it is
-blocked waiting on I/O from other repositories). If you want to provide lower
+blocked waiting on I/O from other repositories).
-commit import latency for some repositories, you can launch additional
-dedicated daemons:
 
-For example, if you want low latency on the repositories with callsigns
+If you want to provide lower commit import latency for some repositories, you
-`A` and `B`, but don't care about latency for the other repositories, you could
+can either launch **more daemons** (which will generally lower latency for all
-launch two daemons like this:
+repositories) or launch additional **dedicated daemons** (which will give you
+very fine-grained control over import latency).
+
+=== More Daemons ===
+
+The coarse approach to reducing import latency is to simply launch more daemons,
+using `phd`:
+
+  phabricator/bin $ ./phd launch RepositoryPullLocal
+
+This will launch another copy of the daemon. The daemons acquire a global lock
+before pulling a repository, so you can launch additional daemons without
+causing contention or race conditions.
+
+=== Dedicated Daemons ===
+
+You can take a more fine-grained approach and launch dedicated daemons for
+specific repositories or groups of repositories. For example, if you want low
+latency on the repositories with callsigns `A` and `B`, but don't care about
+latency for the other repositories, you could launch two daemons like this:
 
   phabricator/bin $ ./phd launch RepositoryPullLocal -- A B
   phabricator/bin $ ./phd launch RepositoryPullLocal -- --not A --not B