revi-archive/phorge-phorge
1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-22 23:02:42 +01:00
Code Issues Releases Wiki Activity

Add a global lock for the RepositoryPullDaemon

Browse source 
Summary: Allow multiple daemons to run without contention.

Test Plan: Ran multiple daemons simultaneously in "debug" mode, observed them acquiring (and sometimes failing to acquire) locks.

Reviewers: btrahan, jungejason, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1400

Differential Revision: https://secure.phabricator.com/D2877
This commit is contained in:
epriestley 2012-06-28 09:54:55 -07:00
parent bbfb686db5
commit 1bd0a1f0d9
2 changed files with 35 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
src/applications/repository/daemon/PhabricatorRepositoryPullLocalDaemon.php
View file

@ -133,6 +133,13 @@ final class PhabricatorRepositoryPullLocalDaemon
}
try {
$callsign = $repository->getCallsign();
$this->log("Updating repository '{$callsign}'.");
$lock_name = get_class($this).':'.$callsign;
$lock = PhabricatorGlobalLock::newLock($lock_name);
$lock->lock();
$this->pullRepository($repository);
if (!$no_discovery) {
@ -141,8 +148,13 @@ final class PhabricatorRepositoryPullLocalDaemon
$this->discoverRepository($repository);
}
$lock->unlock();
$sleep_for = $repository->getDetail('pull-frequency', $min_sleep);
$retry_after[$id] = time() + $sleep_for;
} catch (PhutilLockException $ex) {
$retry_after[$id] = time() + $min_sleep;
$this->log("Failed to acquire lock.");
} catch (Exception $ex) {
$retry_after[$id] = time() + $min_sleep;
phlog($ex);

29
src/docs/userguide/diffusion.diviner
View file

@ -120,13 +120,30 @@ a large number of relatively inactive repositories, but might benefit from
tuning in some cases. The daemon makes a rough effort to respect pull
frequencies defined in repository configuration, but may not be able to import
new commits very quickly if you have a large number of repositories (as it is
blocked waiting on I/O from other repositories). If you want to provide lower
commit import latency for some repositories, you can launch additional
dedicated daemons:
blocked waiting on I/O from other repositories).
For example, if you want low latency on the repositories with callsigns
`A` and `B`, but don't care about latency for the other repositories, you could
launch two daemons like this:
If you want to provide lower commit import latency for some repositories, you
can either launch **more daemons** (which will generally lower latency for all
repositories) or launch additional **dedicated daemons** (which will give you
very fine-grained control over import latency).
=== More Daemons ===
The coarse approach to reducing import latency is to simply launch more daemons,
using `phd`:
phabricator/bin $ ./phd launch RepositoryPullLocal
This will launch another copy of the daemon. The daemons acquire a global lock
before pulling a repository, so you can launch additional daemons without
causing contention or race conditions.
=== Dedicated Daemons ===
You can take a more fine-grained approach and launch dedicated daemons for
specific repositories or groups of repositories. For example, if you want low
latency on the repositories with callsigns `A` and `B`, but don't care about
latency for the other repositories, you could launch two daemons like this:
phabricator/bin $ ./phd launch RepositoryPullLocal -- A B
phabricator/bin $ ./phd launch RepositoryPullLocal -- --not A --not B