1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-25 08:12:40 +01:00
phorge-phorge/src/applications/harbormaster/step
epriestley 96c51028e5 In Harbormaster, release artifacts as soon as no waiting/running build steps will use them
Summary:
Ref T11153. If you have a build plan like this:

  - Lease machine A.
  - Lease machine B.
  - Run client-tests on machine A.
  - Run server-tests on machine B.

...and we get machine A quickly, then finish the tests, we currently do not release machine A until the whole plan finishes.

In the best case, this wastes resources (something else could be using that machine for a while).

In a worse case, this wastes a lot of resources (if machine B is slow to acquire, or the server tests are much slower than the client tests, machine A will get tied up for a really long time).

In the absolute worst case, this might deadlock things.

Instead, release artifacts as soon as no waiting/running steps take them as inputs. In this case, we'd release machine A as soon as we finished running the client tests.

In the case where machines A and B are resources of the same type, this should prevent deadlocks. In all cases, this should improve build throughput at least somewhat.

Test Plan:
I wrote this build plan which runs a "fast" step (10 seconds) and a "slow" step (120 seconds):

{F1691190}

Before the patch, running this build plan held the lease on the "fast" machine for the full 120 seconds, then released both leases at the same time at the very end.

After this patch, I ran this plan and observed the "fast" lease get released after 10 seconds, while the "slow" lease was held for the full 120.

(Also added some `var_dump()` into things to sanity check the logic; it appeared correct.)

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11153

Differential Revision: https://secure.phabricator.com/D16145
2016-06-17 16:13:56 -07:00
..
__tests__ Add a bunch of tests for subclass implementations 2015-06-15 18:13:27 +10:00
HarbormasterArcLintBuildStepImplementation.php Lump Harbormaster build steps into groups 2015-08-06 04:19:42 -07:00
HarbormasterArcUnitBuildStepImplementation.php Lump Harbormaster build steps into groups 2015-08-06 04:19:42 -07:00
HarbormasterBuildStepImplementation.php In Harbormaster, release artifacts as soon as no waiting/running build steps will use them 2016-06-17 16:13:56 -07:00
HarbormasterCircleCIBuildStepImplementation.php Remove all uses of PhutilGitURI in Phabricator 2016-06-13 07:20:58 -07:00
HarbormasterDrydockCommandBuildStepImplementation.php Put Drydock build steps into their own group in Harbormaster 2015-10-05 15:59:35 -07:00
HarbormasterHTTPRequestBuildStepImplementation.php Add a "Build with CircleCI" build step 2016-03-22 12:12:11 -07:00
HarbormasterLeaseWorkingCopyBuildStepImplementation.php Fix an issue with incorrect authorization handling in Working Copy build steps 2015-10-30 16:02:35 +00:00
HarbormasterPublishFragmentBuildStepImplementation.php Add harbormaster.createartifact 2015-08-15 07:28:56 -07:00
HarbormasterSleepBuildStepImplementation.php Lump Harbormaster build steps into groups 2015-08-06 04:19:42 -07:00
HarbormasterThrowExceptionBuildStep.php Lump Harbormaster build steps into groups 2015-08-06 04:19:42 -07:00
HarbormasterUploadArtifactBuildStepImplementation.php Remove call to loadDrydockLease 2015-08-18 17:52:55 -07:00
HarbormasterWaitForPreviousBuildStepImplementation.php Slightly simplify the Harbormaster build log API 2016-03-01 15:25:04 -08:00