revi-archive/phorge-phorge
1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-18 19:40:55 +01:00
Code Issues Releases Wiki Activity
phorge-phorge/resources/sql/patches
History
epriestley f0fdcf1a51 Undumb the Drydock resource allocator pipeline 
Summary:
This was the major goal of D3859/D3855, and to a lesser degree D3854/D3852.

As Drydock is allocating a resource, it may need to allocate other resources first. For example, if it's allocating a working copy, it may need to allocate a host first.

Currently, we have the process basically queue up the allocation (insert a task into the queue) and sleep() until it finishes. This is problematic for a bunch of reasons, but the major one is that if allocation takes more resources (host, port, machine, DNS) than you have daemons, they could all end up sleeping and waiting for some other daemon to do their work. This is really stupid. Even if you only take up some of them, you're spending slots sleeping when you could be doing useful work.

To partially get around this and make the CLI experience less dumb, there's this goofy `synchronous` flag that gets passed around everywhere and pushes the workflow through a pile of special cases. Basically the `synchronous` flag causes us to do everything in-process. But this is dumb too because we'd rather do things in parallel if we can, and we have to have a lot of special case code to make it work at all.

Get rid of all of this. Instead of sleep()ing, try to work on the tasks that need to be worked on. If another daemon grabbed them already that's fine, but in the worst case we just gracefully degrade and do everything in process. So we get the best of both worlds: if we have parallelizable tasks and free daemons, things will execute in parallel. If we have nonparallelizable tasks or no free daemons, things will execute in process.

Test Plan: Ran `drydock_control.php --trace` and saw it perform cascading allocations without sleeping or special casing.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3861
2012-11-01 11:30:42 -07:00
..
0000.legacy.sql Minor, completely remove references to PHID from schema patches so upgrade-from-scratch works. 2012-05-24 13:59:12 -07:00
000.project.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
001.maniphest_projects.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
002.oauth.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
003.more_oauth.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
004.daemonrepos.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
005.workers.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
006.repository.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
007.daemonlog.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
008.repoopt.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
009.repo_summary.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
010.herald.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
011.badcommit.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
012.dropphidtype.sql Minor, completely remove references to PHID from schema patches so upgrade-from-scratch works. 2012-05-24 13:59:12 -07:00
013.commitdetail.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
014.shortcuts.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
015.preferences.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
016.userrealnameindex.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
017.sessionkeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
018.owners.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
019.arcprojects.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
020.pathcapital.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
021.xhpastview.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
022.differentialcommit.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
023.dxkeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
024.mlistkeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
025.commentopt.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
026.diffpropkey.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
027.metamtakeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
028.systemagent.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
029.cursors.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
030.imagemacro.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
031.workerrace.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
032.viewtime.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
033.privtest.sql Improve schema upgrade workflow for unprivileged users 2011-04-30 00:50:48 -07:00
034.savedheader.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
035.proxyimage.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
036.mailkey.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
037.setuptest.sql Add a "setup" mode which guides new users through application configuration 2011-05-10 15:12:30 -07:00
038.admin.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
039.userlog.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
040.transform.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
041.heraldrepetition.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
042.commentmetadata.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
043.pastebin.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
044.countdown.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
045.timezone.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
046.conduittoken.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
047.projectstatus.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
048.relationshipkeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
049.projectowner.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
050.taskdenormal.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
051.projectfilter.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
052.pastelanguage.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
053.feed.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
054.subscribers.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
055.add_author_to_files.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
056.slowvote.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
057.parsecache.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
058.missingkeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
059.engines.php SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
060.phriction.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
061.phrictioncontent.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
062.phrictionmenu.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
063.pasteforks.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
064.subprojects.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
065.sshkeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
066.phrictioncontent.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
067.preferences.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
068.maniphestauxiliarystorage.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
069.heraldxscript.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
070.differentialaux.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
071.contentsource.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
072.blamerevert.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
073.reposymbols.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
074.affectedpath.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
075.revisionhash.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
076.indexedlanguages.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
077.originalemail.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
078.nametoken.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
079.nametokenindex.php Add a name token table so on-demand typeaheads can match last names 2011-10-23 14:25:26 -07:00
080.filekeys.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
081.filekeys.php Use a proper entropy source to generate file keys 2011-10-23 14:42:23 -07:00
082.xactionkey.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
083.dxviewtime.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
084.pasteauthorkey.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
085.packagecommitrelationship.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
086.formeraffil.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
087.phrictiondelete.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
088.audit.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
089.projectwiki.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
090.forceuniqueprojectnames.php Provide wiki pages for projects 2011-12-20 14:03:12 -08:00
091.uniqueslugkey.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
092.dropgithubnotification.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
093.gitremotes.php Make tracked git repositories use an implicit 'origin' remote 2011-12-29 08:35:32 -08:00
094.phrictioncolumn.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
095.directory.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
096.filename.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
097.heraldruletypes.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
098.heraldruletypemigration.php Created personal vs. global herald rule distingtion 2012-01-19 11:21:49 -08:00
099.drydock.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
100.projectxaction.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
101.heraldruleapplied.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
102.heraldcleanup.php Write fewer "applied" rows and clean up excess historical rows 2012-01-25 11:53:39 -08:00
103.heraldedithistory.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
104.searchkey.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
105.mimetype.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
106.chatlog.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
107.oauthserver.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
108.oauthscope.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
109.oauthclientphidkey.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
110.commitaudit.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
111.commitauditmigration.php Make PhabricatorRepositoryCommmit schema changes for audit 2012-02-28 21:06:34 -08:00
112.oauthaccesscoderedirecturi.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
113.lastreviewer.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
114.auditrequest.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
115.prepareutf8.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
116.utf8-backup-first-expect-wait.sql Minor, completely remove references to PHID from schema patches so upgrade-from-scratch works. 2012-05-24 13:59:12 -07:00
117.repositorydescription.php Drop unused column 2012-03-12 12:11:36 -07:00
118.auditinline.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
119.filehash.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
120.noop.sql Add "Flags" to allow users to collect the things they love 2012-03-27 16:22:40 -07:00
121.drydocklog.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
122.flag.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
123.heraldrulelog.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
124.subpriority.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
125.ipv6.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
126.edges.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
127.userkeybody.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
128.phabricatorcom.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
129.savedquery.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
130.denormalrevisionquery.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
131.migraterevisionquery.php Fix patch 131 for db with lots of revisions 2012-04-16 12:35:48 -07:00
132.phame.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
133.imagemacro.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
134.emptysearch.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
135.datecommitted.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
136.sex.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
137.auditmetadata.sql SQL Patch Management: SQL Changes 2012-04-30 07:53:53 -07:00
138.notification.sql SQL patch for notification 2012-06-08 12:42:59 -07:00
daemonstatus.sql Save daemon state to database 2012-08-01 17:06:04 -07:00
daemonstatuskey.sql Show list of non-exited daemons 2012-08-14 18:01:15 -07:00
daemontaskarchive.sql Move completed tasks to an "archive" table and delete them in the GC 2012-10-31 15:22:16 -07:00
differentialbookmarks.sql [Phabricator] track Mercurial bookmarks for differential diffs 2012-06-30 15:41:58 -07:00
draft-metadata.sql Allow using StorageFixtureScopeGuard on Windows 2012-10-24 13:59:22 -07:00
drydocktaskid.sql Undumb the Drydock resource allocator pipeline 2012-11-01 11:30:42 -07:00
edgetype.sql Save edge type as number 2012-08-16 14:43:03 -07:00
emailtable.sql Allow users to have multiple email addresses, and verify emails 2012-05-07 10:29:33 -07:00
emailtableport.php Minor, derrrp. 2012-05-07 10:34:07 -07:00
emailtableremove.sql Allow users to have multiple email addresses, and verify emails 2012-05-07 10:29:33 -07:00
fact-raw.sql Work-in-progress schema for Facts app 2012-08-09 08:40:56 -07:00
harbormasterobject.sql Add a generic multistep Markup cache 2012-07-09 15:20:56 -07:00
holidays.sql Display holidays 2012-05-03 09:22:52 -07:00
ldapinfo.sql Made it possible to login using LDAP 2012-06-13 08:58:46 -07:00
maniphestxcache.sql Use the unified markup cache for Maniphest 2012-07-11 11:40:10 -07:00
markupcache.sql Add a generic multistep Markup cache 2012-07-09 15:20:56 -07:00
migrate-differential-dependencies.php Use Edges to store dependencies between revisions in Differential 2012-07-18 20:42:06 -07:00
migrate-maniphest-dependencies.php Use Edges to store dependencies between tasks in Maniphest 2012-07-18 20:41:42 -07:00
migrate-maniphest-revisions.php Move Task <=> Revision storage to Edges 2012-07-20 08:59:39 -07:00
migrate-project-edges.php Migrate project membership to edges 2012-08-07 18:02:05 -07:00
pastepolicy.sql Add paste policy storage 2012-09-13 10:11:14 -07:00
phameblog.sql Phame - introduce blogs 2012-07-19 09:03:10 -07:00
phamedomain.sql Phame - allow blogs to specify custom URIs 2012-09-30 17:10:27 -07:00
phameoneblog.sql Make posts 1:1 with blogs and implement policy controls 2012-10-15 14:50:04 -07:00
phamepolicy.sql Make PhameBlogs respect policies 2012-10-15 14:49:52 -07:00
phiddrop.sql Remove PHID database, add Harbormaster database 2012-05-20 14:46:01 -07:00
policy-project.sql Add View, Edit and Join policies to PhabricatorProject 2012-08-11 07:05:01 -07:00
ponder-comments.sql adding comments to ponder 2012-09-11 12:13:20 -07:00
ponder-mailkey-populate.php Add mail keys to Ponder questions 2012-10-08 20:14:58 -07:00
ponder-mailkey.sql Add mail keys to Ponder questions 2012-10-08 20:14:58 -07:00
ponder.sql Adding Ponder-related files. 2012-08-10 10:44:04 -07:00
statustxt.sql Create a status tool by giving /calendar/ some teeth 2012-10-24 13:22:24 -07:00
symbolcontexts.sql Add a context field to symbol objects 2012-08-06 12:20:45 -07:00
testdatabase.sql Remove PHID database, add Harbormaster database 2012-05-20 14:46:01 -07:00
threadtopic.sql Make Thread-Topic human readable 2012-06-14 11:36:34 -07:00
userstatus.sql Allowing setting user status 2012-05-03 18:24:30 -07:00
usertranslation.sql Allow user override translation and implement PhutilPerson 2012-06-14 18:33:00 -07:00
xhprof.sql Properly create xhpast database 2012-10-08 16:09:20 -07:00