1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-11 16:16:14 +01:00
Commit graph

303 commits

Author SHA1 Message Date
James Rhodes
d571507651 Update a few things in Drydock for policies
Summary: DrydockResource has been updated to be policy-aware (although there are no policy columns).

Test Plan: Clicked around in Drydock, viewed resources and leases, everything still seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3605, T4111

Differential Revision: https://secure.phabricator.com/D7595
2013-11-28 08:32:51 +11:00
James Rhodes
7c3cb5948c Drydock blueprint for preallocated remote hosts
Summary:
This adds a Drydock blueprint for preallocated, remote hosts.  This will be used by the Harbormaster interface to allow users to specify remote hosts that builds can be run on.

This adds a `canAllocateResource` method to Drydock blueprints; it is used to detect whether a blueprint can allocate a resource for the given type and attributes.

Test Plan:
Ran:

```
bin/drydock lease --type host --attributes remote=true,preallocated=true,host=192.168.56.101,port=22,user=james,keyfile=,path=C:\\Build\\,platform=windows
```

and saw the "C:\Build\<id>" folder appear on the remote Windows machine.  Viewed the lease and resource in Drydock as well.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T4111

Differential Revision: https://secure.phabricator.com/D7593
2013-11-22 14:34:10 -08:00
Chad Little
97c690fc0f PHUIPropertyListView
Summary: This builds out and implements PHUIPropertyListView (container) and PHUIPropertyListItemView (section) as well as adding tabs.

Test Plan: Tested each page I edited with the exception of Releeph and Phortune, though those changes look ok to me diff wise. Updated examples page with tabs.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7283
2013-10-11 07:53:56 -07:00
epriestley
b592630d72 Provide more structure to PHUIObjectBoxView
Summary:
Three changes here.

  - Add `setActionList()`, and use that to set the action list.
  - Add `setPropertyList()`, and use that to set the property list.

These will let us add some apropriate CSS so we can fix the border issue, and get rid of a bunch of goofy `.x + .y` selectors.

  - Replace `addContent()` with `appendChild()`.

This is just a consistency thing; `AphrontView` already provides `appendChild()`, and `addContent()` did the same thing.

Test Plan:
  - Viewed "All Config".
  - Viewed a countdown.
  - Viewed a revision (add comment, change list, table of contents, comment, local commits, open revisions affecting these files, update history).
  - Viewed Diffusion (browse, change, history, repository, lint).
  - Viewed Drydock (resource, lease).
  - Viewed Files.
  - Viewed Herald.
  - Viewed Legalpad.
  - Viewed macro (edit, edit audio, view).
  - Viewed Maniphest.
  - Viewed Applications.
  - Viewed Paste.
  - Viewed People.
  - Viewed Phulux.
  - Viewed Pholio.
  - Viewed Phame (blog, post).
  - Viewed Phortune (account, product).
  - Viewed Ponder (questions, answers, comments).
  - Viewed Releeph.
  - Viewed Projects.
  - Viewed Slowvote.

NOTE: Images in Files aren't on a black background anymore -- I assume that's on purpose?

NOTE: Some jankiness in Phortune, I'll clean that up when I get back to it. Not related to this diff.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D7174
2013-09-30 09:36:04 -07:00
Chad Little
94d0704fdb Add objectheaders to new View
Summary: This adds the 'PHUIObjectBox' to nearly every place that should get it. I need to comb through Diffusion a little more. I've left Differential mostly alone, but may decide to do it anyways this weekend. I'm sure I missed something else, but these are easy enough to update.

Test Plan: tested each new layout.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7162
2013-09-28 15:55:38 -07:00
epriestley
c467cc464f Make most repository reads policy-aware
Summary: Ref T603. This swaps almost all queries against the repository table over to be policy aware.

Test Plan:
  - Made an audit comment on a commit.
  - Ran `save_lint.php`.
  - Looked up a commit with `diffusion.getcommits`.
  - Looked up lint messages with `diffusion.getlintmessages`.
  - Clicked an external/submodule in Diffusion.
  - Viewed main lint and repository lint in Diffusion.
  - Completed and validated Owners paths in Owners.
  - Executed dry runs via Herald.
  - Queried for package owners with `owners.query`.
  - Viewed Owners package.
  - Edited Owners package.
  - Viewed Owners package list.
  - Executed `repository.query`.
  - Viewed "Repository" tool repository list.
  - Edited Arcanist project.
  - Hit "Delete" on repository (this just tells you to use the CLI).
  - Created a repository.
  - Edited a repository.
  - Ran `bin/repository list`.
  - Ran `bin/search index rGTESTff45d13dffcfb3ea85b03aac8cc36251cacdf01c`
  - Pushed and parsed a commit.
  - Skipped all the Drydock stuff, as it it's hard to test and isn't normally reachable.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7132
2013-09-25 16:54:48 -07:00
Chad Little
e8bb24fd60 Policy, Status in PHUIHeaderView
Summary: The adds the ability to set 'properties' such as state, privacy, due date to the header of objects.

Test Plan: Implemented in Paste, Pholio. Tested various states.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7016
2013-09-17 09:12:37 -07:00
Chad Little
5ba20b8924 Move PhabricatorObjectItem to PHUIObjectItem, add 'plain' setting for lists.
Summary: Adds plain support for object lists that just look like lists

Test Plan: review UIexamples and a number of other applications

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6922
2013-09-09 14:14:34 -07:00
Gareth Evans
fcba0c74d9 Replace all "attach first..." exceptions with assertAttached()
Summary:
Ref T3599
Go through everything, grep a bit, replace some bits.

Test Plan: Navigate around a bit

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3599

Differential Revision: https://secure.phabricator.com/D6871
2013-09-03 06:02:14 -07:00
Bob Trahan
9838251515 Make PhabricatorActionListView logged-out user savvy
Summary:
Fixes T2691. Now, all PhabricatorActionListViews in the codebase setObjectHref to $request->getRequestURI. This value is passed over to PhabricatorActionItems right before they are rendered. If a PhabricatorActionItem is a workflow and there is no user OR the user is logged out, we used this objectURI to construct a log in URI.

Potentially added some undesirable behavior to aggressively setUser (and later setObjectURI) from within the List on Actions... This should be okay-ish unless there was a vision of actions having different user objects associated with them. I think this is a safe assumption.

Test Plan: played around with a mock all logged out (Ref T2652) and it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2691

Differential Revision: https://secure.phabricator.com/D6416
2013-07-12 11:39:47 -07:00
epriestley
d4ca508d2b Add PhabricatorWorker->log()
Summary:
Ref T2852. Add a `log()` method to `PhabricatorWorker` to make debugging easier.

I renamed the similar Drydock-specific method.

Test Plan:
Used logging in a future revision:

  ...
  <<< [36] <http> 211,704 us
  Updating main task.
  >>> [37] <http> https://app.asana.com/api/1.0/tasks/6153776820388
  ...

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6296
2013-06-25 16:31:37 -07:00
Chad Little
c51f45fbcd Add actions to mobile crumbs
Summary: Adds in the ActionList into Crumbs for mobile on many applications.

Test Plan: Tested each application except probably drydock since not sure how to test that. Also cleaned up Ponder a little.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5648
2013-04-10 13:08:36 -07:00
epriestley
ef7f16180c Restore merge of phutil_tag. 2013-02-13 14:51:18 -08:00
epriestley
73cce6e131 Revert "Promote phutil-tag again"
This reverts commit 8fbabdc06d, reversing
changes made to 2dab1c1e42.
2013-02-13 14:08:57 -08:00
vrana
9b8da73765 Convert AphrontTableView to safe HTML
Summary:
Lots of killed `phutil_escape_html()`.

Done by searching for `AphrontTableView` and then `$rows` (usually) backwards.

Test Plan:
Looked at homepage.

  echo id(new AphrontTableView(array(array('<'))))->render();

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4884
2013-02-09 15:11:38 -08:00
vrana
58b6e2cac6 Convert AphrontDialogView to safe HTML
Summary:
Done by searching for `AphrontDialogView` and then `appendChild()`.

Also added some `pht()`.

Test Plan: None.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4882
2013-02-09 15:11:35 -08:00
epriestley
40547030a5 render_tag -> tag: PropertyListView
Summary: Converts callsites in PropertyListView (addDetail() and setTextContent()).

Test Plan: Grepped for PhabricatorPropertyListView, addDetail() and setTextContent().

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4695
2013-01-29 11:01:47 -08:00
epriestley
d83257c29a Fix object ObjectItemListView attributes
Summary: This works after pht() + html got sorted out.

Test Plan: Looked at some object attribute lists.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4645
2013-01-28 18:09:00 -08:00
vrana
48561a8b1f Convert phutil_render_tag(X, Y, phutil_escape_html(Z)) to phutil_tag
Summary:
Created with spatch:

  lang=diff
  - phutil_render_tag
  + phutil_tag
    (X, Y,
  - phutil_escape_html(
    Z
  - )
    )

Test Plan: Loaded homepage

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4501
2013-01-24 19:08:55 -08:00
Lauri-Henrik Jalonen
2a6060a763 Added beta status for applications
Summary: Fixes T2338

Test Plan: bjhb

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, chad

Maniphest Tasks: T2338

Differential Revision: https://secure.phabricator.com/D4529
2013-01-19 10:31:28 -08:00
epriestley
c85cc932ef Remove Drydock config options
Summary: Adding random config options has a higher cost now since we can't remove them without raising warnings in installs about missing/unknown config. These are a bit premature to expose just yet -- I might want to put them in web-based config, too.

Test Plan: Grepped for strings.

Reviewers: codeblock, btrahan

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4467
2013-01-16 09:20:23 -08:00
Chad Little
38626dce64 Remove spacer from sidebars.
Summary: This removes all calls to addSpacer and the method. We were applying it inconsistently and it was causing spacing issues with redesigning the sidenav. My feeling is we can recreate the space in CSS if the design dictates, which would apply it consistently.

Test Plan: Go to Applications, click on every application.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4420
2013-01-13 08:17:12 -08:00
epriestley
edd5c208a6 Make all of Drydock work on Mobile
Summary: This is probably not the most useful app to have work on mobile, but get the log view to do something fairly sensible.

Test Plan: Looked at all Drydock views in mobile.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4224
2012-12-17 15:16:44 -08:00
epriestley
53c1483ee5 Make most Drydock web interfaces work with mobile
Summary: The logs bits still need some work but add crumbs/lists to everything else. Also build a propery DrydockResourceQuery.

Test Plan: Looked at lease list/detail; resource list/detail.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4221
2012-12-17 14:47:21 -08:00
epriestley
97045077c7 Show Drydock resource leases, add DrydockLeaseQuery, allow reuse of working copies
Summary: Minor updates to Drydock things to make them work better. In particular, after this patch working copies are correctly allocated or reused.

Test Plan: Ran "reparse.php --harbormaster <derp derp>", saw reuse of working copies when unleased resources were avilable.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4216
2012-12-17 13:53:32 -08:00
epriestley
adfe84ffce Add HarbormasterRunnerWorker, for running CI tests
Summary:
This is very preliminary and doesn't actually do anything useful. In theory, it uses Drydock to check out a working copy and run tests. In practice, it's not actually capable of running any of our tests (because of complicated interdependency stuff), but does check out a working copy and //try// to run tests there.

Adds various sorts of utility methods to various things as well.

Test Plan: Ran `reparse.php --harbormaster --trace <commit>`, observed attempt to run tests via Drydock.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015, T1049

Differential Revision: https://secure.phabricator.com/D4215
2012-12-17 13:43:26 -08:00
epriestley
e4bb9255be Allow leases to be explicitly released via web or CLI
Summary: Permit the forcible release of Drydock leases. The implementation isn't very exciting for now.

Test Plan: Released leases via web and CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4181
2012-12-14 15:42:58 -08:00
epriestley
cce5ebebe9 Improve Drydock's ability to allocate leases correctly
Summary:
Right now, Drydock gives out multiple leases to the same working copy and gives out leases to working copies with repository "P" in them when the user requested some other repository.

Add two callbacks:

  - `canAllocateLease()` - allows a blueprint to reject a lease on a resource because of a fundamental incompatibility, like "it's a working copy with Phabricator in it, but the lease wants a working copy with Javelin in it".
  - `shouldAllocateLease()` - allows a blueprint to reject a lease on a resource because of resource limits, like "only one active lease can own a working copy at a time".

Also cleaned up various other things.

Test Plan:
After implementing the callbacks, Drydock has the correct behavior:

  - It gives multiple leases on `localhost`, but only one lease per working-copy resource.
  - It does not grant leases on resources with repository X to requests for repository Y.

Ran `bin/drydock lease --type working-copy --repositoryID 12` and similar repeatedly and verified results in the web console.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4166
2012-12-12 18:42:12 -08:00
epriestley
d1429488cd Show lease and resource attributes in web view for Drydock
Summary: Show attributes on the view pages.

Test Plan: {F26985} {F26986}

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4165
2012-12-12 07:36:53 -08:00
epriestley
f306cab653 Use application icons for "Eye" menu and Crumbs
Summary:
Issues here:

  - Need an application-sized "eye", or a "home" icon for "Phabricator Home".
  - Some of the "apps_lb_2x" sliced images are the "_dark_" versions, not the light versions.
  - If you slice an application-sized "logout" (power off) icon and application-sized "help" (questionmark in circle) icon I can replace the current menu icons and nearly get rid of "autosprite".
  - To replace the icons on /applications/, the non-retina size is "4x", so we'd need "8x" for retina. Alternatively I can reduce the icon sizes by 50%.
  - The "Help", "Settings" and "Logout" items currently have a "glowing" hover state, which needs a variant (or we can drop it).
  - The /applications/ icons have a white hover state (or we can drop it).
  - The 1x application (14x14) icons aren't used anywhere right now, should they be? Maybe in the feed in the future, etc?
  - The "apps-2x" and "apps-large" sheets are the same image, but getting them to actually use the same file is a bit tricky, so I just left them separate for now.

Test Plan:
{F26698}
{F26699}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4108
2012-12-07 13:37:28 -08:00
epriestley
150f711cc8 Make drydock case sensitive in attribute parsing
Summary: See D4047. Get rid of this strtolower() junk.

Test Plan:
```
$ /bin/drydock lease --type working-copy --attributes repositoryID=12
Acquired Lease 66
```

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4048
2012-11-29 06:05:35 -08:00
epriestley
b04114f95c Allow Drydock to allocate (very basic) working copy resources
Summary: This is missing a lot of features, but technically allows working copy allocation.

Test Plan: Ran `drydock lease --type working-copy --attributes repositoryID=12`, got a working copy of Phabricator allocated on disk.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3999
2012-11-27 12:48:14 -08:00
epriestley
5cbc31644b Add a "close" action to Drydock resources
Summary: This does nothing fancy, just closes the resource and releases/breaks leases. They'll get cleaned up in some to-be-written GC process.

Test Plan: Closed resources from web UI and CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3998
2012-11-27 12:48:03 -08:00
epriestley
b2bf634795 Add a detail view for resources in Drydock
Summary: Nothing fancy here, just add a detail page.

Test Plan:
Looked at detail page.

{F24808}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, vrana

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3995
2012-11-20 13:25:22 -08:00
epriestley
c5e64166da Minor, older version of the linter went crazy. 2012-11-06 15:31:00 -08:00
epriestley
7e0ce08154 Make various Drydock CLI/Allocator improvements
Summary:
  - Remove EC2, RemoteHost, Application, etc., blueprints for now. They're very proof-of-concept and Blueprints are getting API changes I don't want to bother propagating for now. Leave the abstract base class and the LocalHost blueprint. I'll restore the more complicated ones once better foundations are in place.
  - Remove the Allocate controller from the web UI. The original vision here was that you'd manually allocate resources in some cases, but it no longer makes sense to do so as all allocations come from leases now. This simplifies allocations and makes the rule for when we can clean up resources clear-cut (if a resource has no more active leases, it can be cleaned up). Instead, we'll build resources like the localhost and remote hosts lazily, when leases come in for them.
  - Add some configuration to manage the localhost blueprint.
  - Refactor `canAllocateResources()` into `isEnabled()` (for config checks) and `canAllocateMoreResources()` (for quota checks, e.g. too many resources are allocated already).
  - Juggle some signatures to align better with a world where blueprints generally do allocate.
  - Add some more logging and error handling.
  - Fix an issue with log ordering.

Test Plan: Allocated some localhost leases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3902
2012-11-06 15:30:11 -08:00
epriestley
0e774dac93 Make minor improvements to Drydock web interface
Summary: Consolidate some code and start on lease detail pages.

Test Plan: {F22783}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3901
2012-11-06 15:28:33 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
89b37f0357 Make various Drydock improvements
Summary:
Tightens up a bunch of stuff:

  - In `drydock lease`, pull and print logs so the user can see what's happening.
  - Remove `DrydockAllocator`, which was a dumb class that did nothing. Move the tiny amount of logic it held directly to `DrydockLease`.
  - Move `resourceType` from worker task metadata directly to `DrydockLease`. Other things (like the web UI) can be more informative with this information available.
  - Pass leases to `allocateResource()`. We always allocate in response to a lease activation request, and the lease often has vital information. This also allows us to associate logs with leases correctly.

Test Plan: Ran `drydock lease --type host` and saw it perform a host allocation in EC2.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3870
2012-11-01 16:53:17 -07:00
epriestley
07dc943215 Modernize the drydock script
Summary: Add a bin/drydock symlink and break it into workflows. Nothing too special here.

Test Plan: Ran `bin/drydock wait-for-lease`, `bin/drydock lease`, `bin/drydock help`, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3867
2012-11-01 15:30:14 -07:00
epriestley
f0fdcf1a51 Undumb the Drydock resource allocator pipeline
Summary:
This was the major goal of D3859/D3855, and to a lesser degree D3854/D3852.

As Drydock is allocating a resource, it may need to allocate other resources first. For example, if it's allocating a working copy, it may need to allocate a host first.

Currently, we have the process basically queue up the allocation (insert a task into the queue) and sleep() until it finishes. This is problematic for a bunch of reasons, but the major one is that if allocation takes more resources (host, port, machine, DNS) than you have daemons, they could all end up sleeping and waiting for some other daemon to do their work. This is really stupid. Even if you only take up some of them, you're spending slots sleeping when you could be doing useful work.

To partially get around this and make the CLI experience less dumb, there's this goofy `synchronous` flag that gets passed around everywhere and pushes the workflow through a pile of special cases. Basically the `synchronous` flag causes us to do everything in-process. But this is dumb too because we'd rather do things in parallel if we can, and we have to have a lot of special case code to make it work at all.

Get rid of all of this. Instead of sleep()ing, try to work on the tasks that need to be worked on. If another daemon grabbed them already that's fine, but in the worst case we just gracefully degrade and do everything in process. So we get the best of both worlds: if we have parallelizable tasks and free daemons, things will execute in parallel. If we have nonparallelizable tasks or no free daemons, things will execute in process.

Test Plan: Ran `drydock_control.php --trace` and saw it perform cascading allocations without sleeping or special casing.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3861
2012-11-01 11:30:42 -07:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
epriestley
a7da4fad88 Add Drydock Application
Summary: Add an Application class for Drydock and move routing rules there.

Test Plan: Looked at /applications/, clicked around drydock.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3847
2012-10-31 09:57:57 -07:00
vrana
8ff52c0b6c Set viewer for all handles loaded in controllers
Summary:
I've replaced all `id(new PhabricatorObjectHandleData(...))->loadHandles()` by `$this->loadViewerHandles(...)`.
Lint caught one usage in a static method.

Test Plan: Displayed revision with sporadic author.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3432
2012-09-04 23:14:26 -07:00
epriestley
66cee129b6 Remove all code referencing old tab navigation
Summary:
  - Add getHelpURI() to PhabricatorApplication for application user guides.
  - Add a new "help" icon menu item and skeletal Diviner application.
  - Move help tabs to Applications where they exist, document the other ones that don't exist yet.
  - Grep for all tab-related stuff and delete it.

Test Plan: Clicked "help" for some apps. Clicked around randomly in a bunch of other apps.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3267
2012-08-13 15:28:41 -07:00
epriestley
ed4a155c91 Rename "IDPaged" to "CursorPaged", "executeWithPager" to "executeWith[Cursor|Offset]Pager"
Summary:
I'm trying to make progress on the policy/visibility stuff since it's a blocker for Wikimedia.

First, I want to improve Projects so they can serve as policy groups (e.g., an object can have a visibility policy like "Visible to: members of project 'security'"). However, doing this without breaking anything or snowballing into a bigger change is a bit awkward because Projects are name-ordered and we have a Conduit API which does offset paging. Rather than breaking or rewriting this stuff, I want to just continue offset paging them for now.

So I'm going to make PhabricatorPolicyQuery extend PhabricatorOffsetPagedQuery, but can't currently since the `executeWithPager` methods would clash. These methods do different things anyway and are probably better with different names.

This also generally improves the names of these classes, since cursors are not necessarily IDs (in the feed case, they're "chronlogicalKeys", for example). I did leave some of the interals as "ID" since calling them "Cursor"s (e.g., `setAfterCursor()`) seemed a little wrong -- it should maybe be `setAfterCursorPosition()`. These APIs have very limited use and can easily be made more consistent later.

Test Plan: Browsed around various affected tools; any issues here should throw/fail in a loud/obvious way.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3177
2012-08-07 11:54:06 -07:00
vrana
6cc196a2e5 Move files in Phabricator one level up
Summary:
- `kill_init.php` said "Moving 1000 files" - I hope that this is not some limit in `FileFinder`.
- [src/infrastructure/celerity] `git mv utils.php map.php; git mv api/utils.php api.php`
- Comment `phutil_libraries` in `.arcconfig` and run `arc liberate`.

NOTE: `arc diff` timed out so I'm pushing it without review.

Test Plan:
/D1234
Browsed around, especially in `applications/repository/worker/commitchangeparser` and `applications/` in general.

Auditors: epriestley

Maniphest Tasks: T1103
2012-06-01 12:32:44 -07:00
epriestley
09c8af4de0 Upgrade phabricator to libphutil v2
Summary: Mechanical changes from D2588. No "Class.php" moves yet.

Test Plan: See D2588.

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2591
2012-05-30 14:26:29 -07:00
epriestley
914f044b62 More Drydock Stuff
Summary:
  - Still really really rough.
  - Adds a full synchronous mode for debugging.
  - Adds some logging.
  - It can now allocate EC2 machines and put webroots on them in a hacky, terrible way.
  - Adds a base query class.

Test Plan: oh hey look a test page? http://ec2-50-18-65-151.us-west-1.compute.amazonaws.com:2011/

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D2026
2012-03-26 20:54:26 -07:00
epriestley
11cccb98c2 Add "final" to more classes
Summary: No big surprises here, delted the unused "DarkConsole" class.

Test Plan: Ran 'testEverythingImplemented' to verify I wasn't finalizing anything we extend.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T795

Differential Revision: https://secure.phabricator.com/D1876
2012-03-13 11:18:11 -07:00
epriestley
add1ae945d Use setConcreteOnly() in Phabricator and only list/launch concrete Daemons
Summary: We currently allow you to launch abstract daemons; use
setConcreteOnly() to only list/launch concrete daemons.

Test Plan: Ran "phd list" (no abstract daemons listed), "phd launch
PhabricatorRepositoryCommitDiscoveryDaemon" (reasonable error message).

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T801

Differential Revision: https://secure.phabricator.com/D1487
2012-01-25 11:50:59 -08:00
epriestley
7d1f62409d Move resource allocation to task queue
Summary: Run the actual resource allocation for Drydock out-of-process via the
task queue.

Test Plan: Ran "drydock_control.php", saw it insert a task and wait for task
completion. Ran "phd debug taskmaster" and saw it run the task.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1470
2012-01-24 09:44:14 -08:00
epriestley
d1ee08b2df Drydock Rough Cut
Summary:
Rough cut of Drydock. This is very basic and doesn't do much of use yet (it
//does// allocate EC2 machines as host resources and expose interfaces to them),
but I think the overall structure is more or less reasonable.

== Interfaces

Vision: Applications interact with Drydock resources through DrydockInterfaces,
like **command**, **filesystem** and **httpd** interfaces. Each interface allows
applications to perform some kind of operation on the resource, like executing
commands, reading/writing files, or configuring a web server. Interfaces have a
concrete, specific API:

  // Filesystem Interface
  $fs = $lease->getInterface('filesystem'); // Constants, some day?
  $fs->writeFile('index.html', 'hello world!');

  // Command Interface
  $cmd = $lease->getInterface('command');
  echo $cmd->execx('uptime');

  // HTTPD Interface
  $httpd = $lease->getInterface('httpd');
  $httpd->restart();

Interfaces are mostly just stock, although installs might add new interfaces if
they expose different ways to interact with resources (for instance, a resource
might want to expose a new 'MongoDB' interface or whatever).

Currently: We have like part of a command interface.

== Leases

Vision: Leases keep track of which resources are in use, and what they're being
used for. They allow us to know when we need to allocate more resources (too
many sandcastles on the existing hosts, e.g.) and when we can release resources
(because they are no longer being used). They also give applications something
to hold while resources are being allocated.

  // EXAMPLE: How this should work some day.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('sandcastle');
  $allocator->setAttributes(
    array(
      'diffID' => $diff->getID(),
    ));
  $lease = $allocator->allocate();
  $diff->setSandcastleLeaseID($lease->getID());

  // ...

  if ($lease->getStatus() == DrydockLeaseStatus::STATUS_ACTIVE) {
    $sandcastle_link = $lease->getInterface('httpd')->getURI('/');
  } else {
    $sandcastle_link = 'Still building your sandcastle...';
  }
  echo "Sandcastle for this diff: ".$sandcastle_link;

  // EXAMPLE: How this actually works now.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('host');
  // NOTE: Allocation is currently synchronous but will be task-driven soon.
  $lease = $allocator->allocate();

Leases are completely stock, installs will not define new lease types.

Currently: Leases exist and work but are very very basic.

== Resources

Vision: Resources represent some actual thing we've put somewhere, whether it's
a host, a block of storage, a webroot, or whatever else. Applications interact
through resources by acquiring leases to them, and then getting interfaces
through these leases. The lease acquisition process has a side effect of
allocating new resources if a lease can't be acquired on existing resources
(e.g., the application wants storage but all storage resources are full) and
things are configured to autoscale.

Resources may themselves acquire leases in order to allocate. For instance, a
storage resource might first acquire a lease to a host resource. A 'test
scaffold' resource might lease a storage resource and a mysql resource.

Not all resources are auto-allocate: the entry-level version of Drydock is that
you manually allocate a couple boxes and configure them through the web console.
Then, e.g.,  'storage' / 'webroot' resources allocate on top of them, but the
host pool itself does not autoscale.

Resources are completely stock, they are abstract shells representing any
arbitrary thing.

Currently: Resource exist ('host' only) but are very very basic.

== Blueprints

Vision: Blueprints contain instructions for building interfaces to, (possibly)
allocating, updating, managing, and destroying a specific type of resource in a
specific location. One way to think of them is that they are scripts for
creating and deleting resources. For example, the LocalHost, RemoteHost and
EC2Host blueprints can all manage 'host' resources.

Eventually, we will support more types of resources (storage, webroot,
sandcastle, test scaffold, phacility deployment) and more providers for resource
types, some of which will be in the Phabricator mainline and some of which will
be custom.

Blueprints are very custom and specific to application types, so installs will
define new blueprints if they are making significant use of Drydock.

Currently: They exist but have few capabilities. The stock blueprints do nearly
nothing useful. There is a technically functional blueprint for host allocation
in EC2.

== Allocator

This is just the actual code to execute the lease acquisition process.

Test Plan: Ran "drydock_control.php" script, it allocated a machine in EC2,
acquired a lease on it, interfaced with it, and then released the lease. Ran it
again, got a fresh lease on the existing resource.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D1454
2012-01-19 21:12:57 -08:00