Summary:
Ref T2015. Not directly related to Drydock, but I've wanted to do this for a bit.
Introduce a common base class for all the workflows in the scripts in `bin/*`. This slightly reduces code duplication by moving `isExecutable()` to the base, but also provides `getViewer()`. This is a little nicer than `PhabricatorUser::getOmnipotentUser()` and gives us a layer of indirection if we ever want to introduce more general viewer mechanisms in scripts.
Test Plan: Lint; ran some of the scripts.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7838
Summary: Ref T2015. All the Drydock query classes share the application method; move it into a shared base class to slightly shrink the codebase.
Test Plan: Browsed query UIs.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7837
Summary:
Ref T2015. Moves a bunch of raw object loads into modern policy-aware queries.
Also straightens out the Log and Lease policies a little bit: there are legitimate states where these objects are not attached to a resource (particularly, while a lease is being acquired). Handle these more gracefully.
Test Plan: Lint / browsed stuff.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7836
Summary:
Ref T2015. Currently, Drydock has a `wait-for-lease` workflow which is invoked in the background by the `lease` workflow.
The goal of this mechanism is to allow `bin/drydock lease` to print out logs as the lease is acquired. However, this predates the `runAllTasksInProcess` flags, and they provide a simpler and more robust way (potentially with `--trace` and `PhutilConsole`) to do synchronous execution and debug logging.
Simplify this whole mechanism: just run everything in-process in `bin/drydock lease`, and do logging via `--trace`. We could thread a `PhutilConsole` through things too, but this seems good enough for now.
Also various cleanup/etc.
Test Plan: Ran `bin/drydock lease`. Ran `bin/harbormaster build X --plan Y`, for `Y` being a Drydock-dependent build plan.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7835
Summary: Ref T4266. This implements rules similar to the old rules. With D7842, maybe this is reasonable? I think it's not like grotesquely bad, at least.
Test Plan: See screenshot.
Reviewers: chad, wrotte
Reviewed By: chad
CC: aran
Maniphest Tasks: T4266
Differential Revision: https://secure.phabricator.com/D7843
Summary:
Ref T4266. This possibly moves us towards getting reasonable timeline grouping:
- Always sort icon stories to the top.
- Render one timestamp for the whole group, using the earliest tranaction date.
- Move any "Edit", "Edited", or "Preview" links to the top.
- Rendering just one timestamp implicitly fixes the JS issues.
- For stories without an icon, indent them if any member of the group has an icon.
Test Plan: See screenshots.
Reviewers: chad, wotte
Reviewed By: chad
CC: aran
Maniphest Tasks: T4266
Differential Revision: https://secure.phabricator.com/D7842
Summary:
Ref T2015. After introducing ApplicationSearch, the left nav turned into a soupy mess. Split the major sections into four separate areas, and unify them with a simple console.
This also reverts all the prefix stuff, since the results were awful and I don't anticipate it ever being the best solution to any UX problem.
Test Plan:
Browsed blueprints, resources, leases and logs.
Here's the new console:
{F93279}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7833
Summary: Ref T2015. Update DrydockLog for policy awareness and give it a policy query.
Test Plan: Browsed all the log interfaces.
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7831
Summary: Ref T2015. This turns the side nav into a bigger mess for now, but uses ApplicationSearch for blueprints.
Test Plan: Queried blueprints in the UI.
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7829
Summary:
Ref T2015. These never got updated to the new stuff, move them out of the old `Constants` class and let them load handles, etc.
Also some half-cleanup of some Blueprint/BlueprintImplementation stuff.
Test Plan: Used `phid.query` to query a Resource, Lease, and Blueprint.
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7828
Summary: See thread; fixes fatal. The actual name of this method is `getHarbormaster...`.
NOTE: This fixes a fatal in Differential which impedes review, so I'm pushing it as-is.
Test Plan: Browsed a revision.
Reviewers: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7834
Summary:
Ref T2015. Applies ApplicationSearch to DrydockLease.
This makes the left nav in Drydock a little funky. It will probably get worse for a bit before it gets better, since I want to bring everything to ApplicationSearch and then sort out the details.
Test Plan: Queried leases in Drydock.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7827
Summary: Ref T2015. DrydockLease predates widespread adoption of policies. Make it -- and its query -- policy aware.
Test Plan: Browsed leases from the web UI. Grepped for callsites.
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7826
Summary:
Ref T1049. Adds `bin/harbormaster` and `bin/harbormaster build` for applying plans from the console. Since this gets `--trace`, it's much easier to debug what's going on.
This doesn't work properly with some of the Drydock steps yet, I need to look at those. I think `setRunAllTasksInProcess` probably obsoletes some of the mechanisms. It might also not work with "Wait for Builds" but I didn't check.
Test Plan: Used `bin/harbormaster` to run a bunch of builds. Ran builds from web UI.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7825
Summary:
Ref T1049. Generally, it's useful to separate test/trial/manual runs from production/automatic runs.
For example, you don't want to email a bunch of people that the build is broken just because you messed something up when writing a new build plan. You'd rather try it first, then promote it into production once you have some good runs.
Similarly, test runs generally should not affect the outside world, etc. Finally, some build steps (like "wait for other buildables") may want to behave differently when run in production/automation than when run in a testing environment (where they should probably continue immediately).
So, formalize the distinction between automatic buildables (those created passively by the system in response to events) and manual buildables (those created explicitly by users). Add filtering, and stop the automated parts of the system from interacting with the manual parts (for example, we won't show manual results on revisions).
This also moves the "Apply Build Plan" to a third, new home: instead of the sidebar or Buildables, it's now on plans. I think this generally makes more sense given how things have developed. Broadly, this improves isolation of test environments.
Test Plan: Created some builds, browsed around, used filters, etc.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7824
Summary: Ref T1049. Adds "Repository", "Revision", "Diff" and "Commit" as searchable fields.
Test Plan: Used all the fields to filter things.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7823
Summary:
Fixes T4195. Allows you to write a rule against a commit's branches.
This completes outstanding work on T4195.
Test Plan: Pushed to Git and Mercurial repositories and verified branches were selected correctly by examining transcripts.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7820
Summary: See comments in <https://secure.phabricator.com/D6331#comment-3> -- make the Conduit Token and Conduit Certificate interfaces readonly and select-on-click.
Test Plan:
- Viewed `/conduit/token/`, verified it was readonly and selected on click.
- Viewed `/settings/panel/conduit/`, likewise.
Reviewers: Avish, btrahan, wotte
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7819
Summary:
Ref T4257. Currently, the pull logic looks like this:
if (new) {
create();
} else {
if (hosted) {
install_hooks();
} else {
update();
}
}
This means that the first time you run `repository pull`, hooks aren't installed, which makes debugging trickier. Instead, reorganize the logic:
if (new) {
create();
} else {
if (!hosted) {
update();
}
}
if (hosted) {
install_hooks();
}
Test Plan: Ran `bin/repository pull` on a new `hg` repo and got hooks installed immediately.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4257
Differential Revision: https://secure.phabricator.com/D7818
Summary:
Fixes T4257. The `hg heads` command exits with an error code and no output in an empty repository.
Just ignore the error code: we don't have a great way to distinguish between errors, and we ran another `hg` command moments before, so we have at least some confidence it isn't a PATH sort of thing.
Test Plan: Created a new Mercurial repository and pushed to hit the error in T4257. Applied this fix and got a clean push with an accurate push log.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4257
Differential Revision: https://secure.phabricator.com/D7817
Summary: A user is reporting a re-lock in this daemon, which I can't
reproduce, but might be possible if this throws. Stop it from throwing in
a way which evades unlock.
See: <https://github.com/facebook/phabricator/issues/476>
Auditors: btrahan
Summary: add support for "assignee" conditions
Test Plan: Create a Herald rule where condition is assignee, and create a task assign to someone.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7813
Summary:
Fixes two issues:
- When rendering a task's details, we currently issue a policy-oblivious query. Instead, issue a policy-aware query.
- The formatting is a little bit weird, with the top half in a box and the bottom half with an older style. Make them consistent.
Test Plan: Looked at the detail pages for several tasks in queue.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D7812
Summary:
Ref T4195. This allows you to write rules which disallow merge commits.
Also make the reject message a little more useful.
Test Plan:
remote: This push was rejected by Herald push rule H27.
remote: Change: commit/daed0d448404
remote: Rule: No Merges
remote: Reason: No merge commits allowed. If you must push a merge, include "@force-merge" in the commit message.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7809
Summary: Ref T4195. Ref T2783. We have an old-school implementation of this; move it into a LowLevel query and make callers all run through Conduit. I need the LowLevel query for hooks, to implement an "is merge commit" Herald rule.
Test Plan:
- Ran query via Conduit for SVN, Mercurial, Git.
- Parsed a commit which closed a revision, attach/closed worked correctly.
- Browsed Diffusion.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195, T2783
Differential Revision: https://secure.phabricator.com/D7808
Summary: Refs T4195. Fixes T3936. You can't currently write rules like "block commits unless they're attached to an **accepted** revision"; allow that.
Test Plan: Pushed commits into a rule with this field, saw it work / not crash.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, mbishopim3
Maniphest Tasks: T3936, T4195
Differential Revision: https://secure.phabricator.com/D7807
Summary: Ref T4195. Allows you to write revision-based commit hooks, e.g. block all commits with no corresponding revision.
Test Plan:
Here's are the fields populating:
{F90989}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7806
Summary: Ref T4195. I need to query commit metadata to figure out which revision a commit is associated with. Move this out of the MessageParser so the code can be called from the HookEngine.
Test Plan: Used `reparse.php` to reparse a variety of SVN, Mercurial and Git commits. Used `var_dump()` to verify sensible fields were returned.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7805
Summary: Ref T4195. I need this for the Herald pre-commit rules, and it generally simplifies things.
Test Plan: Used `reparse.php` plus `var_dump()` to inspect refs in Git, Mercurial and SVN repos. They all looked correct and reparsed correctly.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7804
Summary:
There's no particular reason to allow the user to edit the clone URI field in Diffusion; editing it has no meaning and if you fat finger the keyboard, it's quite possible that the user will either accidentally clear and/or modify the URI before copying (bit me this morning).
Adding a readonly attribute to the input field allows the same benefit (URI is easily selectable) while preventing such accidental input. Fixes T4246.
Test Plan: Verified that the desired behavior is present in both Chrome, Safari, and Firefox. Field remains selectable with one click, but field is not editable.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4246
Differential Revision: https://secure.phabricator.com/D7810
Summary: Ref T4195. Adds "Author" and "Committer" fields.
Test Plan:
Created a rule using these fields:
{F90897}
...then pushed git, mercurial and svn commits and verified the correct values populated in the transcript:
{F90898}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7802
Summary:
Ref T4195. To implement the "Author" and "Committer" rules, I need to resolve author/committer strings into Phabricator users.
The code to do this is currently buried in the daemons. Extract it into a standalone query.
I also added `bin/repository lookup-users <commit>` to test this query, both to improve confidence I'm getting this right and to provide a diagnostic command for users, since there's occasionally some confusion over how author/committer strings resolve into valid users.
Test Plan:
I tested this using `bin/repository lookup-users` and `reparse.php --message` on Git, Mercurial and SVN commits. Here's the `lookup-users` output:
>>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIS3
Examining commit rINIS3...
Raw author string: epriestley
Phabricator user: epriestley (Evan Priestley )
Raw committer string: null
>>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rPOEMS165b6c54f487c8
Examining commit rPOEMS165b6c54f487...
Raw author string: epriestley <git@epriestley.com>
Phabricator user: epriestley (Evan Priestley )
Raw committer string: epriestley <git@epriestley.com>
Phabricator user: epriestley (Evan Priestley )
>>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIH6d24c1aee7741e
Examining commit rINIH6d24c1aee774...
Raw author string: epriestley <hg@yghe.net>
Phabricator user: epriestley (Evan Priestley )
Raw committer string: null
>>> orbital ~/devtools/phabricator $
The `reparse.php` output was similar, and all VCSes resolved authors correctly.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1731, T4195
Differential Revision: https://secure.phabricator.com/D7801
Summary: Ref T4195. Even though we use `svnlook` in the hook itself, I need this query elsewhere, so provide it and merge the classes into one which does the right thing.
Test Plan:
- Used `reparse.php` to reparse messages for Git, SVN and Mercurial commits, using `var_dump()` to examine the commit refs for sanity.
- Used `reparse.php` to reparse changes for an SVN commit.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7800
Summary: Ref T4249. Currently, a global rule can only trigger project audits. Although there probably aren't a huge number of use cases for triggering users from global rules, it works fine and it's somewhat confusing not to allow it.
Test Plan: {F90902}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4249
Differential Revision: https://secure.phabricator.com/D7803
Summary: There were a number of places that were generating nonsense queries for both hosted and non-hosted subversion repositories.
Test Plan: Attempted several activities in Diffusion with both a hosted and non-hosted subversion repository, including viewing various types of diffs and raw files.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7799
Summary: Ref T4010. I'll hold this for a bit, but we should eventually drop this table once the dust has settled.
Test Plan: Ran storage upgrade.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4010
Differential Revision: https://secure.phabricator.com/D7372
Summary: If you push a large binary and the data crosses multiple data frames, we can end up in a loop in the parser.
Test Plan:
After this change, I was able to push a 95MB binary in 7s, which seems reasonable:
>>> orbital ~/repos/INIS $ svn st
A large2.bin
>>> orbital ~/repos/INIS $ ls -alh
total 390648
drwxr-xr-x 6 epriestley admin 204B Dec 18 17:14 .
drwxr-xr-x 98 epriestley admin 3.3K Dec 16 11:19 ..
drwxr-xr-x 7 epriestley admin 238B Dec 18 17:14 .svn
-rw-r--r-- 1 epriestley admin 80B Dec 18 15:07 README
-rw-r--r-- 1 epriestley admin 95M Dec 18 16:53 large.bin
-rw-r--r-- 1 epriestley admin 95M Dec 18 17:14 large2.bin
>>> orbital ~/repos/INIS $ time svn commit -m 'another large binary'
Adding (bin) large2.bin
Transmitting file data .
Committed revision 25.
real 0m7.215s
user 0m5.327s
sys 0m0.407s
>>> orbital ~/repos/INIS $
There may be room to improve this by using `PhutilRope`.
Reviewers: wrotte, btrahan, wotte
Reviewed By: wotte
CC: aran
Differential Revision: https://secure.phabricator.com/D7798
Summary: Ref T4195. Same as D7793, but for mercurial. (As usual, SVN needs some goofy nonsense instead, so the next diff will just make this field work.)
Test Plan: Ran `reparse.php` on Git and Mercurial commits, var_dump'd the output and it looked correct.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7795
Summary: Ref T4195. I need to issue this command from the pre-commit hook to get commit bodies for hooks.
Test Plan: Ran `reparse.php --message --trace` and dumped the $ref, which looked correct.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7793
Summary: We currently have a lot of calls to `addCrumb(id(new PhabricatorCrumbView())->...)` which can be expressed much more simply with a convenience method. Nearly all crumbs are only textual.
Test Plan:
- This was mostly automated, then I cleaned up a few unusual sites manually.
- Bunch of grep / randomly clicking around.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: hach-que, aran
Differential Revision: https://secure.phabricator.com/D7787
Summary: Ref T4195. Add Mercurial support to the content hook phase.
Test Plan:
Here are some `commit` push logs for a Mercurial repo:
{F90689}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4195
Differential Revision: https://secure.phabricator.com/D7792
