1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-13 10:22:42 +01:00
Commit graph

4831 commits

Author SHA1 Message Date
epriestley
1ba52fac86 Introduce DrydockQuery to slightly reduce code duplication
Summary: Ref T2015. All the Drydock query classes share the application method; move it into a shared base class to slightly shrink the codebase.

Test Plan: Browsed query UIs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7837
2013-12-27 13:15:30 -08:00
epriestley
9b0fa5747b Make Drydock more broadly aware of policies
Summary:
Ref T2015. Moves a bunch of raw object loads into modern policy-aware queries.

Also straightens out the Log and Lease policies a little bit: there are legitimate states where these objects are not attached to a resource (particularly, while a lease is being acquired). Handle these more gracefully.

Test Plan: Lint / browsed stuff.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7836
2013-12-27 13:15:19 -08:00
epriestley
1650874004 Modernize Drydock CLI management of task execution
Summary:
Ref T2015. Currently, Drydock has a `wait-for-lease` workflow which is invoked in the background by the `lease` workflow.

The goal of this mechanism is to allow `bin/drydock lease` to print out logs as the lease is acquired. However, this predates the `runAllTasksInProcess` flags, and they provide a simpler and more robust way (potentially with `--trace` and `PhutilConsole`) to do synchronous execution and debug logging.

Simplify this whole mechanism: just run everything in-process in `bin/drydock lease`, and do logging via `--trace`. We could thread a `PhutilConsole` through things too, but this seems good enough for now.

Also various cleanup/etc.

Test Plan: Ran `bin/drydock lease`. Ran `bin/harbormaster build X --plan Y`, for `Y` being a Drydock-dependent build plan.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7835
2013-12-27 13:15:12 -08:00
epriestley
536c606dde Implment ApplicationTransaction grouping rules
Summary: Ref T4266. This implements rules similar to the old rules. With D7842, maybe this is reasonable? I think it's not like grotesquely bad, at least.

Test Plan: See screenshot.

Reviewers: chad, wrotte

Reviewed By: chad

CC: aran

Maniphest Tasks: T4266

Differential Revision: https://secure.phabricator.com/D7843
2013-12-27 05:51:15 -08:00
epriestley
5fdd800bb6 Provide a Drydock "Console" controller
Summary:
Ref T2015. After introducing ApplicationSearch, the left nav turned into a soupy mess. Split the major sections into four separate areas, and unify them with a simple console.

This also reverts all the prefix stuff, since the results were awful and I don't anticipate it ever being the best solution to any UX problem.

Test Plan:
Browsed blueprints, resources, leases and logs.

Here's the new console:

{F93279}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7833
2013-12-26 12:30:36 -08:00
epriestley
9c9a9a919e Use ApplicationSearch for DrydockLog
Summary: Ref T2015. Move logs over to ApplicationSearch.

Test Plan: Browsed logs in UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7832
2013-12-26 12:30:29 -08:00
epriestley
3db5833622 Make DrydockLog policy-aware
Summary: Ref T2015. Update DrydockLog for policy awareness and give it a policy query.

Test Plan: Browsed all the log interfaces.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7831
2013-12-26 12:30:17 -08:00
epriestley
bc3912e641 Use ApplicationSearch for Drydock Resources
Summary: Ref T2015. Bring ApplicationSearch to Resources, too.

Test Plan: Browsed/queried resources in UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7830
2013-12-26 12:30:10 -08:00
epriestley
fa00e86f87 Use ApplicationSearch for Drydock Blueprints
Summary: Ref T2015. This turns the side nav into a bigger mess for now, but uses ApplicationSearch for blueprints.

Test Plan: Queried blueprints in the UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7829
2013-12-26 12:30:04 -08:00
epriestley
1a82743491 Make Drydock Lease and Resource PHIDs use newer PHID infrastructure
Summary:
Ref T2015. These never got updated to the new stuff, move them out of the old `Constants` class and let them load handles, etc.

Also some half-cleanup of some Blueprint/BlueprintImplementation stuff.

Test Plan: Used `phid.query` to query a Resource, Lease, and Blueprint.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7828
2013-12-26 12:29:58 -08:00
epriestley
f9f70dc2e9 Use correct method name in Harbormaster UI event listener
Summary: See thread; fixes fatal. The actual name of this method is `getHarbormaster...`.

NOTE: This fixes a fatal in Differential which impedes review, so I'm pushing it as-is.

Test Plan: Browsed a revision.

Reviewers: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7834
2013-12-26 11:37:25 -08:00
epriestley
70244d3a12 Use ApplicationSearch to manage DrydockLeases
Summary:
Ref T2015. Applies ApplicationSearch to DrydockLease.

This makes the left nav in Drydock a little funky. It will probably get worse for a bit before it gets better, since I want to bring everything to ApplicationSearch and then sort out the details.

Test Plan: Queried leases in Drydock.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7827
2013-12-26 10:42:00 -08:00
epriestley
6b2d480fe7 Make DrydockLease a policy-aware object
Summary: Ref T2015. DrydockLease predates widespread adoption of policies. Make it -- and its query -- policy aware.

Test Plan: Browsed leases from the web UI. Grepped for callsites.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7826
2013-12-26 10:41:36 -08:00
epriestley
aad6b57c36 Add bin/harbormaster to make builds easier to debug
Summary:
Ref T1049. Adds `bin/harbormaster` and `bin/harbormaster build` for applying plans from the console. Since this gets `--trace`, it's much easier to debug what's going on.

This doesn't work properly with some of the Drydock steps yet, I need to look at those. I think `setRunAllTasksInProcess` probably obsoletes some of the mechanisms. It might also not work with "Wait for Builds" but I didn't check.

Test Plan: Used `bin/harbormaster` to run a bunch of builds. Ran builds from web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7825
2013-12-26 10:40:52 -08:00
epriestley
ac19c55822 Formalize "manual" buildables in Harbormaster
Summary:
Ref T1049. Generally, it's useful to separate test/trial/manual runs from production/automatic runs.

For example, you don't want to email a bunch of people that the build is broken just because you messed something up when writing a new build plan. You'd rather try it first, then promote it into production once you have some good runs.

Similarly, test runs generally should not affect the outside world, etc. Finally, some build steps (like "wait for other buildables") may want to behave differently when run in production/automation than when run in a testing environment (where they should probably continue immediately).

So, formalize the distinction between automatic buildables (those created passively by the system in response to events) and manual buildables (those created explicitly by users). Add filtering, and stop the automated parts of the system from interacting with the manual parts (for example, we won't show manual results on revisions).

This also moves the "Apply Build Plan" to a third, new home: instead of the sidebar or Buildables, it's now on plans. I think this generally makes more sense given how things have developed. Broadly, this improves isolation of test environments.

Test Plan: Created some builds, browsed around, used filters, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7824
2013-12-26 10:40:43 -08:00
epriestley
4a56e26a67 Allow buildables to be queried by container and underlying buildable
Summary: Ref T1049. Adds "Repository", "Revision", "Diff" and "Commit" as searchable fields.

Test Plan: Used all the fields to filter things.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7823
2013-12-26 10:40:34 -08:00
epriestley
60288edf80 Allow Harbormaster build plans to be disabled
Summary: Fixes T4187. Ref T1049. Allows build plans to be enabled or disabled.

Test Plan: Enabled and disabled build plans.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4187, T1049

Differential Revision: https://secure.phabricator.com/D7822
2013-12-26 10:40:22 -08:00
epriestley
adcc4ee1db Add a "branches" rule for Herald commit rules
Summary:
Fixes T4195. Allows you to write a rule against a commit's branches.

This completes outstanding work on T4195.

Test Plan: Pushed to Git and Mercurial repositories and verified branches were selected correctly by examining transcripts.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7820
2013-12-26 10:40:16 -08:00
epriestley
44c9a94abe Make conduit certificate readonly and select-on-click
Summary: See comments in <https://secure.phabricator.com/D6331#comment-3> -- make the Conduit Token and Conduit Certificate interfaces readonly and select-on-click.

Test Plan:
  - Viewed `/conduit/token/`, verified it was readonly and selected on click.
  - Viewed `/settings/panel/conduit/`, likewise.

Reviewers: Avish, btrahan, wotte

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7819
2013-12-23 10:43:53 -08:00
epriestley
81dcf6378d Make repository pull install hooks the first time
Summary:
Ref T4257. Currently, the pull logic looks like this:

  if (new) {
    create();
  } else {
    if (hosted) {
      install_hooks();
    } else {
      update();
    }
  }

This means that the first time you run `repository pull`, hooks aren't installed, which makes debugging trickier. Instead, reorganize the logic:

  if (new) {
    create();
  } else {
    if (!hosted) {
      update();
    }
  }

  if (hosted) {
    install_hooks();
  }

Test Plan: Ran `bin/repository pull` on a new `hg` repo and got hooks installed immediately.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4257

Differential Revision: https://secure.phabricator.com/D7818
2013-12-23 10:43:49 -08:00
epriestley
6daa2b6c2e Fix a commit hook issue with the initial commit to Mercurial repositories
Summary:
Fixes T4257. The `hg heads` command exits with an error code and no output in an empty repository.

Just ignore the error code: we don't have a great way to distinguish between errors, and we ran another `hg` command moments before, so we have at least some confidence it isn't a PATH sort of thing.

Test Plan: Created a new Mercurial repository and pushed to hit the error in T4257. Applied this fix and got a clean push with an accurate push log.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4257

Differential Revision: https://secure.phabricator.com/D7817
2013-12-23 10:43:45 -08:00
epriestley
d9a04fcb53 Minor, make sure we release locks in pull daemon if we fail in unusual ways
Summary: A user is reporting a re-lock in this daemon, which I can't
reproduce, but might be possible if this throws. Stop it from throwing in
a way which evades unlock.

See: <https://github.com/facebook/phabricator/issues/476>

Auditors: btrahan
2013-12-22 08:51:07 -08:00
xiaogaozi
54a0dd8139 Herald - add support for "assignee" conditions
Summary: add support for "assignee" conditions

Test Plan: Create a Herald rule where condition is assignee, and create a task assign to someone.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7813
2013-12-22 08:47:56 -08:00
epriestley
e27bbb9aa5 Enable the "Accepted Differential Revision" field for Herald post-commit hooks
Summary: I implemented this field, but didn't actually enable it.

Auditors: btrahan
2013-12-21 11:11:52 -08:00
epriestley
c462713584 Minor cleanup for task rendering in Daemons
Summary:
Fixes two issues:

  - When rendering a task's details, we currently issue a policy-oblivious query. Instead, issue a policy-aware query.
  - The formatting is a little bit weird, with the top half in a box and the bottom half with an older style. Make them consistent.

Test Plan: Looked at the detail pages for several tasks in queue.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D7812
2013-12-20 18:02:32 -08:00
epriestley
dd3ed6fdd8 Fix breadcrumb issue on Macro Create page
Summary: uhoh

Test Plan: !!!

Reviewers: frgtn, Korvin, btrahan

Reviewed By: Korvin

CC: aran

Differential Revision: https://secure.phabricator.com/D7811
2013-12-20 14:06:21 -08:00
epriestley
a64d127e25 Add "is merge commit" Herald field for pre-commit rules
Summary:
Ref T4195. This allows you to write rules which disallow merge commits.

Also make the reject message a little more useful.

Test Plan:
  remote: This push was rejected by Herald push rule H27.
  remote: Change: commit/daed0d448404
  remote:   Rule: No Merges
  remote: Reason: No merge commits allowed. If you must push a merge, include "@force-merge" in the commit message.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7809
2013-12-20 12:39:40 -08:00
epriestley
9c938701c3 Modernize Diffusion commitparentsquery
Summary: Ref T4195. Ref T2783. We have an old-school implementation of this; move it into a LowLevel query and make callers all run through Conduit. I need the LowLevel query for hooks, to implement an "is merge commit" Herald rule.

Test Plan:
  - Ran query via Conduit for SVN, Mercurial, Git.
  - Parsed a commit which closed a revision, attach/closed worked correctly.
  - Browsed Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195, T2783

Differential Revision: https://secure.phabricator.com/D7808
2013-12-20 12:39:21 -08:00
epriestley
72c73d644b Add an "Accepted Differential revision" field to Commit and pre-commit Content Herald rules
Summary: Refs T4195. Fixes T3936. You can't currently write rules like "block commits unless they're attached to an **accepted** revision"; allow that.

Test Plan: Pushed commits into a rule with this field, saw it work / not crash.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, mbishopim3

Maniphest Tasks: T3936, T4195

Differential Revision: https://secure.phabricator.com/D7807
2013-12-20 12:39:13 -08:00
epriestley
2436458b90 Implement "Differential Revision" fields in Herald pre-commit content adapter
Summary: Ref T4195. Allows you to write revision-based commit hooks, e.g. block all commits with no corresponding revision.

Test Plan:
Here's are the fields populating:

{F90989}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7806
2013-12-20 12:39:01 -08:00
epriestley
d7c4edab28 Move commit message/metadata field query to a separate class
Summary: Ref T4195. I need to query commit metadata to figure out which revision a commit is associated with. Move this out of the MessageParser so the code can be called from the HookEngine.

Test Plan: Used `reparse.php` to reparse a variety of SVN, Mercurial and Git commits. Used `var_dump()` to verify sensible fields were returned.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7805
2013-12-20 12:38:44 -08:00
epriestley
23332241b2 Move commit hash querying to DiffusionLowLevelCommitQuery
Summary: Ref T4195. I need this for the Herald pre-commit rules, and it generally simplifies things.

Test Plan: Used `reparse.php` plus `var_dump()` to inspect refs in Git, Mercurial and SVN repos. They all looked correct and reparsed correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7804
2013-12-20 12:38:15 -08:00
William R. Otte
ff13bb8538 Render the clone URI input field as readonly.
Summary:
There's no particular reason to allow the user to edit the clone URI field in Diffusion; editing it has no meaning and if you fat finger the keyboard, it's quite possible that the user will either accidentally clear and/or modify the URI before copying (bit me this morning).

Adding a readonly attribute to the input field allows the same benefit (URI is easily selectable) while preventing such accidental input.  Fixes T4246.

Test Plan: Verified that the desired behavior is present in both Chrome, Safari, and Firefox. Field remains selectable with one click, but field is not editable.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4246

Differential Revision: https://secure.phabricator.com/D7810
2013-12-20 13:47:25 -06:00
epriestley
134c8f5547 Add "Author" and "Committer" fields to Herald pre-commit content hooks
Summary: Ref T4195. Adds "Author" and "Committer" fields.

Test Plan:
Created a rule using these fields:

{F90897}

...then pushed git, mercurial and svn commits and verified the correct values populated in the transcript:

{F90898}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7802
2013-12-19 11:05:31 -08:00
epriestley
d667b12206 Provide a standalone query for resolution of commit author/committer into Phabricator users
Summary:
Ref T4195. To implement the "Author" and "Committer" rules, I need to resolve author/committer strings into Phabricator users.

The code to do this is currently buried in the daemons. Extract it into a standalone query.

I also added `bin/repository lookup-users <commit>` to test this query, both to improve confidence I'm getting this right and to provide a diagnostic command for users, since there's occasionally some confusion over how author/committer strings resolve into valid users.

Test Plan:
I tested this using `bin/repository lookup-users` and `reparse.php --message` on Git, Mercurial and SVN commits. Here's the `lookup-users` output:

  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIS3
  Examining commit rINIS3...
  Raw author string: epriestley
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rPOEMS165b6c54f487c8
  Examining commit rPOEMS165b6c54f487...
  Raw author string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIH6d24c1aee7741e
  Examining commit rINIH6d24c1aee774...
  Raw author string: epriestley <hg@yghe.net>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $

The `reparse.php` output was similar, and all VCSes resolved authors correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1731, T4195

Differential Revision: https://secure.phabricator.com/D7801
2013-12-19 11:05:17 -08:00
epriestley
f750d5f8dc Provide a low-level SVN commit query, and merge the VCS query types
Summary: Ref T4195. Even though we use `svnlook` in the hook itself, I need this query elsewhere, so provide it and merge the classes into one which does the right thing.

Test Plan:
  - Used `reparse.php` to reparse messages for Git, SVN and Mercurial commits, using `var_dump()` to examine the commit refs for sanity.
  - Used `reparse.php` to reparse changes for an SVN commit.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7800
2013-12-19 11:05:06 -08:00
epriestley
d64dd7e2e8 Allow global Commit herald rules to trigger audits by users
Summary: Ref T4249. Currently, a global rule can only trigger project audits. Although there probably aren't a huge number of use cases for triggering users from global rules, it works fine and it's somewhat confusing not to allow it.

Test Plan: {F90902}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4249

Differential Revision: https://secure.phabricator.com/D7803
2013-12-19 10:33:15 -08:00
William R. Otte
264bef58c4 Various fixes for hosted and non-hosted subversion queries in Diffusion.
Summary: There were a number of places that were generating nonsense queries for both hosted and non-hosted subversion repositories.

Test Plan: Attempted several activities in Diffusion with both a hosted and non-hosted subversion repository, including viewing various types of diffs and raw files.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7799
2013-12-19 10:32:26 -06:00
epriestley
151f01ae94 Implement "Body" field in Herald pre-commit content hooks
Summary: Ref T4195. Adds support for writing rules against commit message bodies.

Test Plan: Pushed git, hg, svn commits and verified their bodies populated correctly in transcripts.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7796
2013-12-19 06:56:01 -08:00
epriestley
f37832aed7 Fix loop in svnserve workflow for large binaries
Summary: If you push a large binary and the data crosses multiple data frames, we can end up in a loop in the parser.

Test Plan:
After this change, I was able to push a 95MB binary in 7s, which seems reasonable:

  >>> orbital ~/repos/INIS $ svn st
  A       large2.bin
  >>> orbital ~/repos/INIS $ ls -alh
  total 390648
  drwxr-xr-x   6 epriestley  admin   204B Dec 18 17:14 .
  drwxr-xr-x  98 epriestley  admin   3.3K Dec 16 11:19 ..
  drwxr-xr-x   7 epriestley  admin   238B Dec 18 17:14 .svn
  -rw-r--r--   1 epriestley  admin    80B Dec 18 15:07 README
  -rw-r--r--   1 epriestley  admin    95M Dec 18 16:53 large.bin
  -rw-r--r--   1 epriestley  admin    95M Dec 18 17:14 large2.bin
  >>> orbital ~/repos/INIS $ time svn commit -m 'another large binary'
  Adding  (bin)  large2.bin
  Transmitting file data .
  Committed revision 25.

  real	0m7.215s
  user	0m5.327s
  sys	0m0.407s
  >>> orbital ~/repos/INIS $

There may be room to improve this by using `PhutilRope`.

Reviewers: wrotte, btrahan, wotte

Reviewed By: wotte

CC: aran

Differential Revision: https://secure.phabricator.com/D7798
2013-12-18 17:48:29 -08:00
epriestley
92bc76aae0 Move mercurial commit metadata parsing into a LowLevel query
Summary: Ref T4195. Same as D7793, but for mercurial. (As usual, SVN needs some goofy nonsense instead, so the next diff will just make this field work.)

Test Plan: Ran `reparse.php` on Git and Mercurial commits, var_dump'd the output and it looked correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7795
2013-12-18 17:48:19 -08:00
epriestley
f048053c75 Move git commit metadata parsing into a LowLevelQuery
Summary: Ref T4195. I need to issue this command from the pre-commit hook to get commit bodies for hooks.

Test Plan: Ran `reparse.php --message --trace` and dumped the $ref, which looked correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7793
2013-12-18 17:48:06 -08:00
epriestley
a5dc9067af Provide convenience method addTextCrumb() to PhabricatorCrumbsView
Summary: We currently have a lot of calls to `addCrumb(id(new PhabricatorCrumbView())->...)` which can be expressed much more simply with a convenience method. Nearly all crumbs are only textual.

Test Plan:
  - This was mostly automated, then I cleaned up a few unusual sites manually.
  - Bunch of grep / randomly clicking around.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: hach-que, aran

Differential Revision: https://secure.phabricator.com/D7787
2013-12-18 17:47:34 -08:00
William R. Otte
dc43123494 Get the proper Subversion URI for hosted repositories.
Summary: The commit change parser worker was incorrectly getting the remote uri; calling getSubversionPathURI instead behaves properly for hosted repositories.  Fixes T4236.

Test Plan: Observed that my hosted subversion repositories finally had history in Diffusion.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4236

Differential Revision: https://secure.phabricator.com/D7794
2013-12-18 15:01:36 -08:00
epriestley
d90f44ef20 Support content pre-commit hooks in Mercurial
Summary: Ref T4195. Add Mercurial support to the content hook phase.

Test Plan:
Here are some `commit` push logs for a Mercurial repo:

{F90689}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7792
2013-12-18 14:19:18 -08:00
epriestley
5f4df0f3e3 Support "changed filename" and "file content" fields for commit content Herald rules
Summary: Ref T4195. Adds support for diff content rules.

Test Plan: Pushed SVN and Git changes through, saw them generate reasonable transcripts. Mercurial still isn't hooked up to this phase.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7791
2013-12-18 14:18:58 -08:00
epriestley
e115f11f80 Provide basic commit content hooks for Herald
Summary: Ref T4195. This doesn't provide any interesting fields yet (content, affected paths, commit message) but fires the hook correctly.

Test Plan: Added a blocking hook and saw it fire.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7789
2013-12-18 14:18:45 -08:00
epriestley
1ff3ef382d Give Herald rules a standard "Hnnn" object name
Summary: Allow Herald rules to be referred to with `H123`, etc., like other object types are. Herald rules now have proper PHIDs and an increasingly prominent role in triggering application actions. Although I suspect users will rarely use `H123` in Remarkup to mention rules, this can simplify some of the interfaces which relate objects across systems.

Test Plan: Looked at various interfaces and saw `H123` names. Mentioned `H123` in remarkup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7786
2013-12-18 12:00:18 -08:00
epriestley
5863f792a6 Remove many redundant implementations of canLoadNamedObject()
Summary:
These just got copy/pasted like crazy, the base class has the correct default implementation.

(I'm adding "H" for Herald Rules, which is why I was in this code.)

I also documented the existing prefixes at [[ Object Name Prefixes ]].

Test Plan: Verified base implementation. Typed some object names into the jump nav.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Differential Revision: https://secure.phabricator.com/D7785
2013-12-18 12:00:01 -08:00
epriestley
181bfffaa1 Truncate object fields in Herald transcripts
Summary:
A few users have hit cases where Herald transcripts of large commits exceed the MySQL packet limit, because one of the fields in the transcript is an enomrous textual diff.

There's no value in saving these huge amounts of data. Transcripts are useful for understanding the action of Herald rules, but can be reconstructed later. Instead of saving all of the data, limit each field to 4KB of data.

For strings, we just truncate at 4KB. For arrays, we truncate after 4KB of values.

Test Plan: Ran unit tests. Artificially decreased limit and ran transcripts, saw them truncate properly.

Reviewers: btrahan

Reviewed By: btrahan

CC: frgtn, aran

Differential Revision: https://secure.phabricator.com/D7783
2013-12-18 11:59:53 -08:00
epriestley
b0f7e7a6af Work around hg echoing warnings to stdout under --debug
Summary:
Ref T615. Ref T4237. With `--debug`, Mercurial will echo an "ignoring untrusted configuration option" warning **to stdout** if `.hgrc` has the wrong owner.

However, we need `--debug` to make `{parents}` usable, at least until the patches I got into the upstream are widely deployed. So after getting `--debug` output, strip off any leading warnings.

These warnings should always be in English, at least, since we set `LANG` explicitly.

Test Plan: Unit tests. @asherkin, maybe you can confirm this? I can't actually get the warning, but I think my `hg` in PATH is just a bit out of date.

Reviewers: asherkin, btrahan

Reviewed By: asherkin

CC: asherkin, aran

Maniphest Tasks: T615, T4237

Differential Revision: https://secure.phabricator.com/D7784
2013-12-17 18:04:01 -08:00
epriestley
3386920971 Add Herald support for blocking ref changes
Summary: Ref T4195. Allows users to write Herald rules which block ref changes. For example, you can write a rule like `alincoln can not create branches`, or `no one can push to the branch "frozen"`.

Test Plan:
This covers a lot of ground. I created and pushed a bunch of rules, then looked at transcripts, in general. Here are some bits in detail:

Here's a hook-based reject message:

  >>> orbital ~/repos/POEMS $ git push
  Counting objects: 5, done.
  Delta compression using up to 8 threads.
  Compressing objects: 100% (3/3), done.
  Writing objects: 100% (3/3), 274 bytes, done.
  Total 3 (delta 2), reused 0 (delta 0)
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: This commit was rejected by Herald pre-commit rule H24.
  remote: Rule: No Branches Called Blarp
  remote: Reason: "blarp" is a bad branch name
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp -> blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Here's a transcript, showing that all the field values populate sensibly:

{F90453}

Here's a rule:

{F90454}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7782
2013-12-17 15:23:55 -08:00
epriestley
baa756a027 Add rulePHID to HeraldEffect
Summary: Ref T4195. Herald rules gained PHIDs only recently, propagate them to HeraldEffect to make some of the hook stuff eaiser.

Test Plan: iiam

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7781
2013-12-17 15:23:45 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
epriestley
2216a5e6ef Add Subversion ref and content logs to pre-commit hooks
Summary: Ref T4195. SVN has no such thing as refs (I was thinking about writing a quasi-ref anyway like `HEAD: r23 -> r24`, but I'm not sure it would actually be useful). And content is very easy to build.

Test Plan: Pushed some stuff to SVN, got logs from it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7766
2013-12-17 11:11:52 -08:00
epriestley
11b8e57ae0 Remove "pretag" hook support in Mercurial
Summary: Ref T4195. This doesn't actually work like I thought it did: it only fires locally, when you run `hg tag`. Mercurial tags are also weird and basically don't make any sense and everyone should use bookmarks instead. We could implement some flavor of this eventually, but I'd like to see users request it first. They can implement their own with content-based hooks once those work, anyway.

Test Plan: This code didn't do anything.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7765
2013-12-17 09:18:48 -08:00
epriestley
74251b3636 Support bookmark hook operations in Mercurial
Summary: Ref T4195. Turns bookmark mutations in Mercurial into log objects.

Test Plan:
Pushed a pile of bookmarks and got logs:

{F89313}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7764
2013-12-17 08:34:30 -08:00
epriestley
6f3a99eb39 Generate ref updates in Mercurial hooks
Summary: Ref T4195. Mercurial is not my favorite VCS.

Test Plan:
Hit the split branches case:

  >>> orbital ~/repos/INIH $ hg push --force
  pushing to ssh://dweller@local.aphront.com/diffusion/INIH
  searching for changes
  remote: adding changesets
  remote: adding manifests
  remote: adding file changes
  remote: added 2 changesets with 2 changes to 1 files (+1 heads)
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push splits the head of branch 'default' into multiple heads: 802c785c3dd9, e73400db39b0. This is inadvisable and dangerous.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  remote: transaction abort!
  remote: rollback completed
  remote: abort: pretxnchangegroup.phabricator hook exited with status 1

Hit the divergent heads case:

  >>> orbital ~/repos/INIH $ hg push --force
  pushing to ssh://dweller@local.aphront.com/diffusion/INIH
  searching for changes
  remote: adding changesets
  remote: adding manifests
  remote: adding file changes
  remote: added 1 changesets with 1 changes to 1 files (+1 heads)
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push creates new, divergent heads for the branch 'default': f56af4232aa9. This is inadvisable and dangerous.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  remote: transaction abort!
  remote: rollback completed
  remote: abort: pretxnchangegroup.phabricator hook exited with status 1

Did a bunch of good/bad pushes:

{F89300}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7763
2013-12-17 08:34:15 -08:00
epriestley
2725586baf Restructure HookEngine to use PushLog records for all operations
Summary:
Ref T4195. This pulls the central logic of HookEngine up one level and makes all the git stuff genrate PushLogs.

In future diffs, everything will generate PushLogs and we can hand those off to Herald.

Test Plan:
Pushed a pile of valid/invalid stuff:

{F89256}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7761
2013-12-17 08:32:33 -08:00
epriestley
537f2eabee Make it harder to misconfigure phpmailer.smtp-protocol
Summary: Until we implement an "enum" type for config, make this a bit harder to get wrong. A user entered "TLS", but the correct value is "tls". The documentation is consistent about this, but the behavior is sitll surprsing.

Test Plan: eyeballed it

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7778
2013-12-16 12:30:56 -08:00
epriestley
00f192cebb Save ApplicationSearch queries generated from GET parameters in the request
Summary:
Fixes T4239. Currently, if you go to `/maniphest/?authors=alincoln`, operations dependent on the query key (like "Save Custom Query..." and "Export to Excel...") don't have a query key to work with. Make sure they have one.

Also remove a stray `phlog()`.

Test Plan: "Save Custom Query...", etc., now work on GET queries.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4239

Differential Revision: https://secure.phabricator.com/D7777
2013-12-16 12:30:51 -08:00
epriestley
0011068d7a Fix an issue in Harbormaster when a buildable has no repository
Summary: Not every revision belongs to a repository, so we might end up here with `$repo` still equal to `null`. Don't fatal if we do.

Test Plan: iiam

Reviewers: btrahan, hach-que, zeeg

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7771
2013-12-13 18:20:42 -08:00
Chris Colborne
707b39c5b5 Fix comitted typo in Diffusion
See: <https://github.com/facebook/phabricator/pull/468>

Reviewed by: epriestley
2013-12-13 06:53:13 -08:00
James Rhodes
8aee78b24d Remove patch preview from Phragment version controller
Summary: Patches can exceed the 30 second time out in most PHP installations.  This removes the patch preview from the version controller so that users can still see the information (although they may not be able to download the actual patch).

Test Plan: Viewed a version and saw that the patch didn't appear.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7767
2013-12-13 15:01:03 +11:00
James Rhodes
86ec4d6021 Implement policies in Phragment
Summary: This implements support for enforcing and setting policies in Phragment.

Test Plan: Set policies and ensured they were enforced successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7751
2013-12-13 14:42:12 +11:00
epriestley
7f45824984 Fix two issues with creating Conpherence threads via mail on some configurations
Summary:
Ref T4107. Two issues:

  - With strict MySQL settings, we try to insert `null` into the non-nullable `messageCount` field. Add an `initializeNew...` method.
  - If we don't create a new conpherence (for example, because the message body is empty), we fatal on `getPHID()` right now.

Also, make this stuff a little easier to test.

Test Plan: Used `mail_handler.php` to receive empty conpherence mail, and new-thread conpherence mail.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4107

Differential Revision: https://secure.phabricator.com/D7760
2013-12-12 10:59:28 -08:00
epriestley
d846f6508b Fix some repository URI handling issues in Git and Mercurial
Summary:
See <https://github.com/facebook/phabricator/issues/467>. @dctrwatson also ran into an issue where we were trying to `setPass()` a GitURI.

  - For Git and Mercurial, properly generate credential URIs where relevant.
  - Don't try to `setPass()` on Git-style URIs.

This isn't perfect but should clean things up a bit.

Test Plan: Added unit tests. Lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: dctrwatson, aran

Differential Revision: https://secure.phabricator.com/D7759
2013-12-12 09:45:27 -08:00
James Rhodes
1d9bf6f82b Fix Phortune so it allows users to create their accounts implicitly
Summary: This is a small fix for Phortune so that policies don't prevent the user accounts from being implicitly created when they first visit Phortune.

Test Plan: Visited Phortune and it worked.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7758
2013-12-12 11:19:03 +11:00
epriestley
61c934449d Fix fatal in Git hook when a --force push completely rewrites a ref
Summary: Fixes T4224. If you `git merge-base A B`, and they have //no// ancestor, the command exits with an error. Assume errors mean "no ancestry" and continue.

Test Plan: Completely rewrite a repository with a `--force` push.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4224

Differential Revision: https://secure.phabricator.com/D7756
2013-12-11 14:46:46 -08:00
epriestley
6b1ec35cf3 Allow Herald rules to check for revisions with no reviewers
Summary: Fixes T4225. Adds the NON_EXISTS condition to Herald for "Reviewers", and adds a few more conditions which have reasonable meanings.

Test Plan: Used test console to check a revision with reviewers, and another without reviewers. Both produced the expected results.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4225

Differential Revision: https://secure.phabricator.com/D7757
2013-12-11 14:46:39 -08:00
Matt Robenolt
c3d9c28382 Specify an ssh port for Diffusion when running against the grain
Summary: We run `git` on a different port than 22, so would like to reflect this change in the UI.

Test Plan: Set diffusion.ssh-port in settings, then make sure it's reflected on the Diffusion repository Clone URI.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, dctrwatson

Differential Revision: https://secure.phabricator.com/D7755
2013-12-11 12:11:13 -08:00
epriestley
052c83a613 Fix error detection for "ls-tree" output
Summary: Fixes T4223. The output of `ls-tree` is partially delimited by spaces
and partially delimited by `\t`. The code I added in D7744 to help debug the
issue in T4159 doesn't work properly for files with 7 or more bytes in their
filesize, because the internals use `%7s`.

Auditors: btrahan
2013-12-11 07:15:10 -08:00
James Rhodes
b8b7bf8ad9 Provide phragment.getstate and phragment.getpatch Conduit methods
Summary:
This provides a `phragment.getstate` and a `phragment.getpatch` Conduit method.

`phragment.getstate` - This returns the current state of the fragment and all of it's children.

`phragment.getpatch` - This accepts a base path and a mapping of paths to hashes.  The mapping is for the caller to specify the current state of the files it has.  This returns a list of patches that the caller needs to apply to it's files to get to the latest version.

Test Plan:
Ran the following script in a folder which had content matching a fragment and it's children:

```
#!/bin/bash

STATE=""
for i in $(find ./ -type f); do
    HASH=$(cat $i | sha1sum | awk '{ print $1 }')
    BASE=${i:2}
    STATE="$STATE,\"$BASE\":\"$HASH\""
done
STATE=${STATE:1}
STATE="{$STATE}"

echo '{"path":"tychaia3.zip","state":'$STATE'}' | arc --conduit-uri=http://phabricator.local/ call-conduit phragment.getpatch
```

and I got:

```
{"error":null,"errorMessage":null,"response":[]}
```

I updated one of the child fragments with a new file and ran the script again (patch has been omitted due to it's size):

```
{"error":null,"errorMessage":null,"response":[{"path":"Content\/TitleFont.xnb","hash_old":"4a927d7b90582e50cdd330de9f4b59b0cc5eb5c7","hash_new":"25867504642a3a403102274c68fbb9b430c1980f","patch":"..."}]}
```

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, staticshock

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7739
2013-12-11 11:19:23 +11:00
James Rhodes
270c8d27ab Implement "Wait for Previous Builds" build step
Summary: This adds a build step which will block a build from continuing if there are previous builds of the build plan still running.

Test Plan: Configured a build plan with a wait of 60 seconds and a "wait for previous builds", then started a build.  While that was still building, reconfigured the plan to have a wait time of 3 seconds, started it, and saw it move into the "Waiting" status.  When the 60 second build finished, both builds passed.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7745
2013-12-10 11:02:34 +11:00
Aviv Eyal
de969ab540 plug octocat icon to landing button
Test Plan: look at shiny button

Reviewers: epriestley, chad, #blessed_reviewers

Reviewed By: chad

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7749
2013-12-09 13:29:34 -08:00
epriestley
b8b7e583ad Show a queue utilization statistic in the Daemon console
Summary:
This came up recently in a discussion with @lifeihuang, and then tangentally with @hach-que. Make it easier for users to get a sense of whether they might need to add more daemons. Although we've improved the transparency of daemons, it's not easy for non-experts to determine at a glance how close to overflowing the queue is.

This number is approximate, but should be good enough for determining if your queue is more like 25% or 95% full.

If this goes over, say, 80%, it's probably a good idea to think about adding a couple of daemons. If it's under that, you should generally be fine.

Test Plan: {F88331}

Reviewers: btrahan, hach-que, lifeihuang

Reviewed By: btrahan

CC: hach-que, lifeihuang, aran, chad

Differential Revision: https://secure.phabricator.com/D7747
2013-12-09 13:22:22 -08:00
epriestley
52462a46c0 Raise a better error for malformed git ls-tree
Summary: Ref T4159. See T4159 for discussion.

Test Plan: Faked the error and generated a reasonable error message.

Reviewers: hach-que, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4159

Differential Revision: https://secure.phabricator.com/D7744
2013-12-09 13:22:13 -08:00
epriestley
5d27aeb240 Fix the exception when watching the first commit of a mercurial repo
Summary: Most checks were actually in place, but `ExecFuture` throws a `CommandException` which wasn't taken into account.

Test Plan: look at the first command and no longer saw an exception. Also, other commits worked as well.

Reviewers: richardvanvelzen

Reviewed By: richardvanvelzen

CC: krisbuist, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7730
2013-12-09 09:20:40 -08:00
James Rhodes
8fd256a1fd Implement step for publishing files as fragments to Phragment in Harbormaster
Summary: This adds a build step in Harbormaster for publishing file artifacts as fragments in Phragment.

Test Plan:
Created a build plan with the following steps:

  * Lease Host
  * Upload Artifact
  * Publish Fragment

Ran the build plan against a buildable and saw the fragment get created in Phragment.  Ran the plan again and saw the fragment get updated with a new version.  Modified the file that got uploaded and ran the plan again, checked the history of the fragment, and saw the differences represented as a Diff-Match-Patch patch.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7742
2013-12-09 09:34:09 +11:00
James Rhodes
8bb6e807f0 Implement snapshots in Phragment
Summary:
Ref T4212.  This implements snapshots in Phragment, which allows you to take a snapshot of a fragment at a given point in time, and download a ZIP of the snapshot as it was in this state.

There's also functionality for deleting and promoting snapshots.  You can promote a snapshot to either the latest version or any other snapshot of the fragment.

Test Plan: Clicked around, took some snapshots, promoted them to different points and deleted snapshots.  Also downloaded ZIPs of the snapshots and saw the right versions coming through for all the files downloaded.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205, T4212

Differential Revision: https://secure.phabricator.com/D7741
2013-12-09 08:24:50 +11:00
James Rhodes
e165787725 Implement "Revert to Version" functionality in Phragment
Summary:
This functionality allows users to revert a fragment to a previous version from the history page.

Reverting a version actually creates a new version pointing at the same file as the version being "reverted" to.  In this sense it acts pretty much like Git and other distributed VCS where once you have published a commit the only way to undo your changes is to create a new commit that reverts those changes.

Test Plan: Reverted a fragment to a version before it was deleted, then reverted it to when it was deleted and saw the new versions have the correct file PHIDs (including null for the deletion).

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7738
2013-12-09 07:57:53 +11:00
James Rhodes
dabc7ea28d Render deleted files in Phragment as disabled
Summary: This updates Phragment so that fragments that are currently considered deleted have a disabled status and have an additional attribute 'Deleted'.  It also places this effect on versions (in the history controller) that actually involve deleting the file.

Test Plan: Viewed deleted fragments and versions.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7737
2013-12-09 07:52:24 +11:00
James Rhodes
67b37a5c1d Fix deletion detection when updating based on ZIP
Summary: When the code to update based on ZIP went to look up the child fragments, it explicitly used the paths provided in the ZIP.  This meant that we could never detect omissions because there'd never be a scenario where a child fragment would return but not exist in the ZIP.  To fix this, the query should be using `withLeadingPath` instead of `withPaths`.

Test Plan: Uploaded a ZIP that omitted a file and saw the `deleteFile` get called (by placing debugging output in the code).

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7736
2013-12-09 07:48:54 +11:00
James Rhodes
acd5d5ae4a Correctly handle downloading ZIP with empty directories in Phragment
Summary: This logic causes an exception because getPHID() is called on a fragment that has no latest version.  This fixes the code so that in this scenario, it returns an empty array (with no path to be added to the ZIP).

Test Plan: Downloaded the ZIP successfully after the patch was applied.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7735
2013-12-08 12:15:13 +11:00
James Rhodes
c48cfb31bc Implement viewing versions and downloading patches in Phragment
Summary:
This adds support for viewing individual versions on a fragment as well as comparing versions and downloading diff_match_patch-based patches.

It does not use the side-by-side diff format as while it works for small changes, it quickly becomes impossible to distingush what changes have been made due to the diff_match_patch format.

Test Plan: Clicked on versions and downloaded patches.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7734
2013-12-08 12:14:34 +11:00
James Rhodes
5728ef7836 Implicitly detect directories when updating a fragment from a ZIP
Summary: This fixes the update-from-ZIP functionality so that it will automatically detect directories in the ZIP that do not have explicit entries.  Some ZIP programs do not create directory entries explicitly, so if we fail to do this then there's no way for users to access the sub-fragments (even though they exist, there is no directory fragment to click through).

Test Plan: Created and updated fragments from a ZIP that had implicit directories in it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, staticshock

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7733
2013-12-08 12:10:00 +11:00
epriestley
7a5c3cc854 Fix undefined variables in Subversion
Summary: These variables won't be in scope in Subversion.

See: <https://secure.phabricator.com/rP2ff5541fc59c4be7abd733a39e12db8358004f7a>

Auditors: btrahan
2013-12-07 11:12:38 -08:00
epriestley
99ad978e90 Add UI for choosing header color
Summary: See D7731. Fixes T4194.

Test Plan: {F88020}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran, mbishopim3

Maniphest Tasks: T4194

Differential Revision: https://secure.phabricator.com/D7740
2013-12-07 10:46:09 -08:00
James Rhodes
25e7b7d53c Implement support for creating and updating fragments from ZIPs
Summary:
This implements support for creating and updating fragments from ZIP files.  It allows you to upload a ZIP via the Files application, create a fragment from it, and have it recursively imported into Phragment.  Updating that folder with another ZIP will recursively create, update and delete files as appropriate.

The logic for creating and updating fragments from files has also been centralized into the PhragmentFragment class.  Directories are also now supported; a directory fragment is simply a fragment that has no patches; thus a directory fragment can be converted to a file fragment by uploading a first patch for it.

Test Plan: Uploaded ZIP files through the interface and saw all of the fragments get created and updated as expected.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7729
2013-12-07 13:37:18 +11:00
James Rhodes
ccd4ae5638 Implement "Download ZIP" controller for Phragment
Summary: Depends on D7727.  This adds support for downloading a fragment and all it's children as a ZIP file.  Fragments that have children automatically become directories in the ZIP file.

Test Plan: Downloaded a fragment as a ZIP and was able to extract the contents successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7728
2013-12-07 13:25:22 +11:00
James Rhodes
f7f5a5dd34 Implement update and history controllers in Phragment
Summary: Depends on D7726.  This adds a history controller (for viewing a list of patches associated with a fragment) and an update controller, for creating a new patch of a fragment.

Test Plan: Updated and viewed history of fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7727
2013-12-07 12:56:55 +11:00
James Rhodes
4c143ad3b2 Phragment v0
Summary: Ref T4205.  This is an initial implementation of Phragment.  You can create and browse fragments in the system (but you can't yet view a fragment's patches / history).

Test Plan: Clicked around and created fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7726
2013-12-07 12:43:49 +11:00
epriestley
f69793184e Fix over-matching of quoted text for message bodies beginning with "On..."
Summary:
A user sent a message to Phabricator which looked like:

  On blah blah blah ?

  On <date>, <user> wrote:
  > blah blah blah

The current algorithm is too aggressive and thinks lines 1-3 are //all// the "On ... wrote:" string. Instead, patch only the most recent "On".

Test Plan: Added a failing test and made it pass.

Reviewers: btrahan, zeeg

Reviewed By: zeeg

CC: aran

Differential Revision: https://secure.phabricator.com/D7732
2013-12-06 15:47:37 -08:00
James Rhodes
79d153b85d Implement explicit build step ordering in Harbormaster
Summary: This implements support for explicitly marking the sequence of build steps.  Users can now drag and re-order build steps in plans, and artifact dependencies are re-calculated so that if you move "Run Command" before "Lease Host", the "Run Command" step has it's artifact setting cleared and thus the step becomes invalid.

Test Plan: Re-ordered build steps and observed dependencies being correctly recalculated.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7715
2013-12-06 14:12:15 +11:00
James Rhodes
dd01535ed6 Implement "Upload Artifact" build step
Summary: This implements a build step for uploading an artifact from a build machine to Phabricator.  It uses SFTP so that it will work on both UNIX and Windows build machines.

Test Plan: Ran an "Upload Artifact" build against a Windows machine (with FreeSSHD installed).  The artifact uploaded to Phabricator, appeared on the build view and the file contents could be viewed from Phabricator.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7582
2013-12-06 14:11:05 +11:00
epriestley
a1f3233481 Don't show client IP in push logs unless viewer can edit the repository
Summary: This locks push logs down a little bit and makes them slightly more administrative. Primarily, don't show IPs to googlebot, etc.

Test Plan: Viewed push logs as edit and non-edit users.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7722
2013-12-05 17:01:07 -08:00
epriestley
faaaff0b6f Fix an error in the PolicyFilter algorithm
Summary:
`PhabricatorPolicyFilter` has a bug right now where it lets through objects incorrectly if:

  - the query requests two or more policies;
  - the object satisfies at least one of those policies; and
  - policy exceptions are not enabled.

This would be bad, but there's only one call in the codebase which satisfies all of these conditions, in the Maniphest batch editor. And it's moot anyway because edit operations get another policy check slightly later. So there is no policy/security impact from this flaw.

(The next diff relies on this behavior, which is how I caught it.)

Test Plan:
  - Added a failing unit test and made it pass.
  - Grepped the codebase for `requireCapabilities()` and verified that there is no security impact. Basically, 99% of callsites use `executeOne()`, which throws anyway and moots the filtering.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7721
2013-12-05 17:00:53 -08:00
epriestley
5ca84589bd Add an SSH access log
Summary: Ref T4107. Ref T4189. This implements an SSH access log, similar to the HTTP access log.

Test Plan:
  [Thu, 05 Dec 2013 13:45:41 -0800]	77841	orbital	::1	dweller	epriestley	epriestley	git-receive-pack	/diffusion/POEMS/	0	324765	402	232
  [Thu, 05 Dec 2013 13:45:48 -0800]	77860	orbital	::1	dweller	epriestley	epriestley	git-receive-pack	/diffusion/POEMS/	0	325634	402	232

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4107, T4189

Differential Revision: https://secure.phabricator.com/D7719
2013-12-05 17:00:48 -08:00
epriestley
39b384041f Add "r <name>" to jump nav to locate repositories by name
Summary: User request.

Test Plan: Searched for `r ph`, `r poetry`.

Reviewers: btrahan, bigo

Reviewed By: bigo

CC: aran

Differential Revision: https://secure.phabricator.com/D7720
2013-12-05 14:26:38 -08:00
epriestley
2ff5541fc5 Record new commits in the push log
Summary:
Ref T4195. Like the previous diffs, these both create a useful log and give us an object to hand off to Herald.

Surface this information in Diffusion, too, and clean things up a little bit.

Test Plan: {F87565}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7718
2013-12-05 11:59:41 -08:00
epriestley
3f50460149 Allow repository push logs to be filtered by pusher and repository
Summary: Ref T4195. Add UI options to filter push logs by pusher and repository. Add a link from the repository view page to the push logs.

Test Plan: Viewed a hosted repository, clicked logs link, saw logs. Filtered lgos by repo/pusher.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7713
2013-12-05 11:59:33 -08:00
epriestley
e28b848ab2 Store pusher remote address and push protocol in PushLog
Summary: Ref T4195. Stores remote address and protocol in the logs, where possible.

Test Plan: Pushed some stuff, looked at the log, saw data.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7711
2013-12-05 11:59:22 -08:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
epriestley
c6fd969416 Fix an edge case when trying to pull duplicate refs via Doorkeeper
Summary:
Report from Asana. In some unclear circumstances, we my attempt to resolve duplicate refs which currently ends up hitting a duplicate key error.

Instead, reference the same external object if we happen to be handed duplicate refs.

Test Plan:
Used this script to reproduce the issue. Applied the fix; issue went away:

  #!/usr/bin/env php
  <?php

  require_once 'scripts/__init_script__.php';

  $args = new PhutilArgumentParser($argv);
  $args->parseStandardArguments();

  $ref = id(new DoorkeeperObjectRef())
    ->setApplicationType(DoorkeeperBridgeAsana::APPTYPE_ASANA)
    ->setApplicationDomain(DoorkeeperBridgeAsana::APPDOMAIN_ASANA)
    ->setObjectType(DoorkeeperBridgeAsana::OBJTYPE_TASK)
    ->setObjectID(7253737283629); // Use a new task ID which we've never pulled.

  $refs = array(clone $ref, clone $ref);

  $asana_user = id(new PhabricatorPeopleQuery())
    ->setViewer(PhabricatorUser::getOmnipotentUser())
    ->withUsernames(array('asana'))
    ->executeOne();

  $resolved_refs = id(new DoorkeeperImportEngine())
    ->setViewer($asana_user)
    ->setRefs($refs)
    ->execute();

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7709
2013-12-05 11:45:59 -08:00
Chad Little
00c6e60c27 Clean up logged out prompt on TransactionComments
Summary: We were getting a weird double box here, missed it my first pass

Test Plan: Review logged in Maniphest and Paste, as well as logged out versions. Test Login flow.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7716
2013-12-05 08:44:43 -08:00
Chad Little
b925fc884a Change closed tasks to view as disabled
Summary: This cleans up the UI of closed tasks in Maniphest task view, removes the Foot and sets view to disabled.

Test Plan: Searched for all tasks

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7712
2013-12-04 20:45:23 -08:00
James Rhodes
48e80f0541 Remove -t -t from Drydock SSH interface
Summary: These arguments prevent stderr from being routed correctly for Linux hosts and break Windows entirely.  Removing them fixes the issue.

Test Plan: Removed those options and both Linux and Windows hosts had their output fed back into Harbormaster correctly.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1029

Differential Revision: https://secure.phabricator.com/D7710
2013-12-05 14:59:46 +11:00
James Rhodes
a8b27130cb Make "Edit Build Plan" more resiliant so old build configurations can be deleted
Summary: Currently the "Edit Build Plan" page crashes if there are any build steps with invalid implementations (because the implementation class has been removed or renamed).  This updates the Edit Build Plan page so that steps with invalid implementations can be deleted.

Test Plan: Looked at a build plan with invalid configurations and deleted it's steps.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7708
2013-12-05 14:20:40 +11:00
James Rhodes
5c02113bf9 Migrate "Run Command" to use Drydock hosts
Summary: This migrates the "Run Remote Command" build step over to use Drydock hosts and Harbormaster artifacts.

Test Plan:
Created a build plan with a "Lease Host" step and a "Run Command" step.  Configured the "Run Command" step to use the artifact from the "Lease Host" step.

Saw the results:

{F87377}

{F87378}

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049, T4111

Differential Revision: https://secure.phabricator.com/D7707
2013-12-05 14:06:22 +11:00
James Rhodes
d8d1173f52 Implement support for leasing from Drydock hosts in Harbormaster
Summary:
This adds LeaseHostBuildStepImplementation for getting leases on hosts in Drydock via Harbormaster.  It stores the resulting lease in an artifact.

There is also a few bug fixes as well.

Test Plan: Created a build plan with a "Lease Host" build step.  Ran the build plan and saw the build pass and the artifact in the database.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049, T4111

Differential Revision: https://secure.phabricator.com/D7706
2013-12-05 12:46:23 +11:00
James Rhodes
53250d84df Introduce HarbormasterBuildTarget to snapshot build steps through a build
Summary: This implements build targets as outlined in D7582.  Build targets represent an instance of a build step particular to the build.  Logs and artifacts have been adjusted to attach to build targets instead of build / build step pairs.

Test Plan: Ran builds and clicked around the interface.  Everything seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7703
2013-12-05 12:01:12 +11:00
epriestley
2aad7289fd Support "ssh-dss" keys
Summary: Ref T4151. These are a (common) variant of "ssh-dsa" keys (which are somewhat theoretical, but show up on Google).

Test Plan: syntax

Reviewers: btrahan, dctrwatson, phpcodemonkey

Reviewed By: phpcodemonkey

CC: aran

Maniphest Tasks: T4151

Differential Revision: https://secure.phabricator.com/D7704
2013-12-04 16:53:16 -08:00
epriestley
e77d5012be Fix two issues with shell/config scripts for hosted repositories
Summary: Ref T4151. `-ne` is numeric in some/most/all shells; `exec --` apparently doens't always work.

Test Plan: Will make @zeeg test.

Reviewers: btrahan, zeeg

Reviewed By: zeeg

CC: zeeg, aran

Maniphest Tasks: T4151

Differential Revision: https://secure.phabricator.com/D7702
2013-12-04 16:45:54 -08:00
epriestley
ff4e965c90 Don't try to download diffs-of-diffs
Summary:
Ref T1715. When the user clicks "Download Raw Diff" in Differential, we try to build a diff of exactly what they're seeing. However:

  - This doesn't work if any of the changes have multiple hunks, and fixing it seems hard.
  - I suspect this diff is never actually useful anyway? And probably kind of confusing in the best case. You can't really apply it to anyhting, since you'd have to apply another diff first.

Instead, just build the right-side diff, which should align well with user expectation and doesn't suffer from the multi-hunk bug.

Some day, we could maybe add some of the fancy options in T1715.

See: <https://github.com/facebook/phabricator/issues/461>

Test Plan: Downloaded a multi-hunk diff, got the original back and applied it cleanly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1715

Differential Revision: https://secure.phabricator.com/D7694
2013-12-04 14:39:07 -08:00
James Rhodes
b111bc039d Support text-based private key credentials in DrydockSSHCommandInterface
Summary: This updates DrydockSSHCommandInterface to correctly hold open the private key credentials for the life of the interface so that remote commands will execute correctly with a text-based private key.

Test Plan: Created a text-based private key, created a resource based on it and leased against it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111

Differential Revision: https://secure.phabricator.com/D7700
2013-12-05 09:19:32 +11:00
James Rhodes
9c6f6043f0 Update preallocated hosts to use Passphrase credentials
Summary: Depends on D7695.  This updates preallocated hosts to use Passphrase credentials.  Due to the way SSH private key text credentials work (the TempFile disappears before SSH commands can be executed), this only supports file-based private keys at the moment.

Test Plan:
Created a Passphrase credential for a file-based SSH key.  Allocated a resource with:

```
bin/drydock create-resource --blueprint 1 --name "My Linux Host" --attributes platform=linux,host=localhost,port=22,path=/var/drydock,credential=2
```

and successfully leased it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7697
2013-12-05 08:17:23 +11:00
James Rhodes
1f53017f1f Validate resource attributes for preallocated hosts before executing leases
Summary: This prevents issues when the user hasn't provided the appropriate attributes for a preallocated host.

Test Plan: Attempted to lease against a resource with omitted attributes, got an exception thrown before any SSH commands occurred.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7695
2013-12-05 08:16:33 +11:00
epriestley
1965bf9e52 Show "Tnnn" crumb on Edit Task screen
Summary: Fixes T4198. We don't currently show "(Maniphest) > T123 > Edit" on the edit screen, which is inconsistent. Add the "T123" crumb.

Test Plan: {F87177}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T4198

Differential Revision: https://secure.phabricator.com/D7699
2013-12-04 08:07:11 -08:00
Richard van Velzen
16a7eaa700 Fix wrong link to "Create Rule" in Herald mobile view
Summary: The link pointed to `create/`, which gives as `404`.

Test Plan: clicked the link. It worked.

Reviewers: epriestley, #blessed_reviewers, chad

Reviewed By: chad

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7698
2013-12-04 04:57:34 -08:00
Chad Little
b2debb14c7 Mobile Notifications
Summary: Touch up /notifications/ for desktop and mobile

Test Plan: Tested read and unread notifications on mobile and desktop

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7671
2013-12-03 11:58:10 -08:00
epriestley
addd0bbf3b Fix a constant in the hook to reduce the number of dragons
Test Plan: Will push.

Auditors: btrahan
2013-12-03 10:31:29 -08:00
epriestley
d2e9aee16d Reject dangerous changes in Git repositories by default
Summary: Ref T4189. This adds a per-repository "dangerous changes" flag, which defaults to off. This flag must be enabled to do non-appending branch mutation (delete branches / rewrite history).

Test Plan:
With flag on and off, performed various safe and dangerous pushes.

  >>> orbital ~/repos/POEMS $ git push origin :blarp
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push deletes the branch 'blarp'.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad, richardvanvelzen

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7689
2013-12-03 10:28:39 -08:00
epriestley
632e1ceda6 Do the heavy lifting for git commit hooks
Summary:
Ref T4189. This doesn't add any rules yet, but does all the heavy lifting to figure out what's changed and put it in a consuamble (if somewhat ad-hoc) datastructure, which lists all the ref and tag modifications and all the new commits in a consistent way.

From here, it should be fairly straightforward to add top-level rules (e.g., ff pushes only).

Test Plan: Output is huge, see comments.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7687
2013-12-03 10:27:45 -08:00
Chad Little
9942c2a39e Add Action Buttons to ObjectHeaders
Summary: This adds the ability to float action buttons inside ObjectHeaderView.

Test Plan: Tested a UI Example on desktop and mobile. Will test on Notifications next.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7684
2013-12-03 08:34:04 -08:00
epriestley
50b5a0c8b7 Add explicit "/" when installing hooks
Summary: There's no guarantee that the local path has a trailing "/". We
should probably guarantee that at some point, but just add one
unconditionally for now.

Auditors: btrahan
2013-12-02 16:18:02 -08:00
James Rhodes
ba16df0fed Restructure Drydock so that blueprints are instances in the DB
Summary:
//(this diff used to be about applying policies to blueprints)//

This restructures Drydock so that blueprints are instances in the DB, with an associated implementation class.  Thus resources now have a `blueprintPHID` instead of `blueprintClass` and DrydockBlueprint becomes a DAO.  The old DrydockBlueprint is renamed to DrydockBlueprintImplementation, and the DrydockBlueprint DAO has a `blueprintClass` column on it.

This now just implements CAN_VIEW and CAN_EDIT policies for blueprints, although they are probably not enforced in all of the places they could be.

Test Plan: Used the `create-resource` and `lease` commands.  Closed resources and leases in the UI.  Clicked around the new and old lists to make sure everything is still working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T2015

Differential Revision: https://secure.phabricator.com/D7638
2013-12-03 11:09:07 +11:00
epriestley
f93c6985ad Support Mercurial pretxnchangegroup hooks
Summary: Ref T4189. Fixes T2066. Mercurial has a //lot// of hooks so I'm not 100% sure this is all we need to install (we may need separate hooks for tags/bookmarks) but it should cover most of what we're after at least.

Test Plan:
  - `bin/repository pull`'d a Mercurial repo and got a hook install.
  - Pushed to a Mercurial repository over SSH and HTTP, with good/bad hooks. Saw hooks fire.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2066, T4189

Differential Revision: https://secure.phabricator.com/D7685
2013-12-02 15:46:03 -08:00
epriestley
017d6ccd07 Support SVN pre-commit hoooks
Summary:
Ref T4189. This adds SVN support, which was a little more messy than I though. Principally, we can not use `PHABRICATOR_USER` for Subversion, because it strips away the entire environment for "security reasons".

Instead, use `--tunnel-user` plus `svnlook author` to figure out the author.

Also fix "ssh://" clone URIs, which needs to be "svn+ssh://".

Test Plan:
  - Made SVN commits through the hook.
  - Made Git commits, too, to make sure I didn't break anything.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7683
2013-12-02 15:45:55 -08:00
epriestley
618b5cbbc4 Install pre-commit hooks in Git repositories
Summary:
Ref T4189. T4189 describes most of the intent here:

  - When updating hosted repositories, sync a pre-commit hook into them instead of doing a `git fetch`.
  - The hook calls into Phabricator. The acting Phabricator user is sent via PHABRICATOR_USER in the environment. The active repository is sent via CLI.
  - The hook doesn't do anything useful yet; it just veifies basic parameters, does a little parsing, and exits 0 to allow the commit.

Test Plan:
  - Performed Git pushes and pulls over SSH and HTTP.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7682
2013-12-02 15:45:36 -08:00
John Watson
49f3ff0e08 Attach TaskPHIDs to commits in diffusion.getcommits
Summary: Uses edge query to attach TaskPHIDs to commit objects

Test Plan: Use conduit to getcommits with attached tasks

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7668
2013-11-27 14:36:07 -08:00
James Rhodes
d571507651 Update a few things in Drydock for policies
Summary: DrydockResource has been updated to be policy-aware (although there are no policy columns).

Test Plan: Clicked around in Drydock, viewed resources and leases, everything still seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3605, T4111

Differential Revision: https://secure.phabricator.com/D7595
2013-11-28 08:32:51 +11:00
Brecht Van Lommel
bf2f599abb Fix Maniphest Next button not working when default query has 100+ results.
Summary: If there is no /query in the URL, the default query would be lost when clicking Next, causing the search form to be shown on the second page. This is not so likely to happen on a standard Phabricator installation because the default query is Assigned, and few people will have 100+ tasks assigned.

Test Plan:
* Go to /maniphest/query/edit/
* Move Open Tasks to the top
* Go to /maniphest/
* Click Next on the bottom right
* See only tasks that are actually open

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7667
2013-11-27 12:18:11 -08:00
Brecht Van Lommel
8f48968c6e Fix missing button to expand project action list on mobile.
Test Plan: Go to a project page, make browser window narrow, click to expand action list.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7666
2013-11-27 12:03:31 -08:00
epriestley
e1bd868142 Minor, fix the PagedForm UIExample
Summary: A couple of features got added when this was used in Diffusion, but
didn't get backported to the UIExample.

See: <https://github.com/facebook/phabricator/issues/454>

Auditors: btrahan
2013-11-26 08:57:17 -08:00
epriestley
209861500f Use "en_US.UTF-8", not "C", as the LANG setting
Summary: `LANG=C` is smooshing UTF-8 in some cases. See IRC.

Test Plan: User confirmed this works.

Reviewers: btrahan, asherkin

Reviewed By: asherkin

CC: aran

Differential Revision: https://secure.phabricator.com/D7659
2013-11-26 08:50:36 -08:00
epriestley
dd4a4a8c45 Fix an error in PassphrasePasswordKey construction
Summary: This uses the wrong class name, and builds the wrong object type.

See: <51fb1ca16d (commitcomment-4703209)>

Auditors: btrahan
2013-11-26 05:56:36 -08:00
Vasyl Vavrychuk
e67302a4c9 Display closed documents with crossed text in search result.
Summary:
By default in search application document status field is "Open and Closed Documents".
Often searching with this default status I get confused that open and closed items in
search result are not distinguished.

Test Plan: Search and see open/closed issues distinguished.

Reviewers: chad, #blessed_reviewers, epriestley

Reviewed By: epriestley

CC: epriestley, aran, Korvin

Differential Revision: https://secure.phabricator.com/D7626
2013-11-25 20:05:36 -08:00
epriestley
6985c71117 Turn the macro selector into a tokenizer
Summary: Ref T3562. Here's the 10 minute "value" option.

Test Plan: See screenshots.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T3562

Differential Revision: https://secure.phabricator.com/D7658
2013-11-25 19:22:06 -08:00
epriestley
325c8853c9 Add a configuration option to put Asana tasks into Asana projects
Summary: Request from Asana. Adds an option for adding tasks to projects.

Test Plan: Used `bin/feed republish` to create and update Asana tasks with projects configured. Saw them end up in the right projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7655
2013-11-25 17:40:12 -08:00
epriestley
b435c03eca Add a config flag to treat "Accepted" revisions as "Closed"
Summary: See D7653. This is exclusively for Asana, who uses Differential for a post-commit, Audit-like workflow but has a small set of requirements for it to be a good fit (just this) and a large set of requirements for Diffusion/Audit to be a good fit.

Test Plan: Set the flag, verified "Accepted" revisions are no longer on the dashboard.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7654
2013-11-25 17:40:01 -08:00
epriestley
1da691113a Normalize the definition of "closed" revision statuses
Summary:
Currently, "Closed" and "Abandoned" are treated as "closed". I want to add a flag which treats "Accepted" as "Closed", too, for Asana and other companies who use an Asana-like workflow.

The background here is that their workflow is a bit weird. They basically do audits, but have a lot of things which Diffusion doesn't do well right now. This one change makes Differential fit their workflow fairly well, even though it's an audit workflow.

To prepare for this, normalize the definition of "closed" better. We have a few callsites which explicitly check for "ABANDONED || CLOSED", and normalizing this is cleaner anyway.

Also delete the very old COMMITTED status, which has been obsolete for over a year.

Test Plan: Browsed around most/all of the affected interfaces.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7653
2013-11-25 17:39:24 -08:00
epriestley
e4920cdf86 Provide an LDAPS example in LDAP auth
Summary: Fixes T4148. LDAPS works with "ldaps://", it just isn't documented or clear.

Test Plan: {F84893}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4148

Differential Revision: https://secure.phabricator.com/D7652
2013-11-25 14:55:44 -08:00
epriestley
e944cf8ff4 Propagate "edit policy" and "view policy" when creating a new template task
Summary:
Fixes T4158. Two possible refinements:

  - Maybe we should make all of these things respect `ManiphestCapabilityEditAssign::CAPABILITY`, etc. I think it's reasonable either way, and this is probably more intuitive and useful for most cases.
  - Maybe we should check that you can see the policies before copying them. Again, this is sort of reasonable either way.

Test Plan: Created a new task from a template, saw that it inherited policies.

Reviewers: btrahan, hach-que

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T4158

Differential Revision: https://secure.phabricator.com/D7649
2013-11-25 14:55:06 -08:00
epriestley
8b19465a9d Render linked JIRA issues with a Doorkeeper tag
Summary: Fixes T3687. Instead of rendering "JIRA Issues" in Differential using plain links, render them using Doorkeeper tags so they get the nice "enhance with object name" effect.

Test Plan: {F84886}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3687

Differential Revision: https://secure.phabricator.com/D7648
2013-11-25 14:54:35 -08:00
Chad Little
3c21375930 Clean up external accounts page
Summary: Touched up the layout, css of this page

Test Plan: Viewed linked and linkable accounts. Tested mobile layout

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7644
2013-11-24 19:14:16 -08:00
Chad Little
b8b80e3896 Update 'Add SSH Key' page
Summary: Uses an ObjectBoxView

Test Plan: added an ssh key

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4150

Differential Revision: https://secure.phabricator.com/D7645
2013-11-24 19:13:57 -08:00
epriestley
de73029e99 Propagate PHABRICATOR_ENV into VCS commands explicitly
Summary: Fixes T4155. See discussion in T4155.

Test Plan: @mbishopim3 confirmed this fixes his issue.

Reviewers: btrahan, chad

Reviewed By: chad

CC: mbishopim3, aran

Maniphest Tasks: T4155

Differential Revision: https://secure.phabricator.com/D7646
2013-11-24 18:11:56 -08:00
Brecht Van Lommel
2a65b3020e Fix error creating repository from file:/// location, due to uninitialized variable.
Summary: This was broken in rP51fb1ca16d7f.

Test Plan: Imported a repository with file:/// location, it worked.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7636
2013-11-23 06:30:52 -08:00
epriestley
a7d39f092b Fix an issue with editing policies in Phlux.
Mostly, this is testing the auto-mirroring stuff.

Auditors: btrahan
2013-11-22 16:55:57 -08:00
epriestley
c978fe5240 Fix an issue where SSH protocols would check legacy credentials
Missed this while grepping. We should use SSH purely based on protocol.

Auditors: btrahan
2013-11-22 16:31:14 -08:00