1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-11 01:12:41 +01:00
Commit graph

1366 commits

Author SHA1 Message Date
epriestley
d667b12206 Provide a standalone query for resolution of commit author/committer into Phabricator users
Summary:
Ref T4195. To implement the "Author" and "Committer" rules, I need to resolve author/committer strings into Phabricator users.

The code to do this is currently buried in the daemons. Extract it into a standalone query.

I also added `bin/repository lookup-users <commit>` to test this query, both to improve confidence I'm getting this right and to provide a diagnostic command for users, since there's occasionally some confusion over how author/committer strings resolve into valid users.

Test Plan:
I tested this using `bin/repository lookup-users` and `reparse.php --message` on Git, Mercurial and SVN commits. Here's the `lookup-users` output:

  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIS3
  Examining commit rINIS3...
  Raw author string: epriestley
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rPOEMS165b6c54f487c8
  Examining commit rPOEMS165b6c54f487...
  Raw author string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIH6d24c1aee7741e
  Examining commit rINIH6d24c1aee774...
  Raw author string: epriestley <hg@yghe.net>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $

The `reparse.php` output was similar, and all VCSes resolved authors correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1731, T4195

Differential Revision: https://secure.phabricator.com/D7801
2013-12-19 11:05:17 -08:00
epriestley
f750d5f8dc Provide a low-level SVN commit query, and merge the VCS query types
Summary: Ref T4195. Even though we use `svnlook` in the hook itself, I need this query elsewhere, so provide it and merge the classes into one which does the right thing.

Test Plan:
  - Used `reparse.php` to reparse messages for Git, SVN and Mercurial commits, using `var_dump()` to examine the commit refs for sanity.
  - Used `reparse.php` to reparse changes for an SVN commit.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7800
2013-12-19 11:05:06 -08:00
epriestley
92bc76aae0 Move mercurial commit metadata parsing into a LowLevel query
Summary: Ref T4195. Same as D7793, but for mercurial. (As usual, SVN needs some goofy nonsense instead, so the next diff will just make this field work.)

Test Plan: Ran `reparse.php` on Git and Mercurial commits, var_dump'd the output and it looked correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7795
2013-12-18 17:48:19 -08:00
epriestley
f048053c75 Move git commit metadata parsing into a LowLevelQuery
Summary: Ref T4195. I need to issue this command from the pre-commit hook to get commit bodies for hooks.

Test Plan: Ran `reparse.php --message --trace` and dumped the $ref, which looked correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7793
2013-12-18 17:48:06 -08:00
epriestley
e115f11f80 Provide basic commit content hooks for Herald
Summary: Ref T4195. This doesn't provide any interesting fields yet (content, affected paths, commit message) but fires the hook correctly.

Test Plan: Added a blocking hook and saw it fire.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7789
2013-12-18 14:18:45 -08:00
epriestley
1ff3ef382d Give Herald rules a standard "Hnnn" object name
Summary: Allow Herald rules to be referred to with `H123`, etc., like other object types are. Herald rules now have proper PHIDs and an increasingly prominent role in triggering application actions. Although I suspect users will rarely use `H123` in Remarkup to mention rules, this can simplify some of the interfaces which relate objects across systems.

Test Plan: Looked at various interfaces and saw `H123` names. Mentioned `H123` in remarkup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7786
2013-12-18 12:00:18 -08:00
epriestley
181bfffaa1 Truncate object fields in Herald transcripts
Summary:
A few users have hit cases where Herald transcripts of large commits exceed the MySQL packet limit, because one of the fields in the transcript is an enomrous textual diff.

There's no value in saving these huge amounts of data. Transcripts are useful for understanding the action of Herald rules, but can be reconstructed later. Instead of saving all of the data, limit each field to 4KB of data.

For strings, we just truncate at 4KB. For arrays, we truncate after 4KB of values.

Test Plan: Ran unit tests. Artificially decreased limit and ran transcripts, saw them truncate properly.

Reviewers: btrahan

Reviewed By: btrahan

CC: frgtn, aran

Differential Revision: https://secure.phabricator.com/D7783
2013-12-18 11:59:53 -08:00
epriestley
3386920971 Add Herald support for blocking ref changes
Summary: Ref T4195. Allows users to write Herald rules which block ref changes. For example, you can write a rule like `alincoln can not create branches`, or `no one can push to the branch "frozen"`.

Test Plan:
This covers a lot of ground. I created and pushed a bunch of rules, then looked at transcripts, in general. Here are some bits in detail:

Here's a hook-based reject message:

  >>> orbital ~/repos/POEMS $ git push
  Counting objects: 5, done.
  Delta compression using up to 8 threads.
  Compressing objects: 100% (3/3), done.
  Writing objects: 100% (3/3), 274 bytes, done.
  Total 3 (delta 2), reused 0 (delta 0)
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: This commit was rejected by Herald pre-commit rule H24.
  remote: Rule: No Branches Called Blarp
  remote: Reason: "blarp" is a bad branch name
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp -> blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Here's a transcript, showing that all the field values populate sensibly:

{F90453}

Here's a rule:

{F90454}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7782
2013-12-17 15:23:55 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
James Rhodes
86ec4d6021 Implement policies in Phragment
Summary: This implements support for enforcing and setting policies in Phragment.

Test Plan: Set policies and ensured they were enforced successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7751
2013-12-13 14:42:12 +11:00
epriestley
d846f6508b Fix some repository URI handling issues in Git and Mercurial
Summary:
See <https://github.com/facebook/phabricator/issues/467>. @dctrwatson also ran into an issue where we were trying to `setPass()` a GitURI.

  - For Git and Mercurial, properly generate credential URIs where relevant.
  - Don't try to `setPass()` on Git-style URIs.

This isn't perfect but should clean things up a bit.

Test Plan: Added unit tests. Lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: dctrwatson, aran

Differential Revision: https://secure.phabricator.com/D7759
2013-12-12 09:45:27 -08:00
James Rhodes
b8b7bf8ad9 Provide phragment.getstate and phragment.getpatch Conduit methods
Summary:
This provides a `phragment.getstate` and a `phragment.getpatch` Conduit method.

`phragment.getstate` - This returns the current state of the fragment and all of it's children.

`phragment.getpatch` - This accepts a base path and a mapping of paths to hashes.  The mapping is for the caller to specify the current state of the files it has.  This returns a list of patches that the caller needs to apply to it's files to get to the latest version.

Test Plan:
Ran the following script in a folder which had content matching a fragment and it's children:

```
#!/bin/bash

STATE=""
for i in $(find ./ -type f); do
    HASH=$(cat $i | sha1sum | awk '{ print $1 }')
    BASE=${i:2}
    STATE="$STATE,\"$BASE\":\"$HASH\""
done
STATE=${STATE:1}
STATE="{$STATE}"

echo '{"path":"tychaia3.zip","state":'$STATE'}' | arc --conduit-uri=http://phabricator.local/ call-conduit phragment.getpatch
```

and I got:

```
{"error":null,"errorMessage":null,"response":[]}
```

I updated one of the child fragments with a new file and ran the script again (patch has been omitted due to it's size):

```
{"error":null,"errorMessage":null,"response":[{"path":"Content\/TitleFont.xnb","hash_old":"4a927d7b90582e50cdd330de9f4b59b0cc5eb5c7","hash_new":"25867504642a3a403102274c68fbb9b430c1980f","patch":"..."}]}
```

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, staticshock

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7739
2013-12-11 11:19:23 +11:00
James Rhodes
270c8d27ab Implement "Wait for Previous Builds" build step
Summary: This adds a build step which will block a build from continuing if there are previous builds of the build plan still running.

Test Plan: Configured a build plan with a wait of 60 seconds and a "wait for previous builds", then started a build.  While that was still building, reconfigured the plan to have a wait time of 3 seconds, started it, and saw it move into the "Waiting" status.  When the 60 second build finished, both builds passed.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7745
2013-12-10 11:02:34 +11:00
James Rhodes
8fd256a1fd Implement step for publishing files as fragments to Phragment in Harbormaster
Summary: This adds a build step in Harbormaster for publishing file artifacts as fragments in Phragment.

Test Plan:
Created a build plan with the following steps:

  * Lease Host
  * Upload Artifact
  * Publish Fragment

Ran the build plan against a buildable and saw the fragment get created in Phragment.  Ran the plan again and saw the fragment get updated with a new version.  Modified the file that got uploaded and ran the plan again, checked the history of the fragment, and saw the differences represented as a Diff-Match-Patch patch.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7742
2013-12-09 09:34:09 +11:00
James Rhodes
8bb6e807f0 Implement snapshots in Phragment
Summary:
Ref T4212.  This implements snapshots in Phragment, which allows you to take a snapshot of a fragment at a given point in time, and download a ZIP of the snapshot as it was in this state.

There's also functionality for deleting and promoting snapshots.  You can promote a snapshot to either the latest version or any other snapshot of the fragment.

Test Plan: Clicked around, took some snapshots, promoted them to different points and deleted snapshots.  Also downloaded ZIPs of the snapshots and saw the right versions coming through for all the files downloaded.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205, T4212

Differential Revision: https://secure.phabricator.com/D7741
2013-12-09 08:24:50 +11:00
James Rhodes
e165787725 Implement "Revert to Version" functionality in Phragment
Summary:
This functionality allows users to revert a fragment to a previous version from the history page.

Reverting a version actually creates a new version pointing at the same file as the version being "reverted" to.  In this sense it acts pretty much like Git and other distributed VCS where once you have published a commit the only way to undo your changes is to create a new commit that reverts those changes.

Test Plan: Reverted a fragment to a version before it was deleted, then reverted it to when it was deleted and saw the new versions have the correct file PHIDs (including null for the deletion).

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7738
2013-12-09 07:57:53 +11:00
James Rhodes
c48cfb31bc Implement viewing versions and downloading patches in Phragment
Summary:
This adds support for viewing individual versions on a fragment as well as comparing versions and downloading diff_match_patch-based patches.

It does not use the side-by-side diff format as while it works for small changes, it quickly becomes impossible to distingush what changes have been made due to the diff_match_patch format.

Test Plan: Clicked on versions and downloaded patches.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7734
2013-12-08 12:14:34 +11:00
epriestley
99ad978e90 Add UI for choosing header color
Summary: See D7731. Fixes T4194.

Test Plan: {F88020}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran, mbishopim3

Maniphest Tasks: T4194

Differential Revision: https://secure.phabricator.com/D7740
2013-12-07 10:46:09 -08:00
James Rhodes
ccd4ae5638 Implement "Download ZIP" controller for Phragment
Summary: Depends on D7727.  This adds support for downloading a fragment and all it's children as a ZIP file.  Fragments that have children automatically become directories in the ZIP file.

Test Plan: Downloaded a fragment as a ZIP and was able to extract the contents successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7728
2013-12-07 13:25:22 +11:00
James Rhodes
f7f5a5dd34 Implement update and history controllers in Phragment
Summary: Depends on D7726.  This adds a history controller (for viewing a list of patches associated with a fragment) and an update controller, for creating a new patch of a fragment.

Test Plan: Updated and viewed history of fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7727
2013-12-07 12:56:55 +11:00
James Rhodes
4c143ad3b2 Phragment v0
Summary: Ref T4205.  This is an initial implementation of Phragment.  You can create and browse fragments in the system (but you can't yet view a fragment's patches / history).

Test Plan: Clicked around and created fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7726
2013-12-07 12:43:49 +11:00
James Rhodes
79d153b85d Implement explicit build step ordering in Harbormaster
Summary: This implements support for explicitly marking the sequence of build steps.  Users can now drag and re-order build steps in plans, and artifact dependencies are re-calculated so that if you move "Run Command" before "Lease Host", the "Run Command" step has it's artifact setting cleared and thus the step becomes invalid.

Test Plan: Re-ordered build steps and observed dependencies being correctly recalculated.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7715
2013-12-06 14:12:15 +11:00
James Rhodes
dd01535ed6 Implement "Upload Artifact" build step
Summary: This implements a build step for uploading an artifact from a build machine to Phabricator.  It uses SFTP so that it will work on both UNIX and Windows build machines.

Test Plan: Ran an "Upload Artifact" build against a Windows machine (with FreeSSHD installed).  The artifact uploaded to Phabricator, appeared on the build view and the file contents could be viewed from Phabricator.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7582
2013-12-06 14:11:05 +11:00
epriestley
5ca84589bd Add an SSH access log
Summary: Ref T4107. Ref T4189. This implements an SSH access log, similar to the HTTP access log.

Test Plan:
  [Thu, 05 Dec 2013 13:45:41 -0800]	77841	orbital	::1	dweller	epriestley	epriestley	git-receive-pack	/diffusion/POEMS/	0	324765	402	232
  [Thu, 05 Dec 2013 13:45:48 -0800]	77860	orbital	::1	dweller	epriestley	epriestley	git-receive-pack	/diffusion/POEMS/	0	325634	402	232

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4107, T4189

Differential Revision: https://secure.phabricator.com/D7719
2013-12-05 17:00:48 -08:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
James Rhodes
5c02113bf9 Migrate "Run Command" to use Drydock hosts
Summary: This migrates the "Run Remote Command" build step over to use Drydock hosts and Harbormaster artifacts.

Test Plan:
Created a build plan with a "Lease Host" step and a "Run Command" step.  Configured the "Run Command" step to use the artifact from the "Lease Host" step.

Saw the results:

{F87377}

{F87378}

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049, T4111

Differential Revision: https://secure.phabricator.com/D7707
2013-12-05 14:06:22 +11:00
James Rhodes
d8d1173f52 Implement support for leasing from Drydock hosts in Harbormaster
Summary:
This adds LeaseHostBuildStepImplementation for getting leases on hosts in Drydock via Harbormaster.  It stores the resulting lease in an artifact.

There is also a few bug fixes as well.

Test Plan: Created a build plan with a "Lease Host" build step.  Ran the build plan and saw the build pass and the artifact in the database.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049, T4111

Differential Revision: https://secure.phabricator.com/D7706
2013-12-05 12:46:23 +11:00
James Rhodes
53250d84df Introduce HarbormasterBuildTarget to snapshot build steps through a build
Summary: This implements build targets as outlined in D7582.  Build targets represent an instance of a build step particular to the build.  Logs and artifacts have been adjusted to attach to build targets instead of build / build step pairs.

Test Plan: Ran builds and clicked around the interface.  Everything seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7703
2013-12-05 12:01:12 +11:00
epriestley
d2e9aee16d Reject dangerous changes in Git repositories by default
Summary: Ref T4189. This adds a per-repository "dangerous changes" flag, which defaults to off. This flag must be enabled to do non-appending branch mutation (delete branches / rewrite history).

Test Plan:
With flag on and off, performed various safe and dangerous pushes.

  >>> orbital ~/repos/POEMS $ git push origin :blarp
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push deletes the branch 'blarp'.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad, richardvanvelzen

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7689
2013-12-03 10:28:39 -08:00
James Rhodes
ba16df0fed Restructure Drydock so that blueprints are instances in the DB
Summary:
//(this diff used to be about applying policies to blueprints)//

This restructures Drydock so that blueprints are instances in the DB, with an associated implementation class.  Thus resources now have a `blueprintPHID` instead of `blueprintClass` and DrydockBlueprint becomes a DAO.  The old DrydockBlueprint is renamed to DrydockBlueprintImplementation, and the DrydockBlueprint DAO has a `blueprintClass` column on it.

This now just implements CAN_VIEW and CAN_EDIT policies for blueprints, although they are probably not enforced in all of the places they could be.

Test Plan: Used the `create-resource` and `lease` commands.  Closed resources and leases in the UI.  Clicked around the new and old lists to make sure everything is still working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T2015

Differential Revision: https://secure.phabricator.com/D7638
2013-12-03 11:09:07 +11:00
epriestley
618b5cbbc4 Install pre-commit hooks in Git repositories
Summary:
Ref T4189. T4189 describes most of the intent here:

  - When updating hosted repositories, sync a pre-commit hook into them instead of doing a `git fetch`.
  - The hook calls into Phabricator. The acting Phabricator user is sent via PHABRICATOR_USER in the environment. The active repository is sent via CLI.
  - The hook doesn't do anything useful yet; it just veifies basic parameters, does a little parsing, and exits 0 to allow the commit.

Test Plan:
  - Performed Git pushes and pulls over SSH and HTTP.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7682
2013-12-02 15:45:36 -08:00
James Rhodes
d571507651 Update a few things in Drydock for policies
Summary: DrydockResource has been updated to be policy-aware (although there are no policy columns).

Test Plan: Clicked around in Drydock, viewed resources and leases, everything still seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3605, T4111

Differential Revision: https://secure.phabricator.com/D7595
2013-11-28 08:32:51 +11:00
epriestley
8b19465a9d Render linked JIRA issues with a Doorkeeper tag
Summary: Fixes T3687. Instead of rendering "JIRA Issues" in Differential using plain links, render them using Doorkeeper tags so they get the nice "enhance with object name" effect.

Test Plan: {F84886}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3687

Differential Revision: https://secure.phabricator.com/D7648
2013-11-25 14:54:35 -08:00
epriestley
4b91c4f7ae Add UI for defining repository mirrors
Summary:
Ref T4038. This adds everything except the actual pushing part for mirrors.

This isn't the most beautiful or sophisticated UI, but I want get the authoritative repositories self-hosted and get users beta-ing hosting as soon as possible. We can do transactions, etc., later on.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7632
2013-11-22 15:23:50 -08:00
epriestley
61b26255bb Add "PassphraseKey" classes for code which needs to actually use credentials
Summary: Ref T4122. These classes provide typed, checked access to credentials, so you can say "give me this password, and throw if anything is funky".

Test Plan: Used in next revision.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7625
2013-11-22 15:23:10 -08:00
James Rhodes
7c3cb5948c Drydock blueprint for preallocated remote hosts
Summary:
This adds a Drydock blueprint for preallocated, remote hosts.  This will be used by the Harbormaster interface to allow users to specify remote hosts that builds can be run on.

This adds a `canAllocateResource` method to Drydock blueprints; it is used to detect whether a blueprint can allocate a resource for the given type and attributes.

Test Plan:
Ran:

```
bin/drydock lease --type host --attributes remote=true,preallocated=true,host=192.168.56.101,port=22,user=james,keyfile=,path=C:\\Build\\,platform=windows
```

and saw the "C:\Build\<id>" folder appear on the remote Windows machine.  Viewed the lease and resource in Drydock as well.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T4111

Differential Revision: https://secure.phabricator.com/D7593
2013-11-22 14:34:10 -08:00
epriestley
13275860b1 When stopping on redirect, show a full stack trace
Summary: Ref T4140. Provide more debugging information so we can figure out what's going on with redirect loops.

Test Plan: {F83868}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4140

Differential Revision: https://secure.phabricator.com/D7620
2013-11-21 14:38:29 -08:00
epriestley
8f715d8edf Add a credential selection control to Passphrase
Summary: Ref T4122. Adds a control for choosing credentials.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, lsave

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7617
2013-11-21 12:35:36 -08:00
Bob Trahan
7b718bb033 Nuance - federate out the design of NuanceSource via NuanceSourceDefinition
Summary: ...and get the basic edit flow "working" for a new NuanceSourceDefinition - the Phabricator Form. ...and fix a dumb bug in the query class so when you redirect to the view page / try to edit an existing NuanceSource you don't fatal.

Test Plan: played around with the edit form and it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7585
2013-11-20 13:41:19 -08:00
epriestley
91d084624b Passphrase v0
Summary:
Ref T4122. Implements a credential management application for the uses described in T4122.

@chad, this needs an icon, HA HA HAHA HA BWW HA HA HA

bwahaha

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7608
2013-11-20 09:13:35 -08:00
Aviv Eyal
dcf909ba56 Land to GitHub + support stuff
Summary:
A usable, Land to GitHub flow.

Still to do:
- Refactor all git/hg stratagies to a sane structure.
- Make the dialogs Workflow + explain why it's disabled.
- Show button and request Link Account if GH is enabled, but user is not linked.
- After refreshing token, user ends up in the settings stage.

Hacked something in LandController to be able to show an arbitrary dialog from a strategy.
It's not very nice, but I want to make some more refactoring to the controller/strategy/ies anyway.

Also made PhabricatorRepository::getRemoteURIObject() public, because it was very useful in getting
the domain and path for the repo.

Test Plan:
Went through these flows:
- load revision in hosted, github-backed, non-github backed repos to see button as needed.
- hit land with weak token - sent to refresh it with the extra scope.
- Land to repo I'm not allowed - got proper error message.
- Successfully landed; Failed to apply patch.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T182

Differential Revision: https://secure.phabricator.com/D7555
2013-11-13 17:25:24 -08:00
epriestley
f5ca647d2c Add bin/repository edit for CLI repository editing
Summary:
Ref T4039. This is mostly to deal with that, to prevent the security issues associated with mutable local paths. The next diff will lock them in the web UI.

I also added a confirmation prompt to `bin/repository delete`, which was a little scary without one.

See one comment inline about the `--as` flag. I don't love this, but when I started adding all the stuff we'd need to let this transaction show up as "Administrator" it quickly got pretty big.

Test Plan: Ran `bin/repository edit ...`, saw an edit with a transaction show up on the web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4039

Differential Revision: https://secure.phabricator.com/D7579
2013-11-13 11:26:05 -08:00
epriestley
c8320923c4 Implement most of the administrative UI for approval queues
Summary:
Nothing fancy here, just:

  - UI to show users needing approval.
  - "Approve" and "Disable" actions.
  - Send "Approved" email on approve.
  - "Approve" edit + log operations.
  - "Wait for Approval" state for users who need approval.

There's still no natural way for users to end up not-approved -- you have to write directly to the database.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7573
2013-11-13 11:24:18 -08:00
epriestley
85f505465e Support serving SVN repositories over SSH
Summary:
Ref T2230. The SVN protocol has a sensible protocol format with a good spec here:

http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_ra_svn/protocol

Particularly, compare this statement to the clown show that is the Mercurial wire protocol:

> It is possible to parse an item without knowing its type in advance.

WHAT A REASONABLE STATEMENT TO BE ABLE TO MAKE ABOUT A WIRE PROTOCOL

Although it makes substantially more sense than Mercurial, it's much heavier-weight than the Git or Mercurial protocols, since it isn't distributed.

It's also not possible to figure out if a request is a write request (or even which repository it is against) without proxying some of the protocol frames. Finally, several protocol commands embed repository URLs, and we need to reach into the protocol and translate them.

Test Plan: Ran various SVN commands over SSH (`svn log`, `svn up`, `svn commit`, etc).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7556
2013-11-11 12:19:06 -08:00
epriestley
8840f60218 Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.

Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.

A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.

Test Plan:
  - Ran `hg clone` over SSH.
  - Ran `hg fetch` over SSH.
  - Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).

Reviewers: btrahan, asherkin

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 12:18:27 -08:00
epriestley
f2938bacd9 Generalize SSH passthru for repository hosting
Summary:
Ref T2230. In Git, we can determine if a command is read-only or read/write from the command itself, but this isn't the case in Mercurial or SVN.

For Mercurial and SVN, we need to proxy the protocol that's coming over the wire, look at each request from the client, and then check if it's a read or a write. To support this, provide a more flexible version of `passthruIO`.

The way this will work is:

  - The SSH IO channel is wrapped in a `ProtocolChannel` which can parse the the incoming stream into message objects.
  - The `willWriteCallback` will look at those messages and determine if they're reads or writes.
    - If they're writes, it will check for write permission.
    - If we're good to go, the message object is converted back into a byte stream and handed to the underlying command.

Test Plan: Executed `git clone`, `git clone --depth 3`, `git push` (against no-write repo, got error), `git push` (against valid repo).

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, asherkin, aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7551
2013-11-11 12:12:21 -08:00
James Rhodes
79ef667dfd Render build status on revisions and commits
Summary:
This uses an event listener to render the status of builds on their buildables.  The revision and commit view now renders out the status of each of the builds.

Currently the revision controller has the results for the latest diff rendered out.  We might want to show the status of previous diffs in the future, but for now I think the latest diff should do fine.

There's also a number of bug fixes in this diff, including a particularly nasty one where builds would have a build plan PHID generated for them, which resulted in handle lookups always returning invalid objects.

Test Plan: Ran builds against diffs and commits, saw them appear on the revision and commit view controllers.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7544
2013-11-09 15:04:00 -08:00
epriestley
56c65e33ef Add a "Make an HTTP Request" build step
Summary:
Ref T1049. This is very minimal, but does what it says.

I merged the variable replacement code so Remote + HTTP can share more stuff.

Test Plan:
Ran "HTTP" and "Remote" build plans.

{F79886}

{F79887}

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: zeeg, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7541
2013-11-09 07:16:12 -08:00
epriestley
f357e159b1 Liberate. 2013-11-08 18:19:48 -08:00
James Rhodes
0ac1be7094 Implemented support for build logs
Summary:
Depends on D7519.

This implements support for build logs in Harbormaster.  This includes support for appending to a log from the "Run Remote Command" build step.

It also adds the ability to cancel builds.

Currently the build view page doesn't update the logs live; I'm sure this can be achieved with Javelin, but I don't have enough experience with Javelin to actually make it poll from updates to content in the background.

{F79151}

{F79153}

{F79150}

{F79152}

Test Plan:
Tested this by setting up SSH on a Windows machine and using a Remote Command configured with:

```
C:\Windows\system32\cmd.exe /C cd C:\Build && mkdir Build_${timestamp} && cd Build_${timestamp} && git clone --recursive https://github.com/hach-que/Tychaia.git && cd Tychaia && Protobuild.exe && C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe Tychaia.Windows.sln
```

and observed the output of the build stream from the Windows machine into Phabricator.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7521
2013-11-08 18:15:07 -08:00