1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-27 07:50:57 +01:00
Commit graph

177 commits

Author SHA1 Message Date
epriestley
611f670720 Add CustomField to Diffusion and add a "branches" field
Summary:
Ref T4588. Request from @zeeg. Adds a "BRANCHES" field to commit emails, so the branches the commit appears on are shown.

I've implemented this with CustomField, but in a very light way.

Test Plan: Used `scripts/repository/reparse.php --herald` to generate mail, got a BRANCHES section where applicable.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: aran, epriestley, zeeg

Maniphest Tasks: T4588

Differential Revision: https://secure.phabricator.com/D8501
2014-03-12 11:30:33 -07:00
Joshua Spence
e11adc4ad7 Added some additional assertion methods.
Summary:
There are quite a few tests in Arcanist, libphutil and Phabricator that do something similar to `$this->assertEqual(false, ...)` or `$this->assertEqual(true, ...)`.

This is unnecessarily verbose and it would be cleaner if we had `assertFalse` and `assertTrue` methods.

Test Plan: I contemplated adding a unit test for the `getCallerInfo` method but wasn't sure if it was required / where it should live.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8460
2014-03-08 19:16:21 -08:00
epriestley
a3c1dcb928 Produce a more tailored checkout URI for Subversion
Summary:
For imported SVN repositories with an "Import Only" path, we produce a `/path/to/root/` URI, but should produce `/path/to/root/then/to/import/only/`.

As it is, the URI instructs the user to check out the whole repository.

Also, don't show the "Clone As" fragment in the URI for remote repositories, and prevent it from being edited for nonhosted repositories. This is generally more consistent with user expectation.

Test Plan:
  - Created a remote SVN repository with "Import Only", saw path include it.
  - Verified no "Clone As" options, no "Clone As" in URI.
  - Switched it to hosted, saw "Clone As" options appear and work properly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, staticshock

Differential Revision: https://secure.phabricator.com/D8375
2014-02-28 13:04:41 -08:00
epriestley
e28b78f5eb Fix two issues with ref discovery
Summary:
See IRC. I'm having trouble figuring out what's going on with b4taylor's report, but fix two possible issues:

  # The commit query is missing a `repositoryID`, which could cause issues if you import two copies of the same repository.
  # I think we may try to close commits on untracked branches right now, as long as they aren't excluded by other autoclose rules.

Test Plan: Ran `bin/repository refs` on a few repos.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, brennantaylor

Differential Revision: https://secure.phabricator.com/D8373
2014-02-28 12:56:18 -08:00
Joshua Spence
6270114767 Various linter fixes.
Summary:
- Removed trailing newlines.
- Added newline at EOF.
- Removed leading newlines.
- Trimmed trailing whitespace.
- Spelling fix.
- Added newline at EOF

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: hach-que, chad, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8344
2014-02-26 12:44:58 -08:00
epriestley
a298a79bda Convert Phabricator to handle "%s" / "%B" properly
Summary:
Ref T1191. I believe we only have three meaningful binary fields across all applications:

  - The general cache may contain gzipped content.
  - The file storage blob may contain arbitrary binary content.
  - The Passphrase secret can store arbitrary binary data (although it currently never does).

This adds Lisk config for binary fields, and uses `%B` where necessary.

Test Plan:
  - Added and executed unit tests.
  - Forced file uploads to use MySQL, uploaded binaries.
  - Disabled the CONFIG_BINARY on the file storage blob and tried again, got an appropraite failure.
  - Tried to register with an account containing a G-Clef, and was stopped before the insert.

Reviewers: btrahan, arice

Reviewed By: arice

CC: arice, chad, aran

Maniphest Tasks: T1191

Differential Revision: https://secure.phabricator.com/D8316
2014-02-23 16:20:46 -08:00
epriestley
88227d26bc Allow CustomField to provide ApplicationTransaction change details
Summary:
Ref T3886. Ref T418. For fields like "Summary" and "Test Plan" where changes can't be summarized in one line, allow CustomField to provide a "(Show Details)" link and render a diff.

Also consolidate some of the existing copy/paste, and simplify this featuer slightly now that we've move to dialogs.

Test Plan:
{F115918}

  - Viewed "description"-style field changes in phlux, pholio, legalpad, maniphest, differential, ponder (questions), ponder (answers), and repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886, T418

Differential Revision: https://secure.phabricator.com/D8284
2014-02-21 11:53:04 -08:00
epriestley
b96ab5aadf Modernize VCS password storage to use shared hash infrastructure
Summary: Fixes T4443. Plug VCS passwords into the shared key stretching. They don't use any real stretching now (I anticipated doing something like T4443 eventually) so we can just migrate them into stretching all at once.

Test Plan:
  - Viewed VCS settings.
  - Used VCS password after migration.
  - Set VCS password.
  - Upgraded VCS password by using it.
  - Used VCS password some more.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4443

Differential Revision: https://secure.phabricator.com/D8272
2014-02-18 14:09:36 -08:00
epriestley
c41b4cfac0 Allow Git and Mercurial repositories to be cloned with names in the URI
Summary:
Ref T4175. This allows these URIs to all be valid for Git and Mercurial:

  /diffusion/X/
  /diffusion/X/anything.git
  /diffusion/X/anything/

This mostly already works, it just needed a few tweaks.

Test Plan: Cloned git and hg working copies using HTTP and SSH.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4175

Differential Revision: https://secure.phabricator.com/D8098
2014-01-30 11:42:25 -08:00
epriestley
ffeee37810 Add "Clone As" to repositories and generate full clone commands in UI
Summary:
Ref T4175.

  - Add a configurable name for the clone-as directory, so you can have "Bits & Pieces" clone as "bits~n~pieces/" or simliar.
  - By default, use "reasonable" heruistics to choose such a name.
  - Generate a copy/pasteable clone commmand with this directory name.

Test Plan: Looked at some repositories.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4175

Differential Revision: https://secure.phabricator.com/D8097
2014-01-30 11:42:10 -08:00
epriestley
96dd530c44 Distinguish between "Remote URI" and "Clone URI" in Repositories
Summary:
Hosted repositories have muddied this distinction somewhat. In some cases, we only want to use the real remote URI, and the call is only relevant for imported repositories.

In other cases, we want the URI we'd plug into `git clone`.

Move this logic into `PhabricatorRepository` and make the distinction more clear.

Test Plan: Viewed SVN, Git, and Mercurial hosted and remote repositories, all the URIs looked reasonable.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, dctrwatson

Differential Revision: https://secure.phabricator.com/D8096
2014-01-30 11:41:21 -08:00
epriestley
f585f6fddd Enable "Allow Dangerous Changes" config for Mercurial repositories
Summary:
Fixes T4357. The Mercurial commit hooks enforce dangerous change protection, but the UI doesn't actually let you configure it.

This was just an oversight (I never went back and enabled it) -- allow it to be configured in the UI.

Test Plan: Clicked "Edit Repository" on a hosted Mercurial repository, saw option to enable dangerous changes.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4357

Differential Revision: https://secure.phabricator.com/D8108
2014-01-30 09:45:03 -08:00
Richard van Velzen
28fcb5711b Add basic support for mirroring Mercurial repositories
Summary:
Ref T4338. Mercurial exits with exit code 1 and "no changes found" in stdout
when there's no changes. I've split up the `pushRepositoryToMirror` to make
the code a tad more readable.

It isn't perfect, but it works for me.

Test Plan:
pushed some changes to my hosted repo. Saw them appearing in the
mirrored repo

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4338

Differential Revision: https://secure.phabricator.com/D8107
2014-01-30 08:37:12 -08:00
epriestley
e293d39b5f Split the difference on remote URIs.
This is slightly trickier than D8082.

Auditors: btrahan
2014-01-27 19:44:39 -08:00
epriestley
f2bc293a2a Use remote URI, not display URI, to match repositories
Summary:
I derped this up, and it passed my tests because the two URIs are the same for hosted repositories.

Match repositories against their remote URI (like `https://github.com/user/repo.git`), not their display URI (like `/diffusion/X/`).

Test Plan: i r dums

Reviewers: talshiri, btrahan

Reviewed By: talshiri

CC: aran

Differential Revision: https://secure.phabricator.com/D8082
2014-01-27 18:49:35 -08:00
epriestley
e34a44a7ed Allow repository lookup by remote URI
Summary: This helps us move away from arcanist projects in normal use, by allowing us to identify repositories based on the remote URI.

Test Plan: Used `repository.query` to query some repos by remote URI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8069
2014-01-25 14:02:38 -08:00
epriestley
ce45c57057 Add more options to repository.query
Summary: Expose more options on `repository.query`. My broad goal here is to move forward on suppressing Arcanist Projects.

Test Plan: Used Conduit console to execute queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8068
2014-01-25 14:02:28 -08:00
Brecht Van Lommel
f007ed6263 Allow repository mirrors even if the repository is hosted elsewhere.
Summary:
I'm not sure if this is desired functionality, but we happen to need
mirroring of our repository which is not hosted by phabricator, and as far as
I can tell the mirroring code does not depend on the hosting code.

Test Plan:
Setup mirror with SSH credentials to github, pushed changes to
elsewhere hosted repository, commits got mirrored to github.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4338

Differential Revision: https://secure.phabricator.com/D7637
2014-01-25 12:02:50 -08:00
epriestley
618da2265d Remove all the multi-pass autoclose-branch separate-cache / seenOnBranches junk
Summary:
Ref T4327. Simplify the git discovery process so I can move it to the DiscoveryEngine, so I can make change parsing testable.

In particular:

  - As an optimization, we process closeable branches ("master") first, then process uncloseable branches ("epriestley-devel"). This means that in the common case we can insert a commit as closeable immediately when it is discovered, the first pass through the pipeline will get it right, and the "ref update" step will never need to do any meaningful work.
  - Commits which do not initially appear on a closeable branch, but later move to one (via merges or ref moves) will now be caught in the ref update step, have the closeable flag set, and have a message step re-queued.
  - We no longer need to do a separate discovery step on closable branches.
  - We no longer need to keep track of `seenOnBranches`.

Test Plan: Ran discovery on repositories after pushing commits, got reasonable results.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7985
2014-01-17 11:48:53 -08:00
epriestley
cec3f44b90 Provide an alternate, more general "closeable" flag for commits
Summary:
Ref T4327. This provides a more general RefCursor-based way to identify closeable commits, and moves away from the messy `seenOnBranches` stuff. Basically:

  - When a closeable ref (like the branch "master") is updated, query the VCS for all commits which are ancestors of the new ref, but not ancestors of the existing closeable heads we previously knew about. This is basically all the commits which have been merged or moved onto the closeable ref.
  - Take these commits and set the "closeable" flag on the ones which don't have it yet, then queue new tasks to reprocess them.

I haven't removed the old stuff yet, but will do that shortly.

Test Plan:
  - Ran `bin/repository discover` and `bin/repository refs` on a bunch of different VCSes and VCS states. The effects seemed correct (e.g., new commits were marked as closeable).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7984
2014-01-17 11:48:53 -08:00
epriestley
f4b9efe256 Introduce ref cursors for repository parsing
Summary:
Ref T4327. I want to make change parsing testable; one thing which is blocking this is that the Git discovery process is still part of `PullLocal` daemon instead of being part of `DiscoveryEngine`. The unit test stuff which I want to use for change parsing relies on `DiscoveryEngine` to discover repositories during unit tests.

The major reason git discovery isn't part of `DiscoveryEngine` is that it relies on the messy "autoclose" logic, which we never implemented for Mercurial. Generally, I don't like how autoclose was implemented: it's complicated and gross and too hard to figure out and extend.

Instead, I want to do something more similar to what we do for pushes, which is cleaner overall. Basically this means remembering the old branch heads from the last time we parsed a repository, and figuring out what's new by comparing the old and new branch heads. This should give us several advantages:

  - It should be simpler to understand than the autoclose stuff, which is pretty mind-numbing, at least for me.
  - It will let us satisfy branch and tag queries cheaply (from the database) instead of having to go to the repository. We could also satisfy some ref-resolve queries from the database.
  - It should be easier to extend to Mercurial.

This implements the basics -- pretty much a table to store the cursors, which we update only for Git for now.

Test Plan:
  - Ran migration.
  - Ran `bin/repository discover X --trace --verbose` on various repositories with branches and tags, before and after modifying pushes.
  - Pushed commits to a git repo.
  - Looked at database tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7982
2014-01-17 11:48:53 -08:00
epriestley
972dfa7bfc Add 'hook.d/' directories to SVN and Git repositories for custom hooks
Summary:
Fixes T4189. Ref T4151. Allows repositories to have additional custom hooks for operations which can't be expressed with Herald (one such operation is lint).

This adds only local hook directories, since they're easier to use with existing hooks than global directories. I might add global directories eventually.

This doesn't support Mercurial since we have no demand for it and it's more complicated (we lose compatibility and power by just dropping a `hooks.d/` somewhere).

Test Plan:
  - Pulled hosted SVN and Git repos to verify the hook directories generate correctly.
  - Added a variety of hooks to the hook directories (echo + pass, fail).
  - Pushed commits and verified the hooks fired (output expected info, or failed).
  - Verified push log reflected the correct error code ("3", external) and detail ("nope.sh") when rejecting.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4151, T4189

Differential Revision: https://secure.phabricator.com/D7884
2014-01-03 12:26:10 -08:00
epriestley
637e3f38f3 Allow repositories to be associated with projects
Summary: Ref T4264. Ref T2628. Ref T3102. Allows you to associate repositories with projects. In the future, you'll be able to write Herald object rules against projects, use Herald fields like "Repository's projects", and search by project.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3102, T4264, T2628

Differential Revision: https://secure.phabricator.com/D7881
2014-01-03 12:24:09 -08:00
epriestley
09341be10f Remove repository shortcuts
Summary:
Repositories currently have a no-UI "shortcut" feature which is only used by Facebook (and I'm not sure it's even used). As implemented, this feature is policy-oblivious and kind of nonsensical. Throw it away.

I'm open to reimplementing this, but I want to see some level of interest in it before I do. The new implementation would add shortcuts to each repository, similar to how mirrors work. My original plan was to follow this up with such an implementation (it's half-implemented in my sandbox), but as I worked through it I'm not sure it's really valuable.

Test Plan: Browsed repository list, grep.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Differential Revision: https://secure.phabricator.com/D7862
2014-01-02 11:59:27 -08:00
epriestley
aad6b57c36 Add bin/harbormaster to make builds easier to debug
Summary:
Ref T1049. Adds `bin/harbormaster` and `bin/harbormaster build` for applying plans from the console. Since this gets `--trace`, it's much easier to debug what's going on.

This doesn't work properly with some of the Drydock steps yet, I need to look at those. I think `setRunAllTasksInProcess` probably obsoletes some of the mechanisms. It might also not work with "Wait for Builds" but I didn't check.

Test Plan: Used `bin/harbormaster` to run a bunch of builds. Ran builds from web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7825
2013-12-26 10:40:52 -08:00
epriestley
b0f7e7a6af Work around hg echoing warnings to stdout under --debug
Summary:
Ref T615. Ref T4237. With `--debug`, Mercurial will echo an "ignoring untrusted configuration option" warning **to stdout** if `.hgrc` has the wrong owner.

However, we need `--debug` to make `{parents}` usable, at least until the patches I got into the upstream are widely deployed. So after getting `--debug` output, strip off any leading warnings.

These warnings should always be in English, at least, since we set `LANG` explicitly.

Test Plan: Unit tests. @asherkin, maybe you can confirm this? I can't actually get the warning, but I think my `hg` in PATH is just a bit out of date.

Reviewers: asherkin, btrahan

Reviewed By: asherkin

CC: asherkin, aran

Maniphest Tasks: T615, T4237

Differential Revision: https://secure.phabricator.com/D7784
2013-12-17 18:04:01 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
epriestley
2216a5e6ef Add Subversion ref and content logs to pre-commit hooks
Summary: Ref T4195. SVN has no such thing as refs (I was thinking about writing a quasi-ref anyway like `HEAD: r23 -> r24`, but I'm not sure it would actually be useful). And content is very easy to build.

Test Plan: Pushed some stuff to SVN, got logs from it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7766
2013-12-17 11:11:52 -08:00
epriestley
74251b3636 Support bookmark hook operations in Mercurial
Summary: Ref T4195. Turns bookmark mutations in Mercurial into log objects.

Test Plan:
Pushed a pile of bookmarks and got logs:

{F89313}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7764
2013-12-17 08:34:30 -08:00
epriestley
2725586baf Restructure HookEngine to use PushLog records for all operations
Summary:
Ref T4195. This pulls the central logic of HookEngine up one level and makes all the git stuff genrate PushLogs.

In future diffs, everything will generate PushLogs and we can hand those off to Herald.

Test Plan:
Pushed a pile of valid/invalid stuff:

{F89256}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7761
2013-12-17 08:32:33 -08:00
epriestley
d846f6508b Fix some repository URI handling issues in Git and Mercurial
Summary:
See <https://github.com/facebook/phabricator/issues/467>. @dctrwatson also ran into an issue where we were trying to `setPass()` a GitURI.

  - For Git and Mercurial, properly generate credential URIs where relevant.
  - Don't try to `setPass()` on Git-style URIs.

This isn't perfect but should clean things up a bit.

Test Plan: Added unit tests. Lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: dctrwatson, aran

Differential Revision: https://secure.phabricator.com/D7759
2013-12-12 09:45:27 -08:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
epriestley
d2e9aee16d Reject dangerous changes in Git repositories by default
Summary: Ref T4189. This adds a per-repository "dangerous changes" flag, which defaults to off. This flag must be enabled to do non-appending branch mutation (delete branches / rewrite history).

Test Plan:
With flag on and off, performed various safe and dangerous pushes.

  >>> orbital ~/repos/POEMS $ git push origin :blarp
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push deletes the branch 'blarp'.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad, richardvanvelzen

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7689
2013-12-03 10:28:39 -08:00
epriestley
209861500f Use "en_US.UTF-8", not "C", as the LANG setting
Summary: `LANG=C` is smooshing UTF-8 in some cases. See IRC.

Test Plan: User confirmed this works.

Reviewers: btrahan, asherkin

Reviewed By: asherkin

CC: aran

Differential Revision: https://secure.phabricator.com/D7659
2013-11-26 08:50:36 -08:00
epriestley
de73029e99 Propagate PHABRICATOR_ENV into VCS commands explicitly
Summary: Fixes T4155. See discussion in T4155.

Test Plan: @mbishopim3 confirmed this fixes his issue.

Reviewers: btrahan, chad

Reviewed By: chad

CC: mbishopim3, aran

Maniphest Tasks: T4155

Differential Revision: https://secure.phabricator.com/D7646
2013-11-24 18:11:56 -08:00
epriestley
c978fe5240 Fix an issue where SSH protocols would check legacy credentials
Missed this while grepping. We should use SSH purely based on protocol.

Auditors: btrahan
2013-11-22 16:31:14 -08:00
epriestley
d09dd23bd7 Actually push to mirrors
Summary: Fixes T4038. Push repositories to mirrors.

Test Plan: Created a functional mirror of a local: https://github.com/epriestley/poems

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7633
2013-11-22 15:24:09 -08:00
epriestley
4b91c4f7ae Add UI for defining repository mirrors
Summary:
Ref T4038. This adds everything except the actual pushing part for mirrors.

This isn't the most beautiful or sophisticated UI, but I want get the authoritative repositories self-hosted and get users beta-ing hosting as soon as possible. We can do transactions, etc., later on.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7632
2013-11-22 15:23:50 -08:00
epriestley
51fb1ca16d Migrate repositories to use Passphrase for credential management
Summary: Fixes T4122. Ref T2230. Instead of storing credentials on each repository, store them in Passphrase. This allows easy creation/management of many repositories which share credentials.

Test Plan:
  - Upgraded repositories.
  - Created and edited repositories.
  - Pulled HTTP and SSH repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230, T4122

Differential Revision: https://secure.phabricator.com/D7629
2013-11-22 15:23:33 -08:00
Saulius Zemaitaitis
4910a36563 Set reasonable defaults when displaying remote repository URIs.
Summary: Show SSH user on git-over-ssh repositories and hide both username and password for other repos.

Test Plan: View repository details page in diffusion, Clone URI should appear with a username (taken from repo config) and any http(s) repos should be without usernames.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4147

Differential Revision: https://secure.phabricator.com/D7631
2013-11-22 11:22:39 -08:00
epriestley
e99c53da2e Fix an issue with SVN path construction in the presence of subpath configuration
Summary: D7590 made path construction more consistent, but affected this callsite if a subpath is configured. Currently, we end up with double `@@` in the URI.

Test Plan:
  - Ran unit tests.
  - Ran `bin/repostitory discover`.

Reviewers: staticshock, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7619
2013-11-21 14:41:38 -08:00
epriestley
a07f444f2a Fix git "origin" remote in more circumstances
Summary:
Fixes T4041. We currently detect when "origin" is incorrect, but can do better:

  - When "origin" is missing, we can add it. This happens for Git 1.7.1 -- see T4041.
  - When "origin" is wrong, we can fix it automatically if we control the repository.

We only need to fail when origin exists, is wrong, and we aren't in charge of the repository.

Test Plan: Ran `bin/repository discover X` on a repository with a good origin, no origin, a bad-but-under-control origin, and a bad-out-of-control origin. Got the right behavior in all cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, champo

Maniphest Tasks: T4041

Differential Revision: https://secure.phabricator.com/D7614
2013-11-20 10:41:56 -08:00
epriestley
ff8b48979e Simplify Repository remote and local command construction
Summary:
This cleans up some garbage:

  - We were specifying environmental variables with `X=y git ...`, but now have `setEnv()` on both `ExecFuture` and `PhutilExecPassthru`. Use `setEnv()`.
  - We were specifying the working directory with `(cd %s && git ...)`, but now have `setCWD()` on both `ExecFuture` and `PhutilExecPassthru`. Use `setCWD()`.
  - We were specifying the Git credentials with `ssh-agent -c (ssh-add ... && git ...)`. We can do this more cleanly with `GIT_SSH`. Use `GIT_SSH`.
  - Since we have to write a script for `GIT_SSH` anyway, use the same script for Subversion and Mercurial.

This fixes two specific issues:

  - Previously, we were not able to set `-o StrictHostKeyChecking=no` on Git commands, so the first time you cloned a git repo the daemons would generally prompt you to add `github.com` or whatever to `known_hosts`. Since this was non-interactive, things would mysteriously hang, in effect. With `GIT_SSH`, we can specify the flag, reducing the number of ways things can go wrong.
  - This adds `LANG=C`, which probably (?) forces the language to English for all commands. Apparently you need to install special language packs or something, so I don't know that this actually works, but at least two users with non-English languages have claimed it does (see <https://github.com/facebook/arcanist/pull/114> for a similar issue in Arcanist).

At some point in the future I might want to combine the Arcanist code for command execution with the Phabricator code for command execution (they share some stuff like LANG and HGPLAIN). However, credential management is kind of messy, so I'm adopting a "wait and see" approach for now. I expect to split this at least somewhat in the future, for Drydock/Automerge if nothing else.

Also I'm not sure if we use the passthru stuff at all anymore, I may just be able to delete that. I'll check in a future diff.

Test Plan: Browsed and pulled Git, Subversion and Mercurial repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7600
2013-11-20 10:41:35 -08:00
epriestley
08bdfacff3 Make Subversion URI construction more consistent
Summary:
Ref T2230. SVN has some weird rules about path construction. Particularly, if you're missing a "/" in the remote URI right now, the change parsing step doesn't build the right paths.

Instead, build the right paths more intelligently.

Test Plan: Added and executed unit tests. Imported an SVN repo.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, jpeffer

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7590
2013-11-20 10:41:25 -08:00
Aviv Eyal
dcf909ba56 Land to GitHub + support stuff
Summary:
A usable, Land to GitHub flow.

Still to do:
- Refactor all git/hg stratagies to a sane structure.
- Make the dialogs Workflow + explain why it's disabled.
- Show button and request Link Account if GH is enabled, but user is not linked.
- After refreshing token, user ends up in the settings stage.

Hacked something in LandController to be able to show an arbitrary dialog from a strategy.
It's not very nice, but I want to make some more refactoring to the controller/strategy/ies anyway.

Also made PhabricatorRepository::getRemoteURIObject() public, because it was very useful in getting
the domain and path for the repo.

Test Plan:
Went through these flows:
- load revision in hosted, github-backed, non-github backed repos to see button as needed.
- hit land with weak token - sent to refresh it with the extra scope.
- Land to repo I'm not allowed - got proper error message.
- Successfully landed; Failed to apply patch.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T182

Differential Revision: https://secure.phabricator.com/D7555
2013-11-13 17:25:24 -08:00
epriestley
d4eca25774 Don't implement SVN over HTTP
Summary:
Ref T2230. As far as I can tell, getting SVN working over HTTP is incredibly complicated. It's all DAV-based and doesn't appear to have any kind of binary we can just execute and pass requests through to. Don't support it for now.

  - Disable it in the UI.
  - Make sure all the error messages are reasonable.

Test Plan: Tried to HTTP an SVN repo. Tried to clone a Git repo with SVN, got a good error message.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7562
2013-11-11 16:10:41 -08:00
epriestley
3607bd487c Survive pull/discover for hosted repositories in all VCSes
Summary:
Hosted repositories only sometimes survive the pull/discover phases right now, due to issues like:

  - Pull tries to `git clone`, but should `git init`.
  - Mercurial doesn't handle empty repositories with on branches.
  - SVN tries to connect to an invalid remote.
  - None of them set the INIT repo flag correctly, so status doesn't get updated properly in the UI.

Fix all this stuff.

Test Plan:
  - For each of Git, SVN and Mercurial:
    - Created a new repository from the web UI in a deactivated state.
    - Made it hosted.
    - Manually ran pull/discover.
    - Verified we end up with initialized, empty repositories in consistent states.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7474
2013-11-01 17:36:30 -07:00
epriestley
a0e820ad9a Improve repository hinting and feedback
Summary:
  - Warn about "Read/Write" instead of disabling it, to prevent edits which mutate it after changing a hosted repository to an unhosted one.
  - Warn about authenticated connections with HTTPS auth disabled, and link to the relevant setting.
  - When "Autoclose" is disabled, show that "Autoclose Branches" won't have an effect.
  - For hosted repositories, show the HTTP and SSH clone URIs.
    - Make them easy to copy/paste.
    - Link to credential management.
    - Show if they're read-only.
    - This could be a bit nicer-looking than it is.

Test Plan: Looked at repositories in a bunch of states and made various edits to them.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7471
2013-11-01 17:35:43 -07:00
epriestley
0278b15ceb Implementation of VCS passwords against user.
Summary: This allows users to set their HTTP access passwords via Diffusion interface.

Test Plan: Clicked the "Set HTTP Access Password" link, set a password and saw it appear in the DB.

Reviewers: #blessed_reviewers, hach-que, btrahan

Reviewed By: hach-que

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7462
2013-11-01 08:34:11 -07:00
epriestley
3a39b01233 Add "RepositoryStatusMessage" and detailed information about initilization
Summary:
`RepositoryStatusMessage` is basically a key/value table associated with a repository that I'm using to let the daemons store the most recent event of a given type, so we can easily show it on the status dashboard. I think this will be a lot easier for users to figure out than digging through logfiles.

I'm also going to write the "this needs a pull" status here eventually, for reducing the time lapse between pushes and discovery.

  - Add storage for these messages.
  - Have the pull engine populate the INIT phase. I'll do the FETCH phase next.
  - Update the status readout to show all the various states.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7461
2013-10-30 16:04:19 -07:00