Summary: these transactions should //never// merge since they are always created for a 1:1 replacement. (ie any merging would be implicitly erroneous). Fixes T4081
Test Plan: made a mock with three images and replaced all three successfully. replaced image A with image B, did not save, replaced image B with image C, then saved and verified transaction correctly showed image A replaced with image C.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4081
Differential Revision: https://secure.phabricator.com/D7536
Summary:
Depends on D7500.
This seemed like a pretty good idea once I thought of it. Instead of having some custom triggering logic instead Harbormaster, I figured it best to leverage all of Herald's power so that users can create rules to apply builds to commits and differential revisions. This gives the added advantage that they can trigger off builds for particular types of revisions and commits, which seems like it could be really useful (e.g. run extra tests against revisions that touch sensitive areas of the code).
Test Plan: Ran the usual daemons + the Harbormaster daemon. Pushed a commit to the repository and saw both the buildable and build get created when the commit worked picked it up. Submitted a diff and saw both the buildable and build get created when the Herald rules were evaluated for the diff.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran, hwinkel
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7501
Summary: another little piece here that basically just adds some permissions to source editing. serving it up before I do anything too complicated to make sure it seems kosher. in terms of what comes next this form needs to be dynamic based on source type so there'll be some fun there. That said, I plan to implement a more simple "phabricator form" only version to start here and flesh out a few other things like queues with that.
Test Plan: set permission to no one for source edit and got a nice error page.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7535
Summary: I've kept this as close as possible to the Git version for ease of review and later refactoring of them both together. At minimum, the functions to get the working dir should probably be cleaned up one day.
Test Plan: Landed a revision.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7534
Summary:
See <https://github.com/facebook/phabricator/issues/433>. We were missing a "^" here.
This should be moved over to transactions soon and then we can get rid of the duplication. :/
Test Plan: Tried to create a repository with callsign "9X", got a helpful error about "ALL UPPERCASE LETTERS".
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7531
Summary:
Fixes T4061. Following the instructions in the documentation with Apache 2.4 (which is installed
with Ubuntu 13.10 and other distributions) will result in a "403 forbidden" error.
The instruction provides information on how to fix it.
Test Plan: Tested on apache 2.4 install
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran, asherkin
Maniphest Tasks: T4061
Differential Revision: https://secure.phabricator.com/D7529
Summary:
After upgrading to PHP 5.5, the conduit list was not fully visible because
INF was being treated as "0" for some reason. Fixed by making it a PHP_MAX_INT
Test Plan: Checked on PHP 5.5
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7530
Summary:
Fixes T4067. The way `DiffusionCommitQuery` works prevents it from loading SVN identifiers in some cases without additional constraints, since "12345" might be an SVN revision 12345, or it might be the first 5 characters of a Git commit hash.
Introduce `withRepository()` as a shorthand for `withDefaultRepository()` + `withRepositoryIDs()`. This tells the query to:
- Only look in the given repository; and
- use the more liberal identifier resolution rules while doing so.
The practical impact this has is that blame tooltips in SVN work again. The other queries which are fixed here were never run in SVN (which doesn't have first-class branches or tags); I've cleaned them up only for completeness.
Test Plan:
- Viewed blame in SVN, saw information again instead of empty tooltip.
- Viewed brnaches/tags in Mercurial and Git.
{F79226}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4067
Differential Revision: https://secure.phabricator.com/D7523
Summary:
we were just checking if projects/ was in the URI before barfing. Use some more fun utility functions such that we only complain if there is no project.
Fixes T4071.
Test Plan: made a subpage under a project - success! tried to make a project wiki page where there was no project - successful failure! tried to make a project wiki sub page where there was no project - successful failure!
Reviewers: epriestley, chad
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4071
Differential Revision: https://secure.phabricator.com/D7527
Summary:
See discussion in <https://github.com/facebook/phabricator/issues/430>.
(If we end up with more than like 5 of these we should probably make this a warning or something instead, the only goal is to prevent user error.)
Test Plan:
{F79196}
{F79197}
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran, enko
Differential Revision: https://secure.phabricator.com/D7522
Summary: The "Reviewers" condition in Differential Revision rules has the wrong typeahead and can't select projects, but should be able to.
Test Plan: {F79273}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7526
Summary: Ref T2230. This is easily the worst thing I've had to write in a while. I'll leave some notes inline.
Test Plan: Ran `hg clone http://...` on a hosted repo. Ran `hg push` on the same. Changed sync'd both ways.
Reviewers: asherkin, btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7520
Summary: This is starting to get a bit sizable and it turns out Mercurial is sort of a beast, so split the VCS serve stuff into a separate controller.
Test Plan: Pushed and pulled an authenticated Git repository.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, hach-que
Differential Revision: https://secure.phabricator.com/D7494
Summary: Ref T4068. Partly, this moves discovery to the more unit-testable PhabricatorRepositoryDiscoveryEngine. It also fixes some issues, see inlines.
Test Plan: In a Mercurial repository, ran `bin/repository discover --repair`, verified commits came out topographically sorted. Ran without `--repair` and in various other contexts, like with no commits to discover and some-but-not-all commits to discover.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4068
Differential Revision: https://secure.phabricator.com/D7518
Summary:
I updated the wiki too - https://secure.phabricator.com/w/projects/pebkac/ - with what I am thinking right now. Rough plan here is
- next diff:
- implement editors and transactions
- implement "web type" for contact source
- /pebkac/item/new/ will be the entry point for this
- implement "actions" on a contact
- probably some "polish" on the scaffolding laid out here; like "create" permissions maybs
- diffs after that:
- implement "twitter" type for source
- implement email reply handler stuff for item and source
Probs a great time to blast huge holes in all this stuff. :D
Test Plan: these pages load and arc lint doesn't complain
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran, chad
Differential Revision: https://secure.phabricator.com/D7465
Summary:
Ref T1049. Ref T2222. `DifferentialDiff` does not currently have a PHID, but we need it for Harbormaster and ApplicationTransactions. See some discussion in D7501.
(I split the SQL into two sections so we can't fail in the middle. At some point, I'd like to do a pass on the migration stuff and get this happening automatically, and also simplify the PatchList.)
Test Plan:
- Ran `bin/storage upgrade`.
- Checked for valid PHIDs in the database.
- Used `phid.query` to look up a diff by PHID.
- Created a new diff and verified it got a PHID.
Reviewers: btrahan, hach-que
Reviewed By: btrahan
CC: aran, vrana
Maniphest Tasks: T2222, T1049
Differential Revision: https://secure.phabricator.com/D7513
Summary: Ref T4068. Adds a command to list all commits in an "importing" status. This will allow users to use `reparse.php` to diagnose and repair issues.
Test Plan:
- Ran `bin/repository importing P`, etc.
- Used `reparse.php` to reparse some commit stages and saw status update correctly.
- Ran on a repo with no importing commits.
- Ran with `... --simple | xargs`, which saves us having to put an `awk` or something in there for users.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4068
Differential Revision: https://secure.phabricator.com/D7515
Summary:
Ref T4068. In some cases like that one, I anticipate a repository not fully importing when a handful of random commits are broken. In the long run we should just deal with that properly, but in the meantime provide an administrative escape hatch so you can mark the repository as imported and get it running normally.
The major reason to do this is that Herald, Feed, Harbormaster, etc., won't activate until a repository is "imported".
Test Plan:
- Tried to mark an imported repository as imported, got an "already imported" message.
- Same for not-imported.
- Marked a repository not-imported.
- Marked a repository imported.
- Marked a repository not-imported, then waited for the daemons to mark it imported again automatically.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, kbrownlees
Maniphest Tasks: T4068
Differential Revision: https://secure.phabricator.com/D7514
Summary: Ref T1049. Nothing fancy, but shows red for fail/error and green for pass. See discussion in D7502.
Test Plan: {F78839}
Reviewers: hach-que, btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7512
Summary: Going to remove the workboard shadow area to keep the design less complex visually, shrink icons on mobile view
Test Plan: test uiexamples for new layout changes
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7516
Summary:
Depends on D7501.
This just renders the buildable's actual object name onto the list, so you can see at a glance what the buildable represents. I'd like to also pull across a list of builds of this buildable and change the bar color, but I'm not quite sure how to do that in the search architecture without N+1 querying.
Test Plan:
Looked at the buildable list and it looked like this:
{F78555}
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7502
Summary: Cleans up some CSS while adding lots of other... Mainly, this allow min-width "tables" that trigger a scroll-bar, but go full width if larger than min.
Test Plan: Tested Workboard Examples and some Project pages, Chrome, Tablet and Mobile Layouts
Reviewers: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7509
Summary: This is a little funky but fixes an issue with Git repos that are
non-bare needing "origin/" to resolve branches other than "master". Eventually
this should get cleaned up.
Test Plan: Reporting user verified this fixed their issue.
Auditors: btrahan
Summary: Ref T4064. The response code here isn't normally relevant, but we can hit these via `git clone http://../`, etc., and it's clearly more correct to use HTTP 500.
Test Plan: Added a fake `throw new Exception()` and verified I got an HTTP 500 response.
Reviewers: jamesr, btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4064
Differential Revision: https://secure.phabricator.com/D7507
Summary: This implements an interface for adding new build steps, editing existing build steps and deleting build steps from build plans. It uses the settings definitions on the build implementation to work out what fields should be displayed on the edit page.
Test Plan:
See screenshots:
{F78529}
{F78532}
{F78528}
{F78531}
{F78527}
{F78530}
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7500
Summary:
Depends on D7498.
This implements support for a "build step implementation". Build steps have an associated class name (which makes the class in PHP) and a details field, which is serialized JSON (same as PhabricatorRepository).
This also implements a SleepBuildStepImplementation which just pauses the build for a specified period of seconds.
Test Plan:
Inserted a build step with `insert into harbormaster_buildstep (phid, buildPlanPHID, className, details, dateCreated, dateModified) values ('', 'PHID-HMCP-zkh5w6czfbfpk2gxwdeo', 'SleepBuildStepImplementation', '{"seconds":5}', NOW(), NOW());` (adjusting the build plan PHID as appropriate).
Started the daemon and applied the build plan to a buildable, and saw the daemon take a 5 second delay after creating `SleepBuildStepImplementation`.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran, chad
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7499
Summary:
ref T182.
Simple approach of clone, patch, push. While waiting for drydock, implement a hackish mutex
setup for the workspace, which should work ok as long as there's only one committer who is
carefull about theses things.
Less obvious note: This is taking the both author and commiter's 'primary email' for the commit -
which might rub some people wrong.
Test Plan:
With a hosted repo, created some diffs and landed them.
Also clicked button for some error cases, got the right error message.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: hach-que, Korvin, epriestley, aran
Maniphest Tasks: T182
Differential Revision: https://secure.phabricator.com/D7486
Summary: This implements a basic Harbormaster daemon that takes pending builds and builds them (currently just sleeps 15 seconds before moving to passed state). It also implements an interface to apply a build plan to a buildable, so that users can kick off builds for a buildable.
Test Plan: Ran `bin/phd debug PhabricatorHarbormasterBuildDaemon` and used the interface to start some builds by applying a build plan. Observed them move from 'pending' to 'building' to 'passed'.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T1049
Differential Revision: https://secure.phabricator.com/D7498
Summary:
Fixes T4060. The logic here is:
- When you take several actions at once, we show a single feed story for all of them.
- We choose the "most interesting" title for the feed story. For example, "close task" is more interesting than "add CC".
Currently, the issue with this is:
- "Add comment" is the //least interesting// title. I think this is correct: all other actions are more interesting than the fact that you added a comment.
- We try to conserve the number of objects we need to load by rendering only the most interesting transaction.
To fix this:
- Stop being so conservative; load all of the transactions and all of their PHIDs.
- Add bodies from any transactions which render bodies. In all cases (I think?) this is a maximum of one comment adding a body.
The end result is a story like this:
epriestley closed T123: the building is on fire.
"Okay guys I put the fire out"
Test Plan: See screenshot.
Reviewers: chad, btrahan
Reviewed By: chad
CC: aran, asherkin
Maniphest Tasks: T4060
Differential Revision: https://secure.phabricator.com/D7504
Summary: Fixes the width and some other minor issues.
Test Plan: Tested Mobile and Desktop
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7496
Summary:
Expands on D7488, which looks way better than the config checks. I'm leaving the config checks for now, but maybe we should just get rid of them? This advice is delivered in a far more timely way.
- Check for normal VCS binaries too.
- Link to `environment.append-paths`.
- Get rid of untranslated names (I think they're probably not too useful?)
Test Plan: See screenshots.
Reviewers: hach-que, btrahan
Reviewed By: hach-que
CC: aran
Differential Revision: https://secure.phabricator.com/D7495
Summary:
Currently if 'git-http-backend' is not on the PATH, there is no visible message to the user other than "info/refs: is this a valid git repository?" when trying to clone. This adds a setup check so that if there are any Git repositories in use, it will check for the existance of the "git-http-backend" binary in the PATH.
I believe this is shipped by default alongside the git package on most distros, but in some (such as OpenSUSE), this binary isn't on the PATH by default.
Test Plan: Removed `/usr/lib/git` from my `environment.append-paths` and saw the message appear. Added it back and the message went away.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4050
Differential Revision: https://secure.phabricator.com/D7488
Summary:
Ref T1493. Diffusion has some garbagey behavior for things we can't resolve. Common cases are:
- Looking at a branch that doesn't exist.
- Looking at a repository with no branches.
- Looking at a commit that doesn't exist.
- Looking at an empty repository.
In these cases, we generally fatal unhelpfully. I want to untangle this mess.
This doesn't help much, but does clean things up a bit. We currently have two separate query paths, "stablecommitname" and "expandshortcommit". These are pretty much doing the same thing -- taking some ref like "master" or "default" or a tag name or part of a commit name, and turning it into a full commit name. Merge them into a single "resolverefs" method.
This simplifies the code a fair bit, and gives us better error messages. They still aren't great, but they're like this now:
Ref "7498aec194ecf2d333e0e2baddd9d5cdf922d7f1" is ambiguous or does not exist.
...instead of just:
ERR-INVALID-COMMIT
Test Plan: Looked at Git, Mercurial and Subversion repositories that were empty and non-empty. Looked at branches/heads. Tried to look at invalid commits. Looked at tags. All of this still works, and some behaviors are a bit better than they used to be.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1493
Differential Revision: https://secure.phabricator.com/D7484
Summary: Fixes the junk I broke in D7484. Before that, tag content was a side effect of resolving the ref name. Now, fetch it explicitly in `diffusion.tagsquery`.
Test Plan: Looked at a tag, saw the annotation/message.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7485
Summary: This puts Conduit calls into the "Services" tab. They aren't always real service calls, but I think they're big enough to belong there and be useful.
Test Plan: Viewed "Services" tab, saw conduit calls.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7482
Summary: Ref T1493. Consolidate these a bit; they might need some more magic once we do `--noupdate` checkouts. Mostly just trying to clean up and centralize this code a bit.
Test Plan: Viewed and `bin/repository discover`'d Mercurial repos with and without any branches.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1493
Differential Revision: https://secure.phabricator.com/D7480
Summary: The idea is to have all `phtize` definitions in applications to allow their separation.
Test Plan: Clicked View Options after mangling the translation.
Reviewers: epriestley
Reviewed By: epriestley
CC: btrahan, Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7345
Summary: Adds summary (description) and test plan icons to make these area's more unique and differentiated over general sections.
Test Plan: Test a diff, a commit, a task
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7493
Summary: When we stack property sections without a header, we still want a thin line. This adds a line on non-initial sections but not when headers exist. Fixes T4051
Test Plan: Tested Macro, File, Diffusion, a Whacked Profile Page and Maniphest. All lines look right
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4051
Differential Revision: https://secure.phabricator.com/D7492
Summary: This disables CSRF checking around the `$repository->writeStatusMessage` so that pushing changes over HTTP to Git repositories doesn't fail miserably.
Test Plan: Applied this fix and I could `git push` to hosted repositories again.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4052
Differential Revision: https://secure.phabricator.com/D7490
Summary: This fixes an issue where Git authentication would always fail on an install with `policy.allow-public` set to false. This is because when public access is allowed, anonymous users can query the user list. However, when public access is not allowed, you have to be authenticated before you can read any of the user objects.
Test Plan:
Prior to this fix, I get:
```
james@james-laptop:~/git/8> git clone http://phabricator.local/diffusion/TEST/
Cloning into 'TEST'...
fatal: unable to access 'http://phabricator.local/diffusion/TEST/': The requested URL returned error: 403
```
when `policy.allow-public` is false. After this fix I get:
```
james@james-laptop:~/git/8> git clone http://phabricator.local/diffusion/TEST/
Cloning into 'TEST'...
remote: Counting objects: 102, done.
remote: Compressing objects: 100% (71/71), done.
remote: Total 102 (delta 6), reused 0 (delta 0)
Receiving objects: 100% (102/102), 9.89 KiB | 0 bytes/s, done.
Resolving deltas: 100% (6/6), done.
Checking connectivity... done
```
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4049
Differential Revision: https://secure.phabricator.com/D7489
Summary: This implements Conduit calls for querying Phame blogs and Phame posts.
Test Plan: Made some calls and they seem to generally work.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T3695
Differential Revision: https://secure.phabricator.com/D7478
Summary: This adds support for a Conduit method to process Remarkup content in bulk. It also updates the `getEngineContexts` methods to support any missing contexts.
Test Plan: Ran the command and processed a few sets of text.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4046
Differential Revision: https://secure.phabricator.com/D7479
Summary: This adds pht's and such english to Phriction email body.
Test Plan: Edited a Document, Moved a Document. Got new emails. Such Wow.
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7477
Summary:
Ref T2230. This will need some more refinement, but basically it adds a "Create" vs "Import" step before we go through the paged workflow.
- If you choose "Create", we skip the remote URI / auth stuff, and then set the "hosted" flag.
- If you choose "Import", we do what we do now.
Test Plan: Created and imported repos.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7475
Summary:
Hosted repositories only sometimes survive the pull/discover phases right now, due to issues like:
- Pull tries to `git clone`, but should `git init`.
- Mercurial doesn't handle empty repositories with on branches.
- SVN tries to connect to an invalid remote.
- None of them set the INIT repo flag correctly, so status doesn't get updated properly in the UI.
Fix all this stuff.
Test Plan:
- For each of Git, SVN and Mercurial:
- Created a new repository from the web UI in a deactivated state.
- Made it hosted.
- Manually ran pull/discover.
- Verified we end up with initialized, empty repositories in consistent states.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7474
Summary:
- Warn about "Read/Write" instead of disabling it, to prevent edits which mutate it after changing a hosted repository to an unhosted one.
- Warn about authenticated connections with HTTPS auth disabled, and link to the relevant setting.
- When "Autoclose" is disabled, show that "Autoclose Branches" won't have an effect.
- For hosted repositories, show the HTTP and SSH clone URIs.
- Make them easy to copy/paste.
- Link to credential management.
- Show if they're read-only.
- This could be a bit nicer-looking than it is.
Test Plan: Looked at repositories in a bunch of states and made various edits to them.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: chad, aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7471
Summary:
See <https://github.com/facebook/phabricator/issues/425>. There are some ways that the change parsers may not reach `finishParse()`, but we now need them to in order to mark the commit imported, advance the progress bar, and eventually kick the repository out of IMPORTING status.
Take all the copy/pasted code in the parsers and move it into the parent. Specifically, this is:
- Printing a status message about starting a parse;
- checking for bad commits;
- queueing the next parse stage; and
- marking the import step complete.
Test Plan: Used `reparse.php --change` to reparse Git, SVN and Mercurial repos.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7470
Summary: Depends on D7642. This updates the authentication logic so that HTTP writes can be made to Git repositories hosted by Phabricator.
Test Plan: Set the policy to allow me to push and I was able to. Changed the policy to disallow push and I was no longer able to push.
Reviewers: #blessed_reviewers, hach-que
Reviewed By: hach-que
CC: Korvin, epriestley, aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7468
Summary: This allows users to set their HTTP access passwords via Diffusion interface.
Test Plan: Clicked the "Set HTTP Access Password" link, set a password and saw it appear in the DB.
Reviewers: #blessed_reviewers, hach-que, btrahan
Reviewed By: hach-que
CC: Korvin, epriestley, aran, jamesr
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7462
Summary:
Ref T2350. Fixes T2231.
- Adds log flags around discovery.
- Adds message flags for "needs update". This is basically an out-of-band hint to the daemons that a repository should be pulled sooner than normal. We set the flag when users push a revision, and expose a Conduit method that `arc land` will be able to use.
Test Plan: See screenshots.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2350, T2231
Differential Revision: https://secure.phabricator.com/D7467
Summary:
I pulled these into the property list recently, which made them more consistent, but that dropped "preserve linebreaks". Since these usually come from the CLI, render with linebreaks preserved.
@csilvers, you'll need to `bin/cache purge --purge-remarkup` after this if you want to fix existing revisions.
Test Plan: Made a revision with some poetry, saw poetry preserved.
Reviewers: btrahan, csilvers
Reviewed By: btrahan
CC: csilvers, aran
Differential Revision: https://secure.phabricator.com/D7464
Summary: At least under GitHub, the token value is stored as "null", and not missing. And `null > anything` is false, so Phabricator thinks the token is expired or not there.
Test Plan: http://ph.vm/settings/panel/external/ before shows "No OAuth Access Token," and after it says "Active OAuth Token".
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7466
Summary:
`RepositoryStatusMessage` is basically a key/value table associated with a repository that I'm using to let the daemons store the most recent event of a given type, so we can easily show it on the status dashboard. I think this will be a lot easier for users to figure out than digging through logfiles.
I'm also going to write the "this needs a pull" status here eventually, for reducing the time lapse between pushes and discovery.
- Add storage for these messages.
- Have the pull engine populate the INIT phase. I'll do the FETCH phase next.
- Update the status readout to show all the various states.
Test Plan: See screenshots.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7461
Summary: This moved into Diffusion in D7458 and is now presented in a much cleaner, more targeted way.
Test Plan: Loaded `/repository/`.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7459
Summary:
Replace the blanket "daemons not running" warning with a lot more specific detail, to try to make it easier for users to figure out how to set up repositories correctly.
The next change here will add some additional status information from the daemons, so this panel can report results in greater detail.
Test Plan: See screenshots.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7458
Summary:
- Use DiffusionCommitQuery
- Get rid of the "Author" column.
- Collapse commit + revision together.
- Better tooltips to cover for the removed information.
- Colorize only the "line" column.
- Generally, reduce the amount of visual noise and non-code-stuff going on in this interface.
- I'd like to make the "<<" thing look nicer too but that might take some actual design.
Test Plan: See screenshots.
Reviewers: btrahan, chad
Reviewed By: chad
CC: chad, aran
Differential Revision: https://secure.phabricator.com/D7457
Summary: Minor cleanup. Make the "imported" check less strict (we don't need owners or herald to show change status). Export the "imported" flag over Conduit.
Test Plan: Viewed tag table. Viewed partially imported repositories.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7455
Summary: Swap to DiffusionCommitQuery, other minor cleanup.
Test Plan: Viewed page, forced error view and looked at it.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7454
Summary: See @scottmac's reply in T3982. It looks like his email client uses the standard quote string, but includes it in the quoted block.
Test Plan: Added a failing unit test, made it pass.
Reviewers: btrahan
Reviewed By: btrahan
CC: scottmac, aran
Differential Revision: https://secure.phabricator.com/D7440
Summary: I'm planning to add more detailed info to Diffusion itself, but catch the big issue here.
Test Plan: Hit config issue locally, then resolved it.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7439
Summary: The new "importStatus" property provides a much stronger and more consistent version of this flag. The only callsite was removed by D7452.
Test Plan: Used `grep` to check for callsites and found none.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7453
Summary:
Ref T2716.
- Serve from `DiffusionCommitQuery`, not `PhabricatorAuditCommitQuery` (which should probably die).
- Fix logic for `limit`, which incorrectly failed to display the "Showing %d branches." text.
- Clean up things a touch.
- I didn't end up actually needing `needCommitData()`, but left it in there since I think it will be needed soon.
- Removed a "TODO" because I don't remember what "etc etc" means.
Test Plan: Looked at branches in several repositories.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2716
Differential Revision: https://secure.phabricator.com/D7451
Summary: If you load Diffusion between a repository being pulled and discovered, you can end up with a valid commit reference that hasn't been discovered yet. Don't fatal.
Test Plan: Saw somewhat-helpful error page instead of fatal.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7448
Summary: Ref T2230. This cleans up D7442, by using `git for-each-ref` everywhere we can, in a basically reasonable way.
Test Plan:
In bare and non-bare repositories:
- Ran discovery with `bin/repository discover`;
- listed branches on `/diffusion/X/`;
- listed tags on `/diffusion/X/`;
- listed tags, branches and refs on `/diffusion/rXnnnn`.
Reviewers: btrahan, avivey
Reviewed By: avivey
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7447
Summary: Fixes T4035. I removed these two "remote/" things in rP59922b7, but we need them for non-bare repositories. Without them, the commands work and run fine and the output looks OK, but the results may not reflect the correct information (e.g., the log shows the working copy's master, which may not be in the same state as origin/master). I'm going to generally clean this up, but unbreak it for now.
Test Plan: Viewed bare and non-bare repositories in Diffusion, got accurate history.
Reviewers: btrahan, hach-que
Reviewed By: btrahan
CC: aran, mbishopim3
Maniphest Tasks: T4035
Differential Revision: https://secure.phabricator.com/D7445
Summary: Makes Legalpad a little easier to grok.
Test Plan: View Document
Reviewers: epriestley, btrahan
Reviewed By: btrahan
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7460
Summary: The warning panel on large commits in diffusion was being overrun with other styles. Fixes T3952
Test Plan: test on a large commit
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T3952
Differential Revision: https://secure.phabricator.com/D7456
Summary: We don't have a section header on `/diffusion/X/` for descriptions right now. Add one to improve consistency.
Test Plan: Looked at a repository.
Reviewers: chad, btrahan
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D7449
Summary: Helps make it seem more q/a like and consistent.
Test Plan: Look at question
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7450
Summary:
Ref T2230. Although all the non-bare commands //run// fine in bare repos, not all of them do exactly the same thing.
This could use further cleanup, but at least get it working again for now.
Test Plan: Ran `bin/repository pull`, `bin/repository discover`, viewed Diffusion (looked at branch table), viewed a commit (looked at "Branches"), for bare and non-bare git repos.
Reviewers: avive, btrahan, avivey
Reviewed By: avivey
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7442
Summary: Updates the review status list to align better inside property lists. Alsu uses the default colors a bit more. This removes an overflow hidden on the value side, but that shouldnt cause any issues, given it has plenty of space.
Test Plan: tested differential and audit, highlighted and not.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7441
Summary:
Relocated files aren't treated as newly created files by the worker. This
can lead to the worker trying to look up information about deleted files
in the wrong location.
Test Plan: See T4030
Reviewers: #blessed_reviewers, epriestley
Reviewed By: epriestley
CC: epriestley, aran
Maniphest Tasks: T4030
Differential Revision: https://secure.phabricator.com/D7432
Summary: Fixes T4028, makes the table a bit easier to read and navigate changesets.
Test Plan: Tested a big history, back to original and latest.
Reviewers: epriestley, btrahan
CC: Korvin, epriestley, aran
Maniphest Tasks: T4028
Differential Revision: https://secure.phabricator.com/D7438
Summary:
Fixes T3619. These URIs are valid:
git@domain.com:/path (Git SCP-style implicit SSH)
ssh://git@domain.com/path (Explicit SSH)
This URI, arrived at by adding "ssh://" to the front of an SCP-style URI, is not:
ssh://git@domain.com:/path
Detect URIs in this form and reject them. See T3619.
Test Plan:
{F75486}
Also set some valid URIs.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3619
Differential Revision: https://secure.phabricator.com/D7431
Summary:
This doesn't really impact anything very much, but is a little cleaner than cloning repositories with a working copy. It's somewhat important for allowing pushes, because you can't push to a checked-out branch.
Mercurial has a similar option (`--noupdate`) but leave that alone for now.
The origin stuff was mostly for sanity/explicitness purposes -- I believe it's safe to remove in all non-ridiculous cases. Git fails with it in bare repositories (it automatically creates an `origin`, but doesn't create the local refs for it, or something).
Test Plan: Nuked a repo, re-cloned it, pulled and updated it several times. Browsed both bare and non-bare repos in Diffusion.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7430
Summary:
Fixes T3217. Ref T776. Ref T1493. Broadly, this introduces a mechanism which works like this:
- When a repository is created, we set an "importing" flag.
- After discovery completes, we check if a repository has no importing commits. Basically, this is the first time we catch up to HEAD.
- If we're caught up, clear the "importing" flag.
This flag lets us fix some issues:
- T3217. Currently, when you import a new repository and users have rules like "Email me on every commit ever" or "trigger an audit on every commit", we take a bunch of publish actions. Instead, implicitly disable publishing during import.
- An imported but un-pulled repository currently has an incomprehensible error on `/diffusion/X/`. Fix that.
- Show more cues in the UI about importing.
- Made some exceptions more specific.
Test Plan:
This is the new screen for a completely new repo, replacing a giant exception:
{F75443}
- Created a repository, saw it "importing".
- Pulled and discovered it.
- Processed its commits.
- Ran discovery again, saw import flag clear.
- Also this repository was empty, which hit some of the other code.
This is the new "parsed empty repository" UI, which isn't good, but is less broken:
{F75446}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, hach-que
Maniphest Tasks: T3607, T1493, T776, T3217
Differential Revision: https://secure.phabricator.com/D7429
Summary:
Fixes T3416. Fixes T1733.
- Adds a flag to the commit table showing whether or not we have parsed it.
- The flag is set to `0` initially when the commit is discovered.
- The flag is set to `1` when the changes are parsed.
- The UI can now use the flag to distinguish between "empty commit" and "commit which we haven't imported changes for yet".
- Simplify rendering code a little bit.
- Fix an issue with the Message parser for empty commits.
- There's a key on the flag so we can do `SELECT * FROM repository_commit WHERE repositoryID = %d AND importStatus = 0 LIMIT 1` soon, to determine if a repository is fully imported or not. This will let us improve the UI (Ref T776, Ref T3217).
Test Plan:
- Ran `bin/storage upgrade -f`.
- Created an empty commit.
- Without the daemons running, ran `bin/repository pull GTEST` and `bin/repository discover GTEST`.
- Viewed web UI to get the first screenshot ("Still Importing...").
- Ran the message and change steps with `scripts/repository/reparse.php`.
- Viewed web UI to get the second screenshot ("Empty Commit").
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T776, T1733, T3416, T3217
Differential Revision: https://secure.phabricator.com/D7428
Summary:
- Don't try to pull hosted repos.
- Also, fix the `--verbose` + `--trace` interaction for `bin/repository`.
- Also, fix a couple of unit tests which got tweaked earlier.
Test Plan:
$ ./bin/repository pull GTEST --verbose
Pulling 'GTEST'...
Repository "GTEST" is hosted, so Phabricator does not pull updates for it.
Done.
Reviewers: btrahan, hach-que
Reviewed By: hach-que
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7427
Summary: We've had support for this for a long time, but it was conditional on config. Since it more-or-less actually does something now, just enable it unconditionally.
Test Plan: Settings -> SSH Public Keys
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7426
Summary: Looks like this is pretty straightforward; same as the reads except mark it as needing PUSH.
Test Plan: Ran `git push`, pushed over SSH to a hosted repo.
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7425
Summary: Like D7423, but for SSH.
Test Plan: Ran `git clone ssh://...`, got a clone.
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7424
Summary: Mostly ripped from D7391. No writes yet.
Test Plan: Ran `git clone` against a local over HTTP, got a clone.
Reviewers: btrahan, hach-que
Reviewed By: hach-que
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7423
Summary:
- Add web UI for configuring SSH hosting.
- Route git reads (`git-upload-pack` over SSH).
Test Plan:
>>> orbital ~ $ git clone ssh://127.0.0.1/
Cloning into '127.0.0.1'...
Exception: Unrecognized repository path "/". Expected a path like "/diffusion/X/".
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
>>> orbital ~ $ git clone ssh://127.0.0.1/diffusion/X/
Cloning into 'X'...
Exception: No repository "X" exists!
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
>>> orbital ~ $ git clone ssh://127.0.0.1/diffusion/MT/
Cloning into 'MT'...
Exception: This repository is not available over SSH.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
>>> orbital ~ $ git clone ssh://127.0.0.1/diffusion/P/
Cloning into 'P'...
Exception: TODO: Implement serve over SSH.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7421
Summary:
Fixes T2229. This sets the stage for a patch similar to D7417, but for SSH. In particular, SSH 6.2 introduced an `AuthorizedKeysCommand` directive, which lets us do this in a mostly-reasonable way without needing users to patch sshd (if they have a recent enough version, at least).
The way the `AuthorizedKeysCommand` works is that it gets run and produces an `authorized_keys`-style file fragment. This isn't ideal, because we have to dump every key into the result, but should be fine for most installs. The earlier patch against `sshd` passes the public key itself, which allows the script to just look up the key. We might use this eventually, since it can scale much better, so I haven't removed it.
Generally, auth is split into two scripts now which mostly do the same thing:
- `ssh-auth` is the AuthorizedKeysCommand auth, which takes nothing and dumps the whole keyfile.
- `ssh-auth-key` is the slightly cleaner and more scalable (but patch-dependent) version, which takes the public key and dumps only matching options.
I also reworked the argument parsing to be a bit more sane.
Test Plan:
This is somewhat-intentionally a bit obtuse since I don't really want anyone using it yet, but basically:
- Copy `phabricator-ssh-hook.sh` to somewhere like `/usr/libexec/openssh/`, chown it `root` and chmod it `500`.
- This script should probably also do a username check in the future.
- Create a copy of `sshd_config` and fix the paths/etc. Point the KeyScript at your copy of the hook.
- Start a copy of sshd (6.2 or newer) with `-f <your config file>` and maybe `-d -d -d` to foreground and debug.
- Run `ssh -p 2222 localhost` or similar.
Specifically, I did this setup and then ran a bunch of commands like:
- `ssh host` (denied, no command)
- `ssh host ls` (denied, not supported)
- `echo '{}' | ssh host conduit conduit.ping` (works)
Reviewers: btrahan
Reviewed By: btrahan
CC: hach-que, aran
Maniphest Tasks: T2229, T2230
Differential Revision: https://secure.phabricator.com/D7419
Summary:
Mostly ripped from D7391, with some changes:
- Serve repositories at `/diffusion/X/`, with no special `/git/` or `/serve/` URI component.
- This requires a little bit of magic, but I got the magic working for Git, Mercurial and SVN, and it seems reasonable.
- I think having one URI for everything will make it easier for users to understand.
- One downside is that git will clone into `X` by default, but I think that's not a big deal, and we can work around that in the future easily enough.
- Accept HTTP requests for Git, SVN and Mercurial repositories.
- Auth logic is a little different in order to be more consistent with how other things work.
- Instead of AphrontBasicAuthResponse, added "VCSResponse". Mercurial can print strings we send it on the CLI if we're careful, so support that. I did a fair amount of digging and didn't have any luck with git or svn.
- Commands we don't know about are assumed to require "Push" capability by default.
No actual VCS data going over the wire yet.
Test Plan:
Ran a bunch of stuff like this:
$ hg clone http://local.aphront.com:8080/diffusion/P/
abort: HTTP Error 403: This repository is not available over HTTP.
...and got pretty reasonable-seeming errors in all cases. All this can do is produce errors for now.
Reviewers: hach-que, btrahan
Reviewed By: hach-que
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7417
Summary:
Basically straight from D7391. The differences are basically:
- Policy stuff is all application-scope instead of global-scope.
- Made a few strings a little nicer.
- Deleted a bit of dead code.
- Added a big "THIS DOESN'T WORK YET" warning.
Test Plan: See screenshots.
Reviewers: hach-que, btrahan
Reviewed By: hach-que
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7416
Summary: No editing or view yet, just adds the schema and a policy default. Part of D7391.
Test Plan: `bin/storage upgrade`
Reviewers: hach-que, btrahan
Reviewed By: hach-que
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7415
