1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-13 10:22:42 +01:00
Commit graph

1396 commits

Author SHA1 Message Date
epriestley
a15df4f8d5 Rename "needReviewerStatus()" into "needReviewers()"
Summary: Ref T10967. The old name was because we had a `getReviewers()` tied to `needRelationships()`, rename this method to use a simpler and more clear name.

Test Plan: `grep`, browsed around.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10967

Differential Revision: https://secure.phabricator.com/D17519
2017-03-20 16:46:16 -07:00
epriestley
d179d0150c Remove obsolete "relationships" code from Differential
Summary:
Ref T10967. There have been two different ways to load reviewers for a while: `needReviewerStatus()` and `needRelationships()`.

The `needRelationships()` stuff was a false start along time ago that didn't really go anywhere. I believe the idea was that we might want to load several different types of edges (subscribers, reviewers, etc) on lots of different types of objects. However, all that stuff pretty much ended up modularizing so that main `Query` classes did not need to know about it, so `needRelationships()` never got generalized or went anywhere.

A handful of things still use it, but get rid of them: they should either `needReviewerStatus()` to get reviewer info, or the ~3 callsites that care about subscribers can just load them directly.

Test Plan:
  - Grepped for removed methods (`needRelationships()`, `getReviewers()`, `getCCPHIDs()`, etc).
  - Browsed Diffusion, Differential.
  - Called `differential.query`.

It's possible I missed some stuff, but it should mostly show up as super obvious fatals ("call needReviewerStatus() before getReviewerStatus()!").

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10967

Differential Revision: https://secure.phabricator.com/D17518
2017-03-20 16:45:48 -07:00
Chad Little
40391d089e Add a sort order to the favorites menu
Summary: These were once ordered, but I think we switched to being defined in the Engine and never implemented the sorts there. This adds sort ordering to Tasks, Projects, and Repositories.

Test Plan: Review Favorites Menu in local install, see order is now set per the engine. Click Edit Favorites, and re-order. See order sticks.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D17490
2017-03-11 09:40:06 -08:00
epriestley
0a0ac1302f Prevent users from taking "edit"-like actions via comment forms if they don't have edit permission
Summary:
Ref T12335. Fixes T11207. Edit-like interactions which are not performed via "Edit <object>" are a bit of a grey area, policy-wise.

For example, you can correctly do these things to an object you can't edit:

  - Comment on it.
  - Award tokens.
  - Subscribe or unsubscribe.
  - Subscribe other users by mentioning them.
  - Perform review.
  - Perform audit.
  - (Maybe some other stuff.)

These behaviors are all desirable and correct. But, particularly now that we offer stacked actions, you can do a bunch of other stuff which you shouldn't really be able to, like changing the status and priority of tasks you can't edit, as long as you submit the change via the comment form.

(Before the advent of stacked actions there were fewer things you could do via the comment form, and more of them were very "grey area", especially since "Change Subscribers" was just "Add Subscribers", which you can do via mentions.)

This isn't too much of a problem in practice because we won't //show// you those actions if the edit form you'd end up on doesn't have those fields. So on intalls like ours where we've created simple + advanced flows, users who shouldn't be changing task priorities generally don't see an option to do so, even though they technically could if they mucked with the HTML.

Change this behavior to be more strict: unless an action explicitly says that it doesn't need edit permission (comment, review, audit) don't show it to users who don't have edit permission and don't let them take the action.

Test Plan:
  - As a user who could not edit a task, tried to change status via comment form; received policy exception.
  - As a user who could not edit a task, viewed a comment form: no actions available (just "comment").
  - As a user who could not edit a revision, viewed a revision form: only "review" actions available (accept, resign, etc).
  - Viewed a commit form but these are kind of moot because there's no separate edit permission.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12335, T11207

Differential Revision: https://secure.phabricator.com/D17452
2017-03-02 16:56:57 -08:00
epriestley
c5fa7421c2 Allow commits to be queried by repository using the tagged(...) typehaead function
Summary:
Fixes T12322. Allows you to search for commits using the `tagged(...)` repository function, so you can find "any commmit in any repository tagged with android" or similar.

I moved the function from Differential (which was the application using it) to Diffusion (which is more accurately the application which provides it).

I fixed a bug where searching for `tagged(xyz)` would have no effect (constraint was ignored) if there were no repositories tagged with "xyz". The fix isn't perfectly clean, but should work properly for the moment.

Test Plan:
  - Searched with `tagged(...)` in Diffusion and Differential.
  - Searched by repository.
  - Searched with `tagged(...)` for a project with no tagged repositories.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12322

Differential Revision: https://secure.phabricator.com/D17426
2017-02-27 10:46:55 -08:00
epriestley
89d1403fe8 Explicitly decline to add commit authors as auditors from Herald
Summary:
Fixes T12304. If you have a Herald rule which tries to add a commit author as an auditor, it fails validation when trying to apply.

Stop trying to apply these transactions, and explicitly tell the user why. Differential already uses a similar ruleset around reviewers, but Audit was using older code.

Test Plan:
  - Wrote a Herald rule to add A, B and C as auditors.
  - Committed as A.
  - After change, saw B and C added with transacript guidance that A was the author.

{F3235660}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12304

Differential Revision: https://secure.phabricator.com/D17404
2017-02-23 15:19:23 -08:00
epriestley
3b6a651b69 Merge multiple Auditors transactions from Herald
Summary:
Fixes T12302. Currently, we aren't merging multiple "AddAuditors" transactions correctly.

This can occur when Herald triggers multiple auditor rules.

Instead, merge them.

Test Plan:
  - Wrote two different Herald rules that add auditors.
  - Pushed a commit which triggered them.
  - After the change, saw all the auditors get added correctly.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12302

Differential Revision: https://secure.phabricator.com/D17403
2017-02-23 15:14:58 -08:00
Jakub Vrana
9f3cde4db7 Fix errors found by PHPStan
Test Plan: None.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D17377
2017-02-18 09:24:56 +00:00
Jakub Vrana
a778151f28 Fix errors found by PHPStan
Test Plan: Ran `phpstan analyze -a autoload.php phabricator/src`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D17371
2017-02-17 10:10:15 +00:00
epriestley
037c749ef3 Fix missing setQuoteRef() on Commit detail pages in Diffusion
Summary: Fixes T12253.

Test Plan:
  - Before change: used "Quote Comment", saw "In null, alice wrote:" in quoted text.
  - After change: used "Quote Comment", saw proper reference to the commit/page. Clicked reference, was sent to the comment properly.

{F2859093}

Reviewers: chad, avivey

Reviewed By: avivey

Maniphest Tasks: T12253

Differential Revision: https://secure.phabricator.com/D17343
2017-02-13 07:44:01 -08:00
Josh Cox
e0675b28d8 Pass exception to PhutilProxyException
Summary: Fixes T12243. That error occured due to network flakiness with some mounted filesystems so I'm not sure how best to simulate it. But you can look and see that the PhutilProxyException does indeed expect an exception as its second arg.

Test Plan: Look at method signature... look at callsite... now back at the method. Smile and nod.

Reviewers: #blessed_reviewers, yelirekim, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Maniphest Tasks: T12243

Differential Revision: https://secure.phabricator.com/D17335
2017-02-08 13:24:44 -05:00
Josh Cox
1b8b64aae6 Stop calling the undefined withIsTag method
Summary: This just cleans up a method call that was missed in D15986. It's been causing fatal errors in one of our workflows.

Test Plan: Grep'd for other instances of `withIsTag` and didn't find any

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, yelirekim

Differential Revision: https://secure.phabricator.com/D17299
2016-12-14 14:56:40 -05:00
epriestley
4890d66795 Excluded authored commits from "Ready to Audit"; handle unreachable commits better
Summary:
Ref T10978. I'm inching toward cleaning up our audit state. Two issues are:

  - Authored commits show up in "Ready to Audit", but should not.
  - Unreachable commits (like that stacked of unsquashed stuff) show up too, but we don't really care about them.

Kick authored stuff out of the "Ready to Audit" bucket and hide unreachable commits by default, with constraints for filtering. Also give them a closed/disabled/strikethru style.

Test Plan:
  - Viewed audit buckets.
  - Searched for reachable/unreachable commits.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17279
2017-01-31 13:37:05 -08:00
epriestley
bcbd4035fd Remove several pieces of audit-related code
Summary: Ref T10978. This code (mostly related to the old ADD_AUDIT transaction and some to the "store English text in the database" audit reasons) is no longer reachable.

Test Plan:
Grepped for removed symbols:

  - withAuditStatus
  - getActionNameMap (unrelated callsites exist)
  - getActionName (unrelated callsites exist)
  - getActionPastTenseVerb
  - addAuditReason
  - getAuditReasons
  - auditReasonMap

Also audited some commits.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17267
2017-01-30 15:26:26 -08:00
epriestley
5e7a091737 Write an explicit edge for commit membership in packages
Summary:
Ref T10978. Currently, during commit import, we write an "Audit Not Required" auditor for commits which don't require an audit.

This auditor is used to power the "Commits in this package" query in Owners.

This conflates audits and commit/package membership. I think it might even predate edges. Code needs to dance around this mess and we get the wrong result in some cases, since auditors are now editable.

Instead, write an explicit edge which just says "this commit is part of such-and-such packages". Then use that to run the query. Logical!

I'll issue guidance on this but I'm not migrating it, since it fixes itself going forward and only really affects the UI in Owners.

Test Plan:
  - Ran `bin/audit update-owners` with various arguments.
  - Viewed packages in web UI, saw them load the proper commits.
  - Queried by packages in Diffusion explicitly.
  - Clicked the "View All" link in Owners and got to the right search UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17264
2017-01-30 15:23:34 -08:00
epriestley
4b248e3545 Make the "Add Auditors" Herald rules use modern transactions
Summary:
Ref T10978. Convert "Add Auditors" rules in Herald to modern modular transactions.

Here and in D17262 (and in the next change), I've removed "audit reasons". There are several reasons for this:

  - They're pretty hacky.
  - They store English-language (well, usually) text in the database, which can't be translated.
  - I think they may not be necessary. When they were written, Herald did not apply transactions, so it was less clear when Herald was doing something. In modern code, it does, so Herald auditors are clear. The owenrs/package rules are now more clear, too. I'd like to see evidence that confusion still exists before rebuilding this feature in a modern, translatable way, since I think we may not need it at all.

Test Plan: Ran `bin/repository reparse --herald <commit>` to re-run Herald rules. Saw rules add auditors appropriately.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17263
2017-01-30 15:23:20 -08:00
epriestley
2e3e078358 Remove "diffusion.createcomment" Conduit API method
Summary: Ref T10978. This was introduced in D6923 in 2013 as a deprecated method (before methods were extensible) and has only ever been deprecated. It no longer works after D17250 (despite my mistaken claim there that we never had an API for actions), and has been superceded by `diffusion.commit.edit` which is a modern, fully-power method.

Test Plan: Viewed Conduit console, no longer saw method.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17254
2017-01-26 12:57:15 -08:00
epriestley
97cac83e9b Add a "Needs Verification" state to Audit
Summary:
Fixes T2393. This allows authors to explicitly say "I think I fixed everything, please accept my commit now thank you".

Also improves behavior of "re-accept" and "re-reject" after new auditors you have authority over get added.

Test Plan:
  - Kicked a commit back and forth between an author and auditor by alternately using "Request Verification" and "Raise Concern".
  - Verified it showed up properly in bucketing for both users.
  - Accepted, added a project, accepted again (works now; didn't before).
  - Audited on behalf of projects / packages.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2393

Differential Revision: https://secure.phabricator.com/D17252
2017-01-25 13:08:59 -08:00
epriestley
ca182c7f48 Clean up "Audit Authority" code, at least mostly
Summary:
Ref T2393. We had three copies of this code ("which packages/projects can a user accept on behalf of?"). I removed one in D17250. This consolidates the other two.

This still isn't perfect and it should probably live in a Query or something some day, but there's some weird stuff going on with the viewer in the editor context, and at least the code handles the viewer correctly now and isn't living somewhere weird and totally unrelated to auditing, and the callsites don't need to do a bunch of extra work.

This also moves towards fixing the "re-accept if you've already accepted but then a new package you have authority over was added" bug, which we fixed recently in Differential. This should be less common in Audit, but should still be fixed.

Test Plan: Viewed and audited commits with a mixture of user, package, and project auditors. Saw actions apply to the expected set of auditors.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2393

Differential Revision: https://secure.phabricator.com/D17251
2017-01-25 13:08:25 -08:00
epriestley
36d936fe8a Remove an unused method in Audit for building comment actions
Summary: Ref T2393. This has been obsoleted by stacked actions and is no longer used.

Test Plan: Grepped for callsites, viwed commits.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2393

Differential Revision: https://secure.phabricator.com/D17249
2017-01-25 13:07:48 -08:00
epriestley
ed38642afc Give Audit an informational "This commit now requires (something)..." transaction
Summary: Ref T2393. This adds a state-change transaction hint to Audit, like we have in Differential. This is partly for consistency and partly to make it more clear what should happen next.

Test Plan: {F2477848}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2393

Differential Revision: https://secure.phabricator.com/D17243
2017-01-25 07:53:18 -08:00
epriestley
a9158d34d4 Show commit audit status in repository history tables, including merge commit lists
Summary:
Fixes T6024. Ref T12121. Currently, we show build status in commit history tables; show audit status alongside it.

Also:

  - Change the "Author/Committer" header to just "Author"; I think it's reasonably obvious what "x/y" means (if you can't guess, you can click the commit and likely figure it out) and this gives us a little more space.
  - Make the audit list look more like the corresponding list in Differential, with similar formatting.

Test Plan:
  - Viewed history of a repostiory, saw audit status.
  - Viewed a merge commit, saw audit status in the list of merged commits.
  - Viewed a commit search results list.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12121, T6024

Differential Revision: https://secure.phabricator.com/D17227
2017-01-19 11:43:21 -08:00
epriestley
545dad319e Add an "Auditors" rule for Commits
Summary: Fixes T5889. You can't write a rule like "if no other Herald rules did anything...", but you can use this rule to check for Owners or an explicit "Auditors" field doing things.

Test Plan: Using the test console, ran an "Auditors" rule against a commit with and without an auditor. Got expected pass/fail outcomes.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5889

Differential Revision: https://secure.phabricator.com/D17221
2017-01-18 10:05:30 -08:00
epriestley
9d3f09ab47 Modularize global quick create builtin items
Summary: Ref T5867. Instead of hard-coding projects, tasks and repositories, let EditEngines say "I want a quick create item" so third-party code can also hook into the menu without upstream changes.

Test Plan: Saw same default items in menu.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5867

Differential Revision: https://secure.phabricator.com/D17215
2017-01-17 15:56:31 -08:00
epriestley
903e37a21b Show yellow "draft" bubble in Audit
Summary: Fixes T6660. Uses the new stuff in Audit to build an EditEngine-aware icon.

Test Plan: {F2364304}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6660

Differential Revision: https://secure.phabricator.com/D17208
2017-01-16 10:28:59 -08:00
epriestley
19525ed81a Add diffusion.commit.search Conduit API method
Summary: Ref T10978. This is bare bones, but the SearchEngine is at least mostly in reasonable shape now, so get it in place and freeze the old stuff. I previously froze `audit.query`, which did much the same thing.

Test Plan: Issued some queries with the API, technically got results back.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17194
2017-01-12 13:23:29 -08:00
epriestley
a635da68d4 Provide bucketing for commits in Audit
Summary:
Fixes T9430. Fixes T9362. Fixes T9544. This changes the default view of Audit to work like Differential, where commits you need to audit or respond to are shown in buckets.

This is a bit messy and probably needs some followups. This stuff has changed from a compatibility viewpoint:

  - The query works differently now (but in a better, modern way), so existing saved queries will need to be updated.
  - I've removed the counters from the home page instead of updating them, since they're going to get wiped out by ProfileMenu soon anyway.
  - When bucketed queries return too many results (more than 1,000) we now show a warning about it. This isn't greaaaat but it seems good enough for now.

Test Plan: {F2351123}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9430, T9362, T9544

Differential Revision: https://secure.phabricator.com/D17192
2017-01-12 12:04:05 -08:00
epriestley
7d3d022407 Restore "[Action]" mail subject lines to Differential/Diffusion
Summary: Ref T11114. Ref T10978. These hadn't made it over to EditEngine yet.

Test Plan:
  - Took various actions on revisions and commits.
  - Used `bin/mail show-outbound --id ...` to examine the "Vary Subject", saw it properly generate "[Accepted]", "[Resigned]", etc.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11114, T10978

Differential Revision: https://secure.phabricator.com/D17191
2017-01-12 11:44:24 -08:00
epriestley
b941331bdf Prevent users from resigning from audits they've already resigned from
Summary: Ref T10978. Since "Resigned" is a status in Audit, you could repeatedly resign. This is confusing; prevent it.

Test Plan: Tried to resign twice; was only allowed to resign once.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17187
2017-01-11 16:28:57 -08:00
epriestley
11861265fe Merge "Audit" more completely into "Diffusion"
Summary:
Fixes T6630. Long ago, "Audit", "Diffusion" and "Repositories" were three totally separate applications.

This separation isn't useful and the three rapidly became intertwined. Ideally, they would all be one application.

This doesn't take us quite that far, but Audit no longer has any controllers and has little actual behavior.

The "Audit" screen has always just been a SearchEngine view of commits with some filters on it, and this formalizes that and puts a link to it in Diffusion. (This view has other uses, too.)

Test Plan:
  - Accessed audit from home page.
  - Accessed audit/commits from Diffusion.
  - Could no longer uninstall Audit on its own.
  - Grepped for `/audit/` and `AuditApplication`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6630

Differential Revision: https://secure.phabricator.com/D17186
2017-01-11 16:28:42 -08:00
epriestley
b5722a9963 Use EditEngine stacked comments in Diffusion
Summary: Ref T10978. Ref T8739. Fixes T10446. Converts Diffusion to modern comment/preview code, like Differential.

Test Plan: {F2342933}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978, T10446, T8739

Differential Revision: https://secure.phabricator.com/D17183
2017-01-11 14:46:48 -08:00
epriestley
82c891f586 Add modern "Accept", "Raise Concern" and "Resign" transactions to Audit
Summary:
Ref T10978. This prepares for swapping the comment UI to stacked actions.

These are only accessible via the API.

Test Plan: Used the API to accept, raise concern with, and reject commits.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17182
2017-01-11 13:56:48 -08:00
epriestley
255e3fb1e4 Allow auditors to be added and removed from commits in a modern way
Summary: Ref T10978. Ref T7676. Make auditors work more like reviewers, so they can be freely added or removed.

Test Plan:
  - Interacted with auditors via "Edit Commit" and API.
  - Comment area is still oldschool and doesn't work yet.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978, T7676

Differential Revision: https://secure.phabricator.com/D17181
2017-01-11 13:56:34 -08:00
epriestley
2941b34acb Add "diffusion.commit.edit", a v3 edit API endpoint for commits
Summary: Ref T10978. This currently does almost nothing, but gets it in place so I can add stuff to it.

Test Plan: Made a comment on a commit using the API.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17178
2017-01-11 10:38:14 -08:00
epriestley
279273dc1c Replace old commit edit controller with new EditEngine controller
Summary: Ref T10978. The new controller now does everything the old one did, so swap 'em and nuke the old one.

Test Plan: Edited a commit, hit the new controller, things worked real good.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17177
2017-01-11 10:37:53 -08:00
epriestley
5e07358826 Preserve "Autoclose?" information on new Commit edit flow
Summary: Ref T10978. The current "Edit" flow has some autoclose info. This isn't necessarily the best place to put it in the long run, but preseve it for now since the documentation refers to it.

Test Plan: {F2340658}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17176
2017-01-11 10:31:20 -08:00
epriestley
a27c824da6 Draw project PHIDs from repositories when evaluating Herald object rules for commits
Summary:
Fixes T12097. In D16413, I simplified this code but caused us to load the //commit's// projects instead of the //repository's// projects, which is incorrect.

Normally, commits don't have any project tags when Herald evaluates, so using the commit's projects is generally meaningless.

Test Plan:
  - Tagged a repository with `#X`.
  - Created a Herald object rule for commits with `#X` as the object ("Always ... do nothing.")
  - Ran a commit from the repository.
  - Before patch: rule failed to evaluate.
  - After patch: rule evaluated and passed.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12097

Differential Revision: https://secure.phabricator.com/D17179
2017-01-11 10:29:39 -08:00
epriestley
7ff0be1bde Bring very basic EditEngine support to commits
Summary:
Ref T10978. After T11114, we have some features (like the old code for the haunted comment panel) which are only used by Diffusion. I want to modernize it so I can nuke them. T10978 also describes many bugs which are only fixable after modernizing.

This adds very basic EditEngine support for commits/audit. You can't create new commits with this workflow, just tag/update existing ones.

Test Plan: {F2340347}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10978

Differential Revision: https://secure.phabricator.com/D17175
2017-01-11 09:34:46 -08:00
epriestley
ccff47682f Provide more useful guidance if a repository is clusterized into an existing multi-device cluster
Summary:
Fixes T12087. When transitioning into a clustered configuration for the first time, the documentation recommends using a one-device cluster as a transitional step.

However, installs may not do this for whatever reason, and we aren't as clear as we could be in warning about clusterizing directly into a multi-device cluster.

Roughly, when you do this, we end up believing that working copies exist on several different devices, but have no information about which copy or copies are up to date. //Usually// they all were already synchronized and are all up to date, but we can't make this assumption safely without risking data.

Instead, we err on the side of caution, and require a human to tell us which copy we should consider to be up-to-date, using `bin/repository thaw --promote`.

Test Plan:
```
$ ./bin/repository clusterize rLOCKS --service repos001.phacility.net
Service "repos001.phacility.net" is actively bound to more than one device
(local002.local, local001.phacility.net).

If you clusterize a repository onto this service it will be unclear which
devices have up-to-date copies of the repository. This leader/follower
ambiguity will freeze the repository. You may need to manually promote a
device to unfreeze it. See "Ambiguous Leaders" in the documentation for
discussion.

    Continue anyway? [y/N]
```

Read other changes.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12087

Differential Revision: https://secure.phabricator.com/D17169
2017-01-10 12:45:55 -08:00
epriestley
d4248d231b Correct "Manage Password" link in Quickling in Diffusion
Summary: Fixes T12080. This was missing a "/", but stop hard-coding these URIs.

Test Plan: Clicked both links with Quickling as a logged-in and logged-out user, ended up in the right place.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12080

Differential Revision: https://secure.phabricator.com/D17151
2017-01-08 08:20:23 -08:00
epriestley
8640ab5fc3 Redirect /source/x (no slash) to /source/x/ (canonical) when viewer is logged out and "x" is public
Summary:
Fixes T12035. Normally, the "abc" -> "abc/" redirect is handled automatically when "abc" hits a 404.

However, in this case, "source/x" does not 404. We route this to a valid controller because some VCS requests omit the slashes, then manually perform the redirect if we aren't serving a VCS request.

Allow this controller to serve public resources so we can serve the redirect to logged-out users instead of prompting them to login so they can be redirected.

Test Plan: Visited `/source/x` as a logged-out user, where `x` is a public repository.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12035

Differential Revision: https://secure.phabricator.com/D17097
2016-12-20 07:48:20 -08:00
Sébastien Santoro
01ac745d9d Fixed typo
Summary: In Settings > Set VCS Pasword: artisinal → artisanal

Test Plan: Read again the sentence.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D17095
2016-12-19 17:56:27 -08:00
epriestley
5f26dd9b66 Use futures to improve clustered repository main page performance
Summary:
Ref T11954. In cluster configurations, we get repository information by making HTTP calls over Conduit.

These are slower than local calls, so clustering imposes a performance penalty. However, we can use futures and parallelize them so that clustering actually improves overall performance.

When not running in clustered mode, this just makes us run stuff inline.

Test Plan:
  - Browsed Git, Mercurial and Subversion repositories.
  - Locally, saw a 700ms wall time page drop to 200ms.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11954

Differential Revision: https://secure.phabricator.com/D17009
2016-12-08 07:26:32 -08:00
epriestley
58ea40ad64 Hash Diffusion README cachekey components
Without this, we end up with an overlong cache key in some cases.

Auditors: chad
2016-12-06 10:03:10 -08:00
epriestley
b869e742b9 Cache README content for repositories
Summary:
Ref T11954. Especially with higher-latency file stores like S3, we can spend a lot of time reading README data and then pulling it out of file storage.

Instead, cache it.

Test Plan: Browsed a repostory with a README, saw faster pages.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11954

Differential Revision: https://secure.phabricator.com/D17002
2016-12-06 09:59:17 -08:00
epriestley
e6ddd6d0e9 Cache Almanac URIs for repositories
Summary:
Ref T11954. This is kind of complex and I'm not sure I want to actually land it, but it gives us a fairly good improvement for clustered repositories so I'm leaning toward moving forward.

When we make (or receive) clustered repository requests, we must first load a bunch of stuff out of Almanac to figure out where to send the request (or if we can handle the request ourselves).

This involves several round trip queries into Almanac (service, device, interfaces, bindings, properties) and generally is fairly slow/expensive. The actual data we get out of it is just a list of URIs.

Caching this would be very easy, except that invalidating the cache is difficult, since editing any binding, property, interface, or device may invalidate the cache for indirectly connected services and repositories.

To address this, introduce `PhabricatorCacheEngine`, which is an extensible engine like `PhabricatorDestructionEngine` for propagating cache updates. It has two modes:

  - Discover linked objects (that is: find related objects which may need to have caches invalidated).
  - Invalidate caches (that is: nuke any caches which need to be nuked).

Both modes are extensible, so third-party code can build repository-dependent caches or whatever. This may be overkill but even if Almanac is the only thing we use it for it feels like a fairly clean solution to the problem.

With `CacheEngine`, make any edit to Almanac stuff propagate up to the Service, and then from the Service to any linked Repositories.

Once we hit repositories, invalidate their caches when Almanac changes.

Test Plan:
  - Observed a 20-30ms performance improvement with `ab -n 100`.
  - (The main page making Conduit calls also gets a performance improvement, although that's a little trickier to measure directly.)
  - Added debugging code to the cache engine stuff to observe the linking and invalidation phases.
  - Made invalidation throw; verified that editing properties, bindings, etc, properly invalidates the cache of any indirectly linked repositories.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11954

Differential Revision: https://secure.phabricator.com/D17000
2016-12-06 09:14:45 -08:00
epriestley
4faa4b451f When viewing a branch, preview differences from master
Summary: Ref T929. When viewing a branch, show a few recent differences from the default branch (usually, "master").

Test Plan: {F2079220}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T929

Differential Revision: https://secure.phabricator.com/D16991
2016-12-06 08:16:41 -08:00
epriestley
fc1adf9875 Modernize UI for "Compare" in Diffusion
Summary: Ref T929. We've made some UI updates since D15330.

Test Plan: {F2079125}

Reviewers: avivey, chad

Reviewed By: chad

Maniphest Tasks: T929

Differential Revision: https://secure.phabricator.com/D16990
2016-12-05 18:10:11 -08:00
Aviv Eyal
43f9927a38 Compare two branches
Summary:
This shows the commits list only (Actual `git diff` will show up at a later date).
The inputs are left as text-fields, to allow the form to accept anything that can be resolved. The form is GET, to allow sharing URIs.

The conduit method response array is compatible with that of `diffusion.historyquery`, to make it easy to build
the "history" table.

The hardest part here was, of course, Naming. I think "from" and "onto" are unconfusing, and I'm fairly confident that the "to merge"
instructions are in sync with the actual content of the page.

Test Plan: Look at several "compare" views, with various values of "from" and "onto".

Reviewers: #blessed_reviewers!, epriestley

Subscribers: caov297, 20after4, Sam2304, reardencode, baileyb, chad, Korvin

Maniphest Tasks: T929

Differential Revision: https://secure.phabricator.com/D15330
2016-12-05 16:25:49 -08:00
epriestley
005d8493b0 Pass GIT_ENVIRONMENTAL_MAGIC through to hook subprocesses to support Git 2.11.0
Summary:
Fixes T11940. In 2.11.0, Git has made a change so that newly-pushed changes are held in a temporary area until the hook accepts or rejects them.

This magic temporary area is only readable if the appropriate `GIT_ENVIRONMENTAL_MAGIC` variables are available. When executing `git` commands, pass them through from the calling context.

We're intentionally conservative about which variables we pass, and with good reason (see "httpoxy" in T11359). I think this continues to be the correct default behavior.

Test Plan:
  - Upgraded to Git 2.11.0.
  - Tried to push over SSH, got a hook error.
  - Applied patch.
  - Pulled and pushed over SSH.
  - Pulled and pushed over HTTP.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11940

Differential Revision: https://secure.phabricator.com/D16988
2016-12-05 12:45:30 -08:00
epriestley
faf983614c Improve error messages for running git clone against a Mercurial repository
Summary:
Fixes T11938.

Note that there's a subcase here: if you `hg clone` or `svn checkout` a short `/source/` URI that ends in `.git`, we miss the lookup and don't get this far, so you still get a generic error message.

Hopefully it is clear enough on its own that `proto://.../blah.git` is, in fact, a Git repository, since it says ".git" at the end.

If that doesn't prove to be true, we can be more surgical about this.

Test Plan:
```
$ git clone ssh://local@localvault.phacility.com/source/quack.notgit/
Cloning into 'quack.notgit'...
phabricator-ssh-exec: This repository ("quack.notgit") is not a Git repository. Use "hg" to interact with this repository.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
```

```
$ hg clone ssh://local@localvault.phacility.com/source/phabx
remote: phabricator-ssh-exec: This repository ("phabx") is not a Mercurial repository. Use "git" to interact with this repository.
abort: no suitable response from remote hg!
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11938

Differential Revision: https://secure.phabricator.com/D16976
2016-12-02 07:30:03 -08:00
epriestley
99c6b53ab2 Explicitly update the repository URI index after making a URI edit
Summary:
Fixes T11936. After editing a repository URI, we were not correctly updating the URI index.

Any other edit to the repository //would// update the index, and this index is only really used by `arc` to figure out which repository a working copy belongs to, so that's how this evaded detection for this long. In particular, creating a repository would usually have an edit after any URI edits, to activate it, which would build the index correctly.

Test Plan:
  - Added a new URI to a repository.
  - Verified it was immediately reflected in the `repository_uriindex` table.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11936

Differential Revision: https://secure.phabricator.com/D16972
2016-12-01 14:29:39 -08:00
epriestley
0ed767b967 Fix a couple of partition migration bugs
Summary:
Ref T11044. Few issues here:

  - The `PhutilProxyException` is missing an argument (hit this while in read-only mode).
  - The `$ref_key` is unused.
  - When you add a new master to an existing cluster, we can incorrectly apply `.php` patches which we should not reapply. Instead, mark them as already-applied.

Test Plan:
  - Poked this locally, but will initialize `secure004` as an empty master to be sure.

Reviewers: chad, avivey

Reviewed By: avivey

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16916
2016-11-22 10:57:24 -08:00
epriestley
bf1cbc2499 Don't let users pick "whatever.git" as a repository short name, make "." work
Summary:
Fixes T11902.

  - Periods now work in short names.
  - If you try to name something ".git", no dice.

Test Plan:
  - Tried to name something "quack.git", was politely rejected.
  - Named something "quack.notgit", and it worked fine.
  - Cloned Mercurial and Git repositories over SSH with ".git" and non-".git" variants without hitting any issues.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11902

Differential Revision: https://secure.phabricator.com/D16908
2016-11-21 15:47:20 -08:00
Aviv Eyal
d5a72ca98e Don't show "clone-name" as "Short Name"
Summary: See D16851 - there's now a difference in their meaning, so don't unite them in the UI.

Test Plan: Load manage page of repos

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D16858
2016-11-14 22:46:40 +00:00
epriestley
6a62fca950 Support slightly prettier repository URIs in Diffusion
Summary: Fixes T4245. When a repository has a short name, use `/source/shortname/` as its primary URI.

Test Plan:
  - Cloned Git repositories from shortnames via HTTP and SSH.
  - Cloned Mercurial repositories from shortnames via HTTP and SSH.
  - Cloned Subversion repositories from shortnames via SSH.
  - Browsed Git, Mercurial and Subversion repositories.
  - Added and removed short names to various repositories.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4245

Differential Revision: https://secure.phabricator.com/D16851
2016-11-13 12:42:12 -08:00
epriestley
ff677c1964 Fix two error strings in the diffusion.uri.edit Conduit method
Summary: Fixes T11839. Both are missing a parameter and one is a copy/paste slop.

Test Plan:
{F1913812}

{F1913813}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11839

Differential Revision: https://secure.phabricator.com/D16837
2016-11-10 08:55:12 -08:00
Aviv Eyal
e634812a6d Remove plain-text file view of Diffusion files.
Summary:
fixes T11792.
There's no good reason any more to have this option, so just drop it.

Test Plan: Load a file, toggle remaining "blame" button. Load search results page and an image too, which are serviced by the same controller.

Reviewers: chad, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T11792

Differential Revision: https://secure.phabricator.com/D16833
2016-11-10 00:40:09 +00:00
epriestley
272046ae77 Write a basic SSH pull log for Git
Summary: Ref T11766. When users run `git pull` or similar, log the operation in the pull log.

Test Plan: Performed SSH pulls, got a log in the database. Today, this event log is purely diagnostic and has no UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11766

Differential Revision: https://secure.phabricator.com/D16738
2016-10-20 13:39:30 -07:00
Nevogd
c7a6cfd87c Fix 'Branches' typo in ActionsManagementPanel
Summary:
Fix typo 'Branches' in the panel header for the Diffusion Actions
management panel.

Test Plan: Saw 'Actions' in the panel heading

Reviewers: chad, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D16654
2016-10-03 10:14:30 -07:00
epriestley
9329e6a12d Stop doing an excessive amount of work in diffusion.rawdiffquery
Ref T11665.

Without `-n 1`, this logs the ENTIRE history of the repository. We
actually get the right result, but this is egregiously slow. Add `-n 1`
to return only one result.

It appears that I wrote this wrong way back in 2011, in D953. This
query is rarely used (until recently) which is likely why it has
escaped notice for so long.

Test Plan: Used Conduit console to execute `diffusion.rawdiffquery`.
Got the same results but spent 8ms instead of 200ms executing this
command, in a very small repository.
2016-09-20 06:00:31 -07:00
epriestley
d3280c406d When repositories hit pull errors, stop updating them as frequently
Summary:
Ref T11665. Currently, when a repository hits an error, we retry it after 15s. This is correct if the error was temporary/transient/config-related (e.g., bad network or administrator setting up credentials) but not so great if the error is long-lasting (completely bad authentication, invalid URI, etc), as it can pile up to a meaningful amount of unnecessary load over time.

Instead, record how many times in a row we've hit an error and adjust backoff behavior: first error is 15s, then 30s, 45s, etc.

Additionally, when computing the backoff for an empty repository, use the repository creation time as though it was the most recent commit. This is a good proxy which gives us reasonable backoff behavior.

This required removing the `CODE_WORKING` messages, since they would have reset the error count. We could restore them (as a different type of message), but I think they aren't particularly useful since cloning usually doesn't take too long and there's more status information avilable now than there was when this stuff was written.

Test Plan:
  - Ran `bin/phd debug pull`.
  - Saw sensible, increasing backoffs selected for repositories with errors.
  - Saw sensible backoffs selected for empty repositories.

Reviewers: chad

Maniphest Tasks: T11665

Differential Revision: https://secure.phabricator.com/D16575
2016-09-19 17:29:56 -07:00
Mukunda Modell
c0bf08058b Check for empty output from git ls-tree
Summary: Fixes T10155

Test Plan: View an empty repository in diffusion, check for the exception.
See T10155 for steps to reproduce

Reviewers: epriestley

Subscribers:
2016-09-10 06:02:48 -05:00
epriestley
8d048f06ab Fix a Herald issue where testing commits against rules with revision-related conditions would fail
Summary:
Fixes T11610. Clean up some sketchy old code from long ago.

If you had rules that use conditions like "Accepted revision exists" and ran them in the test console, we'd never load the "CommitData" and fatal.

Instead, load CommitData in `newTestAdapter()` and generally make these pathways a little more modern.

Test Plan:
  - Wrote an "Accepted Revision Exists" rule.
  - Ran a commit in the test console.
  - Before patch, got fatal from T11610.
  - After patch, got clean test result.
  - Also pushed a commit and reviewed the transcript to make sure the rule ran properly.

Reviewers: joshuaspence, chad

Reviewed By: chad

Maniphest Tasks: T11610

Differential Revision: https://secure.phabricator.com/D16522
2016-09-08 17:16:40 -07:00
epriestley
4dc37bcee0 Ignore repository versions on inactive devices in "Repository Servers" panel in Config
Summary:
Fixes T11590. Currently, we incorrectly consider cluster repository versions that are (or were) on devices which are no longer part of the active cluster service when building this status screen.

Instead, ignore them. This is just a display bug; the actual `ClusterEngine` already had similar logic.

Test Plan:
  - Added a bad leader record to `repository_workingcopyversion`.
  - Before patch, got a bad "Partial (1w)" sync:

{F1802292}

  - After patch, got a good "Sycnchronized":

{F1802293}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11590

Differential Revision: https://secure.phabricator.com/D16492
2016-09-05 11:10:16 -07:00
epriestley
c55de86f0e Return Diffusion diffs through Files, not directly over Conduit
Summary:
Fixes T10423. Ref T11524. This changes `diffusion.rawdiffquery` to return a file PHID instead of a blob of data.

This is better in general, but particularly better for huge diffs (as in T10423) and diffs with non-utf8 data (as in T10423).

Test Plan:
  - Used `bin/differential extract` to extract a latin1 diff, got a clean diff.
  - Used `bin/repository reparse --herald` to rerun herald on a latin1 diff, got a clean result.
  - Pushed latin1 diffs to test commit hooks.
  - Triggered the the too large / too slow logic.
  - Viewed latin1 diffs in Diffusion.
  - Used "blame past this change" in Diffusion to hit the `before` logic.

Reviewers: chad

Reviewed By: chad

Subscribers: eadler

Maniphest Tasks: T10423, T11524

Differential Revision: https://secure.phabricator.com/D16460
2016-08-27 09:11:03 -07:00
epriestley
771579496f Make logic for streaming VCS stuff directly to Files more reusable
Summary:
Ref T11524. Ref T10423. Earlier, I converted `diffusion.filecontentquery` to put the actual file content in Files, then return a PHID for the file, instead of trying to send the content over Conduit.

In T11524, we have a similar set of problems with diffs that contain non-UTF8 data (and, in T10423, diffs that are simply enormous).

I want to provide an API method to do the same sort of thing with diff output (like from `git diff`), so we call the method, it shoves the data in Files, and then we go pull it out of Files.

To support this, take the "shove the output of a Future into Files" logic and put it in a new base `FileFuture` query. This will let me make `RawDiffQuery` share the logic more easily.

Test Plan: Browsed Diffusion, ran `diffusion.filecontentquery` to fetch file content.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10423, T11524

Differential Revision: https://secure.phabricator.com/D16458
2016-08-27 09:10:20 -07:00
epriestley
be235301d0 When commits have a "rewritten" hint, try to show that in handles in other applications
Summary:
Ref T11522. This tries to reduce the cost of rewriting a repository by making handles smarter about rewritten commits.

When a handle references an unreachable commit, try to load a rewrite hint for the commit. If we find one, change the handle name to "OldHash > NewHash" to provide a strong hint that the commit was rewritten and that copy/pasting the old hash (say, to the CLI) won't work.

I think this notation isn't totally self-evident, but users can click it to see the big error message on the page, and it's at least obvious that something weird is going on, which I think is the important part.

Some possible future work:

  - Not sure this ("Recycling Symbol") is the best symbol? Seems sort of reasonable but mabye there's a better one.
  - Putting this information directly on the hovercard could help explain what this means.

Test Plan: {F1780719}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11522

Differential Revision: https://secure.phabricator.com/D16437
2016-08-24 09:35:19 -07:00
epriestley
498fb33103 When a commit has a "rewritten" hint, show it in the UI instead of the generic "deleted" message
Summary:
Ref T11522. When a commit is no longer reachable from any branch/tag, we currently show a "this has been deleted" message.

Instead, go further: check if there is a "rewritten" hint pointing at a commit the current commit was rewritten into. If we find one, show a message about that instead.

(This isn't super pretty, just getting it working for now. I expect to revisit this UI in T9713 if we don't get to it before that.)

Test Plan: {F1780703}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11522

Differential Revision: https://secure.phabricator.com/D16436
2016-08-24 09:33:25 -07:00
epriestley
e4c4724afd Migrate the "badcommit" table to use the less-hacky "hint" mechanism
Summary: Ref T11522. This migrates any "badcommit" data (which probably only exists at Facebook and on 1-2 other installs in the wild) to the new "hint" table.

Test Plan:
  - Wrote some bad commit annotations to the badcommit table.
  - Viewed them in the web UI and used `bin/repository reparse --change ...` to reparse them. Saw "this is bad" messages.
  - Ran migration, verified that valid "badcommit" rows were successfully migrated to become "hint" rows.
  - Viewed the new web UI and re-parsed the change, saw "unreadable commit" messages.
  - Viewed a good commit; reparsed a good commit.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11522

Differential Revision: https://secure.phabricator.com/D16435
2016-08-24 09:32:59 -07:00
epriestley
8a4fbcd8c0 Provide a new "hint" table for weird commits (rewritten, unreadable)
Summary:
Ref T11522. This provides storage for tracking rewritten commits (new feature) and unreadable commits (existing feature, but really hacky).

This doesn't do anything yet, just adds a table and a CLI tool for updating it. I'll document the tool once it works. You just pipe in some JSON, but I need to document the format.

Test Plan:
  - Piped JSON for "none", "rewritten" and "unreadable" hints into `bin/repository hint`.
  - Examined the database to see that the table was written properly.
  - Tried to pipe bad JSON in, invalid hint types, etc. Got reasonable human-readable error messages.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11522

Differential Revision: https://secure.phabricator.com/D16434
2016-08-24 09:31:46 -07:00
epriestley
f659b8743a Fix Herald test adapter for commits
Summary:
Fixes T11488. I broke this in D16360, I think by doing a little extra refactoring after testing it.

This code is very old, before commits always needed to have repositories attached in order to do policy checks.

Modernize it by mostly just using the repository which is present on the Commit object, and using the existing edge cache.

Test Plan: Ran a commit through the Herald test adapter.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11488

Differential Revision: https://secure.phabricator.com/D16413
2016-08-17 09:02:53 -07:00
epriestley
39d4e21eec Fix a bad DiffusionCommandEngine parameter from HTTPEngine conversion
Summary:
I converted this call incorrectly in D16092. We should pass the `PhutilURI` object, not the string version of it.

Specifically, this resulted in hitting an error like this if a replica needed synchronization:

```
[2016-08-11 21:22:37] EXCEPTION: (InvalidArgumentException) Argument 1 passed to DiffusionCommandEngine::setURI() must be an instance of PhutilURI, string given, called in...
#0 PhutilErrorHandler::handleError(integer, string, string, integer, array) called at [<phabricator>/src/applications/diffusion/protocol/DiffusionCommandEngine.php:52]
#1 DiffusionCommandEngine::setURI(string) called at [<phabricator>/src/applications/diffusion/protocol/DiffusionRepositoryClusterEngine.php:601]
...
```

Test Plan: Clusterized an observed repository, demoted a node, ran `bin/repository update Rxxx` to update, saw no typehint fatal.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D16390
2016-08-11 16:41:09 -07:00
epriestley
4d68c0ae04 Make Herald test workflow modular and more clear
Summary:
Fixes T9719. Currently, the Herald "Test Console" has a big `instanceof` thing, so new adapters (like a Calendar adapter, or third-party adapters) aren't available automatically. Instead, do a standard modular thing: load the available adapters, ask which ones can test the object the user selected, then let the user pick which one they want to move forward with.

Additionally, it isn't very clear that you can't test "commit hook" rules because they rely on push state which we don't really have a good way to simulate. When the user picks a commit, we now show them the "Hook" events, but the options are disabled and explain why they can not be selected.

Test Plan:
 - Ran test rules for revisions, commits, mocks, tasks, wiki documents, questions, and outbound mail.
 - Plugged in a commit, got a more-helpful choice screen explaining why you do a test run of hook rules.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9719

Differential Revision: https://secure.phabricator.com/D16360
2016-08-03 16:12:33 -07:00
epriestley
d44a5fa933 In Git, only use "--find-copies-harder" on small diffs
Summary:
Ref T10423. This flag can cause `git diff` to take an enormously long time (the problem case was a 5M line, 20K file commit).

Instead:

  - Run without the flag first.
  - If that shows that the diff is definitely small, try again with the flag.
  - If that works, return the slower, better output.
  - If the fast diff affects too many paths or generating the slow diff takes too long, return the faster, slightly worse output.

The quality of the output differs in how well Git is able to detect "M" and "C" (moves and copies of files).

For example, if you copy `src/` to `srcpro/`, the fast output may not show that you copied files. The slow output will.

I think this is rarely useful for large copies anyway: it's interesting if a 1-2 file diff is a copy, but usually obvious/uninteresting if a 500-file diff is a copy.

Test Plan:
  - Ran `bin/repository reparse --change rXnnn` on Git changes.
  - Saw fast and slow commands execute normally.
  - Tried on a large diff, saw only the fast command execute.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10423

Differential Revision: https://secure.phabricator.com/D16266
2016-07-10 08:03:57 -07:00
epriestley
a5b26104f6 Fix an issue with creating new Repository URIs via the Web UI
Summary I broke this in D16237: that made the CLI workflow work, but we attach the repository earlier in the web workflow and won't have one when we arrive here.

Test Plan: Created a new repository URI from the web UI.

Auditors: chad
2016-07-09 05:55:45 -07:00
epriestley
5c8dabdf80 Add a strong hint about importing or observing repositories to repository creation
Summary: Fixes T11278. Also mention `svnsync`, since we have some evidence that it works.

Test Plan: {F1716250}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11278

Differential Revision: https://secure.phabricator.com/D16255
2016-07-08 07:43:34 -07:00
epriestley
921d56efb0 Make repository URI creation work regardless of "repository" transaction order
Summary: Fixes T11276. This feels slightly iffy (we `attachRepository()` here, and also when applying the TYPE_REPOSITORY transaction) but simpler than trying to reorder things.

Test Plan: Created a repository URI with transactions in `["uri", "repository"]` order.

Reviewers: chad, avivey

Reviewed By: avivey

Maniphest Tasks: T11276

Differential Revision: https://secure.phabricator.com/D16237
2016-07-05 16:45:33 -07:00
epriestley
2a1393c008 Fix impropery history graph trace in Mercurial
Summary: Fixes T11267. This data was coming back weird (in reverse order relative to the graph itself). Previously it worked OK anyway, but the new logic is a little more sensitive to the input.

Test Plan: Viewed a Mercurial repository with linear history, saw linear history.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11267

Differential Revision: https://secure.phabricator.com/D16229
2016-07-04 10:24:14 -07:00
epriestley
d7b4c50941 Fix a flipped higlight vs no-highlight condition
Ref T11257.

Auditors: chad
2016-07-02 05:22:55 -07:00
epriestley
498cb5c096 Fix an XSS issue where Diffusion files exceeding the highlighting byte limit were not properly escaped
Fixes T11257.

Auditors: chad
2016-07-02 05:17:05 -07:00
epriestley
dc37789d53 Build that thing someone posted a screenshot of on Facebook
Summary: Seemed kinda cool.

Test Plan: {F1707244}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D16210
2016-07-01 04:36:24 -07:00
epriestley
dc9283b85d Convert all standard relationship-editing actions to modern Relationships code
Summary: Ref T4788. This moves everything except "merge" to the new code.

Test Plan:
  - Edited relationships in Differential, Diffusion, and Pholio.
  - Uninstalled Pholio, made sure "Edit Mocks..." actions vanished.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4788

Differential Revision: https://secure.phabricator.com/D16193
2016-06-29 11:24:52 -07:00
Aviv Eyal
de6349dd67 Revision substate CLOSED_FROM_ACCEPTED
Summary:
Ref T9838.

Add a Properties field to Revision, and update a `wasAcceptedBeforeClose` when closing a revision.

Test Plan:
A quick run through the obvious steps (Close with commit/manually,  with or w/o accept) and calling `differential.query` shows the `wasAcceptedBeforeClose` property was setup correctly.

Pushing closed + accepted passes the relevant herald, which was my immediate issue; Pushing un-accepted is blocked.
Test the "commit" rule (Different from "pre-commit") by hacking the DB and running the "has accepted revision" rule in a test-console.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T9838

Differential Revision: https://secure.phabricator.com/D15085
2016-06-27 20:29:47 +00:00
epriestley
89f9f97159 Provide basic support for Subversion revprops
Summary:
Ref T11208. See that task for a more detailed description of revprops.

This allows revprop changes in a hosted Subversion repository if the repository has the "allow dangerous changes" flag set.

In the future, we could expand this into real Herald support, but the only use case we have for now is letting `svnsync` work.

Test Plan:
Edited revprops with `svn propset --revprop -r 2 propkey propvalue repositoryuri`:

  - Tried before patch, got a "configure a commit hook" error.
  - Tried after patch, got a "dangerous change" error.
  - Allowed dangerous changes.
  - Did a revprop edit.
  - Prevented dangerous changes.
  - Got an error again.
  - Made a normal commit to an SVN repository.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11208

Differential Revision: https://secure.phabricator.com/D16174
2016-06-24 13:43:32 -07:00
epriestley
6f275ba144 Render browse results with global result style
Summary:
Ref T11034. This seems a little more promising. Two problems at the moment:

  - This doesn't actually provide any useful information at all right now.
  - Many object types have no profile images.

Test Plan:
{F1695254}

{F1695255}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11034

Differential Revision: https://secure.phabricator.com/D16155
2016-06-20 16:49:02 -07:00
epriestley
28eb562899 Ignore unrecognized refs in "refs/remotes/"
Summary: Ref T9028. When selecting refs, pretend refs in "refs/remotes/" that we don't otherwise recognize don't exist, since it looks like these are probably remotes //of the remote// we're observing, and who knows what state they're in.

Test Plan: Used `bin/repository discover --verbose` to verify that these named refs no longer appear in the list.

Reviewers: chad, joshuaspence

Reviewed By: joshuaspence

Maniphest Tasks: T9028

Differential Revision: https://secure.phabricator.com/D16136
2016-06-16 16:03:36 -07:00
epriestley
7c8f9d7ba2 Don't track "phabricator/" staging area tags
Summary: Ref T9028. Ref T6878. This rule should probably be refined in the long term, but for now just ignore "phabricator/diff/12424" and similar staging area tags.

Test Plan: Ran `bin/repository discover --verbose` on a repository with staging area refs, saw Phabricator ignore those refs as untracked.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6878, T9028

Differential Revision: https://secure.phabricator.com/D16134
2016-06-16 11:22:02 -07:00
epriestley
ec89c7d63e Add an "Unreachable" flag for commits and revive them during discovery
Summary:
Ref T9028. This is the easy part of dealing with deleted commits:

  - Add a flag for unreachable commits (nothing sets this flag yet).
  - Ignore unreachable commits when querying for known commits during discovery, so we pretend they do not exist.
  - When recording a commit, try just reviving an existing unreachable commit first. If that works, bail out.

Test Plan:
  - Artificially marked a commit as unreachable with raw SQL.
  - Verified it said "deleted: unreachable" in the UI.
  - Ran `repository discover --trace --verbose`.
  - Saw the discovery process ignore the commit when filling the cache.
  - Saw the discovery process revive the commit instead of trying to record it again.
  - Web UI now shows the commit as normal.
  - Running `repository discover` again doesn't make any further changes.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9028

Differential Revision: https://secure.phabricator.com/D16130
2016-06-16 11:20:37 -07:00
epriestley
2949905c04 Fetch and discover all Git ref types, not just branches
Summary:
Ref T9028. Fixes T6878. Currently, we only fetch and discover branches. This is fine 99% of the time but sometimes commits are pushed to just a tag, e.g.:

```
git checkout <some hash>
nano file.c
git commit -am '...'
git tag wild-wild-west
git push origin wild-wild-west
```

Through a similar process, commits can also be pushed to some arbitrary named ref (we do this for staging areas).

With the current rules, we don't fetch tag refs and won't discover these commits.

Change the rules so:

  - we fetch all refs; and
  - we discover ancestors of all refs.

Autoclose rules for tags and arbitrary refs are just hard-coded for now. We might make these more flexible in the future, or we might do forks instead, or maybe we'll have to do both.

Test Plan:
Pushed a commit to a tag ONLY (`vegetable1`).

<cf508b8de6>

On `master`, prior to the change:

  - Used `update` + `refs` + `discover`.
  - Verified tag was not fetched with `git for-each-ref` in local working copy and the web UI.
  - Verified commit was not discovered using the web UI.

With this patch applied:

  - Used `update`, saw a `refs/*` fetch instead of a `refs/heads/*` fetch.
  - Used `git for-each-ref` to verify that tag fetched.
  - Used `repository refs`.
  - Saw new tag appear in the tags list in the web UI.
  - Saw new refcursor appear in refcursor table.
  - Used `repository discover --verbose` and examine refs for sanity.
  - Saw commit row appear in database.
  - Saw commit skeleton appear in web UI.
  - Ran `bin/phd debug task`.
  - Saw commit fully parse.

{F1689319}

Reviewers: chad

Reviewed By: chad

Subscribers: avivey

Maniphest Tasks: T6878, T9028

Differential Revision: https://secure.phabricator.com/D16129
2016-06-16 11:20:05 -07:00
Shijie Feng
aaf3698666 Add datasources to allow search revisions by project.
Summary:
When having lots of repos, seeing "all revisions in this project" is hard, and we ended up adding herald rules to basically copy project tags to the revisions on a per-project basis. Adding a "tagged: project" function to the Repositories search field allows users to find differentials within a project.

Fix T10850.

Test Plan: search differentials by tagging project and repository in the Repository field

Reviewers: avivey, epriestley, #blessed_reviewers

Reviewed By: avivey, epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T10850

Differential Revision: https://secure.phabricator.com/D16096
2016-06-13 18:08:44 +00:00
epriestley
a5e29f3ffa Fix an ancient ad-hoc string truncation
Summary: Fixes T11139. We missed this years ago when we moved to PhutilUTF8StringTruncator.

Test Plan: {F1686072}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11139

Differential Revision: https://secure.phabricator.com/D16105
2016-06-13 10:16:25 -07:00
epriestley
55a698a28a Use HTTPEngineExtension proxy for git HTTP operations
Summary: Ref T10227. When we perform `git` http operations (fetch, mirror) check if we should use a proxy; if we should, set `http_proxy` or `https_proxy` in the environment to make `git` have `curl` use it.

Test Plan:
  - Configured a proxy extension to run stuff through a local instance of Charles.
  - Ran `repository pull` and `repository mirror`.
  - Saw `git` HTTP requests route through the proxy.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10227

Differential Revision: https://secure.phabricator.com/D16092
2016-06-09 12:17:10 -07:00
epriestley
421bf2e548 Allow administrators to configure global default settings
Summary:
Ref T4103. This just adds a single global default setting group, not full profiles.

Primarily, I'm not sure how administrators are supposed to set profiles for users, since most ways user accounts get created don't really support setting roles.. When we figure that out, it should be reasonably easy to extend this. There also isn't much of a need for this now, since pretty much everyone just wants to turn off mail.

Test Plan:
  - Edited personal settings.
  - Edited global settings.
  - Edited a bot's settings.
  - Tried to edit some other user's settings.
  - Saw defaults change appropriately as I edited global and personal settings.

{F1677266}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16048
2016-06-05 13:15:06 -07:00
epriestley
fc45de29a6 Modernize various menu collapse settings
Summary: Ref T4103. Fully modernize the filetree show/hide, durable column show/hide, and profile menu collapse/wide settings.

Test Plan:
  - Toggled filetree on/off, reloaded page, setting stuck.
  - Same with conpherence column and profile menus.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16034
2016-06-04 14:44:36 -07:00
epriestley
5c8ff3d37c Convert Diffusion blame and color into standard internal settings
Summary: Ref T4103. Modernize the blame/color toggles in Diffusion. These have no separate settings UI.

Test Plan: Toggled blame and colors, reloaded pages, settings stuck.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16026
2016-06-04 14:41:49 -07:00
epriestley
eb673fd783 Formalize and fully modularize settings panel groups
Summary:
Ref T4103. Settings panels are grouped into categories of similar panels (like "Email" or "Sessions and Logs").

Currently, this is done informally, by just grouping and ordering by strings. This won't work well with translations, since it means the ordering is entirely dependent on the language order, so the first settings panel you see might be something irrelvant or confusing. We'd also potentially break third-party stuff by changing strings, but do so in a silent hard-to-detect way.

Provide formal objects and modularize the panel groups completely.

Test Plan: Verified all panels still appear properly and in the same groups and order.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16020
2016-06-04 14:39:11 -07:00
epriestley
edfc6a6934 Convert some loadPreferences() to getUserSetting()
Summary:
Ref T4103. This doesn't get everything, but takes care of most of the easy stuff.

The tricky-ish bit here is that I need to move timezones, pronouns and translations to proper settings. I expect to pursue that next.

Test Plan:
- Grepped for `loadPreferences` to identify callsites.
- Changed start-of-week setting, loaded Calendar, saw correct start.
- Visited welcome page, read "Adjust Settings" point.
- Loaded Conpherence -- I changed behavior here slightly (switching threads drops the title glyph) but it wasn't consistent to start with and this seems like a good thing to push to the next version of Conpherence.
- Enabled Filetree, toggled in Differential.
- Disabled Filetree, no longer visible in Differential.
- Changed "Unified Diffs" preference to "Small Screens" vs "Always".
- Toggled filetree in Diffusion.
- Edited a task, saw sensible projects in policy dropdown.
- Viewed user profile, uncollapsed/collapsed side nav, reloaded page, sticky'd.
- Toggled "monospaced textareas", used a comment box, got appropriate fonts.
- Toggled durable column.
- Disabled title glyphs.
- Changed monospaced font to 18px/36px impact.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16004
2016-06-02 06:29:20 -07:00
epriestley
f5f784f4c1 Version clustered, observed repositories in a reasonable way (by largest discovered HEAD)
Summary:
Ref T4292. For hosted, clustered repositories we have a good way to increment the internal version of the repository: every time a user pushes something, we increment the version by 1.

We don't have a great way to do this for observed/remote repositories because when we `git fetch` we might get nothing, or we might get some changes, and we can't easily tell //what// changes we got.

For example, if we see that another node is at "version 97", and we do a fetch and see some changes, we don't know if we're in sync with them (i.e., also at "version 97") or ahead of them (at "version 98").

This implements a simple way to version an observed repository:

  - Take the head of every branch/tag.
  - Look them up.
  - Pick the biggest internal ID number.

This will work //except// when branches are deleted, which could cause the version to go backward if the "biggest commit" is the one that was deleted. This should be OK, since it's rare and the effects are minor and the repository will "self-heal" on the next actual push.

Test Plan:
  - Created an observed repository.
  - Ran `bin/repository update` and observed a sensible version number appear in the version table.
  - Pushed to the remote, did another update, saw a sensible update.
  - Did an update with no push, saw no effect on version number.
  - Toggled repository to hosted, saw the version reset.
  - Simulated read traffic to out-of-sync node, saw it do a remote fetch.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15986
2016-05-30 09:53:01 -07:00
epriestley
e81637a6c6 Fix some issues with the "Explain Why" dialog
Summary:
Ref T11051. This is still not as clear as it should be, but is at least working as intended now.

I believe this part of the code just never worked. The test plan on D10489 didn't specifically cover it.

Test Plan:
Did this sort of thing in a repository:

```
$ git checkout -b featurex
$ echo x >> y
$ git commit -am wip
$ arc diff
```

Then I simulated just pushing it (this flow is a little more involved than necessary):

```
$ arc land --hold
$ git commit --amend
$ # remove all metadata -- particularly, "Differential Revision"!
$ git push HEAD:master
```

I got a not-great but more-useful dialog:

{F1667318}

Prior to this change, the hash match was incorrectly not reported at all.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11051

Differential Revision: https://secure.phabricator.com/D15989
2016-05-30 09:52:35 -07:00