1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-02 03:32:42 +01:00
Commit graph

1709 commits

Author SHA1 Message Date
epriestley
315d857a8a Add a basic web UI for intracluster sync logs
Summary: Depends on D19798. Ref T13216. This puts at least a basic UI on top of sync logs.

Test Plan:
Viewed logs from the web UI and exported data. Note that these syncs are somewhat simulated since I my local cluster is somewhat-faked (i.e., not actually multiple machines).

{F5995899}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19799
2018-11-10 04:58:36 -08:00
epriestley
1d7c960531 Put push log "hookWait" to data export and add all wait values to UI
Summary:
Depends on D19797. Ref T13216.

  - Put the new `hookWait` in the export and the UI.
  - Put the existing waits in the UI, not just the export.
  - Make order consistent: host, write, read, hook (this is the order the timers start in).

Test Plan: Pushed some stuff, viewed web UI and saw sensible numbers, exported data and got the same values.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19798
2018-11-10 04:47:38 -08:00
epriestley
c32fa06266 Use phutil_microseconds_since(...) to simplify some timing arithmetic
Summary: Depends on D19796. Simplify some timing code by using phutil_microseconds_since() instead of duplicate casting and arithmetic.

Test Plan: Grepped for `1000000` to find these. Pulled, pushed, made a conduit call. This isn't exhaustive but it should be hard for these to break in a bad way since they're all just diagnostic.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19797
2018-11-08 16:46:32 -08:00
epriestley
b12e92e6e2 Add timing information for commit hooks to push logs
Summary:
Depends on D19779. Ref T13216. The push logs currently record the "hostWait", which is roughly "locking + subprocess cost". We also record locking separately, so we can figure out "subprocess cost" alone by subtracting the lock costs.

However, the subprocess (normally `git receive-pack`) runs hooks, and we don't have an easy way to figure out how much time was spent doing actual `git` stuff vs spent doing commit hook processing. This would have been useful in diagnosing at least one recent issue.

Track at least a rough hook cost and record it in the push logs.

Test Plan: Pushed to a repository, saw a reasonable hook cost appear in the database table.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19780
2018-11-08 06:00:26 -08:00
epriestley
966db4d38e Add an intracluster synchronization log for cluster repositories
Summary:
Depends on D19778. Ref T13216. See PHI943, PHI889, et al.

We currently have a push log and a pull log, but do not separately log intracluster synchronization events. We've encountered several specific cases where having this kind of log would be helpful:

  - In PHI943, an install was accidentally aborting locks early. Having timing information in the sync log would let us identify this more quickly.
  - In PHI889, an install hit an issue with `MaxStartups` configuration in `sshd`. A log would let us identify when this is an issue.
  - In PHI889, I floated a "push the linux kernel + fetch timeout" theory. A sync log would let us see sync/fetch timeouts to confirm this being a problem in practice.
  - A sync log will help us assess, develop, test, and monitor intracluster routing sync changes (likely those in T13211) in the future.

Some of these events are present in the pull log already, but only if they make it as far as running a `git upload-pack` subprocess (not the case with `MaxStartups` problems) -- and they can't record end-to-end timing.

No UI yet, I'll add that in a future change.

Test Plan:
  - Forced all operations to synchronize by adding `|| true` to the version check.
  - Pulled, got a sync log in the database.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19779
2018-11-07 18:24:20 -08:00
epriestley
e09d29fb1a Clean up the workflow for some post-push logging code
Summary:
Ref T13216. When a repository is clustered, we run this cleanup code (to tell the repository to update, and log some timing information) on both nodes. Currently, we do slightly too much work, which is unnecessary and can be a bit confusing to human readers.

The double update message doesn't hurt anything, but there's no reason to write it twice.

Likewise, the second timing information update query doesn't do anything: there's no PushEvent object with the right identifier, so it just updates nothing. We don't need to run it, and it's confusing that we do.

Instead, only do these writes if we're actually the final node with the repository on it.

Test Plan: Added some logging, saw double writes/updates before the change and no doubles afterwards, with no other behavioral changes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19778
2018-11-07 17:46:50 -08:00
epriestley
24a061f844 Correct an ambiguous regexp in DiffusionRequest
Summary:
See <https://discourse.phabricator-community.org/t/diffusionrequest-regex-error/2057/>.

The intent of `[\d-,]` is "digits, hyphen, and comma" but `[x-y]` means "character range x-y".

Specify `[\d,-]` instead to disambiguate the hyphen as "literal hyphen", not a character range marker.

Test Plan: I can't reproduce the original error as reported, but browsed around Diffusion for a bit.

Reviewers: amckinley, avivey

Reviewed By: avivey

Differential Revision: https://secure.phabricator.com/D19770
2018-11-01 20:01:39 -07:00
epriestley
65e953658a Expose Audit actions for "transaction.search" in a basic way
Summary: Ref T13210. See PHI841. This mirrors D19509 for Differential.

Test Plan: Called `transaction.search` on a commit with a bunch of audit activity, got appropriate labels in the results.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19760
2018-10-27 07:19:50 -07:00
epriestley
61ec434208 Remove unicode marks for "Accept/Raise Concern" in Audit
Summary:
Ref T13210. The comment action dropdown for audits has a heavy checkmark next to "Accept" and a heavy "X" next to "Raise Concern".

We previously removed similar marks in Differential in D19405 and that seems to have gone fine. For consistency, remove these too.

Test Plan: Viewed the comment action dropdown, no longer saw checkmark and X-mark.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19759
2018-10-27 07:19:18 -07:00
epriestley
bc6c8c0e93 Explicitly shuffle nodes before selecting one for cluster sync
Summary:
Depends on D19734. Ref T13202. Ref T13109. Ref T10884. See PHI905. See PHI889. We currently rank cluster nodes in three cases:

  # when performing a write, we can go to any node (D19734 should make our ranking good);
  # when performing a read, we can go to any node (currently random, but T10884 discusses ideas to improve our ranking);
  # when performing an internal synchronization before a read or a write, we must go to an up-to-date node.

Currently, case (3) is not-exactly-deterministic but not random, and we won't spread intracluster traffic acrosss the cluster evenly if, say, half of it is up to date and half of it is still synchronizing. For a given write, I believe all nodes will tend to synchronize from whichever node first received the write today.

Instead, shuffle the list and synchronize from any up-to-date node.

(I think we could improve upon this only by knowing which nodes actually have load and selecting the least-loaded -- doable, but not trivial.)

Test Plan: Poked at it locally, will deploy to `secure`. This is hard to measure/test terribly convincingly.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202, T13109, T10884

Differential Revision: https://secure.phabricator.com/D19735
2018-10-17 08:11:23 -07:00
Austin McKinley
dbf2302b6c Fix self-cancelling typo
Summary: Ref D18268. This typo cancelled itself out, and I can't find any other callers.

Test Plan: arc unit

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19718
2018-09-28 17:44:33 -07:00
Austin McKinley
8065433ee8 Migrate DiffusionBlameController to use repo identities
Summary:
Now on the blame page, identities get `avatar.png` and there are little tooltips that show a few characters of the committer identity string.

Also add a default icon for repo identities.

Test Plan: Loaded some blame pages for files touched by users with and without repo identities attached.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19587
2018-09-26 14:45:58 -07:00
epriestley
021c612cb2 When we fail to acquire a repository lock, try to provide a hint about why
Summary:
Ref T13202. See PHI889. If the lock log is enabled, we can try to offer more details about lock holders.

When we fail to acquire a lock:

  - check for recent acquisitions and suggest that this is a bottleneck issue;
  - if there are no recent acquisitions, check for the last acquisition and print details about it (what process, how long ago, whether or not we believe it was released).

Test Plan:
  - Enabled the lock log.
  - Changed the lock wait time to 1 second.
  - Added a `sleep(10)` after grabbing the lock.
  - In one window, ran a Conduit call or a `git fetch`.
  - In another window, ran another operation.
  - Got useful/sensible errors for both ssh and web lock holders, for example:

> PhutilProxyException: Failed to acquire read lock after waiting 1 second(s). You may be able to retry later. (This lock was most recently acquired by a process (pid=12609, host=orbital-3.local, sapi=apache2handler, controller=PhabricatorConduitAPIController, method=diffusion.rawdiffquery) 3 second(s) ago. There is no record of this lock being released.)

> PhutilProxyException: Failed to acquire read lock after waiting 1 second(s). You may be able to retry later. (This lock was most recently acquired by a process (pid=65251, host=orbital-3.local, sapi=cli, argv=/Users/epriestley/dev/core/lib/phabricator/bin/ssh-exec --phabricator-ssh-device local.phacility.net --phabricator-ssh-key 2) 2 second(s) ago. There is no record of this lock being released.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19702
2018-09-24 15:20:07 -07:00
epriestley
8268abcb78 Enrich "diffusion.commit.search" with identity, status, and message information
Summary:
Depends on D19657. Ref T13197. See PHI841.

This enriches the results from `diffusion.commit.search` with information similar to the information returned by the "commits" attachment from `differential.diff.search`.

Also include unreachable, imported, message, audit status, and repository PHID.

Test Plan: Called `diffusion.commit.search` and reviewed the results, which looked sensible.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19658
2018-09-12 12:45:44 -07:00
epriestley
c7e7b63f15 Rename "PhabricatorAuditCommitStatusConstants" to "DiffusionCommitAuditStatus"; remove "MODERN_"
Summary:
Depends on D19656. Ref T13197. See PHI851.

  - This class is now a real object, so get rid of the "Constants" part of the name.
  - Rename it for greater consistency with other modern objects.
  - Get rid of the `MODERN_` tag now that the old constants are gone.

Test Plan: Bunch of `grep`, browsed around.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19657
2018-09-12 12:44:43 -07:00
epriestley
d63281cc54 Migrate remaining Audit database status constants
Summary: Depends on D19652. Ref T13197. See PHI851. This migrates the actual `auditStatus` on Commits, and older status transactions.

Test Plan:
  - Ran migrations.
  - Spot-checked the database for sanity.
  - Ran some different queries, got unchanged results from before migration.
  - Reviewed historic audit state transactions, and accepted/raised concern on new audits. All state transactions appeared to generate properly.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19655
2018-09-12 12:21:27 -07:00
epriestley
853a816b3c Continue converting Audit constants, allowing the Query to handle either strings or integers
Summary: Ref T13197. We're almost ready to migrate: let the Query accept either older integer values or new string values. Then move some callsites to use strings.

Test Plan: Called `audit.query`, browsed audits, audited commits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19650
2018-09-10 14:46:47 -07:00
epriestley
bae8a95114 Continue replacing Commit/Audit status checks with object-based checks
Summary: Ref T13195. See PHI851. Continuing down the path toward replacing these legacy numeric constants with more modern string constants.

Test Plan:
- Raised concern, requested verification, verified.
- Looked at commit hovercard with audit status.
- Viewed header on a commit page.
- (Didn't test the Doorkeeper stuff since it requires linking to Asana and seems unlikely to break.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19647
2018-09-10 11:20:31 -07:00
epriestley
16a6fc8341 Allow reviewers to mark their own inlines as "Done" before they submit them
Summary:
Ref T13195. Ref T8573. This allows reviewers to mark their own inline comments as "Done" before they submit them.

If you're leaving a non-actionable comment like "this is good", you can pre-check "Done" to give the author a hint that you don't expect any response.

Test Plan: On revisions and commits, added inlines as the author and a reviewer/auditor. Marked them done/not-done before submitting. As author, marked the not-done ones done after submitting. Checked preivews, toggled done/not done states.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195, T8573

Differential Revision: https://secure.phabricator.com/D19634
2018-09-07 11:17:42 -07:00
epriestley
a1ce23b9f5 Introduce an AuditStatus object for commits and move some callsites to it
Summary:
Ref T13195. See PHI851. Add an object, analogous to the `DifferentialRevisionStatus` object, to handle audit status management.

This will primarily make it easier to swap storage over to strings later, but also cleans things up a bit.

Test Plan: Viewed audit/commit lists, saw sensible state icons. Ran `bin/audit synchronize`, got sensible output.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19646
2018-09-07 10:20:04 -07:00
epriestley
650e74933a Update some inline comment logic to use more modern "Viewer"-oriented calls/variables
Summary:
Ref T13195. Ref T8573. The inline comment controllers currently use outdated `$user = $this->getRequest()->getUser()` calls.

Instead, use `$viewer = $this->getViewer()`.

This is just a small consistency update with no behavioral changes.

Test Plan: Viewed and added inlines in Differential and Diffusion.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195, T8573

Differential Revision: https://secure.phabricator.com/D19633
2018-09-06 07:57:41 -07:00
epriestley
fd0da4c41f Rename "PHUIDocumentViewPro" to "PHUIDocumentView"
Summary: Ref T13077. There is no "PHUIDocumentView" so toss the "Pro" suffix from this classname.

Test Plan: Grepped for `PHUIDocumentView` and `PHUIDocumentViewPro`.

Reviewers: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19616
2018-08-28 14:53:07 -07:00
epriestley
632cafec88 Pass commit authorship information to Buildkite
Summary:
Fixes T12251. Ref T13189. See PHI610. The difficulty here is that we don't want to disclose Phabricator account information to Buildkite. We're comfortable disclosing information from `git`, etc.

  - For commits, use the Identity to provide authorship information from Git.
  - For revisions, use the local commit information on the Diff to provide the Git/Mercurial/etc author of the HEAD commit.

Test Plan:
  - Built commits and revisions in Buildkite via Harbormaster.
  - I can't actually figure out how to see author information on the Buildkite side, but the values look sane when dumped locally.

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13189, T12251

Differential Revision: https://secure.phabricator.com/D19614
2018-08-27 12:52:11 -07:00
epriestley
7ef2bb1b56 Support Mercurial "protocaps" wire command
Summary:
Ref T13187. See PHI834. Mercurial has somewhat-recently (changeset is from Jan 2018) introduced a new "protocaps" command, that appears in Mercurial 4.7 and possibly before then.

We must explicitly enumerate all protocol commands because you can't decode the protocol without knowing how many arguments the command expects, so enumerate it.

(Also fix an issue where the related error message had an extra apostrophe.)

Test Plan:
  - Ran `hg clone ...` with client and server on Mercurial 4.7.
  - Before: fatal on unknown "protocaps" command.
  - Midway: better typography in error message.
  - After: clean clone.

Reviewers: amckinley

Maniphest Tasks: T13187

Differential Revision: https://secure.phabricator.com/D19596
2018-08-23 15:06:25 -07:00
Austin McKinley
5c4c593af3 Update DiffusionLastModifiedController to use identities
Summary: Ref T12164. Updates another controller to use identities.

Test Plan:
Pretty ad-hoc, but loaded the main pages of several different repos with and without repo identities. I'm not totally convinced the `author` from this data structure is actually being used:
```
$return = array(
  'commit'    => $modified,
  'date'      => $date,
  'author'    => $author,
  'details'   => $details,
);
```

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12164

Differential Revision: https://secure.phabricator.com/D19580
2018-08-17 12:24:21 -07:00
Austin McKinley
3b05e920e0 Start changing DiffusionCommitController to use identities
Summary: Depends on D19491.

Test Plan: Viewed some commits where the identity was mapped to a user and another that wasn't; saw the header render either a link to the user or the identity object.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19492
2018-08-13 15:23:31 -07:00
epriestley
3574a55a95 Deprecate Conduit method "diffusion.getrecentcommitsbypath"
Summary:
See D19558. This method has no callers and just wraps `diffusion.historyquery`, since D5960 (2013).

This was introduced in D315 (which didn't make it out of FB, I think) inside Facebook for unclear purposes in 2011.

Test Plan: Grepped for callers, found none.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: artms

Differential Revision: https://secure.phabricator.com/D19559
2018-08-03 09:48:58 -07:00
Arturas Moskvinas
356b2781bc Gracefully fail request if non existing callsign is passed to getrecentcommitsbypath instead of crashing
Summary:
`diffusion.getrecentcommitsbypath` fails with 500 error when non existing callsign is passed:
```
>>> UNRECOVERABLE FATAL ERROR <<<

Call to a member function getCommit() on null

```

Expected Behavior:
Return more graceful error notifying caller that such callsign/repository does not exist

Reproduction steps:
Open conduit: https://secure.phabricator.com/conduit/method/diffusion.getrecentcommitsbypath/
Enter:
callsign: "obviouslynotexisting"
path: "/random"
Click call method

Test Plan: after applying patch - call no longer fails with 500s

Reviewers: Pawka, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19558
2018-08-02 19:49:10 +03:00
epriestley
d8834377be When a Herald rule blocks a push, show which rule fired in the push log UI
Summary:
Ref T13164. See PHI765. We currently show "Rejected: Herald" in the push log UI, but don't show which rule rejected a push.

We store this data, and it's potentially useful: either for hunting down a particular issue, or for getting a general sense of how often a reject rule is triggering (maybe because you want to tune how aggressive it is).

Show this data in the web UI, and include it in the data export payload.

Test Plan:
  - Pushed to a hosted repository so that I got blocked by a Herald rule.
  - Viewed the push logs in the web UI, now saw which rule triggered things.
  - Exported logs to CSV, saw Herald rule PHIDs in the data.

{F5776211}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19555
2018-08-01 17:33:50 -07:00
epriestley
dce6dd5d02 Add an explicit "null" to a missed diffusion.branchquery callsite to fix Diffusion "Branches" page
Summary:
See PHI775. See D19499. Originally, see PHI720.

D19499 broke the standalone "Branches" page for commits. Normally, you reach this by taking these steps:

  - View a commit which is contained by 11 or more branches.
  - Click the "More Branches..." link in the "Branches" field.
  - You should be taken to a list of all branches which contain the commit.

The change to the 'branch' parameter was adjusted in the query that builds the "x, y, z, More Branches..." list, but not on the actual "Branches" list with the full list. Adjust it.

Test Plan:
  - Set display limit to 1, viewed a commit on "master" and "stable", clicked "More Branches".
  - Before: saw only "master".
  - After: saw both "master" and "stable".

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19532
2018-07-23 11:21:11 -07:00
epriestley
8ab8c390b7 If "branch" is provided to "diffusion.branchquery", use it as the "<pattern>" argument to "git branch --contains ..."
Summary:
Ref T13151. See PHI720. If you want to test if commit X appears on specific branch Y, `git branch --contains X -- Y` is faster than (effectively) `git branch --contains X | grep Y`.

Since this call has a "branch" parameter anyway, use it as the pattern argument if provided.

Test Plan:
  - Called the API method with no parameters, got all branches.
  - Called the API method with `master`, got just master.
  - Called the API method with `maste*`, got master. This behavior is not officially supported and may change in the future.
  - Viewed a commit, still saw all branches.
    - Grepped for `diffusion.branchquery` and verified that no remaining callsites pass a default "branch" parameter.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13151

Differential Revision: https://secure.phabricator.com/D19499
2018-06-22 17:38:19 -07:00
epriestley
6136b83275 Fix changeset construction special case for empty commits in pre-commit hooks
Summary: Fixes T13155. Ref T13151. A recent change (D19455) changed the return format here, but I missed this special case for empty commits.

Test Plan:
  - T13155 has a good set of reproduction instructions.
  - Pushed an empty commit.
    - Before: bunch of warning log spew.
    - After: clean logs.

Reviewers: amckinley, avivey

Reviewed By: avivey

Maniphest Tasks: T13155, T13151

Differential Revision: https://secure.phabricator.com/D19500
2018-06-21 16:43:20 -07:00
Austin McKinley
05f333dfba Attach identities to commits and users to identities
Summary: Ref T12164. Make it easier to work with identity objects by attaching them to commits and attaching users to identities.

Test Plan: Loaded some commits with `->needIdentities(true)` and checked the resulting objects.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12164

Differential Revision: https://secure.phabricator.com/D19491
2018-06-18 15:31:41 -07:00
Austin McKinley
fe5fde5910 Assign RepositoryIdentity objects to commits
Summary: Depends on D19429. Depends on D19423. Ref T12164. This creates new columns `authorIdentityPHID` and `committerIdentityPHID` on commit objects and starts populating them. Also adds the ability to explicitly set an Identity's assignee to "unassigned()" to null out an incorrect auto-assign. Adds more search functionality to identities. Also creates a daemon task for handling users adding new email address and attempts to associate unclaimed identities.

Test Plan: Imported some repos, watched new columns get populated. Added a new email address for a previous commit, saw daemon job run and assign the identity to the new user. Searched for identities in various and sundry ways.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T12164

Differential Revision: https://secure.phabricator.com/D19443
2018-05-31 07:28:23 -07:00
Austin McKinley
f191a66490 Add controllers/search/edit engine functionality to RepositoryIdentity
Summary: Depends on D19423. Ref T12164. Adds controllers capable of listing and editing `PhabricatorRepositoryIdentity` objects. Starts creating those objects when commits are parsed.

Test Plan: Reparsed some revisions, observed objects getting created in the database. Altered some `Identity` objects using the controllers and observed effects in the database. No attempts made to validate behavior under "challenging" author/committer strings.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T12164

Differential Revision: https://secure.phabricator.com/D19429
2018-05-31 07:03:25 -07:00
epriestley
de999af614 Improve some behaviors around memory pressure when pushing many and/or large changes
Summary:
Ref T13142. When commits are pushed, we try to handle them on one of two pathways:

  - Normal changes: we load these into memory and potentially apply Herald content rules to them.
  - "Enormous" changes: we don't load these into memory and skip content rules for them.

The goal is to degrade gracefully when users push huge changes: they should work, just not support all the features.

However, some changes can slip through the cracks right now:

  - If you push a lot of commits at once, we'll try to cache all of the changes smaller than 1GB in memory. This can require an arbitrarily large amount of RAM.
  - We calculate sizes by just looking at the `strlen()` of the diff, but a changeset takes more RAM in PHP than the raw diff does. So even if a diff is "only" 500MB, it can take much more memory than that. On systems with relatively little memory available, this may result in OOM while processing changes that are close to the "enormous" limit.

This change makes two improvements:

  - Instead of caching everything, cache only 64MB of things.
    - For most pushes, this is the same, since they have less than 64MB of diffs.
    - For pushes of single very large changes, this is a bit slower (more CPU) since we have to do some work twice.
    - For pushes of many changes, this is slower (more CPU) since we have to do some work twice, but, critically, doesn't require unlimited memory.
  - Instead of flagging changes as "enormous" at 1GB, flag them as "enormous" at 256MB.
    - This reduces how much memory is required to process the largest "non-enormous" changes.
    - This also gets us under Git's hard-coded 512MB "always binary" cutoff; see T13143.
    - This is still completely gigantic and way larger than any normal change should be.

An additional improvement would be to try to reduce the amount of memory we need to use to hold a change in process memory. I think the other changes here alone will fix the immediate issue in PHI657, but it would be nice if the "largest non-enormous change" required only a couple gigs of RAM.

Test Plan:
- Used `ini_set('memory_limit', '1G')` to artificially limit memory to 1GB.
- Pushed a series of two commits which add two 550MB text files (Temporarily, I added a `--binary` flag to trick Git into showing real diffs for these, see T13143.)
- Got a memory limit error.
- Applied the "cache only 64MB of stuff" and "consider 256MB, not 1GB, to be enormous" changes.
- Pushed again, got properly rejected as enormous.
- Added `memory_get_usage()` calls to measure how actual memory size and reported "size" estimate compare. For these changes, saw a 639MB diff require 31,479MB of memory, i.e. a factor of about 50x. This is, uh, pretty not great.
- Allowed enormous changes, pushed again, push went through.

Reviewers: amckinley

Maniphest Tasks: T13142

Differential Revision: https://secure.phabricator.com/D19455
2018-05-18 17:15:34 -07:00
epriestley
3544620209 Parse unusual Subversion protocol frames which contain extra whitespace
Summary:
Fixes T13140. See PHI660.

Recent versions of Subversion can send a `(get-file true false  false )` protocol frame with extra space between "false" and "false". This is allowed by the protocol spec, but never normally happens, and we do not parse it correctly.

Instead, parse it correctly.

Test Plan:
  - Added unit tests.
  - Ran `svn proplist svn+ssh://.../diffusion/X/file.c` under SVN 1.10 before and after the change.
    - Before: indefinite hang.
    - After: completed in finite time.

Reviewers: amckinley, asherkin

Reviewed By: amckinley, asherkin

Maniphest Tasks: T13140

Differential Revision: https://secure.phabricator.com/D19451
2018-05-16 17:12:41 -07:00
epriestley
5b640a434c Support an "Ancestors Of: ..." constraint in commit queries
Summary:
Ref T13137. See PHI609. An install would like to filter audit requests on a particular branch, e.g. "master".

This is difficult in the general case because we can not apply this constraint efficiently under every conceivable data shape, but we can do a reasonable job in most practical cases.

See T13137#238822 for more detailed discussion on the approach here.

This is a bit rough, but should do the job for now.

Test Plan:
- Filtered commits by various branches, e.g. "master"; "lfs". Saw correct-seeming results.
- Stubbed out the "just list everything" path to hit the `diffusion.internal.ancestors` path, saw the same correct-seeming results.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13137

Differential Revision: https://secure.phabricator.com/D19431
2018-05-08 15:51:42 -07:00
epriestley
afc3099ee7 Add a view option to disable blame in Diffusion and fix some view transition bugs
Summary:
See PHI604. Ref T13130. Ref T13105. There's currently no way to turn blame off in Diffusion. Add a "Hide Blame" option to the "View Options" dropdown so it can be toggled off.

Also fix a couple of bugs around this: for example, if you loaded a Jupyter notebook and then switched to "Source" view, blame would incorrectly fail to activate because the original rendering of the "stage" used an asynchronous engine so `willRenderRef()` wasn't called to populate blame.

Test Plan:
  - Viewed a source file, toggled blame off/on, reloaded page to see state stick in URL.
  - Viewed a Jupyter notebook, toggled to "Source" view, saw blame.
  - Viewed stuff in Files (no blame UI options).
  - Tried to do some invalid stuff like toggle blame on a non-blame engine (options disable properly).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13130, T13105

Differential Revision: https://secure.phabricator.com/D19414
2018-04-30 15:32:23 -07:00
epriestley
8c78cde32f Stop "git blame" from printing "^" markers on root repository commits
Summary: Depends on D19391. Ref T13126. See that task for some details on what's going on here.

Test Plan:
  - Viewed a file which includes lines that were added during the first commit to the repository.
  - Before D19391: fatal.
  - After D19391: blank.
  - After this patch: accurate blame information.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13126

Differential Revision: https://secure.phabricator.com/D19392
2018-04-20 14:13:10 -07:00
epriestley
95e179d9a4 Fix a fatal in the document engine blame view with files that blame to the initial commit
Summary:
Ref T13126. When you view a file using the new document engine view and some lines were introduced in the initial commit to the repository, Git renders "^abc123" in the blame output.

We currently don't do anything about this, and later fail to look it up and fatal.

It's also unlikely-but-conceivably-possible to end up here if a commit has not imported yet or has been nuked with `bin/remove destroy`.

Let the whole thing run without fataling even if a `$commit` is missing. Future refinements could improve this behavior.

Test Plan: Viewed a file with lines introduced in the initial commit, got empty blame instead of a fatal.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13126

Differential Revision: https://secure.phabricator.com/D19391
2018-04-20 14:12:50 -07:00
epriestley
665529ab60 Restore coverage reporting to Diffusion browse UI
Summary:
Depends on D19377. Ref T13125. Ref T13124. Ref T13105. Coverage reporting in Diffusion didn't initially survive the transition to Document Engine; restore it.

This adds some tentative/theoretical support for multiple columns of coverage, but no way to actually produce them in the UI. For now, the labels, codes, and colors are hard coded.

Test Plan:
Added coverage with `diffusion.updatecoverage`, saw coverage in the UI:

{F5525542}

Hovered over coverage, got labels and highlighting.

Double-checked labels for "N" (Not Executable) and "U" (Uncovered). See PHI577.

Faked some multi-column coverage, but you can't currently get this yourself today:

{F5525544}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13125, T13124, T13105

Differential Revision: https://secure.phabricator.com/D19378
2018-04-17 14:51:47 -07:00
epriestley
6556536d06 Allow repository cluster bindings to be marked as not "writable", making them read-only
Summary:
Depends on D19356. Fixes T10883. Ref T13120.

  - Add a "writable" property to the bindings, defaulting to "true" with a nice dropdown.
  - When selecting hosts, allow callers to request a writable host.
  - If the caller wants a writable host, only return hosts if they're writable.
  - In SVN and Mercurial, we sometimes return only writable hosts when we //could// return read-only hosts, but figuring out if these request are read-only or read-write is currently tricky. Since these repositories can't really cluster yet, this shouldn't matter too much today.

Test Plan:
  - Without any config changes, viewed repositories via web UI and pushed/pulled via SSH and HTTP.
  - Made all nodes in the cluster read-only by disabling "writable", pulled and hit the web UI (worked), tried to push via SSH and HTTP (got errors about read-only).
  - Put everything back, pulled and pushed.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T10883

Differential Revision: https://secure.phabricator.com/D19357
2018-04-12 16:10:36 -07:00
epriestley
7c7e6d555b Give getAlmanacServiceURI() an "options" parameter to prepare for read-only devices
Summary:
Depends on D19355. Ref T10883. Ref T13120. Rather than adding a million parameters here, wrap the selector-parameters in an `$options`.

The next change adds a new "writable" option to support forcing selection of writable hosts.

Test Plan: Pulled and pushed via HTTP and SSH, viewed repositories via Diffusion.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T10883

Differential Revision: https://secure.phabricator.com/D19356
2018-04-12 16:10:12 -07:00
epriestley
472bc3d90a Colorize lines in blame under DocumentEngine, to show relative age of changes
Summary:
Depends on D19313. Ref T13105. Fixes T13015. We lost the coloration for ages in the switch to Document Engine.

Restore it, and use a wider range of colors to make the information more clear.

Test Plan: Viewed some blame, saw a nice explosion of bright colors. This is a cornerstone of good design.

Maniphest Tasks: T13105, T13015

Differential Revision: https://secure.phabricator.com/D19314
2018-04-09 06:11:47 -07:00
epriestley
eca7dc25f2 Use javelin_tag(), not phutil_tag(), to render revision blame tooltips properly
Summary: Depends on D19310. Ref T13105. The "meta" value was not populating correctly because this used `phutil_tag()`.

Test Plan: Will verify on `secure`.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19311
2018-04-09 06:10:09 -07:00
epriestley
09c6d42b95 Mostly make blame work with DocumentEngine
Summary: Ref T13105. This needs refinement but blame sort of works again, now.

Test Plan: Viewed files in Diffusion and Files; saw blame in Diffusion when viewing in source mode.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19309
2018-04-09 04:48:21 -07:00
epriestley
90a614778c Make repository symbol references work with DocumentEngine
Summary: Ref T13105. Ref T13047. This makes symbol indexes work with DocumentEngine in Files, and restores support in Diffusion.

Test Plan: Command-clicked stuff, got taken to the symbol index with reasonable metadata in Diffusion, Differential and Files.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105, T13047

Differential Revision: https://secure.phabricator.com/D19307
2018-04-09 04:47:28 -07:00
epriestley
6dea2ba3b3 Fix DocumentEngine line behaviors in Diffusion
Summary:
Ref T13105. Fixes some issues with line linking and highlighting under DocumentEngine:

  - Adding `$1-3` to the URI didn't work correctly with query parameters.
  - Reading `$1-3` from the URI didn't work correctly because Diffusion parses them slightly abnormally.

Test Plan: Clicked/dragged lines to select them. Observed URI. Reloaded page, got the right selection.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19305
2018-04-09 04:46:47 -07:00
epriestley
1fde4a9450 Move Diffusion browse rendering to DocumentEngine, breaking almost all features
Summary:
Ref T13105. This breaks about 9,000 features but moves Diffusion to DocumentEngine for rendering. See T13105 for a more complete list of all the broken stuff.

But you can't bake a software without breaking all the features every time you make a change, right?

Test Plan: Viewed various files in Diffusion, used DocumentEngine features like highlighting and rendering engine selection.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Subscribers: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19302
2018-04-09 04:46:26 -07:00
epriestley
7d4e25614d Remove the ability to disable blame in Diffusion
Summary: Ref T13105. Given that we now load blame with AJAX, it's not clear that there's any benefit to disabling it. This would also interact oddly with the document engine.

Test Plan: Viewed files in Diffusion, no longer saw blame-related options.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19300
2018-04-09 04:45:16 -07:00
epriestley
9bb338c038 Revert the alternate menu names for applications
Summary: This reverts D18524. See that revision for discussion.

Test Plan: Viewed home menu, saw application names as menu items.

Differential Revision: https://secure.phabricator.com/D19308
2018-04-08 10:20:24 -07:00
epriestley
3de002c841 Fix a commit hook issue where pushing dangerous changes would fatal before hitting the dragon bureaucrats
Summary: See <https://discourse.phabricator-community.org/t/php-fatal-when-using-git-push-d/1317>. The behavioral changes for Herald on initial import from D19265 could leave `$all_updates` undefined if we throw early enough.

Test Plan: Pushed a dangerous change, saw dragon bureaucrats again.

Differential Revision: https://secure.phabricator.com/D19297
2018-04-05 06:19:49 -07:00
epriestley
e40aec0210 When a revision has more than 7 reviewers, render only the first 7 in the list view
Summary:
See PHI489. Ref T13110. At least for now, this just shows "..." at the end since you can click the revision to see the whole list anyway.

Also remove the older-style external Handle passing in favor of lazy construction via HandlePool.

Test Plan: Viewed revisions, fiddled with the 7 limit, got sensible-seeming "..." behavior.

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19293
2018-04-03 12:47:43 -07:00
epriestley
ada0c9126c Provide a modular buildable transaction in Diffusion
Summary:
Depends on D19279. Ref T13110. This implements the existing publishing logic for buildables, but does so via ModularTransactions instead of a core transaction type.

Since each application is implementing build transactions independently, this removes the core type.

Next, Differential will get a similar treatment.

Test Plan: Used `bin/harbormaster publish` (with some commenting-out-guard-clauses) to publish a commit Buildable; saw unchanged feed behavior.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19280
2018-04-03 11:01:37 -07:00
epriestley
95c9d403f4 Make objects implementing BuildableInterface produce a BuildableEngine
Summary:
Ref T13110. Currently, build status is published the same way for every Buildable by the BuildEngine.

I want to change this to delegate publishing to each Buildable, particularly so that Differential may use more detailed rules for handling builds and drafts.

Rather than add additional methods to the existing `BuildableInterface`, add an engine generator method instead. This is a pattern which has seen more use recently (e.g., in Ferret) and lets us pay a little more upfront to pull complex pieces of logic out of the main class and let them use inheritence more easily. If we had Traits that might cover this to some degree.

I'd expect to eventually reduce the size of `BuildableInterface` and move the `CircleCI` and `BuildKite` interfaces so that the `BuildableEngine` implements them instead of the main object.

Here, this new engine does nothing and is never instantiated. In upcoming changes, publishing logic will move into it so that Differential can handle publishing differently.

Test Plan: Ran `arc liberate`, loaded pages, grepped for `BuildableInterface`.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19278
2018-04-03 10:57:51 -07:00
epriestley
74216ea8e0 Disable Herald and enormous change protection for repository initial imports
Summary: See PHI514. Ref T13114. Ref T8951. When a push is an "initial import" (a push of at least 7 commits to an empty repository) don't run Herald or enormous change protection.

Test Plan: Pushed some non-initial changes to a repository, and some initial changes.

Maniphest Tasks: T13114, T8951

Differential Revision: https://secure.phabricator.com/D19265
2018-03-29 08:05:07 -07:00
epriestley
df3c937dab Record lock timing information on PushEvents
Summary:
Depends on D19249. Ref T13109. Add timing information to the `PushEvent`:

  - `writeWait`: Time spent waiting for a write lock.
  - `readWait`: Time spent waiting for a read lock.
  - `hostWait`: Roughly, total time spent on the leaf node.

The primary goal here is to see if `readWait` is meaningful in the wild. If it is, that motivates smarter routing, and the value of smarter routing can be demonstrated by looking for a reduction in read wait times.

Test Plan: Pushed some stuff, saw reasonable timing values in the table. Saw timing information in "Export Data".

Maniphest Tasks: T13109

Differential Revision: https://secure.phabricator.com/D19250
2018-03-22 13:46:01 -07:00
epriestley
69bff489d4 Generate a random unique "Request ID" for SSH requests so processes can coordinate better
Summary:
Depends on D19247. Ref T13109. When we receive an SSH request, generate a random unique ID for the request. Then thread it down through the process tree.

The immediate goal is to let the `ssh-exec` process coordinate with `commit-hook` process and log information about read and write lock wait times. Today, there's no way for `ssh-exec` to interact with the `PushEvent`, but this is the most helpful place to store this data for users.

Test Plan: Made pushes, saw the `PushEvent` table populate with a random request ID. Exported data and saw the ID preserved in the export.

Maniphest Tasks: T13109

Differential Revision: https://secure.phabricator.com/D19249
2018-03-22 13:44:30 -07:00
epriestley
859b274970 Provide more information to users during git push while waiting for write locks
Summary:
Ref T13109. Make it slightly more clear what the scope of the write and read locks are, and slightly more clear that we're actively acquiring locks, not just sitting around waiting.

While waiting on another writer, show who we're waiting on so you can walk over to their desk and glare at them.

Test Plan:
Added `sleep(15)` after `willWrite()`. Pushed in two windows. Saw new, more informative messages. In the second window, saw the new guidance:

> # Waiting for hector to finish writing (on device "repo1.local.phacility.net" for 11s)...

Reviewers: asherkin

Reviewed By: asherkin

Subscribers: asherkin

Maniphest Tasks: T13109

Differential Revision: https://secure.phabricator.com/D19247
2018-03-22 13:42:18 -07:00
epriestley
7e43b74055 Give all commands from DiffusionCommandEngine a default 15 minute timeout
Summary:
Ref T13108. See PHI364. See the task and issue for discussion.

If a `git fetch` during synchronization hangs, the whole node currently hangs. While the causes of a `git fetch` hang aren't clear, we don't expect synchronization to ever reasonably take more than 15 minutes, so add a default timeout.

Test Plan: Will deploy and observe; this is difficult to reproduce or test directly.

Maniphest Tasks: T13108

Differential Revision: https://secure.phabricator.com/D19235
2018-03-16 17:22:03 -07:00
epriestley
a4cc1373d3 Use a tokenizer, not a gigantic poorly-ordered "<select />", to choose repositories in Owners
Summary: Depends on D19190. Fixes T12590. Ref T13099. Replaces the barely-usable, gigantic, poorly ordered "<select />" control with a tokenizer. Attempts to fix various minor issues.

Test Plan:
  - Edited paths: include/exclude paths, from different repositories, different actual paths.
  - Used "Add New Path" to add rows, got repository selector prepopulated with last value.
  - Used "remove".
  - Used validation typeahead, got reasonable behaviors?

The error behavior if you delete the repository for a path is a little sketchy still, but roughly okay.

Maniphest Tasks: T13099, T12590

Differential Revision: https://secure.phabricator.com/D19191
2018-03-07 20:57:24 -08:00
epriestley
b41a0e6ddd Fix broken suggestion/validation for Owners paths in repositories with short names
Summary:
Depends on D19189. Ref T12590. The "validate" and "complete" endpoints for this UI could incorrectly return redirect responses. These aren't critical to the behavior of Owners, but they're nice to have, and shouldn't redirect.

Instead, skip the canonicalizing redirect for AJAX requests.

Test Plan: Edited Owners paths in a repository with a short name, got completion/validation again.

Maniphest Tasks: T12590

Differential Revision: https://secure.phabricator.com/D19190
2018-03-07 18:31:25 -08:00
epriestley
c6a042b59a Correct line highlighting behavior in Diffusion
Summary: See <https://discourse.phabricator-community.org/t/line-highlighting-in-diffusion-breaks-url/1207>. Ref T13088. This was disrupted by changes for the new Harbormaster build logs and now needs an explicit base URI.

Test Plan: Clicked lines and dragged across line ranges in Diffusion, observed correct URI behavior.

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19187
2018-03-07 07:07:06 -08:00
epriestley
ab579f2511 Never generate file download forms which point to the CDN domain, tighten "form-action" CSP
Summary:
Depends on D19155. Ref T13094. Ref T4340.

We can't currently implement a strict `form-action 'self'` content security policy because some file downloads rely on a `<form />` which sometimes POSTs to the CDN domain.

Broadly, stop generating these forms. We just redirect instead, and show an interstitial confirm dialog if no CDN domain is configured. This makes the UX for installs with no CDN domain a little worse and the UX for everyone else better.

Then, implement the stricter Content-Security-Policy.

This also removes extra confirm dialogs for downloading Harbormaster build logs and data exports.

Test Plan:
  - Went through the plain data export, data export with bulk jobs, ssh key generation, calendar ICS download, Diffusion data, Paste data, Harbormaster log data, and normal file data download workflows with a CDN domain.
  - Went through all those workflows again without a CDN domain.
  - Grepped for affected symbols (`getCDNURI()`, `getDownloadURI()`).
  - Added an evil form to a page, tried to submit it, was rejected.
  - Went through the ReCaptcha and Stripe flows again to see if they're submitting any forms.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13094, T4340

Differential Revision: https://secure.phabricator.com/D19156
2018-02-28 17:20:12 -08:00
epriestley
8cbfb386bb When evaluating the "Branches" Herald field from the test console, use the current viewer
Summary: Ref T13093. Depends on D19145. See PHI398. Previously, see D18933. This provides the current viewer to `ConduitCall` so that we don't try to use device credentials from unprivileged web hosts.

Test Plan: Evaluated the "Branches" field locally, saw an appropriate field value.

Maniphest Tasks: T13093

Differential Revision: https://secure.phabricator.com/D19146
2018-02-27 14:37:36 -08:00
epriestley
80fe382e3d Add "Committer's projects" and "Author's projects" fields to Herald commit rules
Summary:
Ref T13093. See PHI396. These are possibly somewhat niche, but reasonable to support and consistent with the existing "Pusher's projects".

Also relabel "Pusher's projects" and "Project tags" for consistency and, hopefully, clarity.

Test Plan:
  - Created new "commit" and "hook: commit content" Herald rules which run against "Author's projects" and "Committer's projects".
  - Test console'd the "Commit" rules.
  - Pushed through the "Hook" rule.
  - In all cases, saw fields populate appropriately.

Maniphest Tasks: T13093

Differential Revision: https://secure.phabricator.com/D19145
2018-02-27 14:33:12 -08:00
epriestley
4c7370a1a3 Make the filetree view width sticky across show/hide and reload
Summary:
Ref T13090. The default width changed recently to become much wider, but the behavior on this control isn't great. Instead:

  - Pick a default width somewhere between the two.
  - Make the width sticky across show/hide (pressing "f" twice remembers your width instead of resetting it).
  - Make the width sticky across reloads (dragging the bar, then reloading the page keeps the bar in the same place).

Test Plan:
  - Without settings, loaded page: got medium-width bar.
  - Dragged bar wide/narrow, toggled on/off with "f", got persistent width.
  - Dragged bar wide/narrow, reloaded page, got persistent width.
  - Dragged bar wide/narrow, toggled it off, reloaded page, toggled it on, got persistent width.

Maniphest Tasks: T13090

Differential Revision: https://secure.phabricator.com/D19129
2018-02-22 13:47:41 -08:00
epriestley
0dee34b3fa Make Facts more modern, DRY, and dimensional
Summary:
Ref T13083. Facts has a fair amount of weird hardcoding and duplication of responsibilities. Reduce this somewhat: no more hard-coded fact aggregates, no more database-driven list of available facts, etc. Generally, derive all objective truth from FactEngines. This is more similar to how most other modern applications work.

For clarity, hopefully: rename "FactSpec" to "Fact". Rename "RawFact" to "Datapoint".

Split the fairly optimistic "RawFact" table into an "IntDatapoint" table with less stuff in it, then dimension tables for the object PHIDs and key names. This is primarily aimed at reducing the row size of each datapoint. At the time I originally wrote this code we hadn't experimented much with storing similar data in multiple tables, but this is now more common and has worked well elsewhere (CustomFields, Edges, Ferret) so I don't anticipate this causing issues. If we need more complex or multidimension/multivalue tables later we can accommodate them. The queries a single table supports (like "all facts of all kinds in some time window") don't make any sense as far as I can tell and could likely be UNION ALL'd anyway.

Remove all the aggregation stuff for now, it's not really clear to me what this should look like.

Test Plan: Ran `bin/fact analyze` and viewed web UI. Nothing exploded too violently.

Subscribers: yelirekim

Maniphest Tasks: T13083

Differential Revision: https://secure.phabricator.com/D19119
2018-02-19 12:05:19 -08:00
epriestley
f713e1dfc1 Add Owners Package support for "Commit Hook: Content" Herald rules
Summary:
See PHI370. Support the "Affected packages" and "Affected package owners" Herald fields in pre-commit hooks.

I believe there's no technical reason these fields aren't supported and this was just overlooked.

Test Plan: Wrote a rule which makes use of the new fields, pushed commits through it. Checked transcripts and saw sensible-looking values.

Differential Revision: https://secure.phabricator.com/D19104
2018-02-16 09:49:24 -08:00
epriestley
4bccb1547d Modularize the "jump nav" behaviors in global search
Summary: Depends on D19087. Ref T13079. This still doesn't feel like the most clean, general system in the world, but is a step forward from hard-coded `switch()` stuff.

Test Plan:
- Jumped to `r`.
- Jumped to `a`.
- Jumped to `r poe` (multiple results).
- Jumped to `r poetry` (one result).
- Jumped to `r syzygy` (no results).
- Jumped to `p`.
- Jumped to `p robot` (multiple results); `p assessment` (one result).
  - The behavior for `p <string>` has changed slightly but should be more powerful now (it's consistent with `r <string>`).
- Jumped to `s <symbol>` and `s <context>-><symbol>`.
- Jumped to `d`.
- Jumped to `f`.
- Jumped to `t`.
- Jumped to `T123`, `D123`, `@dog`, `PHID-DREV-abcd`, etc.

Maniphest Tasks: T13079

Differential Revision: https://secure.phabricator.com/D19088
2018-02-14 18:08:07 -08:00
epriestley
abe5fd57b0 Rename "QuickSearch" Engine/EngineExtension to "Datasource"
Summary: Ref T13079. This recently-introduced Engine/EngineExtension are a good fit for adding more datasource functions in general, but we didn't think quite big enough in naming them.

Test Plan: Used quick search typeahead, hit applications/users/monograms/symbols/etc.

Maniphest Tasks: T13079

Differential Revision: https://secure.phabricator.com/D19087
2018-02-14 18:03:03 -08:00
epriestley
f939a2b12e Make Harbormaster buildable status more of a nice flexible map and less of a bunch of switch statements
Summary: Depends on D19063. Ref T13054. Prepare for the addition of a new `PREPARING` status by getting rid of the "scattered mess of switch statements" pattern of status management.

Test Plan: Searched/browsed buildables. Viewed buildables. Viewed revisions. Grepped for all affected symbols.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13054

Differential Revision: https://secure.phabricator.com/D19064
2018-02-12 12:18:06 -08:00
epriestley
4fa99374be Prevent "Call webhooks" Herald action from appearing in UI for adapters which can't fire it
Summary:
See <https://discourse.phabricator-community.org/t/herald-webhook-is-not-called-but-herald-transcript-tells-me-the-opposite/1098>.

The "Commit Hook" events don't operate on objects and don't use TransactionEditors. They can't call webhooks in a normal way and currently don't call webhooks at all. Stop offering these actions in the UI.

The  "Outbound Mail" event also fires oddly and likely doesn't make much sense to hook anyway.

Test Plan: Verified that these events no longer offer "Call webhooks", while normal events still do.

Differential Revision: https://secure.phabricator.com/D19061
2018-02-11 06:15:29 -08:00
epriestley
d1e273daf6 Remove completely pointless load of every repository when viewing a repository URI
Summary:
See D18176. This query has no effect (other than wasting resources) and the result is unused.

`$repository` already has the URI loaded because we load them unconditionally during request initialization.

Test Plan: Viewed repository URIs.

Subscribers: jmeador

Differential Revision: https://secure.phabricator.com/D19036
2018-02-08 12:47:48 -08:00
epriestley
bca9c08953 Add an "Acting user" field to Herald
Summary:
Ref T13053. Fixes T7804. Adds "Acting user" so you can have "always email me" stuff skip things you did or keep an eye on suspicious interns.

For the test console, the current user is the acting user.

For pushes, the pusher is the acting user.

Test Plan: Wrote acting user rules, triggered them via test console and via multiple actors on real objects.

Maniphest Tasks: T13053, T7804

Differential Revision: https://secure.phabricator.com/D19031
2018-02-08 09:52:18 -08:00
epriestley
a5bbadbaba Fix another Git 2.16.0 CLI compatibility issue
Summary:
This command also needs a "." instead of an empty string now.

(This powers the file browser typeahead in Diffusion.)

Test Plan: Will test in production since there's still no easy 2.16 installer for macOS.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19010
2018-02-07 17:54:39 -08:00
epriestley
032f5b2294 Allow revisions to revert commits and one another, and commits to revert revisions
Summary:
Ref T13057. This makes "reverts" syntax more visible and useful. In particular, you can now `Reverts Dxx` in a revision or commit, and `Reverts <hash>` from a revision.

When you do, the corresponding object will get a more-visible cross-reference marker in its timeline:

{F5405517}

From here, we can look at surfacing revert information more heavily, since we can now query it on revision/commit pages via edges.

Test Plan: Used "reverts <hash>" and "reverts <revision>" in Differential and Diffusion, got sensible results in the timeline.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13057

Differential Revision: https://secure.phabricator.com/D18978
2018-02-02 08:25:58 -08:00
epriestley
f535981c0d Fix a missing getSSHUser() callsite
Summary:
See <https://discourse.phabricator-community.org/t/after-upgrade-git-lfs-push-ends-up-in-call-to-undefined-method-on-diffusion-git-lfs-authenticate-workflow/1047/1>.

I renamed this method in D18912 but missed this callsite since the workflow doesn't live alongside the other ones.

Test Plan: Ran `git push` in an LFS repository over SSH. Before: fatal; after: clean push.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18977
2018-01-31 15:34:12 -08:00
epriestley
1e3d1271ad Make push log "flags", "reject code" human readable; add crumbs to pull/push logs
Summary:
Depends on D18972. Ref T13049.

Currently, the "flags" columns renders an inscrutible bitmask which you have to go hunt down in the code. Show a list of flags in human-readable text instead.

The "code" column renders a meaningless integer code. Show a text description instead.

The pull logs and push logs pages don't have a crumb to go back up out of the current query. Add one.

Test Plan: Viewed push logs, no more arcane numbers. Saw and clicked crumbs on each log page.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13049

Differential Revision: https://secure.phabricator.com/D18973
2018-01-30 15:45:58 -08:00
epriestley
8a2863e3f7 Change the "can see remote address?" policy to "is administrator?" everywhere
Summary:
Depends on D18970. Ref T13049. Currently, the policy for viewing remote addresses is:

  - In activity logs: administrators.
  - In push and pull logs: users who can edit the corresponding repository.

This sort of makes sense, but is also sort of weird. Particularly, I think it's kind of hard to understand and predict, and hard to guess that this is the behavior we implement. The actual implementation is complex, too.

Instead, just use the rule "administrators can see remote addresses" consistently across all applications. This should generally be more strict than the old rule, because administrators could usually have seen everyone's address in the activity logs anyway. It's also simpler and more expected, and I don't really know of any legit use cases for the "repository editor" rule.

Test Plan: Viewed pull/push/activity logs as non-admin. Saw remote addresses as an admin, and none as a non-admin.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13049

Differential Revision: https://secure.phabricator.com/D18971
2018-01-30 15:45:23 -08:00
epriestley
75bc86589f Add date range filtering for activity, push, and pull logs
Summary: Ref T13049. This is just a general nice-to-have so you don't have to export a 300MB file if you want to check the last month of data or whatever.

Test Plan: Applied filters to all three logs, got appropriate date-range result sets.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13049

Differential Revision: https://secure.phabricator.com/D18970
2018-01-30 15:36:22 -08:00
epriestley
5b22412f24 Support data export on push logs
Summary: Depends on D18967. Ref T13049. Nothing too fancy going on here.

Test Plan: Exported push logs, looked at the export, seemed sensible.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13049

Differential Revision: https://secure.phabricator.com/D18968
2018-01-30 11:19:20 -08:00
epriestley
213eb8e93d Define common ID and PHID export fields in SearchEngine
Summary:
Ref T13049. All exportable objects should always have these fields, so make them builtins.

This also sets things up for extensions (like custom fields).

Test Plan: Exported user data, got the same export as before.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13049

Differential Revision: https://secure.phabricator.com/D18951
2018-01-29 15:17:00 -08:00
epriestley
d8f51dff6e Use the configured viewer more consistently in the Herald commit adapter
Summary: See PHI276. Ref T13048. The fix in D18933 got one callsite, but missed the one in the `callConduit()` method, so the issue isn't fully fixed in production. Convert this adapter to use a real viewer (if one is available) more thoroughly.

Test Plan: Ran rules in test console, saw field values. Will test in production again.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13048

Differential Revision: https://secure.phabricator.com/D18950
2018-01-29 15:00:26 -08:00
epriestley
162563d40b Move the fix for Git 2.16.0 from the "Mercurial" part of the code to the "Git" part of the code
Summary: Ref T13050. Oh boy. Both of them run `grep`!

Test Plan: Will push again.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13050

Differential Revision: https://secure.phabricator.com/D18945
2018-01-26 13:48:35 -08:00
epriestley
a80d1e7e7d Pass "." to git grep to satisfy "all paths" for Git 2.16.0
Summary:
Ref T13050. See <https://discourse.phabricator-community.org/t/issues-with-git-2-16-0/1004/2>.

`secure` picked up 2.16.0 so this reproduces now: <https://secure.phabricator.com/source/phabricator/browse/master/?grep=dog>

Test Plan: Will push.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13050

Differential Revision: https://secure.phabricator.com/D18944
2018-01-26 13:38:18 -08:00
epriestley
7e7720803c Set GIT_SSH_VARIANT alongside GIT_SSH
A recent version of Git has changed some piece of behavior here and we
now get "fatal: ssh variant 'simple' does not support setting port"
when using a port. Explicitly setting GIT_SSH_VARIANT to `ssh` likely
fixes this.
2018-01-26 13:21:10 -08:00
epriestley
ad7755d9a9 Fix an issue with symbol lookup identifying path names in Diffusion
Summary:
Depends on D18939. Ref T13047. Symbol lookup can be activated from a diff (in Differential or Diffusion) or from the static view of a file at a particular commit.

In the latter case, we need to figure out the path a little differently. The character and line number approaches still work as written.

Test Plan:
  - Command-clicked symbols in the Diffusion browse view with blame on and off; saw path, line and char populate properly.
  - Command-clicked symbols in Differential diff view to check I didn't break anything.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13047

Differential Revision: https://secure.phabricator.com/D18940
2018-01-26 13:02:20 -08:00
epriestley
fdc36677ba Provide character position information to symbol queries
Summary: Depends on D18937. Ref T13047. When available, provide character positions so external indexers can return more accurate results.

Test Plan: Clicked symbols in Safari, Firefox and Chrome, got sensible-looking character positions.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13047

Differential Revision: https://secure.phabricator.com/D18939
2018-01-26 13:01:57 -08:00
epriestley
d606eb1c38 When available, pass path, line and repository hints to external symbol queries
Summary:
Depends on D18936. Ref T13047. Third parties can define external symbol sources that let users jump to PHP or Python documentation or query some server.

Give these queries more information so they can try to get better results: the path and line where the symbol appeared, and any known repository scope.

Test Plan: Wrote a fake external source that used this data, command-clicked a symbol in Differential, saw a fake external symbol result.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13047

Differential Revision: https://secure.phabricator.com/D18937
2018-01-26 12:00:44 -08:00
epriestley
a79bb55f3f Support CSV, JSON, and tab-separated text as export formats
Summary: Depends on D18919. Ref T13046. Adds some simple modular exporters.

Test Plan: Exported pull logs in each format.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13046

Differential Revision: https://secure.phabricator.com/D18934
2018-01-26 11:16:52 -08:00
epriestley
c0b8e4784b Add a basic, general-purpose export workflow for all objects with SearchEngine support
Summary:
Depends on D18918. Ref T13046. Ref T5954. Pull logs can currently be browsed in the web UI, but this isn't very powerful, especially if you have thousands of them.

Allow SearchEngine implementations to define exportable fields so that users can "Use Results > Export Data" on any query. In particular, they can use this workflow to download a file with pull logs.

In the future, this can replace the existing "Export to Excel" feature in Maniphest.

For now, we hard-code JSON as the only supported datatype and don't actually make any effort to format the data properly, but this leaves room to add more exporters (CSV, Excel) and data type awareness (integer casting, date formatting, etc) in the future.

For sufficiently large result sets, this will probably time out. At some point, I'll make this use the job queue (like bulk editing) when the export is "large" (affects more than 1K rows?).

Test Plan: Downloaded pull logs in JSON format.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13046, T5954

Differential Revision: https://secure.phabricator.com/D18919
2018-01-26 11:15:59 -08:00
epriestley
5058cfb972 Pass a real viewer to HeraldAdapter when doing test console runs
Summary:
Depends on D18932. Ref T13048. See PHI276. In the cluster, we don't have device keys on `web` nodes. This is generally good, since they don't need them, and it means that we aren't putting more credentials than we need on those hosts.

However, it means that when we pull diff content to test "Commit" rules via the Herald test console, we use the omnipotent user and try to use device credentials, and this fails since we don't have any.

Instead, pass the real viewer in this case so we just sign the request as them, like we do for normal Diffusion requests.

Test Plan:
Wrote and ran a commit content rule locally, no issues.

This isn't completely convincing since my local setup does have device credentials, but I'll double-check in production once this deploys.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13048

Differential Revision: https://secure.phabricator.com/D18933
2018-01-26 11:08:12 -08:00
epriestley
a9f87857af Mark the "Reviewer" field for Commits as deprecated
Summary:
Depends on D18931. Ref T13048. Ref T13041. This field means "the first accepting reviewer, where order is mostly arbitrary". Modern rules should almost certainly use "Accepting Reviewers" instead.

Getting rid of this completely is a pain, but we can at least reduce confusion by marking it as not-the-new-hotness. Add a "Deprecated" group, move it there, and mark it for exile.

Test Plan:
Edited a commit rule, saw it in "Deprecated" group at the bottom of the list:

{F5395001}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13048, T13041

Differential Revision: https://secure.phabricator.com/D18932
2018-01-26 11:07:02 -08:00
epriestley
778dfff277 Make minor correctness and display improvements to pull logs
Summary:
Depends on D18915. Ref T13046.

  - Distinguish between HTTP and HTTPS.
  - Use more constants and fewer magical strings.
  - For HTTP responses, give them better type information and more helpful UI behaviors.

Test Plan: Pulled over SSH and HTTP. Reviewed resulting logs from the web UI. Hit errors like missing/invalid credentials.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13046

Differential Revision: https://secure.phabricator.com/D18917
2018-01-23 14:13:18 -08:00
epriestley
e6a9db56a9 Add a basic view for repository pull logs
Summary:
Depends on D18912. Ref T13046. Add a UI to browse the existing pull log table.

The actual log still has some significant flaws, but get the basics working.

Test Plan: {F5391909}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13046

Differential Revision: https://secure.phabricator.com/D18914
2018-01-23 14:10:10 -08:00
epriestley
2914613444 Fix failure to record pullerPHID in repository pull logs
Summary:
See PHI305. Ref T13046.

The SSH workflows currently extend `PhabricatorManagementWorkflow` to benefit from sharing all the standard argument parsing code. Sharing the parsing code is good, but it also means they inherit a `getViewer()` method which returns the ommnipotent viewer.

This is appropriate for everything else which extends `ManagementWorkflow` (like `bin/storage`, `bin/auth`, etc.) but not appropriate for SSH workflows, which have a real user.

This caused a bug with the pull logs where `pullerPHID` was not recorded properly. We used `$this->getViewer()->getPHID()` but the correct code was `$this->getUser()->getPHID()`.

To harden this against future mistakes:

  - Don't extend `ManagementWorkflow`. Extend `PhutilArgumentWorkflow` instead. We **only** want the argument parsing code.
  - Rename `get/setUser()` to `get/setSSHUser()` to make them explicit.

Then, fix the pull log bug by calling `getSSHUser()` instead of `getViewer()`.

Test Plan:
  - Pulled and pushed to a repository over SSH.
  - Grepped all the SSH stuff for the altered symbols.
  -  Saw pulls record a valid `pullerPHID` in the pull log.
  - Used `echo {} | ssh ... conduit conduit.ping` to test conduit over SSH.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13046

Differential Revision: https://secure.phabricator.com/D18912
2018-01-23 14:09:42 -08:00
epriestley
b8a515cb29 Bring new password validation into AuthPasswordEngine
Summary:
Ref T13043. We have ~4 copies of this logic (registration, lost password recovery, set password, set VCS password).

Currently it varies a bit from case to case, but since it's all going to be basically identical once account passwords swap to the new infrastructure, bring it into the Engine so it can live in one place.

This also fixes VCS passwords not being affected by `account.minimum-password-length`.

Test Plan: Hit all errors in "VCS Password" panel. Successfully changed password.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13043

Differential Revision: https://secure.phabricator.com/D18902
2018-01-23 10:58:37 -08:00
epriestley
dd8f588ac5 Migrate VCS passwords to new shared password infrastructure
Summary:
Ref T13043. Migrate VCS passwords away from their dedicated table to new the new shared infrastructure.

Future changes will migrate account passwords and remove the old table.

Test Plan:
- Ran migrations.
  - Cloned with the same password that was configured before the migrations (worked).
  - Cloned with a different, invalid password (failed).
- Changed password.
  - Cloned with old password (failed).
  - Cloned with new password (worked).
- Deleted password in web UI.
  - Cloned with old password (failed).
- Set password to the same password as it currently is set to (worked, no "unique" collision).
- Set password to account password. !!This (incorrectly) works for now until account passwords migrate, since the uniqueness check can't see them yet.!!
- Set password to a new unique password.
  - Cloned (worked).
  - Revoked the password with `bin/auth revoke`.
  - Verified web UI shows "no password set".
  - Verified that pull no longer works.
  - Verified that I can no longer select the revoked password.
- Verified that accounts do not interact:
  - Tried to set account B to account A's password (worked).
  - Tried to set account B to a password revoked on account A (worked).
- Spot checked the `password` and `passwordtransaction` tables for saniity.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13043

Differential Revision: https://secure.phabricator.com/D18898
2018-01-23 10:56:13 -08:00
epriestley
6a62797056 Fix some issues with Diffusion file data limits
Summary:
See <https://discourse.phabricator-community.org/t/files-created-from-repository-contents-slightly-over-one-chunk-in-size-are-truncated-to-exactly-one-chunk-in-size/988/1>. Three issues here:

  - When we finish reading `git cat-file ...` or whatever, we can end up with more than one chunk worth of bytes left in the internal buffer if the read is fast. Use `while` instead of `if` to make sure we write the whole buffer.
  - Limiting output with `setStdoutSizeLimit()` isn't really a reliable way to limit the size if we're also reading from the buffer. It's also pretty indirect and confusing. Instead, just let the `FileUploadSource` explicitly implement a byte limit in a straightforward way.
  - We weren't setting the time limit correctly on the main path.

Overall, this could cause >4MB files to "write" as 4MB files, with the rest of the file left in the UploadSource buffer. Since these files were technically under the limit, they could return as valid. This was intermittent.

Test Plan:
  - Pushed a ~4.2MB file.
  - Reloaded Diffusion a bunch, sometimes saw the `while/if` buffer race and produce a 4MB file with a prompt to download it. (Other times, the buffer worked right and the page just says "this file is too big, sorry").
  - Applied patches.
  - Reloaded Diffusion a bunch, no longer saw bad behavior or truncated files.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18885
2018-01-22 11:52:37 -08:00
epriestley
86a0c7daa2 Fix a fatal in blame when the viewer can't see a revision because of a permission issue
Summary:
Fixes T13040. To reproduce:

  - View a file with blame enabled, where some line has an associated revision (say, `D123`).
  - Edit `D123` so it exists and is a valid revision, but the viewer can't see it.
  - Reload the page.

Instead, only add revisions to the map if we actually managed to load them.

Test Plan: Page no longer fatals.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13040

Differential Revision: https://secure.phabricator.com/D18884
2018-01-19 14:19:06 -08:00
Alex Vandiver
2140741e25 Fix typo in new setting description
Summary:
Noticed by @amckinley in
https://secure.phabricator.com/D18850#inline-57246 but not fixed
before landing.

Test Plan: ispell

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, amckinley, epriestley

Differential Revision: https://secure.phabricator.com/D18861
2018-01-06 07:25:27 -08:00
epriestley
5543592034 Add a couple of clarifying comments to the Mercurial protocol parser
Summary: See D18857. Ref T13036. See PHI275. Explain what's going on here a little better since it isn't entirely obvious and debugging these stream parsers is a gigantic pain.

Test Plan: Read text.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13036

Differential Revision: https://secure.phabricator.com/D18859
2018-01-04 14:23:28 -08:00
epriestley
13c8963dab Fix a Mercurial wire protocol parser issue when we receive a length frame before any data
Summary:
Depends on D18856. Ref T13036. See PHI275. When we receive a length frame but the buffer doesn't have any data yet, we currently emit a pointless 0-length data frame on the channel.

For normal chatter this is harmless/valid, but it causes problems when a channel has transitioned into bundle2 mode (probably it indicates "end of stream")?

In any case, it's never helpful, so if we're about to read a data block and don't have any data, just bail out until we see some more data.

Note that we can't end up here //expecting// a 0-length data block: both the `data-length` and `data-bytes` states already handle that properly.

Test Plan: Pushed 4MB changes to a Mercurial repository with Mercurial 4.1.1, was no longer able to hit channel errors.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13036

Differential Revision: https://secure.phabricator.com/D18857
2018-01-04 14:06:52 -08:00
epriestley
3a4e14431f Remove an obsolete comment about Mercurial SSH error behavior
Summary:
Depends on D18855. Ref T13036. This comment no longer seems to be accurate: anything we send over `stderr` is faithfully shown to the user with recent clients.

From [[ https://www.mercurial-scm.org/repo/hg/file/default/mercurial/help/internals/wireprotocol.txt | this document ]], the missing sauce may have been:

```
A generic error response type is also supported. It consists of a an error
message written to ``stderr`` followed by ``\n-\n``. In addition, ``\n`` is
written to ``stdout``.
```

That is, writing "\n" to stdout in addition to writing the error to stderr. However, this no longer appears to be necessary.

I think the modern client behavior is generally sensible (and consistent with the behavior of Git and Subversion) so this //probably// isn't a bug or me making a mistake.

Test Plan: With a modern client, threw some arbitrary exception during execution. Observed a helpful message on the client with no additional steps.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13036

Differential Revision: https://secure.phabricator.com/D18856
2018-01-04 14:05:44 -08:00
epriestley
0f02d79ffa Remove nonfunctional Mercurial "bundle2" capability filtering from SSH pathway
Summary:
Ref T13036. This code attempts to filter the "capabilities" message to remove "bundle2", but I think this has never worked.

Specifically, the //write// pathway is hooked, and "write" here means "client is writing a message to the server". However, the "capabilities" frame is part of the response, not part of the request. Thus, this code never fires, at least on recent versions of Mercurial.

Since I plan to support bundle2 and don't want to decode response frames, just get rid of this, assuming we'll achieve those goals.

I think this was just overlooked in D14241, which probably focused on the HTTP version. This code does (at least, potentially) do something for HTTP.

I'm leaving the actual "strip stuff" code in place for now since I think it's still used on the HTTP pathway.

Test Plan:
  - Added debug logging, saw this code never hit even though `hg push --debug` shows the client believing bundle2 is supported.
  - Logged both halves of the wire protocol and saw this come from the server, not the client.
  - Ran the failing `hg push` of a 4MB file under hg 4.4.1, got the same error as before.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: cspeckmim

Maniphest Tasks: T13036

Differential Revision: https://secure.phabricator.com/D18855
2018-01-04 14:05:13 -08:00
epriestley
53b25db918 Prevent enormous changes from being pushed to repositoires by default
Summary:
Fixes T13031. "Enormous" changes are basically changes which are too large to hold in memory, although the actual definition we use today is "more than 1GB of change text or `git diff` runs for more than 15 minutes".

If an install configures a Herald content rule like "when content matches /XYZ/, do something" and then a user pushes a 30 GB source file, we can't put it into memory to `preg_match()` it. Currently, the way to handle this case is to write a separate Herald rule that rejects enormous changes. However, this isn't obvious and means the default behavior is unsafe.

Make the default behavior safe by rejecting these changes with a message, similar to how we reject "dangerous" changes (which permanently delete or overwrite history) by default.

Also, change a couple of UI strings from "Enormous" to "Very Large" to reduce ambiguity. See <https://discourse.phabricator-community.org/t/herald-enormous-check/822>.

Test Plan: Changed the definition of "enormous" from 1GB to 1 byte. Pushed a change; got rejected. Allowed enormous changes, pushed, got rejected by a Herald rule. Disabled the Herald rule, pushed, got a clean push. Prevented enormous changes again. Grepped for "enormous" elsewhere in the UI.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: joshuaspence

Maniphest Tasks: T13031

Differential Revision: https://secure.phabricator.com/D18850
2018-01-04 10:02:29 -08:00
epriestley
ead5f4fd9c Add an "Accepting reviewers" Herald field for commits
Summary:
See PHI262. Fixes T12578. Although this is a bit niche and probably better accomplished through advisory/soft measures ("Add blocking reviewers") in most cases, it isn't difficult to implement and doesn't create any technical or product tension.

If installs write a rule that blocks commits, that will probably also naturally lead them to an "add reviewers" rule anyway.

Also, allow packages to be hit with the typeahead. They're valid reviewers but previously you couldn't write rules against them, for no actual reason.

Test Plan: Used test console to run this against commits, got sensible results for the field value.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12578

Differential Revision: https://secure.phabricator.com/D18839
2017-12-26 15:59:36 -08:00
epriestley
e411d75964 Fix an issue where blame could fatal for unrecognized authors
Summary: See PHI255. See <https://discourse.phabricator-community.org/t/error-generating-blame-data/766>.

Test Plan:
  - Viewed a file contributed to by users with no Phabricator user accounts, in Diffusion.
  - Enabled blame.
  - Before patch: blame failed, fatal in logs.
  - After patch: blame worked.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18833
2017-12-20 11:20:23 -08:00
epriestley
7cbbe2ccf7 When users browse to a submodule path in Diffusion explicitly, don't fatal
Summary: Ref T13030. See PHI254. This behavior could be cleaner than I've made it, but it fixes the "this is totally broken" issue, replacing a fatal/exception with an informative (just not terribly useful) page.

Test Plan:
  - Added a submodule to a repository.
  - In Diffusion, clicked some other file next to the submodule, then edited the URI to the submodule path instead.
    - Before patch: fatal.
    - After patch: relatively useful message about this being a submodule.

Note that it's normally hard to hit this URI directly. In the browse view, submodules are marked up as directories and linked to a separate submodule resolution flow.

{F5321524}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13030

Differential Revision: https://secure.phabricator.com/D18831
2017-12-18 09:18:22 -08:00
epriestley
5295840a4c Restore the "Download from Git LFS" UI button to Diffusion
Summary: Depends on D18827. Ref T7789. See PHI204. See PHI131. This button got accidentally removed in Diffusion refactoring (`$data` is no longer used).

Test Plan: {F5321459}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T7789

Differential Revision: https://secure.phabricator.com/D18828
2017-12-18 09:13:19 -08:00
epriestley
8e416474c0 Add a Herald pre-commit field for detecting LFS usage
Summary: Depends on D18825. Ref T7789. See PHI131. Allows installs to selectively disable LFS by adding Herald rules to block commits that use LFS.

Test Plan:
  - Wrote an LFS rule ("When commit uses git lfs, block commit").
  - Pushed an LFS commit: rejected.
  - Pushed a non-lFS commit: success.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T7789

Differential Revision: https://secure.phabricator.com/D18827
2017-12-18 09:12:52 -08:00
epriestley
e34b4bbd90 Move the Git LFS gate to dedicated (non-prototype) config
Summary: See PHI131. Ref T7789. Although this probably isn't 100% complete, there don't seem to be any actual, known, practical blocking issues remaining (everything is either heresay or not reproducible).

Test Plan: Tried to push LFS locally, got blocked with a helpful message. Enabled setting, tried to push LFS locally, got a successful push.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T7789

Differential Revision: https://secure.phabricator.com/D18825
2017-12-18 09:12:22 -08:00
epriestley
a989dd181d Fix Mercurial commit history ordering
Summary:
See <https://discourse.phabricator-community.org/t/diffusion-observed-mercurial-repository-history-broken/825>.

In D18769, I rewrote this from using the `--branch` flag (which is unsafe and does not function on branches named `--config=x.y` and such).

However, this rewrite accidentally changed the result order, which impacted Mercurial commit hisotry lists and graphs. Swap the order of the constraints so we get newest-to-oldest again, as expected.

Test Plan: Viewed a Mercurial repository's history graph, saw sensible chronology after the patch.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18817
2017-12-05 12:12:44 -08:00
lkassianik
46d496b8cc Fix error for URL's that could mean several commits
Summary: Ref T13001, URLs that return multiple commits should show a list of those commits. Not sure if the actual list looks very pretty this way, but was wondering if this approach was vaguely correct.

Test Plan:
- Navigate to `install/rPbd3c23`
- User should see a list view providing links to `install/rPbd3c2355e8e2b220ae5e3cbfe4a057c8088c6a38` and `install/rPbd3c239d5aada68a31db5742bbb8ec099074a561`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T13001

Differential Revision: https://secure.phabricator.com/D18816
2017-12-05 19:24:57 +00:00
lkassianik
42034e6739 Property list view on Diffusion commits should show build status but not Subscriptions, Projects, or Tokens
Summary: Ref T13019, adds build status back to Diffusion commits

Test Plan: Open a Diffusion commit that has a build status, property list view should show the build status, but not Subscriptions, Projects, or Tokens.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T13019

Differential Revision: https://secure.phabricator.com/D18813
2017-12-01 18:16:26 +00:00
epriestley
5240cffd9c Fix an issue where Diffusion could fatal if the default branch was deleted
Summary: See PHI234. In T12931 we improved the behavior of Diffusion when a repository's default branch is set to a branch that does not exist, but in T11823 the way refcursors work changed, and we can now get a cursor (just with no positions) back for a deleted branch. When we did, we didn't handle things gracefully.

Test Plan:
  - Set default branch to a deleted branch, saw nice error instead of fatal.
  - Set default branch to a nonexistent branch which never existed, saw nice error.
  - Set default branch to existing "master", saw repository normally.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18811
2017-11-30 14:06:41 -08:00
epriestley
14cc0abeb3 Fix several safety issues with repository URIs
Summary:
See PHI234. Several issues here:

  - The warning about observing a repository in Read/Write mode checks the raw I/O type, not the effective I/O type. That means we can fail to warn if other URIs are set to "Default", and "Default" is "Read/Write" in practice.
  - There's just an actual typo which prevents the "Observe" version of this error from triggering properly.

Additionally, add more forceful warnings that "Observe" and "Mirror" mean that you want to //replace// a repository with another one, not that we somehow merge branches selectively. It isn't necessarily obvious that "Observe" doesn't mean "merge/union", since the reasons it can't in the general case are somewhat subtle (conflicts between refs with the same names, detecting ref deletion).

Test Plan:
Read documentation. Hit the error locally by trying to "Observe" while in Read/Write mode:

{F5302655}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18810
2017-11-30 14:06:21 -08:00
Aviv Eyal
d8f2630d5c Modernize QuickSearch typeahead
Summary:
Use ClassQuery to find datasources for the quick-search.

Mostly, this allows extensions to add quicksearches.

Test Plan:
using `/typeahead/class/`, tested several search terms that make sense.
Removed the tag interface from a datasource, which removed it from results.

Reviewers: epriestley, amckinley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18760
2017-11-30 15:07:49 +00:00
epriestley
2d4a158356 Fix a bad link target in Diffusion content search results
Summary: See <https://discourse.phabricator-community.org/t/broken-links-in-diffusion-pattern-search-results-page/757>. This links to the display path, which is incorrect.

Test Plan:
  - In any repository, browsed into a directory.
  - Used pattern search to search for something that hits results.
  - Clicked the title (filename/path) of a result table.
    - Before patch: URL omits path context, 404 or wrong result.
    - After patch: taken to proper page.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18779
2017-11-22 11:16:14 -08:00
epriestley
a7921a4448 Filter and reject "--config" and "--debugger" flags to Mercurial in any position
Summary:
Ref T13012. These flags can be exploited by attackers to execute code remotely. See T13012 for discussion and context.

Additionally, harden some Mercurial commands where possible (by using additional quoting or embedding arguments in other constructs) so they resist these flags and behave properly when passed arguments with these values.

Test Plan:
  - Added unit tests.
  - Verified "--config" and "--debugger" commands are rejected.
  - Verified more commands now work properly even with branches and files named `--debugger`, although not all of them do.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13012

Differential Revision: https://secure.phabricator.com/D18769
2017-11-10 08:42:07 -08:00
epriestley
80ebe401e5 Tweak padding/spacing on Diffusion blame view for profile pictures
Summary: Give profile images a little more space, fix "/" spacing, add a tooltip.

Test Plan: {F5251205}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18749
2017-10-31 12:56:36 -07:00
epriestley
bde71324f8 Show small author portraits in Diffusion blame view
Summary: Depends on D18746. See PHI174. Adds small author portraits next to each blame line (this is similar to GitHub).

Test Plan:
My local test data isn't that great since I don't have commits from a lot of accounts, but looks functional:

{F5251056}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18747
2017-10-31 12:10:45 -07:00
epriestley
90d0f8ac6c Revert changes to Diffusion blame view
Summary:
Ref PHI174. This reverts most of these changes:

- 37843127e9 / D18481
- 94cad30ac3 / D18474
- 12ae08b6b1 / D18473
- 0a01334172 / D18462
- ac91ab1ef9 / D18452

These changes made the Diffusion blame view very similar to GitHub's blame view. See D18452 for a before/after of the bulk of these changes; the other revisions are bugfixes.

I think this was generally a step backward, and not motivated by solving a specific problem. I've found the new UI less usable than the old one, and at least one install (see PHI174) also has.

In particular, the revision/commit titles are very bulky and not terribly useful; the date column also isn't terribly useful; the "age" color actually IS pretty useful and was heavily de-emphasized.

I've kept one bugfix here (missing `'a'` tag type) and kept the upgraded icon for "Skip Past This Commit".

I'm going to follow this up with some additional changes:

  - Show a small author profile icon, similar to GitHub, to address PHI174 more directly.
  - Try a zebra-stripe on blocks of rows to make it more clear where changes affected by a particular commit begin and end.
  - Try a hue shift, not just a brightness/saturation shift, to make the "age" color more distinct.
  - Try computing colors as even steps, not based purely on age. Currently, if a file has one long-distant commit and several recent commits, all the recent ones show up as very bright green. I think this would probably be more useful if they were distributed more evenly across the available color bands.

Test Plan:
Viewed blame views in Diffusion, saw a more compact UI similar to the old UI.

{F5251019}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18746
2017-10-31 11:54:47 -07:00
epriestley
157f47cd14 Rewrite CommitQuery to use UNION for performance
Summary:
Ref T12680. See PHI167. See that task for discussion.

Rewrite `DiffusionCommitQuery` to work more like `DifferentialRevisionQuery`, and use a UNION to find "all revisions you need to audit OR respond to".

I tried to get this working a little more cleanly than RevisionQuery does, and can probably simplify that now.

Test Plan: Poked at the UI locally without hitting any apparent issues, but my local data is pretty garbage at this point. I'll take a look at how the query plans work on `secure`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12680

Differential Revision: https://secure.phabricator.com/D18722
2017-10-23 10:32:24 -07:00
epriestley
65f13b156f Improve "refengine" performance for testing large numbers of Mercurial branches
Summary:
See PHI158. In the RefEngine, we test if any old branch positions have been removed from the repository. This is uncommon (but not impossible) in Mercurial, and corresponds to users deleting branches in Git.

Currently, we end up running `hg log` for each position, in parallel. Because of Python's large startup overhead, this can be resource intensive for repositories with a large number of branches.

We have to do this in the general case because the caller may be asking us to resolve `tip`, `newfeature`, `tip~3`, `9`, etc. However, in the specific case where the refs are 40-digit hashes, we can bulk resolve them if they exist, like this:

```
hg log ... --rev (abcd or def0 or ab12 or ...)
```

In the general case, we could probably do less of this than we currently do (instead of testing all old heads, we could prune the list by removing commits which we know are still pointed to by current heads) but that's a slightly more involved change and the effect here is already dramatic.

Test Plan:
Verified that CPU usage drops from ~110s -> ~0.9s:

Before:

```
epriestley@orbital ~/dev/phabricator $ time ./bin/repository refs nss
Updating refs in "nss"...
Done.

real	0m14.676s
user	1m24.714s
sys	0m21.645s
```

After:

```
epriestley@orbital ~/dev/phabricator $ time ./bin/repository refs nss
Updating refs in "nss"...
Done.

real	0m0.861s
user	0m0.882s
sys	0m0.213s
```

  - Manually resolved `blue`, `tip`, `9`, etc., got expected results.
  - Tried to resolve invalid hashes, got expected result (no resolution).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18717
2017-10-20 11:09:14 -07:00
Dmitri Iouchtchenko
9bd6a37055 Fix spelling
Summary: Noticed a couple of typos in the docs, and then things got out of hand.

Test Plan:
  - Stared at the words until my eyes watered and the letters began to swim on the screen.
  - Consulted a dictionary.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D18693
2017-10-09 10:48:04 -07:00
epriestley
85011a46d0 Bail out of PhabricatorRepositoryGraphCache more aggressively after cache fills
Summary:
Ref PHI109. Ref T11786. We currently test elapsed time every 64 iterations (since iterations are normally very fast), but at least one install is seeing the page timeout after 30 seconds.

One reason could be that cache fills may occur, and are likely to be much slower than normal iterations. In an extreme case, we could do 64 cache fills before checking the time. Tweak thing so that we always check the time after doing a cache fill, regardless of how many iterations have elapsed since the last attempt.

Additionally, this API method currently accepts an arbitrary number of paths, but implicitly limits each cache query to 500ms. If more than 60 paths are passed, this may exceed 30s. Only let the cache churn for a maximum of 10s across all paths.

If this is more the latter issue than the former, this might replace the GraphCache timeouts with `git` timeouts, but at least our understanding of what's going on here will improve.

Test Plan: This is difficult to test convincingly locally, since I can't reproduce the original issue. It still works after these changes, but it worked fine before these changes too.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T11786

Differential Revision: https://secure.phabricator.com/D18692
2017-10-06 14:12:58 -07:00
epriestley
89fe84f978 Add a "/source/..." URI for Diffusion commits which redirects
Summary:
See PHI112. The install presumably wants to generate links to Diffusion commits from an external tool, but only knows the short name of the repository.

Provide a `/source/phabricator/commit/abcdef908273` URI which redirects to the canonical URI for the commit.

Test Plan:
  - Visited `/source/` URI for a commit, got a redirect.
  - Visited normal URI for a commit, got a commit page.
  - Visited `/branches/` and `/tags/` for a `/source/` repository, got proper pages.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18676
2017-10-03 13:27:03 -07:00
epriestley
a39f5e1113 Correct bad context path when doing pattern search inside a repository
Summary:
Ref PHI101. It looks like this was maybe copy/pasted by mistake in recent design refactoring.

We need to pass the full path, not the `basename()` of the path, to the search form.

Test Plan: Searched inside `scripts/test/`, found results inside `scripts/test/`.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18664
2017-09-29 14:51:49 -07:00
epriestley
03e5d69817 Fix an error in Diffusion when the Owners application is uninstalled
Summary:
See <https://discourse.phabricator-community.org/t/undefined-view-when-owners-is-uninstalled/451>.

When Owners is not installed, Diffusion can fatal with a bad `$view`.

Test Plan:
  - Uninstall Owners.
  - View the content of any file in Diffusion.
  - Before: fatal on `$view` undefined.
  - After: Valid page with no owners information.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18629
2017-09-19 09:42:42 -07:00
epriestley
b352cacdd9 Swap "-R" and "serve" argument order for Mercurial
Summary: See <https://discourse.phabricator-community.org/t/unable-to-use-current-mercurial-on-debian-stretch/391/13>. I missed that `-R` is order-sensitive.

Test Plan: Verified both orders work on 3.5.2.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18616
2017-09-15 13:07:43 -07:00
epriestley
8982e3e52d Update major RefCursor callsites to work properly with RefPosition
Summary:
Ref T11823. This is the meaty part of the change, and updates `RefEngine` to use separate RefCursor (for names) and RefPosition (for actual commit positions) tables.

I'll hold this whole series until after the release cut so it has some time to bake on `secure` to look for issues. It's also not a huge problem if there are bugs here since these tables are just caches anyway, although they do feed into some other things, and obviously it's never good to have bugs.

Test Plan:
  - This logic can be invoked directly with `bin/repository refs <repository> --trace --verbose`.
  - Ran that on unchanged repositories, new branches, removed branches, and modified branches. Saw appropriate output and cursor positions.
  - Ran on a mercurial repository to test the close/open logic, saw it correct open/closed state of incorrect positions.
  - Browed around Diffusion in various repositories.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T11823

Differential Revision: https://secure.phabricator.com/D18614
2017-09-15 10:21:32 -07:00
epriestley
bd923d1ce0 Provide an explicit "-R" flag to "hg serve"
Summary:
See <https://discourse.phabricator-community.org/t/unable-to-use-current-mercurial-on-debian-stretch/391>.

The Mercurial commit is helpful in particular: <https://www.mercurial-scm.org/repo/hg/rev/77eaf9539499>

We weren't vulnerable to the security issue (users can not control any part of the command) but pass the working directory explicitly to get past the new safety check.

I left `setCWD()` in place (a few lines below) just because it can't hurt, and in some other contexts it sometimes matter (for example, if commit hooks execute, they might inherit the parent CWD here or in other VCSes).

Test Plan:
  - Cloned from a Mercurial repo locally over HTTP.
  - Verified that SSH cloning already uses `-R` (it does, see `DiffusionMercurialServeSSHWorkflow`).
  - Did not actually upgrade to Mercurial 4.0/4.1.3 to completely verify this, but a user in the Discourse thread asserted that a substantially similar fix worked correctly.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18611
2017-09-15 08:57:11 -07:00
epriestley
5ae3af6691 Fix an outdated HTML anchor link in Diffusion table of contents
Summary:
See <https://discourse.phabricator-community.org/t/navigating-to-changed-files-in-diffusion-does-not-work-anymore/433>.

In D18465, I updated these but this hard-coded the anchor for some reason (???) and I missed it while `grep`-ing.

Test Plan: Viewed a commit (`/rXYZaaaa`) and clicked a file link in the table of contents. Got modern `#change-...` anchor and navigation into the document.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18609
2017-09-15 08:37:36 -07:00
Chad Little
a46a9ff165 Update VCS password UI
Summary: Miss this with earlier pass, updates the VCS password page.

Test Plan: Try to set a vcs password

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18574
2017-09-07 15:50:05 -07:00
Chad Little
3b2accee7c Add a border on diffusion uri page
Summary: This should have a border

Test Plan: Reload page

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18549
2017-09-06 20:44:37 +00:00
Chad Little
fc893658b8 Update menu item names for Applications -> Favorites
Summary: Adds a `MenuName` method to applications that `ProfileMenuItem` uses instead of the application name if set. This improves the home/menu/new user experience at little cost. Also renamed the label from Applications to Favorites, since this menu gets altered to provide more than just applications. This also allows instances to set back to Maniphest if they so choose. Overall I think this direction resolves 95% of my concerns, with maybe a small potential downside which I don't really anticipate. We already name Dashboard panels by their object, and that hasn't really caused confusion. I think these links are similar. I click 'Tasks' and get presented a list of my tasks from Maniphest.

Test Plan: Review each of the name changes as a default new install and a modified install.

Reviewers: epriestley, amckinley

Reviewed By: epriestley

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18524
2017-09-05 19:05:03 -07:00
Chad Little
e1fd74ddb5 Update Repository Management pages to new fixed UI
Summary: Simplifies the Repository Management pages to the new fixed column layout. I've also moved "Status" into the Basics page, which feels better, and moved "Documentation" as a nav item to a button in the header. This removed "action list" and "curtain view" from the management panels and uses the new bits from Config/Phacility. Undecided if the icons should stay or go for the nav. Left them in for Diffusion. I want to update the EditEngine pages to display in this UI and not leave the portal, but I haven't dug into that this page. I'm a bit worried it will not easily be possible.

Test Plan:
Generate a svn, git, hg repository, test each of the new pages and each of the new buttons. Activate, deactivate, etc.

{F5164674}

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18523
2017-09-05 19:01:27 -07:00
Chad Little
2ba5968b76 Mobile layouts for Diffusion
Summary: Implements a new mobile view thats more fullscreen, not boxed, so more space. Fixes issues with mobile tables when scrolling overflowed content.

Test Plan: Test home, branch, tags, code, file browse, graph, compare, history, readme, open revisions, owners.

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18505
2017-08-30 12:28:00 -07:00
Chad Little
67c658a7ed Use selected button state on blame button
Summary: Visually selects the button if blame is on.

Test Plan: Turn blame on and off in Diffusion on a file.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18504
2017-08-30 18:33:42 +00:00
Chad Little
b8b701faf7 Clarify language when Autoclose is disabled for a repository
Summary: Fixes T12051, adds additional language.

Test Plan:
Disable Autoclose in Actions, see updated language under Branches.

{F5147291}

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Maniphest Tasks: T12051

Differential Revision: https://secure.phabricator.com/D18482
2017-08-28 12:00:46 -07:00
Chad Little
37843127e9 Widen blame age line in blame view
Summary: 50% more line, no additional cost! Order Now! Operators are standing by.

Test Plan: Blame a file

Reviewers: epriestley, avivey

Reviewed By: avivey

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18481
2017-08-28 11:32:06 -07:00
Chad Little
94cad30ac3 Fix bad tables in diffusion blame
Summary: My fake data was 100%, and not all tables have full revision history. This leads to a broken table. Instead check if we have //any// revisions at all, then always show the column, with or without a link inside.

Test Plan: going on a limb this is the correct fix and test on secure... again ...

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18474
2017-08-24 20:02:35 -07:00
Chad Little
12ae08b6b1 Move differential revision to its own table column in blame view
Summary: There is still some layout issues with revisions, so I've tested it better and moved it to it's own column

Test Plan: Fake in some revision data, test various sizes and shapes.

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18473
2017-08-24 19:36:42 -07:00
Chad Little
8c4f5aba33 Fix Back to HEAD link
Summary: I missed an anchor tag here, adds it back

Test Plan: View blame, click a previous version of the file, click Back to HEAD link.

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Differential Revision: https://secure.phabricator.com/D18451
2017-08-23 09:47:52 -07:00
Chad Little
ac91ab1ef9 Update blame view in Diffusion
Summary: Ref T12824, adds more information to the blame view, exposes date, commit summary, lighter colors.

Test Plan:
Review many diffs with and without blame on.

{F5111758}

{F5111759}

Reviewers: epriestley

Reviewed By: epriestley

Spies: Korvin

Maniphest Tasks: T12824

Differential Revision: https://secure.phabricator.com/D18452
2017-08-23 09:47:35 -07:00
Chad Little
a145d00be6 Update Diffusion File UI for single column
Summary: Moves browseFile to single column, implements Owners as a list under the file (and now directory as well), improved information listed in Owners, and moves actions into the Diffusion action bar instead of the header.

Test Plan:
Test browsing directories, files, text, images, binaries, enabling blame. Mobile and desktop.

{F5111045}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18448
2017-08-21 13:35:25 -07:00
Chad Little
d2a3f2da73 Add indication of hg branch open/closed in branch list
Summary: Adds some basic UI for open / closed state when viewing a list of branches in Mercurial. Fixes T12838

Test Plan: Close and open branches, view list.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12838

Differential Revision: https://secure.phabricator.com/D18447
2017-08-21 09:09:16 -07:00
Chad Little
295c806219 Hide branch status if repository is not hg
Summary: Better table layouts here for branches view

Test Plan: Test git, hg repositories. See column go away.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18444
2017-08-17 14:44:55 -07:00
Chad Little
864dd9a196 List branch on main repository view
Summary: This is in the crumbs, but a little hidden. Puts branch name at the top of the browse table header.

Test Plan: Review a few branchs, change branch, see new name.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18441
2017-08-17 12:15:53 -07:00
Chad Little
281fc19f3f Add more information to Branch status page in Manage Repository
Summary: Adds an icon for default branch, status for branch status

Test Plan: Review `hg` and `git` repositories, change default branch, etc.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18443
2017-08-17 12:09:20 -07:00
Chad Little
053cab4d59 Update VCS Password settings page
Summary: Use proper background.

Test Plan: Visit page, see correct background.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18435
2017-08-17 08:51:38 -07:00
Chad Little
5019960b61 Set border on crumbs on Lint page
Summary: Minor, sets the border, corrects a page header.

Test Plan: View lint pages

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18433
2017-08-16 12:19:35 -07:00
Chad Little
19dae88728 Add branch, tag info to Diffusion Headers
Summary: Improves overall UX of browsing Diffusion. Clarifies branch and tag when possible, changes 'home' to 'code', uses tabs in more locations. Fixes T12837

Test Plan: Review branchs, tags, git, hg, search, browse, history.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12837

Differential Revision: https://secure.phabricator.com/D18434
2017-08-16 12:16:15 -07:00
Chad Little
7bbd26427f Add pattern search to diffusion home
Summary: Moves the method up to DiffusionController, so it can be more universally used. Also now center aligns tabs on mobile. Still todo, get search nicely toggled on mobile

Test Plan: Test mobile, desktop. Test search from home, from browse, and browsing a specific path.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18432
2017-08-15 14:16:33 -07:00
Chad Little
f4fdb92e13 Move Diffusion Actions into action bar on home
Summary: Moving this down the the "bar" to allow pattern search on home. Rebuilds the mobile layout a little.

Test Plan:
Test actions on mobile, desktop, tablet.

{F5100460}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18431
2017-08-15 12:19:49 -07:00
Chad Little
3a50ea4f47 Simplify Create Repository page
Summary: Also adds images, nice images.

Test Plan: Create a repository, test mobile, tablet, desktop layouts.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18430
2017-08-15 11:05:50 -07:00
Chad Little
4d335b7bef Build a basic DiffusionPatternSearchView
Summary: Roughs this in a little, kinda basic. Allows for grouping results by page. A bit better on mobile. Would like more content return from conduit though.

Test Plan:
Test `CMS`, `cms`, and `OMGLOLWTFBBQ`, desktop and mobile

{F5099081}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18429
2017-08-15 06:38:47 -07:00
Chad Little
0a9ad6d5e7 Move pattern search into Diffusion header
Summary: This is only on browse pages, but I think could be global (home) also. Moves it from a button, field, to just a field.

Test Plan:
Review search on desktop, mobile.

{F5098886}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18428
2017-08-14 19:03:56 -07:00
Chad Little
37489865d4 Remove "File Name" search tool
Summary: Removing this cleanly in event we want to put it back later. 99% of these cases are likely workable either by command line or the typeahead. Will gauge feedback if users notice.

Test Plan: Reload page, perform file grep search.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18425
2017-08-14 18:58:25 +00:00
Chad Little
07c0032491 Add a link directly to Browse in Diffusion
Summary: Getting to the straight browse view went away, this adds a link back. I'll look at more long term solution for getting to grep search.

Test Plan: Click on header, get take to browse view.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18421
2017-08-14 11:14:14 -07:00
epriestley
0b1d6a3f6e Convert straggling Herald rules to modern revision status constants
Summary:
Ref T2543. These are the last `ArcanistDifferentialRevisionStatus` callsites.

This removes the very old legacy `precommitRevisionStatus` field, which has no other readers. This was obsoleted by the `CLOSED_FROM_ACCEPTED` stuff, but retained for compatibility.

Test Plan:
  - Poked these with the test console, although they're a little tricky to be sure about.
  - Grepped for `ArcanistDifferentialRevisionStatus`, no more hits.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2543

Differential Revision: https://secure.phabricator.com/D18416
2017-08-11 17:22:05 -07:00
epriestley
50dfdb8d03 Replace legacy Differential queries for "open" revisions with a modern mechanism
Summary: Ref T2543. Several queries want only open revisions. Provide a tailored, non-legacy way to issue that query.

Test Plan: Viewed some of these callsites (e.g., "Similar open revisions affecting these files"), saw only open revisions.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2543

Differential Revision: https://secure.phabricator.com/D18395
2017-08-11 13:37:11 -07:00
epriestley
794e185bf9 Pass SSH wrappers to VCS commands unconditonally, not just if there's an SSH remote
Summary:
Ref T12961. In Mercurial, it's possible to have "subrepos" which may use a different protocol than the main repository.

By putting an SSH repository inside an HTTP repository, an attacker can theoretically get us to execute `hg` without overriding `ui.ssh`, then execute code via the SSH hostname attack.

As an immediate mitigation to this attack, specify `ui.ssh` unconditionally. Normally, this will have no effect (it will just be ignored). In the specific case of an SSH repo inside an HTTP repo, it will defuse the `ssh` protocol.

For good measure and consistency, do the same for Subversion and Git. However, we don't normally maintain working copies for either Subversion or Git so it's unlikely that similar attacks exist there.

Test Plan:
  - Put an SSH subrepo with an attack URI inside an HTTP outer repo in Mercurial.
  - Ran `hg up` with and without `ui.ssh` specified.
  - Got dangerous badness without `ui.ssh` and safe `ssh` subprocesses with `ui.ssh`.

I'm not yet able to confirm that `hg pull -u -- <uri>` can actually trigger this, but this can't hurt and our SSH wrapper is safer than the native behavior for all Subversion, Git and Mercurial versions released prior to today.

Reviewers: chad

Reviewed By: chad

Subscribers: cspeckmim

Maniphest Tasks: T12961

Differential Revision: https://secure.phabricator.com/D18389
2017-08-10 17:49:55 -07:00
Chad Little
a7124f8f7a Add status table to Diffusion Branch manage page
Summary: Fixes T12832. Adds a basic table (not paginated?) to view tracking and autoclose status.

Test Plan:
Review a large repository (Krita) with setting various states of tracking and autoclose.

{F5092117}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12832

Differential Revision: https://secure.phabricator.com/D18386
2017-08-10 13:26:30 -07:00
epriestley
46d1596bf7 Pull legacy revision query status filters out of the main Query class
Summary:
Ref T2543. Currently, Differential uses a set of hard-coded query filters (like "open" and "closed") to query revisions by status (for example, "open" means any of "review, revision, changes planned, accepted [usually]").

In other applications, like Maniphest, we've replaced this with a low level list of the actual statuses, plus higher level convenience UI through tokenizer functions. This basically has all of the benefits of the hard-coded filters with none of the drawbacks, and is generally more flexible.

I'd like to do that in Differential, too, although we'll need to keep the legacy maps around for a while because they're used by `differential.find` and `differential.getrevision`. To prepare for this, pull all the legacy stuff out into a separate class. Then I'll modernize where I can, and we can get rid of this junk some day.

Test Plan: Grepped for `RevisionQuery::STATUS`. Ran queries via Differential UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2543

Differential Revision: https://secure.phabricator.com/D18343
2017-08-09 11:06:15 -07:00
epriestley
2e36653965 Reduce callsites to "ArcanistDifferentialRevisionStatus" in Phabricator
Summary:
Ref T2543. These are currently numeric values, like "0" and "3". I want to replace them with strings, like "accepted", and move definitions from Arcanist to Phabricator.

To set the stage for this, reduce the number of callsites where Phabricator invokes `ArcanistDifferentialRevisionStatus`.

This is just the easy ones. I'll hold this until the release cut.

Test Plan:
- Called `differential.find`.
- Called `differential.getrevision`.
- Called `differential.query`.
- Removed all reviewers from a revision, saw warning.
- Abandoned the no-reviewers revision, no more warning.
- Attached a revision to a task to get it to show the state icon with the status on a tooltip.
- Viewed revision bucketing on dashboard.
- Used `bin/search index` to reindex a revision.
- Hit the "Land Revision" endpoint.

I didn't explicitly test these cases:

  - Doorkeeper Asana integration, since setup takes a thousand years.
  - Disambiguation logic when multiple hashes match, since setup is also very involved.
  - Releeph because it's Releeph.

Reviewers: chad

Reviewed By: chad

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T2543

Differential Revision: https://secure.phabricator.com/D18339
2017-08-09 11:04:52 -07:00
Chad Little
b1e3cf627d Add more repo images
Summary: Just a few more.

Test Plan: Edit Picture, see new image, choose image.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18370
2017-08-08 17:51:15 -07:00
epriestley
f48f2dae9f Move Phabricator to use PhutilBinaryAnalyzer and show binary versions
Summary:
Fixes T12942.

  - Adds binary version and path information to {nav Config > Version Information}.
  - Replaces old code all over the place with new consolidated code.

Test Plan:
{F5073531}

Also faked some cases of missing binaries, bad versions, etc.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12942

Differential Revision: https://secure.phabricator.com/D18306
2017-08-01 07:14:48 -07:00
epriestley
ee884db1f9 Don't fatal when viewing tags pointing at commits we haven't imported/parsed yet
Summary:
In Diffusion, the "Tags" view may read commits which haven't imported or parsed yet, and thus don't have loadable objects.

Most of this logic tests for `if ($commit)`, but the author part did not. Instead, don't render author information if `$commit` is not present.

Test Plan:
  - Loaded tags view with commits present.
  - Faked `$commit = null;`, loaded tag view, got this instead of a fatal:

{F5068432}

Reviewers: chad, amckinley

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D18290
2017-07-28 10:43:24 -07:00
epriestley
8034b9d819 Don't require a device be registered in Almanac to do cluster init/resync steps
Summary:
Fixes T12893. See also PHI15. This is complicated but:

  - In the documentation, we say "register your web devices with Almanac". We do this ourselves on `secure` and in the production Phacility cluster.
  - We don't actually require you to do this, don't detect that you didn't, and there's no actual reason you need to.
  - If you don't register your "web" devices, the only bad thing that really happens is that creating repositories skips version initialization, creating the bug in T12893. This process does not actually require the devices be registered, but the code currently just kind of fails silently if they aren't.

Instead, just move forward on these init/resync phases even if the device isn't registered. These steps are safe to run from unregistered hosts since they just wipe the whole table and don't affect specific devices.

If this sticks, I'll probably update the docs to not tell you to register `web` devices, or at least add "Optionally, ...". I don't think there's any future reason we'd need them to be registered.

Test Plan:
This is a bit tough to test without multiple hosts, but I added this piece of code to `AlmanacKeys` so we'd pretend to be a nameless "web" device when creating a repository:

```
if ($_REQUEST['__path__'] == '/diffusion/edit/form/default/') {
  return null;
}
```

Then I created some Git repositories. Before the patch, they came up with `-` versions (no version information). After the patch, they came up with `0` versions (correctly initialized).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12893

Differential Revision: https://secure.phabricator.com/D18273
2017-07-25 05:12:10 -07:00
Chad Little
69a7d57c3f Add a branch selector to Diffusion
Summary: Fixes T12931. Adds a branch selector that's always visible if the repo has commits.

Test Plan:
Test a plain hg, svn, git repository. Test setting a bad default branch. Test a good default branch. Test on desktop, mobile layouts.

{F5058061}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12931

Differential Revision: https://secure.phabricator.com/D18267
2017-07-24 13:41:23 -07:00
epriestley
b8cd5b0eb8 Use a less-esoteric spelling of "capabilities" in several places
Summary: This spelling can definitely feel a little overplayed at times, but I still think it's a gold standard in spellings of "capabilities".

Test Plan: Felt old and uncool.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D18215
2017-07-12 15:27:57 -07:00
Chad Little
0c4cff28df Clean up NUX a bit on Diffusion
Summary: Just some cleanup. Make sure action-bar has consistent space if locate is there or not, hide tabs if repository has no content. Use clone or checkout language depending on SCM. Fixes T12915.

Test Plan:
Test git, hg, svn blank states.

{F5042707}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12915

Differential Revision: https://secure.phabricator.com/D18208
2017-07-12 07:05:33 -07:00
Chad Little
db57da0f74 Fix SVN form_box error
Summary: Fixes T12915.

Test Plan: Test a SVN repository locally, ensure page loads.

Reviewers: epriestley

Subscribers: Korvin

Maniphest Tasks: T12915

Differential Revision: https://secure.phabricator.com/D18207
2017-07-11 19:51:34 -07:00
Chad Little
a6b550ba03 Move Clone Repository to Dialog
Summary: This moves the clone details on the Repository Home to a button / dialog. Functionally this is to pull content on the page way up, while giving full space to all the clone options. I think we can build this into some FancyJS if needed, but this seems to clean ui the UI dramatically with little overhead. I don't want to attempt the JS dropdown unless we're sure that's the best path (it exposes the most common URI by default, saving a click).

Test Plan: Tested hg, svn, git repositories and the raw URL page. Test close button.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18203
2017-07-11 13:16:47 -07:00
Chad Little
646ad36b15 Move actions into Diffusion header
Summary: This moves actions into the Diffusion main header, removes the locate file box, and widens description and cloning details. Projects are not currently in this layout, but will follow up in another diff. Trying to keep these changes small and iterative.

Test Plan:
Locate some files, test actions dropdown, repository with and without description. Also tablet, mobile layouts.

{F5040026}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18193
2017-07-10 06:51:40 -07:00
Chad Little
39e5da7ea7 Simplify Diffusion Browse Table
Summary: Cleans up colors, removes commit hash and links the text instead. Also unsure how valuable "lint" column is here, but left it. I'd maybe like to understand that workflow since it just seems like clutter overall. Also Fixes T12905

Test Plan:
Review Phabricator, hg, and a few other test repositories locally. Holler if anything here seems bad, but this feels easier to read and use to me.

{F5038425}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12905

Differential Revision: https://secure.phabricator.com/D18189
2017-07-09 09:43:57 -07:00
Chad Little
0b3117bb68 Fix comparison check for SVN in browsing Diffusion
Summary: Fixes T12905. Missed setting this variable.

Test Plan: Browse an SVN repository.

Reviewers: epriestley

Subscribers: Korvin

Maniphest Tasks: T12905

Differential Revision: https://secure.phabricator.com/D18190
2017-07-09 06:43:43 -07:00
Chad Little
e516358d54 Add tabs to Diffusion for consistent navigation
Summary:
Adds a responsive tab bar navigation to Diffusion. Working through the new design here in pieces, so keep in mind M1477 is the target. Notably:

- Removes "branches" and "tags" from RevisionView, now on tabs
- Keeps "browse", "history", "readme" on RevisionView
- Adds tabs for all main views, including Graph... unless how that feels, so let me know.

Test Plan: Browse all pages, desktop and mobile. Test hg, svn, git repositories.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18161
2017-07-05 22:09:36 +01:00
epriestley
a6f0182104 Fix an issue where repositories with hyphens could sort improperly in typeaheads
Summary: Fixes T12894. See that task for discussion.

Test Plan:
  - Created repositories `abcdef`, then `abcdef-a` through `abcdef-f`.
  - Before patch, awkward sort order.
  - After patch, query for `abcdef` hits `abcdef` first.
  - See T12894 for details and screenshots.

Reviewers: chad, amckinley

Reviewed By: chad

Maniphest Tasks: T12894

Differential Revision: https://secure.phabricator.com/D18179
2017-07-03 09:28:02 -07:00
Chad Little
b25b379ca0 Move Diffusion Browse to a single column layout
Summary: The main change here is moving (compare, search, history) into buttons in the header bar on all browse views. This allows Directory Browsing to be full width, since there is no other curtain information. File, Image, LFS, Binary all stay in TwoColumn layouts with the same buttons in the header.

Test Plan: Test viewing a directory, file, image, binary file, readme, and fake a gitlfs.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D17766
2017-07-01 20:45:56 +02:00
epriestley
4e047f7b31 Correct a datasource issue when viewing repository URIs in "Manage Repository"
Summary:
Fixes T12884. In cases other than this UI, applications access URIs through the Repository they're part of. This means that applications interact with URIs which have gone through the correction/adjustment logic in `PhabricatorRepository->attachURIs()`, which fixes up "builtin" URIs to have the right values based on configuration.

In this case (and, as far as I can tell, only this case) we load the URI directly //and// act on its properties which depend on configuration and repository state.

This can mean we're using a different view of the URI than we should be.

To fix this: after loading the URI, reload it through the repository so the relevant adjustments are applied.

I think this is the most reasonable fix. We could try to make `RepositoryURIQuery` somehow enforce this, but the cost of this error is small (mild confusion about display state), the other things which do direct loads don't depend on this state (editing), and everything else loads via a repository and is likely to continue doing that forever.

Test Plan: {F5026633}

Reviewers: chad, amckinley

Reviewed By: chad

Maniphest Tasks: T12884

Differential Revision: https://secure.phabricator.com/D18176
2017-06-30 07:09:53 -07:00
Chad Little
d0898116d8 Add a graph view page to Diffusion
Summary: Fixes T12840. This adds a parallel "graph" button next to history on home and on the history list page. I'll think more about better placement of how to get to this page with the upcoming redesign that's still sitting in Pholio.

Test Plan: View History, View Graph, Try pager, go to a file, click view history, see no graph button.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12840

Differential Revision: https://secure.phabricator.com/D18131
2017-06-19 17:57:20 +02:00
Chad Little
6f7b31fbf8 Add a DiffusionTagListView
Summary: Moves DiffusionTagsListView to uhhh, list. Separates out table view which is still in use now, implements mobile friendly UI for tags.

Test Plan:
Review KDE's Krita repository locally with lots of tags, desktop and mobile.

{F4997708}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12824

Differential Revision: https://secure.phabricator.com/D18115
2017-06-13 11:32:18 -07:00
Chad Little
df6ad07566 Add DiffusionBranchListView for browsing branches
Summary: Adds a new DiffusionBranchListView which replaces the BranchTable when browsing all branches in Diffusion. Has all the same capabilities, but is easier to read, adds a Compare button, and plays nicely on mobile. It does take up more space, but I think that's generally OK here since we expect our branches to not be heaping piles of intern revert branches.

Test Plan:
Follow a few repositories with branches, like Phabricator and KDE's Krita. View layouts on mobile, tablet, desktop. Try out new compare button.

{F4996207}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: avivey, Korvin

Maniphest Tasks: T12824

Differential Revision: https://secure.phabricator.com/D18113
2017-06-13 11:07:03 -07:00
Chad Little
83a89166ee Add profile images to Repositories
Summary: Builds out some images to use to identify repositories. Fixes T12825.

Test Plan:
Try setting custom, built in, and null images.

{F4998175}

{F4998192}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12825

Differential Revision: https://secure.phabricator.com/D18116
2017-06-12 07:51:39 -07:00
epriestley
d5163d0143 Write patterns to "git grep" on stdin instead of passing them with "-e"
Summary:
Fixes T12807. Some shells may apparently mangle/strip UTF8 characters? Just dodge this whole problem by sending the pattern over stdin rather than actually figuring out the particulars.

Related tasks, like T7339 and T5554, discuss finding broader fixes for this class of issue, and this definitely isn't exactly a fully legitimate fix, but in many cases (as here) we can reasonably just avoid the problem rather than actually fixing it, at least for a long time.

Test Plan: Searched for emoji and non-emoji locally, but this worked fine (on OSX) for me before the patch too.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12807

Differential Revision: https://secure.phabricator.com/D18105
2017-06-08 15:25:55 -07:00
Chad Little
b7f147ea0f Clean up user profile commit list view
Summary: Porting over a fix that we could miss the tail end of commits. Also use the new tag borderless option.

Test Plan: Review various commit pages in profile.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18086
2017-06-06 11:07:04 -07:00
Chad Little
ece651255c Optimize mobile layout of DiffusionHistoryView
Summary: Little nits and spacing changes to viewing diffusion commit history on phones.

Test Plan:
Review in Chrome, iOS Simulator.

{F4990749}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18085
2017-06-06 10:29:11 -07:00
Chad Little
65c9d789d2 Add a borderless tag style
Summary: Formally support borderless tags in PHUITagView.

Test Plan: Used in Diffusion History List

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18074
2017-06-04 11:52:35 -07:00
epriestley
5335f29ff2 Correct an issue where the commit list could group commits by server time
Summary:
Commits in the list are grouped by the date they occurred in server time. This may not be the date they occurred in client time.

Use client time, not server time, to group commits.

Test Plan:
- Set server timezone to "Asia/Famagusta".
- Set client timezone to "America/Los_Angeles".
- Viewed Phabricator repository history.

Here's what it looks like before the change:

{F4987094}

Note that the headers of the first two groups both say "Yesterday".

This is because the first commits in each group occurred on June 1 and June 2, respectively, in Famagusta, but both occurred on June 1 in Los Angeles.

Here's what it looks like after the change:

{F4987095}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D18067
2017-06-02 08:39:13 -07:00
epriestley
335c3a7d12 In commit history list view, show all commits
Summary:
Currently, the last group of commits is not shown in the list view because the final `$list` is never added to `$view`.

For example, if the first page would contain commits from "April 7", "April 6", and "April 5", commits from "April 5" are not shown.

(If a repository has 100 commits in a single day, nothing is shown.)

On this server, here's the bottom of page 1:

{F4987087}

Here's the top of page 2:

{F4987088}

However, here's `git log` between those commits:

```
$ git log --oneline 7e46^..5f49f
5f49f9c793 Add sound to logged out Conpherence
1644b45050 Disperse task subpriorities in blocks
c6a7bcfe89 Make Pholio description behave as a remarkup field (e.g., subscribe mentioned users)
bbc5f79227 Make membership lock/unlock feed stories read more naturally
789d57522b Make editing project images redirect to "Manage" more consistently
10b3879232 Make Project slug/hashtag transactions render a little more nicely
abd791889c Update Maniphest title transaction again
5a34b299e4 Update Maniphest title language
601622013d Clarify milestone/subproject creation language
c9889e3d55 Fix an issue in Phriction where moving a document just copied it instead
fdf00f6df4 Clean up some minor UI behaviors in Differential
6c46f27d98 Add quest objectives to the minimap
d783299a19 Fix Phriction status not set property on new document
93e28da76e Add more "disabled" UI to PHUIObjectItemView
7e46d7ab6a Migrate Project color to modular transactions
```

This group of commits does not currently appear anywhere in the list.

Test Plan: Viewed a page of commits, saw 100 commits.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D18066
2017-06-02 08:38:29 -07:00
Chad Little
0d8aba8550 Revert some changes to Diffusion History List
Summary: Ref rPf2fcafb40dde94ddf4ee22716fea74fca0334a64#38208, I think this is a more usable layout. Gets rid of clippy, audit. Adds back Differential link as tag, Build Status as button.

Test Plan: Faked data on this for Differential, Builds, should all work though. Test on real and fake repositories.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18061
2017-06-01 12:39:25 -07:00
Chad Little
fbb7673439 Diffusion History List cleanup
Summary: Removes the odd circle buttons, adds copy-pasta button.

Test Plan: Review new layout locally.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18057
2017-06-01 06:47:32 -07:00
Chad Little
6295e37857 Have Browse button in History actually work
Summary: Ref T12780. Makes the button do something useful, like link to the history at the right spot in the graph.

Test Plan: Click on various browse buttons, get correct url.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12780

Differential Revision: https://secure.phabricator.com/D18054
2017-05-30 20:18:21 -07:00
Chad Little
c5bb69fd7d Use a list view for DiffusionHistory
Summary: This moves Diffusion History to use an easier to parse list view for commits and their (diff, audit, build) status. I left TableView around, which is used on a repositories home, and we can maybe add a "graph view" history back as another controller. Not sure what the real use is for that kind of feature though. I don't have Harbormaster set up locally so I could use another install to give this a run. I also expect to maybe not live with this UI as final, I like the UX, but the icons for indicating status don't really feel great to me, just OK.

Test Plan:
pull various repositories, check various history displays.

{F4980356}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18039
2017-05-30 17:31:48 -07:00
Chad Little
aefc006ba5 Move DiffusionHistoryListView to DiffusionCommitListView
Summary: I think this name is more accurate, also add proper links to author image.

Test Plan: Review commits in sandbox, see new URL on image.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D18026
2017-05-26 13:11:59 -07:00