1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-25 15:00:58 +01:00
Commit graph

14199 commits

Author SHA1 Message Date
epriestley
da40f80741 Update PhabricatorLiskDAO::chunkSQL() for new %Q semantics
Summary:
Ref T13217. This method is slightly tricky:

  - We can't safely return a string: return an array instead.
  - It no longer makes sense to accept glue. All callers use `', '` as glue anyway, so hard-code that.

Then convert all callsites.

Test Plan: Browsed around, saw fewer "unsafe" errors in error log.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13217

Differential Revision: https://secure.phabricator.com/D19784
2018-11-13 08:59:18 -08:00
epriestley
315d857a8a Add a basic web UI for intracluster sync logs
Summary: Depends on D19798. Ref T13216. This puts at least a basic UI on top of sync logs.

Test Plan:
Viewed logs from the web UI and exported data. Note that these syncs are somewhat simulated since I my local cluster is somewhat-faked (i.e., not actually multiple machines).

{F5995899}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19799
2018-11-10 04:58:36 -08:00
epriestley
1d7c960531 Put push log "hookWait" to data export and add all wait values to UI
Summary:
Depends on D19797. Ref T13216.

  - Put the new `hookWait` in the export and the UI.
  - Put the existing waits in the UI, not just the export.
  - Make order consistent: host, write, read, hook (this is the order the timers start in).

Test Plan: Pushed some stuff, viewed web UI and saw sensible numbers, exported data and got the same values.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19798
2018-11-10 04:47:38 -08:00
epriestley
2a7ac8e388 Make "bin/repository thaw" workflow more clear when devices are disabled
Summary:
Ref T13216. See PHI943. If autoscale lightning strikes all your servers at once and destroys them, the path to recovery can be unclear. You're "supposed" to:

  - demote all the devices;
  - disable the bindings;
  - bind the new servers;
  - put whatever working copies you can scrape up back on disk;
  - promote one of the new servers.

However, the documentation is a bit misleading (it was sort of written with "you lost one or two devices" in mind, not "you lost every device") and demote-before-disable is unnecessary and slightly risky if servers come back online. There's also a missing guardrail before the promote step which lets you accidentally skip the demotion step and end up in a confusing state. Instead:

  - Add a guard rail: when you try to promote a new server, warn if inactive devices still have versions and tell the user to demote them.
  - Allow demotion of inactive devices: the order "disable, demote" is safer and more intuitive than "demote, disable" and there's no reason to require the unintuitive order.
  - Make the "cluster already has leaders" message more clear.
  - Make the documentation more clear.

Test Plan:
  - Bound a repository to two devices.
  - Wrote to A to make it a leader, then disabled it (simulating a lightning strike).
  - Tried to promote B. Got a new, useful error ("demote A first").
  - Demoted A (before: error about demoting inactive devices; now: works fine).
  - Promoted B. This worked.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19793
2018-11-10 04:46:46 -08:00
epriestley
c32fa06266 Use phutil_microseconds_since(...) to simplify some timing arithmetic
Summary: Depends on D19796. Simplify some timing code by using phutil_microseconds_since() instead of duplicate casting and arithmetic.

Test Plan: Grepped for `1000000` to find these. Pulled, pushed, made a conduit call. This isn't exhaustive but it should be hard for these to break in a bad way since they're all just diagnostic.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19797
2018-11-08 16:46:32 -08:00
epriestley
b12e92e6e2 Add timing information for commit hooks to push logs
Summary:
Depends on D19779. Ref T13216. The push logs currently record the "hostWait", which is roughly "locking + subprocess cost". We also record locking separately, so we can figure out "subprocess cost" alone by subtracting the lock costs.

However, the subprocess (normally `git receive-pack`) runs hooks, and we don't have an easy way to figure out how much time was spent doing actual `git` stuff vs spent doing commit hook processing. This would have been useful in diagnosing at least one recent issue.

Track at least a rough hook cost and record it in the push logs.

Test Plan: Pushed to a repository, saw a reasonable hook cost appear in the database table.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19780
2018-11-08 06:00:26 -08:00
epriestley
966db4d38e Add an intracluster synchronization log for cluster repositories
Summary:
Depends on D19778. Ref T13216. See PHI943, PHI889, et al.

We currently have a push log and a pull log, but do not separately log intracluster synchronization events. We've encountered several specific cases where having this kind of log would be helpful:

  - In PHI943, an install was accidentally aborting locks early. Having timing information in the sync log would let us identify this more quickly.
  - In PHI889, an install hit an issue with `MaxStartups` configuration in `sshd`. A log would let us identify when this is an issue.
  - In PHI889, I floated a "push the linux kernel + fetch timeout" theory. A sync log would let us see sync/fetch timeouts to confirm this being a problem in practice.
  - A sync log will help us assess, develop, test, and monitor intracluster routing sync changes (likely those in T13211) in the future.

Some of these events are present in the pull log already, but only if they make it as far as running a `git upload-pack` subprocess (not the case with `MaxStartups` problems) -- and they can't record end-to-end timing.

No UI yet, I'll add that in a future change.

Test Plan:
  - Forced all operations to synchronize by adding `|| true` to the version check.
  - Pulled, got a sync log in the database.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19779
2018-11-07 18:24:20 -08:00
epriestley
e09d29fb1a Clean up the workflow for some post-push logging code
Summary:
Ref T13216. When a repository is clustered, we run this cleanup code (to tell the repository to update, and log some timing information) on both nodes. Currently, we do slightly too much work, which is unnecessary and can be a bit confusing to human readers.

The double update message doesn't hurt anything, but there's no reason to write it twice.

Likewise, the second timing information update query doesn't do anything: there's no PushEvent object with the right identifier, so it just updates nothing. We don't need to run it, and it's confusing that we do.

Instead, only do these writes if we're actually the final node with the repository on it.

Test Plan: Added some logging, saw double writes/updates before the change and no doubles afterwards, with no other behavioral changes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19778
2018-11-07 17:46:50 -08:00
epriestley
b645af981b Correct a missing parameter in "Outbound Mail" documentation
Summary: See <https://discourse.phabricator-community.org/t/documentation-error/2079>. This command is missing the configuration key.

Test Plan: Ran `bin/config set --stdin` with no other arguments, got an error about missing key. Ran with `... cluster.mailers`, got read from stdin.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19791
2018-11-07 17:43:01 -08:00
epriestley
8a4bf38655 Use 160-bit TOTP keys rather than 80-bit TOTP keys
Summary:
See <https://hackerone.com/reports/435648>. We currently use 80-bit TOTP keys. The RFC suggests 128 as a minimum and recommends 160.

The math suggests that doing the hashing for an 80-bit key is hard (slightly beyond the reach of a highly motivated state actor, today) but there's no reason not to use 160 bits instead to put this completely out of reach.

See some additional discussion on the HackerOne report about enormous key sizes, number of required observations, etc.

Test Plan: Added a new 160-bit TOTP factor to Google Authenticator without issue.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19792
2018-11-07 15:44:02 -08:00
epriestley
1f6a4cfffe Prevent users from selecting excessively bad passwords based on their username or email address
Summary:
Ref T13216. We occasionally receive HackerOne reports concerned that you can select your username as a password. I suspect very few users actually do this and that this is mostly a compliance/checklist sort of issue, not a real security issue.

On this install, we have about 41,000 user accounts. Of these, 100 have their username as a password (account or VCS). A substantial subset of these are either explicitly intentional ("demo", "bugmenot") or obvious test accounts ("test" in name, or name is a nonsensical string of gibberish, or looks like "tryphab" or similar) or just a bunch of numbers (?), or clearly a "researcher" doing this on purpose (e.g., name includes "pentest" or "xss" or similar).

So I'm not sure real users are actually very inclined to do this, and we can't really ever stop them from picking awful passwords anyway. But we //can// stop researchers from reporting that this is an issue.

Don't allow users to select passwords which contain words in a blocklist: their username, real name, email addresses, or the install's domain name. These words also aren't allowed to contain the password (that is, neither your password nor your username may be a substring of the other one). We also do a little normalization to try to split apart email addresses, domains, and real names, so I can't have "evan1234" as my password.

Test Plan:
  - Added unit tests and made them pass.
  - Tried to set my password to a bunch of variations of my username / email / domain name / real name / etc, got rejected.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19776
2018-11-06 12:44:07 -08:00
epriestley
c206b066df When {meme ...} embed has no text, just use the raw file data unmodified
Summary:
Ref T13216. See PHI948. When you use the remarkup hint button to embed a meme with no text, you get `{meme src=X}`.

If the source is a GIF, we currently split the source apart into frame-by-frame images, process them, and stitch them back together. The end result is the same image we started with, but this process can be slow/expensive, and may timeout for sufficiently large GIFs.

Instead: when there's no text, just return the original image data.

Test Plan:
  - Used `{meme src=X}` with no text, got an image faster.
  - Used `{meme src=X, above=...}` to add text, got an attempt to add text (which didn't get very far locally since I don't have GD configured).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19777
2018-11-06 09:40:22 -08:00
epriestley
d38e768ed8 Prevent users from voting for invalid Slowvote options
Summary:
Depends on D19773. See <https://hackerone.com/reports/434116>. You can currently vote for invalid options by submitting, e.g., `vote[]=12345`.

By doing this, you can see the responses, which is sort of theoretically a security problem? This is definitely a bug, regardless.

Instead, only allow users to vote for options which are actually part of the poll.

Test Plan:
  - Tried to vote for invalid options by editing the form to `vote[]=12345` (got error).
  - Tried to vote for invalid options by editing the radio buttons on a plurality poll into checkboxes, checking multiple boxes, and submitting (got error).
  - Voted in approval and plurality polls the right way, from the main web UI and from the embed (`{V...}`) UI.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19774
2018-11-06 09:21:18 -08:00
epriestley
5e1d94f336 Remove nonfunctional AJAX embed behavior for Slowvote
Summary:
See <https://hackerone.com/reports/434116>. Slowvote has a piece of Javascript that attempts to let you vote on `{V123}` polls inline.

It does not work: nothing ever triggers it (nothing renders a control with a `slowvote-option` sigil).

At least for now, just remove it. It has a completely separate pathway in the controller and both pathways are buggy, so this makes fixing them easier.

Test Plan: Voted in plurality and approval polls via Slowvote and the embedded widget.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19773
2018-11-06 09:20:07 -08:00
epriestley
798a391e5a Add test coverage for "%R" in qsprintf() and convert LiskDAO to support it
Summary:
Ref T13210. Ref T11908. Add some basic test coverage for the new "%R" introduced in D19764, then convert LiskDAO to implement the "Database + Table Ref" interface.

To move forward, we need to convert all of these (where `%T` is not a table alias):

```counterexample
qsprintf($conn, '... %T ...', $thing->getTableName());
```

...to these:

```
qsprintf($conn, '... %R ...', $thing);
```

The new code is a little simpler (no `->getTableName()` call) which is sort of nice. But we also have a //lot// of `%T` so this is probably going to take a while.

(I'll hold this until after the release cut.)

Test Plan:
  - Ran unit tests.
  - Browsed around and edited some objects without issues. This change causes a reasonably large percentage of our total queries to use the new code since the LiskDAO builtin queries are some of the most commonly-constructed queries, although there are still ~700 callsites which need to be examined for possible conversion.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210, T11908

Differential Revision: https://secure.phabricator.com/D19765
2018-11-05 10:59:50 -08:00
epriestley
d2316e8025 Fix an errant "switch ... continue"
Summary:
See <https://discourse.phabricator-community.org/t/unhandled-exception-on-create-task/2062>.

This construction has the same behavior as "switch ... break" but is unconventional. PHP 7.3 started warning about it because it's likely a mistake.

Test Plan: Created a task, edited a task owner. The new code is functionally identical to the old code.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19772
2018-11-05 10:26:27 -08:00
epriestley
24a061f844 Correct an ambiguous regexp in DiffusionRequest
Summary:
See <https://discourse.phabricator-community.org/t/diffusionrequest-regex-error/2057/>.

The intent of `[\d-,]` is "digits, hyphen, and comma" but `[x-y]` means "character range x-y".

Specify `[\d,-]` instead to disambiguate the hyphen as "literal hyphen", not a character range marker.

Test Plan: I can't reproduce the original error as reported, but browsed around Diffusion for a bit.

Reviewers: amckinley, avivey

Reviewed By: avivey

Differential Revision: https://secure.phabricator.com/D19770
2018-11-01 20:01:39 -07:00
Tim Hirsh
9bea00c159 Add harbormaster.buildplan.search api method
Summary: This revision adds a conduit search method for build plans.  Other api methods (eg: `harbormaster.build.search`) support build plan phid's as a constraint, but they weren't exposed anywhere, so this provides a way to fetch them.

Test Plan:
Used the api console to run some searches.  Output:
```
{
  "data": [
    {
      "id": 1,
      "type": "HMCP",
      "phid": "PHID-HMCP-q2c25wvegzdkxs7gzor6",
      "fields": {
        "name": "my build plan",
        "planStatus": "active",
        "dateCreated": 1538085249,
        "dateModified": 1538085249,
        "policy": {
          "view": "users",
          "edit": "admin"
        }
      },
    {
      "id": 1,
      "type": "HMCP",
      "phid": "PHID-HMCP-q2c25wvegzdkxs7gzor6",
      "fields": {
        "name": "my build plan",
        "status": {
          "value": "active"
        },
        "dateCreated": 1538085249,
        "dateModified": 1538085249,
        "policy": {
          "view": "users",
          "edit": "admin"
        }
      },
      "attachments": {}
    },
    ...
  ],
  "maps": {},
  "query": {
    "queryKey": null
  },
  "cursor": {
    "limit": 100,
    "after": null,
    "before": null,
    "order": null
  }
}
```

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, yelirekim

Differential Revision: https://secure.phabricator.com/D19769
2018-11-02 02:57:38 +00:00
epriestley
5d4970d6b2 Fix a bug where "View as Query" could replace a saved query row by ID, causing workboard 404s
Summary:
Fixes T13208. See that task for details.

The `clone $query` line is safe if `$query` is a builtin query (like "open").

However, if it's a saved query we clone not only the query parameters but the ID, too. Then when we `save()` the query later, we overwrite the original query.

So this would happen in the database. First, you run a query and save it as the workboard default (query key "abc123"):

| 123 | abc123 | {"...xxx..."} |

Then we `clone` it and change the parameters, and `save()` it. But that causes an `UPDATE ... WHERE id = 123` and the table now looks like this:

| 123 | def456 | {"...yyy..."} |

What we want is to create a new query instead, with an `INSERT ...`:

| 123 | abc123 | {"...xxx..."} |
| 124 | def456 | {"...yyy..."} |

Test Plan:
  - Followed reproduction steps from above.
    - With just the new `save()` guard, hit the guard error.
    - With the `newCopy()`, got a new copy of the query and "View as Query" remained functional without overwriting the original query row.
  - Ran migration, saw an affected board get fixed.

Reviewers: amckinley, joshuaspence

Reviewed By: joshuaspence

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13208

Differential Revision: https://secure.phabricator.com/D19768
2018-11-01 05:44:49 -07:00
epriestley
b950f877c5 Allow Drydock Blueprints to control "supplemental allocation" behavior so all hosts in an Almanac pool get used
Summary:
Fixes T12145. Ref T13210. See PHI570. See PHI536.

Currently, when you give Drydock an Almanac host pool with more than one host, it never voluntarily builds a second host resource: there is no way to say "maximum X working copies per host" (only "maximum X global working copies") to make the first host overflow, and the allocator tries to pack resources as tightly as possible.

If you can force it to allocate the 2nd..Nth host, things will work reasonably well from there (it will spread working copies across the hosts randomly), but tricking it is very hard, especially before D19761.

To deal with this, give blueprints a new behavior around "supplemental allocations". The idea here is that a blueprint may decide that it would prefer to allocate a fresh new resource instead of allowing an otherwise valid acquisition to occur.

These supplemental allocations follow all the normal allocation rules (they can't exceed limits or actually replace existing resources), so they can only happen if there's free space in the resource pool. But a blueprint can elect for a supplemental allocation to provide a "grow the pool" hint.

The only useful policies here are probably "true" (immediately use all resources, like Almanac) or "false" (pack resources as efficiently as possible) but some other policies //might// be useful (perhaps "start growing the pool when we're getting a bit full even if we aren't at the limit yet, since our workload is bursty").

Then, give Almanac host resources a "true" policy (always allocate supplemental resources) so they use all hosts once a similar number of concurrent jobs arrive.

One aspect of this approach is that we only do supplemental resources if the normal allocation algorithm already decided that the best resource to acquire was part of the same blueprint. I started with an approach like "look at all the blueprints and see if any of them want to be greedy", but then a not-very-desirable blueprint would end up filling up its whole pool before we skipped the supplemental allocation part and ended up picking a different resource. That felt a bit silly and this feels a little cleaner and more focused.

Test Plan:
  - Without changing the Almanac blueprint policy, allocated hosts. Got A, A, A, A, ... (second host never used).
  - Changed the Almanac policy.
  - Allocated hosts, got A, B, random mix of A and B.
  - Destroyed B. Destroyed all leases on A. Allocated. Got A. This tests the "don't build a supplemental resource if there are no leases on the natural resource".

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13210, T12145

Differential Revision: https://secure.phabricator.com/D19762
2018-10-31 18:06:47 -07:00
epriestley
57b4b59819 When a Drydock host based on an Almanac blueprint has its binding disabled, stop handing out leases
Summary:
Ref T13210. Ref T12145. The "Almanac Host" blueprint currently hands out new leases on a given host even if the binding has been disabled.

Although there are some more complicated cases here (e.g., involving cleanup of the existing resource and existing leases), this one seems clear cut: if the binding has been disabled, we should stop handing out new leases on it.

Test Plan:
  - Created a service with two hosts.
  - Requested a lease, got host A.
  - Requested more leases, always got host A (we never build a new host when we don't have to, and we currently never have to).
  - Disabled the binding to host A.
  - Requested a lease.
    - Before patch: got host A.
    - After patch: got host B.
  - Also disabled the other binding to host B, requested a lease, got an indefinite wait for resources (which is expected and reasonable).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13210, T12145

Differential Revision: https://secure.phabricator.com/D19761
2018-10-27 07:20:30 -07:00
epriestley
65e953658a Expose Audit actions for "transaction.search" in a basic way
Summary: Ref T13210. See PHI841. This mirrors D19509 for Differential.

Test Plan: Called `transaction.search` on a commit with a bunch of audit activity, got appropriate labels in the results.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19760
2018-10-27 07:19:50 -07:00
epriestley
61ec434208 Remove unicode marks for "Accept/Raise Concern" in Audit
Summary:
Ref T13210. The comment action dropdown for audits has a heavy checkmark next to "Accept" and a heavy "X" next to "Raise Concern".

We previously removed similar marks in Differential in D19405 and that seems to have gone fine. For consistency, remove these too.

Test Plan: Viewed the comment action dropdown, no longer saw checkmark and X-mark.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19759
2018-10-27 07:19:18 -07:00
epriestley
a7c008708d Correct a mangled translation string in "bin/phd log --id X"
Summary:
Ref T13210. See PHI930. This translation is wrong: the parameter is a comma-separated list as a string, but the USEnglish translation provides alternatives. We can't select among alternatives based on a random string (it isn't a plurality value to let us select "chair" vs "chairs", and isn't a gender value to let us select "his profile" vs "her profile") so we get an error.

But the string itself is also misleading, since "bin/phd log --id A --id B --id C" will say "none of these are valid" if //any// of them are invalid.

Instead, just tell the user explicitly about the first problem.

Test Plan:
  - Ran `bin/phd log --id` with good (got logs) and bad IDs (got sensible error).
  - Ran `bin/phd log` with any logs (got logs) and (simluated) without any logs (got error).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19755
2018-10-26 06:13:18 -07:00
Mike Riley
5f3a7cb41b Expose Drydock leases via Conduit
Summary:
See T13212 for some context and discussion on this being revived.
See T11694 for original context.

Add a query constraint for lease owners and implement the conduit search method for Drydock leases.

Ref T11694. Fixes T13212.

Test Plan:
- Called the API method from conduit and browsed lease queries from the UI.
- Used the new "ownerPHIDs" constraint via API console.

{F5963044}

Reviewers: yelirekim, amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam, epriestley

Maniphest Tasks: T11694, T13212

Differential Revision: https://secure.phabricator.com/D16594
2018-10-26 06:12:38 -07:00
epriestley
f6122547d7 When a lease triggers a resource allocation for a resource which must activate, awaken the lease task after the resource activates
Summary:
Depends on D19753. Ref T13210. This is a small optimization that saves us from waiting up to 15 seconds for a yield.

When there are no Working Copy resources and a new lease comes in, we'll allocate one and yield until it activates.

If activating it (SSH'ing and running `git clone`) takes less than 15 seconds, the resource will activate (say, at T+4) but the lease won't update again for a while (say, until T+15). This leaves us with a pointless wait (in this example, we're sitting around for 9 seconds when we could move forward).

To improve this a little bit, let resources wake up the lease update tasks that triggered allocation after they activate. In the best case, that task runs ~15 seconds sooner. In the worst case, the awaken is just a no-op.

With a more-full queue, this has a smaller effect (it's likely something else will run and be able to use the resource in those 9 seconds).

With already-activated resources, this has no effect (when resources are already activated, we can lease immediately).

Test Plan:
  - Cleaned up all working copy resources.
  - Requested a new "A" working copy.
  - Before patch: got a working copy after 17-18 seconds, most of which was spent yielded.
  - After patch: got a working copy after 3-4 seconds.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19754
2018-10-26 06:11:43 -07:00
epriestley
78ab675bd8 After a Drydock lease triggers a resource to be reclaimed, stop it from triggering another reclaim until the first one completes
Summary:
Depends on D19752. Ref T13210. If resources take a long time to reclaim/destroy (normally, more than 15 seconds) a single new lease may update several times during the reclaim/destroy process and end up reclaiming multiple resources.

Instead: after a lease triggers a reclaim, prevent it from triggering another reclaim as long as the resource it is reclaiming hasn't finished its reclaim/destroy cycle. Basically, each lease only gets to destroy one resource at a time.

Test Plan:
  - Added a `sleep(120)` to `destroyResource()` to simulate a long reclaim/destroy cycle.
  - Allocated A, A, A working copies. Leased a B working copy.
  - Before patch: saw "B" lease destroy all three "A" working copies after ~0, ~15, and ~30 seconds, then build a new "B" resource after ~120 seconds (when the first reclaim/destroy finished).
  - After patch: saw "B" lease destroy one "A" working copy after ~0 seconds, then wait patiently until it finished up, then build a new "B" resource.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19753
2018-10-26 06:11:05 -07:00
epriestley
e9309fdd6a When a Drydock lease schedules a resource to be reclaimed, awaken the lease update task when the reclaim completes
Summary:
Depends on D19751. Ref T13210. When Drydock needs to reclaim an existing unused resource in order to build a new resource to satisfy a lease, the lease which triggered the reclaim currently gets thrown back into the pool with a 15-second yield.

If the queue is pretty empty and the reclaim is quick, this can mean that we spend up to 15 extra seconds just waiting for the lease update task to get another shot at building a resource (the resource reclaim may complete in a second or two, but nothing else happens until the yield ends).

Instead, when a lease triggers a reclaim, have the reclaim reawaken the lease task when it completes. In the best case, this saves us 15 seconds of waiting. In other cases (the task already completed some other way, the resource gets claimed before the lease gets to it), it's harmless.

Test Plan:
  - Allocated A, A, A working copies with limit 3. Leased a B working copy.
  - Before patch: allocation took ~32 seconds.
  - After patch: allocation takes ~17 seconds (i.e., about 15 seconds less).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19752
2018-10-26 06:09:52 -07:00
epriestley
1f6869a765 In "bin/drydock lease", take a JSON "--attributes" so we can accept complex values
Summary:
Depends on D19750. See T13210. The `bin/drydock lease` command makes it easier to request ad-hoc leases, but currently takes lease attributes in the form `--attributes x=y,a=b`.

This was okay for all leases at the time, but doesn't really work for modern WorkingCopy resources since they take a `repositories.map` which has a dictionary as a value. You can't specify that with `repositories.map=...`.

Instead, point `--attributes` at a JSON file or use `--attributes -` to read from stdin.

Test Plan: Used `--attributes` with a file and stdin to allocate working copy leases with repositories.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D19751
2018-10-26 06:09:20 -07:00
epriestley
6deb09efcd When leasing a Working Copy in Drydock, just "git reset --hard" so empty repositories work
Summary:
Ref T13210. We currently "git reset --hard HEAD" during working copy leasing, mostly by convention/familiarity.

However, this command does not work in an empty repository, because there is no HEAD yet.

The command "git reset --hard" appears to have the same meaning and effect in all cases, except that it also works correctly in an empty repository.

The manual suggests that omitting HEAD should be the same as specifying HEAD, too:

> The <tree-ish>/<commit> defaults to HEAD in all forms.

Test Plan: Successfully leased a working copy for an empty repository using Drydock.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19750
2018-10-26 06:08:47 -07:00
epriestley
e2cf1e4288 Skip copied code detection for changes that are too large for it to be useful
Summary:
Ref T13210. See PHI944. When parsing certain large diffs (the case in PHI944 is an 2.5-million line JSON diff), we spend ~66% of runtime and ~80% of memory doing copy detection (the little yellow bar which shows up to give you a hint that code was moved around within a diff).

This is pretty much pointless and copy hints are almost certainly never useful on large changes. Instead, just bail if the change is larger than some arbitrary "probably too big for copy hints to ever be useful" threshold (here, 65535 lines).

Test Plan:
Roughly, ran this against a 2.5 million line JSON diff:

```
$changes = id(new ArcanistDiffParser())->parseDiff($raw_diff);
$diff = DifferentialDiff::newFromRawChanges($viewer, $changes);
```

Before the changes, it took 20s + 2.5GB RAM to parse. After the changes, it took 7s + 500MB RAM to parse.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19748
2018-10-20 03:36:34 -07:00
epriestley
e0dea4c486 Fix packages(project) to work properly and add it to "MailableFunctionDatasource"
Summary:
Ref T13210. See PHI937. This function datasource isn't quite implemented correctly: it doesn't resolve `package(project)` properly, since the logic only handles users.

This blames back to D14013, where it looks like `packages(..)` was added mostly as a general nice-to-have as part of a larger modernization change.

Test Plan: Ran a `packages(project)` query in Differential, got accurate results (previously: no results).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19747
2018-10-19 13:53:27 -07:00
epriestley
bc6c8c0e93 Explicitly shuffle nodes before selecting one for cluster sync
Summary:
Depends on D19734. Ref T13202. Ref T13109. Ref T10884. See PHI905. See PHI889. We currently rank cluster nodes in three cases:

  # when performing a write, we can go to any node (D19734 should make our ranking good);
  # when performing a read, we can go to any node (currently random, but T10884 discusses ideas to improve our ranking);
  # when performing an internal synchronization before a read or a write, we must go to an up-to-date node.

Currently, case (3) is not-exactly-deterministic but not random, and we won't spread intracluster traffic acrosss the cluster evenly if, say, half of it is up to date and half of it is still synchronizing. For a given write, I believe all nodes will tend to synchronize from whichever node first received the write today.

Instead, shuffle the list and synchronize from any up-to-date node.

(I think we could improve upon this only by knowing which nodes actually have load and selecting the least-loaded -- doable, but not trivial.)

Test Plan: Poked at it locally, will deploy to `secure`. This is hard to measure/test terribly convincingly.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202, T13109, T10884

Differential Revision: https://secure.phabricator.com/D19735
2018-10-17 08:11:23 -07:00
epriestley
51073b972e Try to route cluster writes to nodes which won't need to synchronize first
Summary:
Ref T13109. Ref T13202. See PHI905. See PHI889. When we receive a write to a repository cluster, we currently send it to a random writable node.

Instead, we can prefer:

  - the node currently holding the write lock; or
  - any node which is already up to date.

These should simply be better nodes to take writes in all cases. The write lock is global for the repository, so there's no scaling benefit to spreading writes across different nodes, and these particular nodes will be able to accept the write more quickly.

Test Plan:
  - This is observable by using `fprintf(STDERR, "%s\n", ...)` in the logic, then running `git push`. I'd like to pull this routing logic out of `PhabricatorRepository` at some point, probably into a dedicated `ClusterURIQuery` sort of class, but that is a larger change.
  - Added some `fprintf(...)` stuff around which nodes were being selected.
  - Added a `sleep(10)` after grabbing the write lock.
  - In one window, pushed. Then pushed in a second window.
    - Saw the second window select the lock holder as the write target based on it currently holding the lock.
    - Without a concurrent push, saw pushes select up-to-date nodes based on their up-to-date-ness.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: joshuaspence, timhirsh

Maniphest Tasks: T13202, T13109

Differential Revision: https://secure.phabricator.com/D19734
2018-10-17 08:08:25 -07:00
epriestley
0a51bc4f05 Add a space after "View Inline" in mail to prevent double-click on the filename from selecting "Inline"
Summary:
See PHI920. Ref T13210. Since the HTML is just:

```
<a>View Inline</a><span>filename.txt</span>
```

..double-clicking "filename.txt" in email selects "Inlinefilename.txt".

Add a space to stop this. At least in Safari, a space between the tags is not sufficient (perhaps because the parent is a `<div>`?). I couldn't find an authoritative-seeming source on what the rules for this actually are and adding a space here fixes the issue without changing the visual rendering, so just put it here.

Test Plan:
  - Made an inline.
  - Used `bin/mail show-outbound --id ... --dump-html` to dump the HTML.
  - Double-clicked the filename.

{F5929186}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19742
2018-10-11 13:44:20 -07:00
epriestley
8bffc9ea0e In "bin/bulk export", require "--output <path>" by default
Summary:
Depends on D19743. Ref T13210. Since this command can easily dump a bunch of binary data (or just a huge long blob of nonsense) to stdout, default to requiring "--output <file>".

Using `--output -` will print to stdout.

Test Plan: Ran with: no `--output`, `--output file`, `--output -`, `--output - --overwrite`. Got sensible results or errors in all cases.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19744
2018-10-11 13:35:16 -07:00
epriestley
4f54d483d5 Support export of revisions to Excel/CSV/JSON/etc
Summary: Ref T13210. See PHI806. This enables basic export of revisions into flat data formats. This isn't too fancy, but just covers the basics since the driving use case isn't especially concerned about getting all the fields and details.

Test Plan: Exported some revisions into JSON, got sensible output.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19743
2018-10-11 13:34:33 -07:00
epriestley
4f557ff075 When using "bin/bulk export --overwrite", actually overwrite the file
Summary: Depends on D19738. Ref T13210. Currently, when you use "--overwrite", we just //append// the new content. Instead, actually overwrite the file.

Test Plan: Used `--overwrite`, saw an actual clean overwrite instead of an append.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19739
2018-10-11 08:13:43 -07:00
epriestley
4928c34d00 Allow "bin/bulk export" to merge multiple queries and accept more flexible flags
Summary:
Ref T13210. Minor usability improvements to "bin/bulk export":

  - Allow `--class task` to work (previously, only `--class ManiphestTaskSearchEngine` worked).
  - If you run `--query jXIlzQyOYHPU`, don't require `--class`, since the query identifies the class on its own.
  - Allow users to call `--query A --query B --query C` and get a union of all results.

Test Plan:
  - Ran `--class task`, `--query A --query B`, `--query X` (with no `--class`), got good results.
  - Ran various flavors of bad combinations (queries from different engines, invalid engines, query and class differing, ambiguous/invalid `--class` name) and got sensible errors.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210

Differential Revision: https://secure.phabricator.com/D19738
2018-10-10 09:14:14 -07:00
epriestley
99034efa8b Make Pholio mail render without a ton of over-escaped HTML
Summary:
Ref T13202. See PHI900. Fixes T12814. Pholio currently builds HTML comments in an older way that can dump a bunch of over-escaped HTML into mail bodies.

Update the logic to be more similar to the Differential rendering logic and stop over-escaping things.

The result isn't perfect, but is dramatically less broken.

Test Plan: {F5919860}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202, T12814

Differential Revision: https://secure.phabricator.com/D19733
2018-10-05 13:37:26 -07:00
epriestley
c6c1893dc0 Allow revisions to be filtered by created date
Summary:
Ref T13202. See PHI906. This is a reasonable capability which we support in some other applications already.

(The only real reason not to support this is that it creates some clutter in the UI, but I think we're generally in better shape now than we were in the past, and we could make this UI collapse/fold at some point.)

Test Plan: Ran queries with a minimum date, a maximum date, both, and neither. Saw appropriate results in all cases.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19732
2018-10-05 12:28:20 -07:00
epriestley
4858d43d16 Add 'autocomplete="off"' to MFA TOTP inputs
Summary:
Ref T13202. See <https://discourse.phabricator-community.org/t/2fa-input-box-isnt-hinted-as-a-password-so-browsers-suggest-auto-fills/1959>.

If browsers are autofilling this, I think browser behavior here is bad, but behavior is probably better on the balance if we hint this as `autocomplete="off"` and this is a minor concesssion.

Test Plan:
  - I couldn't immediately get any browser to try to autofill this field (perhaps I've disabled autofill, or just not enabled it aggressively?), but this change didn't break anything.
  - After the change, answered a TOTP prompt normally.
  - After the change, inspected page content and saw `autocomplete="off"` on the `<input />` node.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19722
2018-10-01 13:08:54 -07:00
epriestley
39b85c0be0 Use the changeset parse cache when rendering inline comments in mail
Summary:
Ref T13202. See PHI903 and PHI894. When a bot leaves 100 inline comments on the same file and the revision has a 30-member recipient list, we currently highlight the file 3000 times when building mail.

Instead, engage the parse cache so we highlight it once and reuse the cache 2,999 times.

Test Plan:
  - Added debugging code to stop after mail generation and show cache hits/misses.
    - Left a bunch of inlines in a file.
    - Ran the worker with `bin/worker execute --id ...`.
    - Before change: saw 4 comments x 2 recipients = 8 cache misses
    - After change: saw 8 cache hits (cache already filled from web UI rendering)
  - Removed debugging code, ran worker to completion.
  - Used `bin/mail show-outbound --id ... --dump-html` to inspect resulting mail, saw no functional differences.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19721
2018-10-01 13:08:27 -07:00
Austin McKinley
dbf2302b6c Fix self-cancelling typo
Summary: Ref D18268. This typo cancelled itself out, and I can't find any other callers.

Test Plan: arc unit

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19718
2018-09-28 17:44:33 -07:00
Austin McKinley
8065433ee8 Migrate DiffusionBlameController to use repo identities
Summary:
Now on the blame page, identities get `avatar.png` and there are little tooltips that show a few characters of the committer identity string.

Also add a default icon for repo identities.

Test Plan: Loaded some blame pages for files touched by users with and without repo identities attached.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19587
2018-09-26 14:45:58 -07:00
epriestley
021c612cb2 When we fail to acquire a repository lock, try to provide a hint about why
Summary:
Ref T13202. See PHI889. If the lock log is enabled, we can try to offer more details about lock holders.

When we fail to acquire a lock:

  - check for recent acquisitions and suggest that this is a bottleneck issue;
  - if there are no recent acquisitions, check for the last acquisition and print details about it (what process, how long ago, whether or not we believe it was released).

Test Plan:
  - Enabled the lock log.
  - Changed the lock wait time to 1 second.
  - Added a `sleep(10)` after grabbing the lock.
  - In one window, ran a Conduit call or a `git fetch`.
  - In another window, ran another operation.
  - Got useful/sensible errors for both ssh and web lock holders, for example:

> PhutilProxyException: Failed to acquire read lock after waiting 1 second(s). You may be able to retry later. (This lock was most recently acquired by a process (pid=12609, host=orbital-3.local, sapi=apache2handler, controller=PhabricatorConduitAPIController, method=diffusion.rawdiffquery) 3 second(s) ago. There is no record of this lock being released.)

> PhutilProxyException: Failed to acquire read lock after waiting 1 second(s). You may be able to retry later. (This lock was most recently acquired by a process (pid=65251, host=orbital-3.local, sapi=cli, argv=/Users/epriestley/dev/core/lib/phabricator/bin/ssh-exec --phabricator-ssh-device local.phacility.net --phabricator-ssh-key 2) 2 second(s) ago. There is no record of this lock being released.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19702
2018-09-24 15:20:07 -07:00
epriestley
7db265cd5d Parameterize the repository read and write locks
Summary:
Ref T13202. See PHI889. Update the read and write locks to the modern parameterized verison, which handles hashing/normalization and can store better logs.

This parameterized mode was added in D19173 and has been used successfully for some time, but not all locks have switched over to it yet.

Test Plan:
- Added an `fprintf(STDERR, $full_name)` to the lock code.
- Pulled a repository.
- Saw sensible lock name on stdout before "acquired read lock...".
- Additional changes in this patch series will vet this more completely.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19701
2018-09-24 15:14:28 -07:00
epriestley
3244324cb1 Fix comment box borders in timelines after Phriction commenting
Summary:
Ref T13202. In D19660, I added comments to Phriction and tweaked some CSS.

One of these tweaks was getting rid of an extra border which was rendering under the comment area. However, I took off too much and ended up removing borders from other applications.

I think we don't actually need this `setNoBorder()` stuff after all -- a later change was sufficient to stop the actual border I was trying to get rid of from rendering. So this mostly just reverts part of D19660.

This rendering still isn't perfect, but I'm fine leaving that for another day for now.

Test Plan:
  - Viewed comment areas in Phriction. Saw correct number of borders (1).
  - Viewed comment areas in Maniphest. Saw correct number of borders (1).
  - Grepped for extraneous removed classs, no hits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19684
2018-09-19 13:56:58 -07:00
epriestley
5ba66e56fd Fix Phriction rendering for initial install and 404 pages
Summary:
Depends on D19682. Ref T13202. We currently fatal when trying to render a timeline if:

  - an install is fresh, so there are no pages yet, and you look at "/w/"; or
  - you're looking at a Phriction page which doesn't exist (yet) like "/w/aadsflknadsflnf/".

Rendering a timeline and comment area doesn't make sense in these cases, so don't render them.

Test Plan: Hit both cases described above, got "new/empty page" prompts instead of fatals.

Reviewers: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19683
2018-09-17 20:02:59 -07:00
epriestley
e5c6a5749a Fix fatal in rendering Phriction "Moved Away" stories
Summary:
Ref T13202. See PHI881. These stories have bad rendering methods, but they didn't previously render into the timeilne (since Phriction documents didn't have a timeline).

Update the rendering to work.

The rendered outcome isn't great (it isn't very clear or explicit about exactly what moved where), but I'll fix that in a followup. This is a net improvement since it doesn't fatal the page, at least.

Test Plan:
  - Moved page "X" to "Y".
  - Viewed the old page "X".
  - Before patch: bad timeline story would fatal rendering.
  - After patch: story renders, at least, just not great.

Reviewers: amckinley

Maniphest Tasks: T13202

Differential Revision: https://secure.phabricator.com/D19682
2018-09-17 20:02:06 -07:00
epriestley
0167f357b7 Provide a convenient way to log arbitrary text in Drydock without needing structured log classes
Summary: Depends on D19673. Ref T13197. See PHI873.

Test Plan:
Added some code like this:

```
$operation->logText('Nice convenient text logging.');
```

...then got:

{F5887712}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19674
2018-09-15 07:59:50 -07:00
epriestley
a7e060f062 Write a trivial log when starting a repository operation
Summary:
Depends on D19672. Ref T13197. See PHI873. This writes a trivial log when we begin acting on a working copy and makes it look reasonable in the UI.

This is mostly just to prove that logging works properly.

Test Plan: {F5887697}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19673
2018-09-15 07:57:11 -07:00
epriestley
92bcf85974 Add Drydock logs to the RepositoryOperation UI
Summary:
Depends on D19671. Ref T13197. See PHI873.

Expose logs in the RepositoryOperation UI. Nothing writes the logs yet, so these interfaces are currently always empty.

Test Plan:
{F5887102}

{F5887103}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19672
2018-09-15 07:56:35 -07:00
epriestley
10f219fb82 Allow Drydock logs to be associated with RepositoryOperation objects
Summary: Ref T13197. See PHI873. I want to give RepositoryOperation objects access to Drydock logging like leases, resources, and blueprints currently have. This just does the schema/query changes, no actual UI or new logging yet.

Test Plan: Ran storage upgrade, poked around the UI looking for anything broken.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19671
2018-09-15 07:55:14 -07:00
epriestley
40d5d5c984 Remove "mailKey" from "PhabricatorRepositoryCommit"
Summary: Ref T13197. Ref T13065. This continues the gradual purge of dedicated "mailKey" columns in favor of shared infrastructure.

Test Plan:
  - Ran migration.
  - Visually inspected database.
  - Grepped for `mailKey`.
  - Added some comments, saw the daemons generate corresponding mail.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13197, T13065

Differential Revision: https://secure.phabricator.com/D19670
2018-09-15 07:54:15 -07:00
epriestley
2ddc770fd8 Update Phriction documentation for drafts
Summary: Depends on D19668. Ref T13197. See PHI840. This updates the documentation to describe how drafts work in more detail.

Test Plan: Read documentation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19669
2018-09-13 14:16:19 -07:00
epriestley
0abf29765e Make Phriction diff-of-changes view draft-aware and clarify some language
Summary:
Ref T13077. Ref T13197. See PHI840.

  - In the "History > Diff/Compare" view, the button language wasn't draft-aware.
  - Revise language to avoid the word "Revert", since this can be ambiguous.
    - "Edit this page, starting with an older version of the text" is now "Edit Older|Current|Draft Version X".
    - "Mark this older version of the page as the current published version" is now "Publish Older Version".
  - Let the user edit the current published version, too, since this is a reasonable operation if there are drafts.

Test Plan: Navigated the history diff view, saw better button and action text.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197, T13077

Differential Revision: https://secure.phabricator.com/D19668
2018-09-13 14:12:49 -07:00
epriestley
0089ef4b60 Don't show Phriction draft edit events in feed
Summary: Depends on D19663. Ref T13077. When you edit a Phriction draft, don't publish a feed story. (The eventual "Publish" event gets a story.)

Test Plan: Made draft / non-draft / publish edits, only saw feed stories for non-draft and publish edits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19664
2018-09-12 13:56:10 -07:00
epriestley
b6ba75d991 Add a Phriction hint when a draft exists, and fix some draft editing bugs
Summary:
Depends on D19662. Ref T13077. See PHI840.

  - If you're looking at the published version of a document, but a draft version exists and you can edit it, add a hint/link.
  - Fix an issue where the "draft" transaction would complain when you created a document since the initial content is empty and no "draft" transaction is adding any content.

Test Plan: Created new documents, viewed documents with current published versions and unpublished drafts.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19663
2018-09-12 13:45:33 -07:00
epriestley
550028a882 Allow Phriction document edits to be saved as drafts
Summary:
Depends on D19661. Ref T13077. See PHI840.

When a user edits a page normally, add a "Save as Draft" button. Much of this change is around making that button render and behave properly: it needs to be an `<input type="submit" ...>` so browsers submit it and we can figure out which button the user clicked.

Then there are a few minor rules:

  - If you're editing a page which is already a draft, we only give you "Save as Draft". This makes edits to update/revise a draft more natural.
  - Highlight "Publish" if it's a likely action that you might want to take.

Internally, there are two types of edits. Both types create a new version with the new content. However:

  - A "content" edit sets the version shown on the live page to the newly-created version.
  - A "draft" edit does not update the version shown on the live page.

Test Plan: Edited a published document, edited the draft. Published documents. Reverted documents.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19662
2018-09-12 13:30:40 -07:00
epriestley
152e7713eb Remove obsolete, nonfunctional draft auto-saving in Phriction
Summary:
Depends on D19660. Ref T5811. Ref T13077.

Long ago, if you started editing a Phriction document but didn't save it, we'd save the draft in the background as part of the preview.

D11169 updated the preview to use shared infrastructure and broke this function, since we never save drafts.

Since this doesn't work right now, I want to add another thing called "draft", and the future of this feature should be more integrated with modern drafts and EditEngine (which fixed some bugs related to versioning), just get rid of this code for the moment.

Test Plan: Edited documents. This code doesn't do anything since D11169, so no behavior changed.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13077, T5811

Differential Revision: https://secure.phabricator.com/D19661
2018-09-12 13:22:16 -07:00
epriestley
e19c555913 Support (basic) commenting on Phriction documents
Summary:
Depends on D19659. Fixes T1894. Ref T13077. See PHI840.

  - Add an EditEngine, although it currently supports no fields.
  - Add (basic, top-level-only) commenting (we already had the table in the database).

This will probably create some issues. I'm most concerned about documents accumulating a ton of old, irrelevant comments over time which are hard to keep track of and no longer relevant. But I think this is probably a step forward in almost all cases, and a good thing on the balance.

This also moves us incrementally toward putting all editing on top of EditEngine.

Test Plan: {F5877347}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13077, T1894

Differential Revision: https://secure.phabricator.com/D19660
2018-09-12 13:20:52 -07:00
epriestley
f5e90a363e When a user takes actions while in a high security session, note it on the resulting transactions
Summary:
Ref T13197. See PHI873. Record when a user has MFA'd and add a little icon to the transaction, similar to the exiting "Silent" icon.

For now, this just makes this stuff more auditable. Future changes may add ways to require MFA for certain specific transactions, outside of the ones that already always require MFA (like revealing credentials).

Test Plan: {F5877960}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19665
2018-09-12 12:57:02 -07:00
epriestley
8268abcb78 Enrich "diffusion.commit.search" with identity, status, and message information
Summary:
Depends on D19657. Ref T13197. See PHI841.

This enriches the results from `diffusion.commit.search` with information similar to the information returned by the "commits" attachment from `differential.diff.search`.

Also include unreachable, imported, message, audit status, and repository PHID.

Test Plan: Called `diffusion.commit.search` and reviewed the results, which looked sensible.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19658
2018-09-12 12:45:44 -07:00
epriestley
c7e7b63f15 Rename "PhabricatorAuditCommitStatusConstants" to "DiffusionCommitAuditStatus"; remove "MODERN_"
Summary:
Depends on D19656. Ref T13197. See PHI851.

  - This class is now a real object, so get rid of the "Constants" part of the name.
  - Rename it for greater consistency with other modern objects.
  - Get rid of the `MODERN_` tag now that the old constants are gone.

Test Plan: Bunch of `grep`, browsed around.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19657
2018-09-12 12:44:43 -07:00
epriestley
aaf2269551 Remove legacy numeric Audit status constants
Summary: Depends on D19655. Ref T13197. These no longer have callers.

Test Plan: Grepped for these constants.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19656
2018-09-12 12:40:55 -07:00
epriestley
d63281cc54 Migrate remaining Audit database status constants
Summary: Depends on D19652. Ref T13197. See PHI851. This migrates the actual `auditStatus` on Commits, and older status transactions.

Test Plan:
  - Ran migrations.
  - Spot-checked the database for sanity.
  - Ran some different queries, got unchanged results from before migration.
  - Reviewed historic audit state transactions, and accepted/raised concern on new audits. All state transactions appeared to generate properly.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19655
2018-09-12 12:21:27 -07:00
epriestley
cc3b6d5790 Fix a "withHasTransactions()" typo in AuditEditor
Summary: See <https://discourse.phabricator-community.org/t/typo-in-phabricatorauditeditor-php/1910>. This is trivial and reproduces easily, I just missed it in testing.

Test Plan:
  - Left a comment on a commit which I was the author of.
  - Before change: fatal with obvious typo.
  - After change: smooth sailing.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19667
2018-09-12 12:17:58 -07:00
epriestley
2379c21fbb Give Phriction documents a normal timeline
Summary:
Ref T13077. See PHI840. Ref T1894. I'm planning to just let you comment on Phriction documents. I think this will create a few problems (e.g., around popular documents which collect long comment threads that are eventually obsolete) but nothing should be too terribly critical (e.g., we handle it gracefully when objects have very large number of comments/transactions) and for most documents this is likely just a net improvement.

"Just enable comments" is probably not the final iteration on this, but I think it's probably a step forward on the balance, not a step sideways or a slippery slope down into a dark hole or anything.

Test Plan: {F5877316}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13077, T1894

Differential Revision: https://secure.phabricator.com/D19659
2018-09-11 13:31:57 -07:00
epriestley
185e72c881 Add aural section headers for "Event Timeline", "Add Comment", and "Comment Preview"
Summary: See PHI871. Ref T13197. These sections are only divided visually and don't have textual headers. Add aural headers.

Test Plan: {F5875471}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19654
2018-09-11 13:30:10 -07:00
epriestley
8eb8e8e1d8 Make DiffusionCommitSearch accept modern (string) constants
Summary:
Depends on D19650. Ref T13197. Allow `SearchCheckboxesField` to have a "deprecated" map of older aliases, then convert them to modern values.

On the API method page, show all the values.

This technically resolves the issue in PHI841, although I still plan to migrate behind this.

Test Plan:
{F5875363}

- Queried audits, fiddled with `?status=1,audited`, etc.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19651
2018-09-10 16:25:42 -07:00
epriestley
853a816b3c Continue converting Audit constants, allowing the Query to handle either strings or integers
Summary: Ref T13197. We're almost ready to migrate: let the Query accept either older integer values or new string values. Then move some callsites to use strings.

Test Plan: Called `audit.query`, browsed audits, audited commits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19650
2018-09-10 14:46:47 -07:00
epriestley
bae8a95114 Continue replacing Commit/Audit status checks with object-based checks
Summary: Ref T13195. See PHI851. Continuing down the path toward replacing these legacy numeric constants with more modern string constants.

Test Plan:
- Raised concern, requested verification, verified.
- Looked at commit hovercard with audit status.
- Viewed header on a commit page.
- (Didn't test the Doorkeeper stuff since it requires linking to Asana and seems unlikely to break.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19647
2018-09-10 11:20:31 -07:00
epriestley
6dc721009d Layout Phriction actions without floats, to avoid conflicts with floating content
Summary:
Ref T13195. If a Phriction page begins with a code block, the `clear: both;` currently makes it clear the action list.

Instead, use table-cell layout on desktops.

Test Plan: Viewed a Phriction page with an initial code block on desktop/tablet/mobile/printable layouts. Now got more sensible layouts in all cases.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: GoogleLegacy

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19649
2018-09-10 11:19:53 -07:00
epriestley
2a03367a50 When authors add inlines to their own revisions/commits, mark them as "Done" by default
Summary:
Ref T13195. Fixes T8573. When you're adding inlines to your own stuff, mark them "Done" by default. You can unmark them as "Done" if you're legitimately leaving TODOs for yourself, although I think this is unusual.

(If this turns out to be less unusual than I think, we could consider an alternate rule: mark replies by the author as "Done" by default.)

Test Plan: Added some inlines as an author and a non-author. Saw my author inlines marked as "Done" by default. Submitted them; unmarked and submittted them.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195, T8573

Differential Revision: https://secure.phabricator.com/D19635
2018-09-07 11:18:14 -07:00
epriestley
16a6fc8341 Allow reviewers to mark their own inlines as "Done" before they submit them
Summary:
Ref T13195. Ref T8573. This allows reviewers to mark their own inline comments as "Done" before they submit them.

If you're leaving a non-actionable comment like "this is good", you can pre-check "Done" to give the author a hint that you don't expect any response.

Test Plan: On revisions and commits, added inlines as the author and a reviewer/auditor. Marked them done/not-done before submitting. As author, marked the not-done ones done after submitting. Checked preivews, toggled done/not done states.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195, T8573

Differential Revision: https://secure.phabricator.com/D19634
2018-09-07 11:17:42 -07:00
epriestley
046c1b5b82 Expose "isDraft" and "holdAsDraft" revision properties over Conduit
Summary:
Ref T13195. See PHI861. The "+ Draft" flag is not currently exposed over the API, but seems stable enough to expose.

Also expose the "hold as draft" flag, normally `arc diff --draft`.

Today, you can get "+ Draft" with some other state by:

  - abandoning a draft revision ("Abandoned + Draft"); or
  - using `arc diff --plan-changes` with an older `arc` version ("Changes Planned + Draft").

Test Plan: Called `differential.revision.search` via Conduit and got sensible-looking results for revisions in various states.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19648
2018-09-07 11:16:42 -07:00
epriestley
a1ce23b9f5 Introduce an AuditStatus object for commits and move some callsites to it
Summary:
Ref T13195. See PHI851. Add an object, analogous to the `DifferentialRevisionStatus` object, to handle audit status management.

This will primarily make it easier to swap storage over to strings later, but also cleans things up a bit.

Test Plan: Viewed audit/commit lists, saw sensible state icons. Ran `bin/audit synchronize`, got sensible output.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19646
2018-09-07 10:20:04 -07:00
epriestley
ab4d33bede Allow underscores to appear in Harbormaster variable names
Summary:
See PHI859. Ref T13195. The regexp for replacing variables currently does not include underscores.

Include underscores.

I also made a note in T13088: we should (almost certainly?) throw immediately if you try to pass a bogus variable name as a custom parameter, but this is a slightly larger change.

Test Plan:
  - Made an "http request" build plan with `?x=${initiator.phid}&y={$some_variable}`.
  - Added `some_variable` as a parameter to the parameter collection.
  - Before patch: `initiator.phid` was replaced, but `some_variable` was not.
  - After patch: both variables are replaced.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19645
2018-09-07 09:53:59 -07:00
epriestley
e8e5dc0f56 Make a language improvement ("inlines" -> "inline comments")
Summary: See D19632. Agreed that this is more clear.

Test Plan: Read carefully.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19644
2018-09-06 11:43:52 -07:00
epriestley
ef26b06ca8 Begin transitioning audits to modern (string) status constants, from legacy (integer) status constants
Summary:
Ref T13195. See PHI851. Audits currently have older integer status constants. We've moved almost all object types away from this to string constants (which are better in basically every way, and particularly way better for exposing over the API).

Commits/audits are currently accessible over the API and expose these constants via a "statuses" constraint.

Prepare to move toward modern string constants by defining a new, more modern map of status details and defining the existing methods in terms of it.

Test Plan: Browsed audits checking for icons/names/open-ness, saw no changes. This change should have no user-visible effects, as it just reorganizes code.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19642
2018-09-06 10:56:47 -07:00
epriestley
5a38b75f16 In Conduit, let checkbox constraints self-document
Summary:
Ref T13195. Ref PHI851. Currently, checkbox constraints don't tell you what values are supported on the API page.

You can figure this out with "Inspect Element" or by viewing the source code, but just render a nice table instead.

Test Plan: {F5862969}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19641
2018-09-06 08:44:16 -07:00
epriestley
a20f0674a9 Improve some documentation for "diffusion.commit.search"
Summary:
Ref T13195. See PHI851. Start by making some minor improvements here:

  - Clarify that the example of what constraints look like is just an example.
  - Add descriptions for parameters missing descriptions.

Test Plan: Looked at API method page, saw more helpful/complete instructions.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19640
2018-09-06 08:39:21 -07:00
epriestley
e6ee5ee9a1 When applying repository operations via Drydock, provide more context on OperationType
Summary: Ref T13195. See PHI845. For custom OperationTypes, provide access to the Interface and Operation via getters. This is just for convenience, since passing these around everywhere can be a bit of a pain if you have a deep-ish stack of things or love using callbacks or whatever.

Test Plan: Landed a revision via upstream Drydock operations.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19636
2018-09-06 08:15:16 -07:00
epriestley
041392988e When a transaction adds more than 100 inline comments, include only the first 100 in email
Summary:
Ref T13195. An install had a user apply a transaction which added about 1,000 inline comments. Rendering the email for this transaction took a very long time because the context section for each comment must be highlighted separately.

We can make the highlighting faster (in this case, by porting the lexer to PHP) but it's also sort of silly to include more than 100 inlines in an email. These emails are likely to be truncated by outbound size rules at some point anyway. Instead, limit inlines rendered directly into email to the first 100 per transaction group.

Test Plan:
Set limit to 2, added 4 comments, viewed text and HTML emails:

{F5859967}

{F5859968}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19632
2018-09-06 07:58:05 -07:00
epriestley
650e74933a Update some inline comment logic to use more modern "Viewer"-oriented calls/variables
Summary:
Ref T13195. Ref T8573. The inline comment controllers currently use outdated `$user = $this->getRequest()->getUser()` calls.

Instead, use `$viewer = $this->getViewer()`.

This is just a small consistency update with no behavioral changes.

Test Plan: Viewed and added inlines in Differential and Diffusion.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195, T8573

Differential Revision: https://secure.phabricator.com/D19633
2018-09-06 07:57:41 -07:00
epriestley
26d9ee456f Hide the new Phriction "Publish" operation behind the "Prototype" toggle
Summary: Ref T13077. This is currently a little too confusing to go out into the world, mostly because there's no way to edit documents without auto-publishing them. Keep it out of the spotlight for this release.

Test Plan: Viewed Phriction, saw publish operation marked as a prototype.

Reviewers: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19627
2018-08-31 10:30:20 -07:00
epriestley
18e8d9a452 Make the Phriction "History" view more aware of drafts
Summary: Ref T13077. Updates the "History" view to be slightly better organized and draft-aware.

Test Plan: Viewed page history in Phriction.

Reviewers: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19626
2018-08-30 10:36:55 -07:00
epriestley
3b1294cf45 Store Phriction max version on Document, improve editing rules for editing documents with drafts
Summary:
Ref T13077. We need to know the maximum version of a document in several cases, so denormalize it onto the Document object.

Then clean up some behaviors where we edit a document with, e.g., 7 versions but version 5 is currently published. For now, we: edit starting with version 7, save as version 8, and immediately publish the new version.

Test Plan:
  - Ran migration.
  - Edited a draft page without hitting any weird version errors.
  - Checked database for sensible `maxVersion` values.

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19625
2018-08-30 10:12:51 -07:00
epriestley
0a77b0e53e Work around an issue in MariaDB where dropping a column from a UNIQUE KEY fails
Summary:
See T13193. See T13077. If we drop a column which is part of a UNIQUE KEY, MariaDB raises an error.

This is probably a bad idea on our side anyway, but in this case it wasn't an obviously bad idea.

To get around this:

  - Drop the unique key, if it exists, before dropping the column.
  - Explicitly add the new unique key afterward.

Test Plan: Ran `bin/storage upgrade` locally without issue, but I'm on MySQL. Will follow up on T13193.

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D19624
2018-08-30 06:25:39 -07:00
epriestley
75a0455152 Add "Revision test plan" as a Herald field; remove test plan from the "Revision summary" field
Summary:
See PHI844. Ref T13189.

Add "Revision test plan" as an available field for Herald. This is a little niche -- and a little odd because it sticks around even if you fully disable test plans -- but probably broadly reasonable.

The existing "Revision summary" field counterintuitively included the test plan. Separate this out since it's now a separate field and the behavior was weird historic nonsense. I'll note this in the changelog.

Test Plan: Wrote a rule using both fields, verified they generated the expected values.

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19623
2018-08-29 14:17:38 -07:00
epriestley
876638e428 Add a UI element for navigating between versions of a Phriction document
Summary: Depends on D19621. Ref T13077. Fixes T4815. This adds previous/current/next/draft buttons and makes navigation between unpublished and published versions of a document more clear.

Test Plan: {F5841997}

Reviewers: amckinley

Maniphest Tasks: T13077, T4815

Differential Revision: https://secure.phabricator.com/D19622
2018-08-29 13:49:15 -07:00
epriestley
349686319e Allow the published version of a Phriction document to differ from the most recent version
Summary:
Depends on D19620. Ref T13077. This adds a "Publish" operation which points the current version at some historical version of the document -- not necessarily the most recent version. Newer versions become "drafts".

This is still quite rough and missing a lot of hinting in the UI, I'm just making it work so I can start making the UI understand it.

Test Plan: Used the "Publish" action to publish older versions of a document, saw the document revert. Many UI hints are missing and this operation is puzzling and not yet usable for normal users.

Reviewers: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19621
2018-08-29 13:47:36 -07:00
epriestley
50f4adef64 Remove on-object mailkeys from Phriction
Summary: Depends on D19619. Ref T13065. Ref T13077. Migrate Phriction mail keys to the new infrastructure and drop the column.

Test Plan: Ran migrations, spot-checked the database.

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13077, T13065

Differential Revision: https://secure.phabricator.com/D19620
2018-08-29 13:43:13 -07:00
epriestley
64cee4a902 Move Phriction internal document/content references from IDs to PHIDs
Summary:
Ref T13077. This is mostly just a small cleanup change, even though the actual change is large.

We currently reference content and document objects from one another with `contentID` and `documentID`, but this means that `contentID` must be nullable. Switching to PHIDs allows the column to be non-nullable.

This also supports reorienting some current and future transactions around PHIDs, which is preferable for the API. In particular, I'm adding a "publish version X" transaction soon, and would rather callers pass a PHID than an ID or version number, since this will make the API more consistent and powerful.

Today, `contentID` gets used as a cheaty way to order documents by (content) edit time. Since PHIDs aren't orderable and stuff is going to become actually-revertible soon, replace this with an epoch timestamp.

Test Plan:
  - Created, edited, moved, retitled, and deleted Phriction documents.
  - Grepped for `documentID` and `contentID`.
  - This probably breaks //something// but I'll be in this code for a bit and am likely to catch whatever breaks.

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19619
2018-08-29 13:41:24 -07:00
epriestley
04f8270a74 Remove some unusual UI policy hints in Phriction
Summary:
Ref T13077. We currently have these weird policy hints in Phriction that we don't use in other applications. Just remove them for consistency to make the eventual swap to EditEngine a little easier.

Also nuke some unreacahble code.

Test Plan: Loaded edit page, saw more standard UI.

Reviewers: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19618
2018-08-29 07:32:45 -07:00
epriestley
4afb6446d9 Allow DocumentView to render with a curtain, and make Phriction use a curtain
Summary:
Depends on D19616. Ref T13077. Fixes T8172. In the last round of design updates, a lot of actions got stuffed into "Actions" menus.

I never really got used to these and think they're a net usability loss, and broadly agree with the feedback in T8172. I'd generally like to move back toward a state where actions are available on the page, not hidden in a menu.

For now, just put a curtain view on these pages. This could be refined later (e.g., stick this menu to the right hand side of the screen) depending on where other Phriction changes go.

(Broadly, I'm also not satisfied with where we ended up on the fixed-width pages like Diffusion > Manage, Config, and Instances. In contrast, I //do// like where we ended up with Phortune in terms of overall design. I anticipate revisiting some of this stuff eventually.)

Test Plan:
  - Looked at Phriction pages on desktop/tablet/mobile/printable -- actions are now available on the page.
  - Looked at other DocumentView pages (like Phame blogs) -- no changes for now.

Reviewers: amckinley

Maniphest Tasks: T13077, T8172

Differential Revision: https://secure.phabricator.com/D19617
2018-08-28 14:58:05 -07:00
epriestley
fd0da4c41f Rename "PHUIDocumentViewPro" to "PHUIDocumentView"
Summary: Ref T13077. There is no "PHUIDocumentView" so toss the "Pro" suffix from this classname.

Test Plan: Grepped for `PHUIDocumentView` and `PHUIDocumentViewPro`.

Reviewers: amckinley

Maniphest Tasks: T13077

Differential Revision: https://secure.phabricator.com/D19616
2018-08-28 14:53:07 -07:00
epriestley
614f9ba1fb Allow unit test results to specify that their details are formatted with remarkup when reporting to "harbormaster.sendmessage"
Summary: Ref T13189. See PHI710. Ref T13088. Fixes T9951. Allow callers to `harbormaster.sendmessage` to specify that the test details are remarkup so they can use rich formatting and include links, files, etc.

Test Plan: {F5840098}

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13189, T13088, T9951

Differential Revision: https://secure.phabricator.com/D19615
2018-08-28 13:26:11 -07:00
epriestley
632cafec88 Pass commit authorship information to Buildkite
Summary:
Fixes T12251. Ref T13189. See PHI610. The difficulty here is that we don't want to disclose Phabricator account information to Buildkite. We're comfortable disclosing information from `git`, etc.

  - For commits, use the Identity to provide authorship information from Git.
  - For revisions, use the local commit information on the Diff to provide the Git/Mercurial/etc author of the HEAD commit.

Test Plan:
  - Built commits and revisions in Buildkite via Harbormaster.
  - I can't actually figure out how to see author information on the Buildkite side, but the values look sane when dumped locally.

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13189, T12251

Differential Revision: https://secure.phabricator.com/D19614
2018-08-27 12:52:11 -07:00
epriestley
2f5c6541fc Add an "Activated Epoch" and an "Acquired Epoch" to Drydock Leases
Summary: Ref T13189. See PHI690. When a lease is first acquired or activated, note the time. This supports better visibility into queue lengths. For now, this is only queryable via DB and visible in the UI, but can be more broadly exposed in the future.

Test Plan: Landed a revision, saw the leases get sensible timestamps for acquisition/activation.

Reviewers: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19613
2018-08-27 11:27:45 -07:00
epriestley
ee823982a4 Remove another old remarkup engine callsite in Config
Summary: Ref T13189. Summaries do not appear to be meaningfully rendered with Remarkup so just drop the engine. See D19610 for the previous change in this vein.

Test Plan: Viewed config list with option summaries.

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19612
2018-08-27 11:10:14 -07:00
epriestley
b87a809b0b Make some remarkup handling in Config cleaner, fixing {{other.option} links
Summary:
Depends on D19609. Ref T13189. At some point, we switched from RemarkupEngine to RemarkupView and lost this piece of hack-magic.

Restore the hack-magic. It's still hack-magic instead of a real rule, but things are at least cleaner than they were before.

Test Plan: Viewed `auth.require-approval`, etc. Saw references to other config options linked properly.

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19610
2018-08-27 09:54:19 -07:00
epriestley
0295a00229 When there are no setup issues, don't show a weird empty box
Summary: Ref T13189. When there are no setup issues, we currently double-render a weird setup issues box underneath the notice. Get rid of it.

Test Plan: Viewed page with and without setup issues, saw less awkward UI.

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19609
2018-08-27 09:53:52 -07:00
epriestley
cd8b5b82c8 Stop requiring CAN_EDIT to reach the TransactionEditor via "*.edit" in EditEngine
Summary:
Depends on D19607. Ref T13189. See PHI642. Ref T13186.

Some transactions can sometimes be applied to objects you can not edit. Currently, using `*.edit` to edit an object always explicitly requires CAN_EDIT.

Now that individual transactions require CAN_EDIT by default and can reduce or replace this requirement, stop requiring CAN_EDIT to reach the editor.

The only expected effect of this change is that low-permission edits (like disabling a user, leaving a project, or leaving a thread) can now work via `*.edit`.

Test Plan:
  - Tried to perform a normal edit (changing a task title) against an object with no CAN_EDIT. Still got a permissions error.
  - As a non-admin, disabled other users while holding the "Can Disable Users" permission.
  - As a non-admin, got a permissions error while trying to disable other users while not holding the "Can Disable Users" permission.

Reviewers: amckinley

Maniphest Tasks: T13189, T13186

Differential Revision: https://secure.phabricator.com/D19608
2018-08-27 08:10:08 -07:00
epriestley
f9192d07f2 Align web UI "Disable" and "Approve/Disapprove" flows with new "Can Disable Users" permission
Summary:
Depends on D19606. Ref T13189. See PHI642.

  - Disabling/enabling users no longer requires admin. Now, you just need "Can Disable Users".
  - Update the UI to appropriately show the action in black or grey depending on what clicking the button will do.
  - For "Approve/Disapprove", fix a couple bugs, then let them go through without respect for "Can Disable Users". This is conceptually a different action, even though it ultimately sets the "Disabled" flag.

Test Plan:
  - Disabled/enabled users from the web UI as various users, including a non-administrator with "Can Disable Users".
  - Hit permissions errors from the web UI as various users, including an administrator without "Can Disable Users".
  - Saw the "Disable/Enable" action activate properly based on whether clicking the button would actually work.
  - Disapproved a user without "Can Disable Users" permission, tried to re-disapprove a user.
  - Approved a user, tried to reapprove a user.

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19607
2018-08-27 08:09:42 -07:00
epriestley
058952e72e Add a "Can Disable Users" capability to the "People" application
Summary:
Depends on D19605. Ref T13189. See PHI642. This adds a separate "Can Disable Users" capability, and makes the underlying transaction use it.

This doesn't actually let you weaken the permission, since all pathways need more permissions:

  - `user.edit` needs CAN_EDIT.
  - `user.disable/enable` need admin.
  - Web UI workflow needs admin.

Upcoming changes will update these pathways.

Without additional changes, this does let you //strengthen// the permission.

This also fixes the inability to disable non-bot users via the web UI.

Test Plan:
  - Set permission to "No One", tried to disable users. Got a tailored policy error.
  - Set permission to "All Users", disabled/enabled a non-bot user.

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19606
2018-08-27 08:01:27 -07:00
epriestley
8cf56913d8 Deprecate "user.enable" and "user.disable" API methods, redefine them in terms of "user.edit"
Summary:
Depends on D19604. Ref T13189. See PHI642. Deprecates these in favor of "user.edit", redefines them in terms of it, and removes the old `disableUser()` method.

I kept the "is admin" permissions check for consistency, since these methods have always said "(admin only)". This check may not be the most tailored check soon, but we can just keep executing it in addition to the real check.

For now, this change stops this method from actually disabling non-bot users (since it implicitly adds a CAN_EDIT requirement, and even administrators don't have that). An upcoming change will fix that.

Test Plan: Enabled and disabled a (bot) user via these methods. Checked API UI, saw them marked as "disabled".

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19605
2018-08-27 08:00:48 -07:00
epriestley
2f7b10c023 Replace "Disable User" web UI flow with transactions
Summary:
Ref T13189. See PHI642. Upgrades the "Disable" action in the web UI to be transaction-based.

This technically breaks things a little (you can't disable non-bot users, since they now require CAN_EDIT and you won't have it) but an upcoming change will fix the permissions issue.

Test Plan: Disabled and enabled a (bot) user from the web UI.

Reviewers: amckinley

Maniphest Tasks: T13189

Differential Revision: https://secure.phabricator.com/D19604
2018-08-27 08:00:21 -07:00
epriestley
4d89afcc61 Remove requireCapabilities() from ApplicationTransactionEditor and require CAN_EDIT by default
Summary:
Depends on D19585. Ref T13164.

Almost all transactions require CAN_EDIT on the object, but they generally do not enforce this directly today. Instead, this is effectively enforced by Controllers, API methods, and EditEngine doing a `CAN_EDIT` check when loading the object to be edited.

A small number of transactions do not require CAN_EDIT, and instead require only a weaker/lesser permission. These are:

  - Joining a project which you have CAN_JOIN on.
  - Leaving a project which isn't locked.
  - Joining a Conpherence thread you can see (today, no separate CAN_JOIN permission for Conpherence).
  - Leaving a Conpherence thread.
  - Unsubscribing.
  - Using the special `!history` command from email.

Additionally, these require CAN_INTERACT, which is weaker than CAN_EDIT:

  - Adding comments.
  - Subscribing.
  - Awarding tokens.

Soon, I want to add "disabling users" to this list, so that you can disable users if you have "Can Disable User" permission, even if you can not otherwise edit users.

It's possible this list isn't exhaustive, so this change might break something by adding a policy check to a place where we previously didn't have one. If so, we can go weaken that policy check to the appropriate level.

Enforcement of these special cases is currently weird:

  - We mostly don't actually enforce CAN_EDIT in the Editor; instead, it's enforced before you get to the editor (in EditEngine/Controllers).
  - To apply a weaker requirement (like leaving comments or leaving a project), we let you get through the Controller without CAN_EDIT, then apply the weaker policy check in the Editor.
  - Some transactions apply a confusing/redundant explicit CAN_EDIT policy check. These mostly got cleaned up in previous changes.

Instead, the new world order is:

  - Every transaction has capability/policy requirements.
  - The default is CAN_EDIT, but transactions can weaken this explicitly they want.
  - So now we'll get requirements right in the Editor, even if Controllers or API endpoints make a mistake.
  - And you don't have to copy/paste a bunch of code to say "yes, every transaction should require CAN_EDIT".

Test Plan:
- Tried to add members to a Conpherence thread I could not edit (permissions error).
- Left a Conpherence thread I could not edit (worked properly).
- Joined a thread I could see but could not edit (worked properly).
- Tried to join a thread I could not see (permissions error).
- Implemented `requireCapabilites()` on ManiphestTransactionEditor and tried to edit a task (upgrade guidance error).
- Mentioned an object I can not edit on another object (works).
- Mentioned another object on an object I can not edit (works).
- Added a `{F...}` reference to an object I can not edit (works).
- Awarded tokens to an object I can not edit (works).
- Subscribed/unsubscribed from an object I can not edit (works).
- Muted/unmuted an object I can not edit (works).
- Tried to do other types of edits to an object I can not edit (correctly results in a permissions error).
- Joined and left a project I can not edit (works).
- Tried to edit and add members to a project I can not edit (permissions error).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19586
2018-08-24 17:45:56 -07:00
epriestley
b584834b19 In Differential: when the file tree is enabled, default to the "History" tab instead of "Files"
Summary:
Ref T13187. See PHI811. If the file tree is enabled and visible, set the default tab to "History".

  - This is a bit magic.
  - It won't work entirely on mobile (we can't tell that you're on mobile on the server, so we'll pick the "History" tab even though the file tree isn't actually visible on your device).
  - There's no corresponding logic in Diffusion. Diffusion doesn't have the same tab layout, but this makes things somewhat inconsistent.

So I don't love this, but we can try it and see if it's confusing or helpful on the balance.

Test Plan: With filetree on and off, reloaded revisions. Saw appropriate tab selected by default.

Reviewers: amckinley

Maniphest Tasks: T13187

Differential Revision: https://secure.phabricator.com/D19601
2018-08-24 10:29:35 -07:00
epriestley
b6fa009cf0 Enrich "priority" transactions in Maniphest for "transaction.search"
Summary:
Ref T13187. See <https://discourse.phabricator-community.org/t/task-priority-change-info-missing-in-firehose-webhook/1832/2>. We can reasonably enrich these transactions.

Since priorities don't have unique authorative string identifiers, I've mostly mimicked the `maniphest.search` structure.

Test Plan: Called `transaction.search` on tasks which were: created normally, created with a priority change, saw a priority change after creation. All the output looked useful and sensible.

Reviewers: amckinley

Maniphest Tasks: T13187

Differential Revision: https://secure.phabricator.com/D19599
2018-08-24 10:05:05 -07:00
epriestley
5e4d9dfa92 Condition "Changes Since Last Action" Differential link on "first broadcast", not "new object"
Summary: Ref T13187. Ref T13176. With drafts, we actually want to suppress this link on "first broadcast" (the first time we send mail), not on "new object" (when the revision is created as a draft).

Test Plan: Poked at this locally, will keep an eye on it in production.

Reviewers: amckinley

Maniphest Tasks: T13187, T13176

Differential Revision: https://secure.phabricator.com/D19598
2018-08-24 10:03:55 -07:00
epriestley
ca618a8679 Document that phd.taskmasters is a local setting, per daemon
Summary: Ref T13187. See PHI807. The documentation currently does not make it very clear that this is a local setting, per `phd` process. Make it more clear.

Test Plan: {F5827757}

Reviewers: amckinley

Maniphest Tasks: T13187

Differential Revision: https://secure.phabricator.com/D19597
2018-08-24 08:08:19 -07:00
epriestley
7ef2bb1b56 Support Mercurial "protocaps" wire command
Summary:
Ref T13187. See PHI834. Mercurial has somewhat-recently (changeset is from Jan 2018) introduced a new "protocaps" command, that appears in Mercurial 4.7 and possibly before then.

We must explicitly enumerate all protocol commands because you can't decode the protocol without knowing how many arguments the command expects, so enumerate it.

(Also fix an issue where the related error message had an extra apostrophe.)

Test Plan:
  - Ran `hg clone ...` with client and server on Mercurial 4.7.
  - Before: fatal on unknown "protocaps" command.
  - Midway: better typography in error message.
  - After: clean clone.

Reviewers: amckinley

Maniphest Tasks: T13187

Differential Revision: https://secure.phabricator.com/D19596
2018-08-23 15:06:25 -07:00
epriestley
75a5dd8d8c Add more accessibility labels for screen readers
Summary:
Depends on D19594. See PHI823. Ref T13164.

  - Add a label for the "X" button in comment areas, like "Remove Action: Change Subscribers".
  - Add a label for the floating header display options menu in Differential.
  - Add `role="button"` to `PHUIButtonView` objects that we render with an `<a ...>` tag.

Test Plan:
Viewed a revision with `?__aural__=true`:

  - Saw "Remove Action: ..." label.
  - Saw "Display Options" label.
  - Used inspector to verify that some `<a class="button" ...>` now have `<a class="button" role="button" ...>`. This isn't exhaustive, but at least improves things. A specific example is the "edit", "reply", etc., actions on inline comments.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19595
2018-08-17 13:31:51 -07:00
Austin McKinley
5c4c593af3 Update DiffusionLastModifiedController to use identities
Summary: Ref T12164. Updates another controller to use identities.

Test Plan:
Pretty ad-hoc, but loaded the main pages of several different repos with and without repo identities. I'm not totally convinced the `author` from this data structure is actually being used:
```
$return = array(
  'commit'    => $modified,
  'date'      => $date,
  'author'    => $author,
  'details'   => $details,
);
```

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T12164

Differential Revision: https://secure.phabricator.com/D19580
2018-08-17 12:24:21 -07:00
epriestley
438edde031 Add some missing aural button labels for accessibility
Summary:
Ref T13164. See PHI823. (See that issue for some more details and discussion.)

Add aural labels to various buttons which were missing reasonable aural labels.

The "Search" button (magnifying glass in the global search input) had an entire menu thing inside it. I moved that one level up and it doesn't look like it broke anything (?). All the other changes are pretty straightforward.

Test Plan:
{F5806497}

{F5806498}

  - Will follow up on the issue to make sure things are in better shape for the reporting user.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19594
2018-08-17 11:00:29 -07:00
epriestley
a48e6897a4 Remove obsolete setup check call to Maniphest "Can Edit <X>" field checks
Summary: Ref T13164. Missed this in D19581.

Test Plan:
  - Forced setup checks to re-run by visiting {nav Config > Setup Issues} explicitly.
  - Before patch: fatal on call to nonexistent method.
  - After patch: setup issues.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19593
2018-08-16 13:56:20 -07:00
epriestley
0ccf1410e0 Give PhabricatorAuthPassword a formal CAN_EDIT policy
Summary:
Depends on D19585. Ref T13164. This is a precursor for D19586, which causes Editors to start doing more explicit CAN_EDIT checks.

Passwords have an Editor, but don't actually define a CAN_EDIT capability. Define one (you can edit a password if you can edit the object the password is associated with).

(Today, this object is always a User -- this table just unified VCS passwords and Account passwords so they can be handled more consistently.)

Test Plan:
  - With D19586, ran unit tests and got a pass.
  - Edited my own password.
  - Tried to edit another user's password and wasn't permitted to.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19592
2018-08-16 11:53:24 -07:00
epriestley
7e29ec2e2a Move the "Can Lock Projects" check from requireCapabilities() to transaction validation
Summary: Depends on D19584. Ref T13164. This check is an //extra// check: you need EDIT //and// this capability. Thus, we can do it in validation without issues.

Test Plan:
  - This code isn't reachable today: all methods of applying this transaction do a separate check for "Can Lock" upfront.
  - Commented out the "Can Lock" check in the LockController, tried to lock as a user without permission. Was rejected with a policy exception.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19585
2018-08-16 10:56:00 -07:00
epriestley
3b92da22f4 Move the hierarchical edit policy check in Phriction from requireCapabilities() to validateTransactions()
Summary:
Depends on D19583. Ref T13164. This continues the work of getting rid of `requireCapabilities()`.

This check is valid, but can be a `validateTransactions()` check instead. This is generally more consistent with how other applications work (e.g., creating subprojects).

The UI for this isn't terribly great: you get a policy error //after// you try to create the object. But that's how it worked before, so this isn't any worse than it was. The actual policy exception is (very) slightly more clear now (raised against the right object).

Test Plan:
  - Created a child as a user with permission to do so to make sure I didn't break that.
  - Set edit permission on `a/` to just me, tried to create `a/b/` as another user, got a policy exception since they can't edit the parent.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19584
2018-08-16 10:55:11 -07:00
epriestley
24d4445845 Remove pointless requireCapabilities() method from PhabricatorRepositoryEditor
Summary: Depends on D19582. Ref T13164. It's not possible to reach the editor without passing through a CAN_EDIT check, and it shouldn't be necessarily to manually specify that edits require CAN_EDIT by default.

Test Plan: Grepped for `RepositoryEditor`, verified that all callsites pass through a CAN_EDIT check.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19583
2018-08-16 10:53:42 -07:00
epriestley
a39852ae1b Remove pointless requireCapabilities() method from PhabricatorProjectColumnTransactionEditor
Summary:
Depends on D19581. Ref T13164. This method has no effect:

  - You must always have CAN_EDIT to reach an Editor in the first place.
  - Per previous change, I'm going to restructure this so transactions explicitly check CAN_EDIT by default anyway.

Test Plan: Tried to edit and hide a project column as a user without permission, hit global permission checks long before reaching this method.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19582
2018-08-16 10:51:57 -07:00
epriestley
296bf046a8 Remove deprecated Maniphest "Can Edit <Specific Property>" capabilities
Summary:
Depends on D19579. Fixes T10003. These have been deprecated with a setup warning about their impending removal for about two and a half years.

Ref T13164. See PHI642. My overall goal here is to simplify how we handle transactions which have special policy behaviors. In particular, I'm hoping to replace `ApplicationTransactionEditor->requireCapabilities()` with a new, more clear policy check.

A problem with `requireCapabilities()` is that it doesn't actually enforce any policies in almost all cases: the default is "nothing", not CAN_EDIT. So it ends up looking like it's the right place to specialize policy checks, but it usually isn't.

For "Disable", I need to be able to weaken the check selectively (you can disable users if you have the permission, even if you can't edit them otherwise). We have a handful of other edits which work like this (notably, leaving and joining projects) but they're very rare.

Test Plan: Grepped for all removed classes. Edited a Maniphest task.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164, T10003

Differential Revision: https://secure.phabricator.com/D19581
2018-08-16 10:51:06 -07:00
epriestley
f9673a72a8 Allow "user.edit" to enable or disable users
Summary:
Depends on D19577. Ref T13164. See PHI642. This adds modern transaction-oriented enable/disable support.

Currently, this also doesn't let you disable normal users even when you're an administrator. I'll refine the policy model later in this change series, since that's also the goal here (let users set "Can Disable Users" to some more broad set of users than "Administrators").

This also leaves us with two different edit pathways: the old UserEditor one and the new UserTransactionEditor one. The next couple diffs will redefine the other pathways in terms of this pathway.

Test Plan:
  - Enabled/disabled a bot.
  - Tried to disable another non-bot user. This isn't allowed yet, since even as an administrator you don't have CAN_EDIT on them and currently need it: right now, there's no way for a particular set of transactions to say they can move forward with reduced permissions.
  - Tried to enable/disable myself. This isn't allowed since you can't enable/disable yourself.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19579
2018-08-16 10:49:35 -07:00
epriestley
65904d7c51 Add a modern "user.edit" API method for users
Summary:
Depends on D19576. Ref T13164. See PHI642. This adds an EditEngine for users and a `user.edit` modern API method.

For now, all it supports is editing real name, blurb, title, and icon (same as "Edit Profile" from the UI).

Test Plan:
  - Edited my stuff via the new API method.
  - Tried to edit another user, got rejected by policies.
  - Tried to create a user, got rejected by policies.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19577
2018-08-16 10:48:38 -07:00
epriestley
39d415e90e Move users to modular transactions
Summary:
Ref T13164. See PHI642. I'd like to provide a third-generation `user.edit` API endpoint and make `user.enable` and `user.disable` obsolete before meddling with policy details, even if it isn't full-fledged yet.

Users do already have a transactions table and a Transaction-based editor, but it's only used for editing title, real name, etc. All of these are custom fields, so their support comes in automatically through CustomField extension code.

Realign it for modular transactions so new code will be fully modern. There are no actual standalone transaction types yet so this diff is pretty thin.

Test Plan:
  - Grepped for `UserProfileEditor`.
  - Edited a user's title/real name/icon.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19576
2018-08-16 10:47:47 -07:00
Austin McKinley
cc1def6cea Remove some array typehints for passing around
Summary: See discussion at https://secure.phabricator.com/D19492#241996

Test Plan: Refreshed a few Diffusion tabs; nothing broke.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19578
2018-08-13 16:07:56 -07:00
Austin McKinley
3b05e920e0 Start changing DiffusionCommitController to use identities
Summary: Depends on D19491.

Test Plan: Viewed some commits where the identity was mapped to a user and another that wasn't; saw the header render either a link to the user or the identity object.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19492
2018-08-13 15:23:31 -07:00
epriestley
92a29f72c1 Make the Drydock repository operation page slightly richer
Summary:
Ref T13164. See PHI788. The issue requests a "created" timestamp.

Also add filtering for repository, state, and author.

Test Plan:
Used all filters.

{F5795085}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19574
2018-08-13 11:42:10 -07:00
epriestley
fb3ae72e36 When cancelling addition of an Almanac interface, return to the Device page
Summary:
Fixes T13184. In Almanac, interfaces are always added to devices. However, if you "Add New Interface" and then "Cancel", you go to the nonexistent `/interface/` page.

Instead, return to the device page.

Test Plan: From a device page, clicked "Add Interface" and then "Cancel". Ended up back where I was.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13184

Differential Revision: https://secure.phabricator.com/D19573
2018-08-13 11:39:37 -07:00
epriestley
b86dae6214 Fix an issue with error handling when no mailers are available
Summary:
Ref T13164. See PHI785. See D19546. I think I didn't test the updated error messaging here entirely properly, since I have some tasks in queue which error out here ("Missing argument 1 to newMailers(...)").

This is an error condition already, but we want to get through this call so we can raise a tailored message.

Test Plan: Tasks which errored out here now succeed. This condition is only reachable if you misconfigure things in the first place.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19572
2018-08-13 11:39:13 -07:00
epriestley
e5906f4e12 In Differential standalone views, disable some keyboard shortcuts which don't work
Summary:
Ref T13164. See PHI693. In Differential, you can {nav View Options > View Standalone} to get a standalone view of a single changeset. You can also arrive here via the big changeset list for revisions affecting a huge number of files.

We currently suggest that all the keyboard shortcuts work, but some do not. In particular, the "Next File" and "Previous File" keyboard shortcuts (and some similar shortcuts) do not work. In the main view, the next/previous files are on the same page. In the standalone view, we'd need to actually change the URI.

Ideally, we should do this (and, e.g., put prev/next links on the page). As a first step toward that, hide the nonfunctional shortcuts to stop users from being misled.

Test Plan:
  - Viewed a revision in normal and standalone views.
  - No changes in normal view, and all keys still work ("N", "P", etc).
  - In standalone view, "?" no longer shows nonfunctional key commands.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19571
2018-08-13 08:59:05 -07:00
Austin McKinley
a6951a0a5a Add migration to encourage rebuilding repository identities
Summary: Ref T12164. Defines a new manual activity that suggests rebuilding repository identities before Phabricator begins to rely on them.

Test Plan:
- Ran migration, observed expected setup issue: {F5788217}
- Ran `bin/config done identities` and observed setup issue get marked as done.
- Ran `/bin/storage upgrade --apply phabricator:20170912.ferret.01.activity.php` to make sure I didn't break the reindex migration; observed reindex setup issue appear as expected.
- Ran `./bin/config done reindex` and observed reindex issue cleared as expected.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T12164

Differential Revision: https://secure.phabricator.com/D19497
2018-08-10 13:47:03 -07:00
Austin McKinley
2951694c27 Correctly spell 'committer'
Summary: It's a funny word. h/t @joshuaspence

Test Plan: Inspection of correct spelling.

Reviewers: epriestley, joshuaspence

Reviewed By: joshuaspence

Subscribers: Korvin, joshuaspence

Differential Revision: https://secure.phabricator.com/D19570
2018-08-09 17:52:43 -07:00
epriestley
6df278bea8 In "bin/ssh-auth", cache a structure instead of a flat file because paths may change at runtime
Summary:
Fixes T12397. Ref T13164. See PHI801.

Several installs have hit various use cases where the path on disk where Phabricator lives changes at runtime. Currently, `bin/ssh-auth` caches a flat file which includes the path to `bin/ssh-exec`, so this may fall out of date if `phabricator/` moves.

These use cases have varying strengths of legitimacy, but "we're migrating to a new set of hosts and the pool is half old machines and half new machines" seems reasonably compelling and not a problem entirely of one's own making.

Test Plan:
  - Compared output on `master` to output after change, found them byte-for-byte identical.
  - Moved `phabricator/` to `phabricator2/`, ran `bin/ssh-auth`, got updated output.
  - Added a new SSH key, saw it appear in the output.
  - Grepped for `AUTHFILE_CACHEKEY` (no hits).
  - Dropped the cache, verified that the file regenerates cleanly.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164, T12397

Differential Revision: https://secure.phabricator.com/D19568
2018-08-09 13:33:23 -07:00
epriestley
df31405d64 Improve compatibility of "Config > Cache Status" across APCu versions
Summary:
Ref T13164. See PHI790. Older versions of APCu reported cache keys as "key" from `apcu_cache_info()`. APC and newer APCu report it as "info".

Check both indexes for compatibility.

Test Plan:
  - Locally, with newer APCu, saw no behavioral change.
  - Will double check on `admin`, which has an older APCu with the "key" behavior.
  - (I hunted this down by dumping `apcu_cache_info()` on `admin` to see what was going on.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19569
2018-08-08 15:07:03 -07:00
epriestley
3ca3f09a1a Add an "--as" flag to "bin/conduit call ..." to improve flexibility and ease of profiling
Summary:
Ref T13164. In PHI801, an install reported a particular slow Conduit method call.

Conduit calls aren't easily profilable with normal tools (for example, `arc call-conduit --xprofile ...` gives you a profile of the //client//). They can be profiled most easily with `bin/conduit call ... --xprofile`.

However, `bin/conduit call` currently doesn't let you pick a user to execute the command on behalf of, so it's not terribly useful for profiling `*.edit`-style methods which do a write: these need a real acting user.

Test Plan:
Ran `bin/conduit call --method user.whoami --as epriestley ...` with valid, invalid, and no acting users.

```
$ echo '{}' | ./bin/conduit call --method user.whoami --as epriestley --input -
Reading input from stdin...
{
  "result": {
    "phid": "PHID-USER-icyixzkx3f4ttv67avbn",
    "userName": "epriestley",
    "realName": "Evan Priestley",
...
```

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19566
2018-08-08 09:51:21 -07:00
epriestley
201f29fbf4 Fix truncation in "bin/storage probe" of tables larger than 100GB
Summary:
Ref T13164. PHI805 incidentally includes some `bin/storage probe` output for 100GB+ tables which renders wrong.

We have the tools to render it properly, so stop doing this manually and let ConsoleTable figure out the alignment.

Test Plan:
Faked very large table sizes, ran `bin/storage probe`:

{F5785946}

(Then, un-faked the very large table sizes and ran it again, got sensible output.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19567
2018-08-08 09:50:59 -07:00
epriestley
91abc0f027 Stop indexing the chunk data objects for large Files stored in multiple chunks
Summary:
Ref T13164. See PHI766. Currently, when file data is stored in small chunks, we submit each chunk to the indexing engine.

However, chunks are never surfaced directly and can never be found via any search/query, so this work is pointless. Just skip it.

(It would be nice to do this a little more formally on `IndexableInterface` or similar as `isThisAnIndexableObject()`, but we'd have to add like a million empty "yes, index this always" methods to do that, and it seems unlikely that we'll end up with too many other objects like these.)

Test Plan:
  - Ran `bin/harbormaster rebuild-log --id ... --force` before and after change, saw about 200 fewer queries after the change.
  - Uploaded a uniquely named file and searched for it to make sure I didn't break that.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19563
2018-08-03 14:36:12 -07:00
epriestley
5839a54b60 Raise a tailored error when calling "transaction.search" with empty "phids" constraint
Summary:
Ref T13164. See PHI725. For real "*.search" methods, parameters get validated and you get an error if you use an empty list as a constraint.

Since "transaction.search" isn't really a normal "*.search" method, it doesn't benefit from this. Just do the check manually for now.

Test Plan: Made `transaction.search` calls with no constraints (got results); a valid costraint (got fewer results); and an invalid empty constraint (got an exception).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19562
2018-08-03 14:29:36 -07:00
epriestley
f3fa164882 Add a "Last Edited" property to Wiki pages
Summary:
Ref T13164. See PHI797. The last edit is available in the page header, but it's not precise (just says "180 days ago") and a little weird (it's unusual for us to put that kind of information in the header).

Add a precise timestamp to the footer for now. I'd imagine re-examining this the next time Phriction gets some UI work and maybe trying to integrate timeline/transactions more cleanly (see also T1894).

Test Plan: Looked at a wiki page, then edited it. Saw precise "Last Edit" timestamp adjacent to "Last Author".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19560
2018-08-03 14:29:17 -07:00
epriestley
3574a55a95 Deprecate Conduit method "diffusion.getrecentcommitsbypath"
Summary:
See D19558. This method has no callers and just wraps `diffusion.historyquery`, since D5960 (2013).

This was introduced in D315 (which didn't make it out of FB, I think) inside Facebook for unclear purposes in 2011.

Test Plan: Grepped for callers, found none.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: artms

Differential Revision: https://secure.phabricator.com/D19559
2018-08-03 09:48:58 -07:00
Arturas Moskvinas
356b2781bc Gracefully fail request if non existing callsign is passed to getrecentcommitsbypath instead of crashing
Summary:
`diffusion.getrecentcommitsbypath` fails with 500 error when non existing callsign is passed:
```
>>> UNRECOVERABLE FATAL ERROR <<<

Call to a member function getCommit() on null

```

Expected Behavior:
Return more graceful error notifying caller that such callsign/repository does not exist

Reproduction steps:
Open conduit: https://secure.phabricator.com/conduit/method/diffusion.getrecentcommitsbypath/
Enter:
callsign: "obviouslynotexisting"
path: "/random"
Click call method

Test Plan: after applying patch - call no longer fails with 500s

Reviewers: Pawka, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19558
2018-08-02 19:49:10 +03:00
epriestley
e72296f927 Support querying Herald rules by monogram in typeahead datsources
Summary:
Depends on D19556. See PHI765. Ref T13164. Currently, if you type `H1` in this datasource, it isn't smart enough to pull up the right object.

Add support for querying by monogram. This is similar to existing support in Owners packages, etc.

Test Plan: Typed `H1` in the new push log filter, got the right object as a result in the typeahead.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19557
2018-08-01 17:52:27 -07:00
epriestley
06380e8079 Allow push events to be filtered by which Herald rule blocked the push
Summary: Depends on D19555. Ref T13164. See PHI765. An install is interested in getting a sense of the impact of a particular blocking rule, which seems reasonable. Support filtering for pushes blocked by a particular rule or set of rules.

Test Plan: {F5776385}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19556
2018-08-01 17:38:12 -07:00
epriestley
d8834377be When a Herald rule blocks a push, show which rule fired in the push log UI
Summary:
Ref T13164. See PHI765. We currently show "Rejected: Herald" in the push log UI, but don't show which rule rejected a push.

We store this data, and it's potentially useful: either for hunting down a particular issue, or for getting a general sense of how often a reject rule is triggering (maybe because you want to tune how aggressive it is).

Show this data in the web UI, and include it in the data export payload.

Test Plan:
  - Pushed to a hosted repository so that I got blocked by a Herald rule.
  - Viewed the push logs in the web UI, now saw which rule triggered things.
  - Exported logs to CSV, saw Herald rule PHIDs in the data.

{F5776211}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19555
2018-08-01 17:33:50 -07:00
epriestley
96e3c73159 Put "Subprojects" on top of "Milestones" in the Project UI
Summary:
Depends on D19550. Ref T13164. See T12144#226172, mostly. We get some requests to make milestones reorderable, but in most cases users probably wanted subprojects, not milestones.

One reason to end up here is that we put "Milestones" on top. Instead, put "Subprojects" on top, since they're the less specialized option and we aren't terribly consistent about it anyway.

Test Plan: Viewed project subprojects page, saw "Subprojects" above "Milestones".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19551
2018-08-01 13:49:42 -07:00
epriestley
45babe82f3 Add Spaces information to the project list UI
Summary: Depends on D19552. Ref T13164. We need this little `setObject(...)` hook to get the Space name into the search list UI.

Test Plan: Viewed project list, saw some Spaces listed.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19554
2018-07-31 10:24:51 -07:00