1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-25 16:22:43 +01:00
Commit graph

16393 commits

Author SHA1 Message Date
epriestley
9e096cd274 Give the workboard "default" workflows more modern state handling
Summary:
Depends on D20628. Ref T4900. Currently, the "Save Current Order/Filter As Default" flows on workboards duplicate some state construction, and require parameters to be passed to them explicitly.

Now that state management is separate, they can reuse a bit more code and be made to look more like other similar controllers.

Test Plan:
  - Changed the default order of a workboard.
  - Changed the default filter of a workboard.
  - Changed the order of a board to something non-default, then changed the filter, then saved the new filter as the default. Saw the modified order preserved and the modified filter removed, so I ended up in the right ("most correct") place: on the board, with my custom order in a URI parameter, and no filter URI parameter so I could see my new default filter behavior. This is an edge case that's not terribly important to get right, but we do get it right.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20629
2019-07-02 14:36:15 -07:00
epriestley
9c190d68ed Separate workboard view state (ordering, filtering, hidden columns) from the View controller
Summary:
Depends on D20627. Ref T4900. If a user orders a board by "Sort by Title", then toggles the visibility of hidden columns, we want to keep the board sorted by title. To accomplish this, we pass the board state around to all the workflows here.

Pull the "bag of state properties" code out of the View controller. This class basically:

  - reads state from a request (order, hidden, filter);
  - manages defaults;
  - provides the application with the current settings; and
  - generates URIs with "?order=X&hidden=Y&filter=Z" to preserve state.

This is still a little questionable/transitional since some of the controllers need more cleanup.

Test Plan:
Toggled state, order, filters, clicked around various workflows and saw the filters preserved.

A lot of these workflows are pretty serious edge cases. For example, here's a feature this implements:

  - Changed workboard order to "Title".
  - Selected "Bulk Edit Tasks..." in an empty column and command-clicked it to open the link in a new window.
  - Hovered over "Cancel".
  - Saw the link properly generate with "?order=title", preserving the order.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20628
2019-07-02 14:34:33 -07:00
epriestley
0ae9e2c75d Remove property "id" from Workboard View controller
Summary: Depends on D20626. Ref T4900. On this controller, "id" is a separate property, but serves little purpose and complicates separating state management. Remove it.

Test Plan: Bulk edited a column, managed filters, did show/hide on columns, edited a column.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20627
2019-07-02 05:17:01 -07:00
epriestley
df29b82ad6 Remove unused property "slug" from Workboard View controller
Summary:
Ref T4900. The Workboard view controller currently has a lot of different responsibilities (it's ~1,500 lines long) because it has to manage the board filter/sort state.

I'd like to split it up and make it easier to move some workboard features (like "move all tasks in column...") to other Controllers, so we can have smaller controllers implementing specific workflows.

I think the state handling isn't really all that bad, it just needs to be separated a little better than it currently is.

To start with, remove the unused "slug" property.

Test Plan: Searched for "slug", got no hits. This class is final and the property is private, so this is certainly unused.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20626
2019-07-02 05:16:34 -07:00
epriestley
d22f6d219b Lightly modernize OAuth server application view pages
Summary: Depends on D20624. Fixes T13330. The OAuth client pages are using some out-of-date rendering conventions; update them to modern conventions.

Test Plan:
Viewed a page, saw a modern header layout + curtain:

{F6534135}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13330

Differential Revision: https://secure.phabricator.com/D20625
2019-07-02 05:15:37 -07:00
epriestley
0e2cb6e7c4 Fix missing URI for "OAuthServerClient" object handles, causing dialog with no button
Summary:
Ref T13330. Handles for "OAuthServerClient" objects currently do not have a URI, which causes some obscure fallout like a missing "Close" button when examining their transactions.

Add a URI.

Test Plan:
  - Viewed an OAuth server client detail page.
  - Edited a policy, changing it to a custom policy.
  - Clicked "Custom Policy" in the resulting transaction to view a dialog explaining the changes.
  - Before change: dialog has no close button.
  - After change: dialog has a close button.

{F6534121}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13330

Differential Revision: https://secure.phabricator.com/D20624
2019-07-02 05:14:14 -07:00
epriestley
d6dc5d8e68 Fix the "x" link in tokenizer tokens incorrectly closing dialogs
Summary:
See <https://discourse.phabricator-community.org/t/removing-tags-or-subscribers-from-a-maniphest-modal-causes-the-modal-to-close/2874>.

My Javascript is rusty: `'' + null == 'null'`. Same for `undefined`. Use an explicit typecheck instead.

Test Plan: Clicked the "x" in a tokenizer token in a dialog, saw the token removed instead of the dialog closed.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20623
2019-06-28 12:38:51 -07:00
epriestley
159fd44203 Correct transaction strengths after inconsitent scaling by 100 vs 1000
Summary:
See D20540. I mistakenly multiplied some strenghts by 100 and others by 1000 when converting them to integers for `PhutilSortVector`.

Multiply them all by 100 (that is, divide the ones which were multiplied by 1000 by 10) to put things back the way they were.

Test Plan: quick mafs

Reviewers: amckinley, richardvanvelzen

Reviewed By: richardvanvelzen

Differential Revision: https://secure.phabricator.com/D20622
2019-06-26 07:19:33 -07:00
epriestley
987e104610 Bump the remarkup cache version after JIRA/Asana rule changes
Summary: See PHI1319. Ref T13291. Bump the remarkup cache version, since the old JIRA / Asana rules may exist in the partial cached representation of remarkup blocks from older versions.

Test Plan: Typed some comments with various formatting, saw remarkup work fine.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291

Differential Revision: https://secure.phabricator.com/D20619
2019-06-25 14:56:29 -07:00
epriestley
eaa60334ec Limit the read buffer size in bin/storage dump
Summary:
Ref T13328. Currently, we read from `mysqldump` something like this:

```
until (done) {
  for (100 ms) {
    mysqldump > in-memory-buffer;
  }

  in-memory-buffer > disk;
}
```

This general structure isn't great. In this use case, where we're streaming a large amount of data from a source to a sink, we'd prefer to have a "select()"-like way to interact with futures, so our code is called after every read (or maybe once some small buffer fills up, if we want to do the writes in larger chunks).

We don't currently have this (`FutureIterator` can wake up every X milliseconds, or on future exit, but, today, can not wake for readable futures), so we may buffer an arbitrary amount of data into memory (however much data `mysqldump` can write in 100ms).

Reduce the update frequency from 100ms to 10ms, and limit the buffer size to 32MB. This effectively imposes an artificial 3,200MB/sec limit on throughput, but hopefully that's fast enough that we'll have a "wake on readable" mechanism by the time it's a problem.

Test Plan:
  - Replaced `mysqldump` with `cat /dev/zero` as the source command, to get fast input.
  - Ran `bin/storage dump` with `var_dump()` on the buffer size.
  - Before change: saw arbitrarily large buffers (300MB+).
  - After change: saw consistent maximum buffer size of 32MB.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13328

Differential Revision: https://secure.phabricator.com/D20617
2019-06-25 07:17:14 -07:00
epriestley
da0dfc057d Make "bin/files" parsing of working set arguments more consistent
Summary:
Fixes T13326. In D20571, I slightly generalized construction of an iterator over a set of files, but missed some code in other "bin/files ..." commands which was also affected.

Today, basically all of these workflows define their own "--all" and "names" flags. Pull these definitions up and implement them more consistently.

Test Plan: Ran multiple different `bin/files` commands with different combinations of arguments, saw consistent handling of iterator construction.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13326

Differential Revision: https://secure.phabricator.com/D20614
2019-06-24 16:02:39 -07:00
epriestley
a3397fb876 Consider "all account members are disabled" to be a permanent failure when billing a Phortune subscription
Summary:
Fixes T13327. Currently, when we try to bill an account and all members are disabled, we fail temporarily and the task retries forever.

At least for now, just treat this as a permanent failure.

Test Plan:
  - Used `bin/phortune invoice` to generate a normal invoice for a regular subscription.
  - Disabled all the account members, then tried again. Got a helpful permanent failure:

```
$ ./bin/phortune invoice --subscription PHID-PSUB-kbedwt5cyepoc6tohjq5 --auto-range
Set current time to Mon, Jun 24, 2:47 PM.
Preparing to invoice subscription "localb.phacility.com" from Fri, May 31, 10:14 AM to Sun, Jun 30, 10:14 AM.
 WARNING
Manually invoicing will double bill payment accounts if the range overlaps an
existing or future invoice. This script is intended for testing and
development, and should not be part of routine billing operations. If you
continue, you may incorrectly overcharge customers.

    Really invoice this subscription? [y/N] y

[2019-06-24 14:47:57] EXCEPTION: (PhabricatorWorkerPermanentFailureException) All members of the account ("PHID-ACNT-qp54y3unedoaxgkkjpj4") for this subscription ("PHID-PSUB-kbedwt5cyepoc6tohjq5") are disabled. at [<phabricator>/src/applications/phortune/worker/PhortuneSubscriptionWorker.php:88]
arcanist(head=experimental, ref.master=d92fa96366c0, ref.experimental=db4cd55d4673), corgi(head=master, ref.master=6371578c9d32), instances(head=stable, ref.master=ba9e4a19df1c, ref.stable=37fb1f4917c7), libcore(), phabricator(head=master, ref.master=65bc481c91de, custom=11), phutil(head=master, ref.master=7adfe4e4f4a3), services(head=master, ref.master=5424383159ac)
  #0 PhortuneSubscriptionWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:124]
  #1 PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:163]
  #2 PhabricatorWorker::scheduleTask(string, array, array) called at [<phabricator>/src/applications/phortune/management/PhabricatorPhortuneManagementInvoiceWorkflow.php:169]
  #3 PhabricatorPhortuneManagementInvoiceWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:457]
  #4 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:349]
  #5 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/setup/manage_phortune.php:21]
$
```

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13327

Differential Revision: https://secure.phabricator.com/D20613
2019-06-24 15:29:31 -07:00
epriestley
65bc481c91 Remove "phd.pid-directory" configuration and stop passing "piddir" to daemons
Summary: Ref T13321. The daemons no longer write PID files, so we no longer need to pass any of this stuff to them.

Test Plan: Grepped for affected symbols.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20608
2019-06-24 11:28:58 -07:00
epriestley
2498e373b9 Make "phd start" and "phd reload" use the process list, not PID files
Summary:
Ref T13321. This gets rid of the last pidfile readers in Phabricator; we just use the process list instead.

These commands always only work on the current instance since they don't make much sense otherwise.

Test Plan: Ran `bin/phd start` and `bin/phd reload` with and without daemons running.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20606
2019-06-24 11:27:25 -07:00
epriestley
08b9e70bea Make "bin/phd status" report local daemons from the process list, not a mess of local/remote information
Summary:
Ref T13321. Fixes T11037. Realign "bin/phd status" to just mean "show daemon processes on this host".

The value of `bin/phd status` as a mixed remote/local command isn't clear, and the current output is a confusing mess (see T11037).

This also continues letting us move away from PID files.

Test Plan: Ran `bin/phd status`, saw sensible local process status.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321, T11037

Differential Revision: https://secure.phabricator.com/D20604
2019-06-24 11:25:53 -07:00
epriestley
b99c240aa3 Deprecate "bin/phd ... --gently" and update documentation
Summary:
Ref T13321. Previously, the behavior was:

  - `bin/phd stop --gently`: Stop all daemons with PID files that belong to the current instance.
  - `bin/phd stop`: Stop all daemons with PID files that belong to the current instance. Complain if there are more processes.
  - `bin/phd stop --force`: Stop all processes that look like daemons, ignoring instances.

The new behavior is:

  - `bin/phd stop`: Stop all processes that look like daemons and belong to the current instance.
  - `bin/phd stop --force`: Stop all processes that look like daemons, period.

Test Plan: Grep / documentation only.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20602
2019-06-24 11:16:00 -07:00
epriestley
d98bf8ef8e Drive "phd stop" entirely from the process list, not PID files on disk
Summary:
Ref T13321. Depends on D20600. Make `bin/phd stop` mean:

  - `bin/phd stop`: Stop all processes which have daemon process titles. If we're instanced, only stop daemons for the current instance.
  - `bin/phd stop --force`: Stop all processes which have deamon process titles for any instance.

We no longer read or care about PID files on disk, and this moves us away from PID files.

This makes unusual flag `--gently` do nothing. A followup will update the documentation and flags to reflect actual usage/behavior.

This also removes the ability to stop specific PIDs. This was somewhat useful long, long ago when you might explicitly run different copies of the `PullLocal` daemon with flags to control which repositories they updated, but with the advent of clustering it's no longer valid to run custom daemon loadouts.

Test Plan: Ran `bin/phd start`, then `bin/phd stop`. Saw instance daemons stop. Ran `bin/phd stop --force`, saw all daemons stop.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20601
2019-06-24 11:05:56 -07:00
epriestley
75c3598359 Require commit identities when editing commits to resolve an issue with audit actions not applying properly
Summary:
See <https://discourse.phabricator-community.org/t/cannot-audit-a-git-commit/2848>. In D20581, I made some audit behavior dependent upon identities, but the actual edit flow doesn't load them. This can cause us to raise an "attach identities first" exception in the bowels of the edit workflow and trigger unexpected behavior at top level.

Load identities when editing a commit so that the transaction flows have access to identity information and can use it to figure out if a user is an author, etc.

Test Plan:
  - As an auditor, applied an "Accept Commit" action to an open audit after D20581.
  - Before patch: accept no-ops internally since the preconditions throw.
  - After patch: accept works properly.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20612
2019-06-24 10:50:23 -07:00
epriestley
ca56e8590a Don't handle JIRA/Asana URIs with anchors or query parameters in a special way (with Doorkeeper)
Summary:
Ref T13291. See PHI1312. Currently, if you link to a JIRA or Asana issue with an anchor (`#asdf`) or query parameters (`?a=b`), we:

  - treat the link as an external object reference and attempt a lookup on it;
  - if the lookup succeeds, we discard the fragment or parameters when re-rendering the rich link (with the issue/task title).

Particularly, the re-rendering part uses the canonical URI of the object, and can discard these parameters/fragments, which is broken/bad.

As a first pass at improving this, just don't apply special behavior for links with anchors or parameters -- simply treat them as links.

In some future change, we could specialize this behavior and permit certain known parameters or anchors or something, but these use cases are likely fairly marginal.

Test Plan:
Before:

{F6516392}

After:

{F6516393}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291

Differential Revision: https://secure.phabricator.com/D20592
2019-06-21 06:34:44 -07:00
Austin McKinley
6b9f4a918b Modularize PhabricatorEditEngineConfigurationTransaction
Summary: Ref T13319. Ref PHI1302. Migrate `PhabricatorEditEngineConfigurationTransaction` to modular transactions and add some additional transaction rendering to make these edits less opaque.

Test Plan: Hit all the form edit controllers, viewed resulting transaction timeline.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T13319

Differential Revision: https://secure.phabricator.com/D20595
2019-06-20 16:25:21 -07:00
epriestley
c0dc411d23 Update "phabricator/" for "topological" API changes
Summary: Ref T13325.

Test Plan:
  - Grepped for `topograph`.
  - Viewed a task graph since that's easy, looked fine.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13325

Differential Revision: https://secure.phabricator.com/D20599
2019-06-20 16:11:56 -07:00
epriestley
53f8ad14fa Fix an issue in Owners where a transaction change could show too many effects
Summary:
Fixes T13324. Ref PHI1288. Currently, if you edit an Owners package that has some paths with no trailing slashes (like `README.md`) so their internal names and display names differ (`/README.md` display, vs `/README.md/` internal), the "Show Details" in the transaction log shows the path as re-normalized even if you didn't touch it.

Instead, be more careful about handling display paths vs internal paths.

(This code on the whole is significantly less clear than it probably could be, but this issue is so minor that I'm hesitant to start ripping things out.)

Test Plan:
  - In a package with some paths like `/src/` and some paths like `/src`:
  - Added new paths.
  - Removed paths.
  - Changed paths from `/src/` to `/src`.
  - Changed paths from `/src` to `/src/`.

In all cases, the "paths" list and the transaction record identically reflected the edit in the way I expected them to.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13324

Differential Revision: https://secure.phabricator.com/D20596
2019-06-20 16:08:31 -07:00
epriestley
37e26f1b45 Improve rendering of "default value changed" custom form transactions to at least have all the information
Summary:
Ref T13319. Currently, transactions about changes to a default form value use a raw internal key for the affected field and don't show the actual value change.

An ideal implementation will likely require us to specialize a great deal of rendering, but we can do much better than we currently do without too much work:

  - Try to pull the actual `EditField` object for the key so we can `getLabel()` it and get a human-readable label (like `Visible To` instead of `policy.view`).
  - Add a "(Show Changes)" action that dumps the raw values as more-or-less JSON, so you can at least figure out what happened if you're sophisticated enough.

Test Plan:
Before:

{F6516640}

After:

{F6516642}

The quality of "Show Details" varies a lot. For some fields, like "Description", it's pretty good:

{F6516645}

For others, like "Assigned To", it's better than nothing but pretty technical:

{F6516647}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13319

Differential Revision: https://secure.phabricator.com/D20594
2019-06-19 13:47:07 -07:00
epriestley
caaa1394ef Don't count "Cc: x@y.com" as a legitimate recipient if the user who has "x@y.com" attached to their account has not verified the address
Summary:
Fixes T13317. On `admin.phacility.com`, an enterprising user added `noreply@admin.phacility.com` to their account. This caused them to become CC'd on several support issues over the last year, because we send mail "From" this address and it can get CC'd via reply/reply all/whatever else.

The original driving goal here is that if I reply to a task email and CC you on my reply, that should count as a CC in Phabricator, since this aligns with user intent and keeps them in the loop.

This misfire on `noreply@` is ultimately harmless (being CC'd does not grant the user access permission, see T4411), but confusing and undesirable. Instead:

  - Don't allow reserved addresses ("noreply@", "ssladmin@", etc) to trigger this subscribe-via-CC behavior.
  - Only count verified addresses as legitimate user recipients.

Test Plan:
  - Added a `bin/mail receive-test --cc ...` flag to make this easier to test.
  - Sent mail as `bin/mail receive-test --to X --as alice --cc bailey@verified.com`. Bailey was CC'd both before and after the change.
  - Sent mail as `bin/mail receive-test --to X --as alice --cc unverified@imaginary.com`, an address which Bailey has added to her account but not verified.
    - Before change: Bailey was CC'd on the task anyway.
    - After change: Bailey is not CC'd on the task.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13317

Differential Revision: https://secure.phabricator.com/D20593
2019-06-19 12:51:00 -07:00
epriestley
9f44ee3933 Use "link.getAttribute('href')", not "link.href", to bypass dark browser magic
Summary:
Ref T13302. In at least some browsers (including Safari and Chrome), when you write this:

```
<a href="#">...</a>
```

...and then access `<that node>.href`, you get `http://local-domain-whatever.com/path/to/current/page#` back.

This is wonderful, but not what we want. Access the raw attribute value instead, which is `#` in all browsers.

Test Plan:
  - In Safari, Chrome, and Firefox:
  - Clicked "Edit Subtasks" from a task.
  - Clicked "Select" buttons to select several tasks.
  - Before: Clicking these button incorrectly closed the dialog (because of D20573).
  - After: Clicking these buttons now selects tasks without closing the dialog.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13302

Differential Revision: https://secure.phabricator.com/D20590
2019-06-19 10:44:42 -07:00
epriestley
5d8ee504d6 Replace weird, redundant list of branches in Diffusion "Manage" UI with a link to the main branch list
Summary:
Fixes T13312. Currently, {nav Manage > Branches} has a list of branches on the same page. This has a few minor issues:

  - Pager is at the top (see T13312), which is weird.
  - "Default" icon is mystery meat.
  - Table is kind of pointless/redundant in general?

Previously, this table had more information about technical status of each branch (autoclose/track/publish) but most of these details have been simplified/eliminated, and the main "Branches" view now has more information than it did before.

Get rid of this and just link to the main view.

Test Plan: Viewed "Branches" in UI, saw a link to the main view instead of a weird table.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13312

Differential Revision: https://secure.phabricator.com/D20584
2019-06-18 15:19:53 -07:00
epriestley
dcf3ca8e04 When a user clicks a navigation link in a dialog, close the dialog
Summary:
Ref T13302. Currently, if you enable Quicksand (by clicking "Persistent Chat"), open a dialog with links in it (like "Create Subtask" with multiple available subtypes), and then follow a navigation link, the page content reloads behind the dialog but the dialog stays in the foreground.

Fix this by closing dialogs when users click navigation links inside them.

Test Plan:
With Quicksand enabled and disabled, clicked a subtask type in the "Create Subtask" dialog.

  - Before, Quicksand Disabled: Dialog stays on screen, then navigation occurs.
  - After, Quicksand Disabled: Dialog vanishes, then navigation occurs.
  - Before, Quicksand Enabled: Dialog stays on screen, navigation occurs behind it.
  - After, Quicksand Enabled: Dialog vanishes, then navigation occurs.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13302

Differential Revision: https://secure.phabricator.com/D20573
2019-06-18 15:16:32 -07:00
epriestley
dda5c13ef5 Parse "shallow" frames in the Git "upload-pack" wire protocol parser
Summary:
Fixes T13309. If you void the warranty on a repository on disk and turn it into a shallow clone, Phabricator currently can't serve it.

We don't support hosting shallow working copies, but we should still parse and proxy the protocol rather than breaking in a mysterious way.

Test Plan:
  - Created a shallow working copy with `mv X X.full; git clone --depth Y file://.../X.full X` in the storage directory on disk.
  - Cloned it with `git clone <uri>`.
  - Deleted all the refs inside it so the wire only has "shallow" frames; cloned it.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13309

Differential Revision: https://secure.phabricator.com/D20577
2019-06-18 15:14:52 -07:00
epriestley
7538286499 Fix missing link targets for "View Object" header buttons in HTML email
Summary:
See <https://discourse.phabricator-community.org/t/view-task-from-maniphest-e-mail-doesnt-have-url/2827>.

I added "View Task" / "View Commit" buttons recently but the logic for generating URIs isn't quite right. Fix it up.

Test Plan:
  - Commented on a task.
  - Used `bin/mail show-outbound --id ... --dump-html > out.html` to dump the HTML.
  - Previewed the HTML in a browser.
  - This time, actually clicked the button to go to the task.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20586
2019-06-18 13:20:56 -07:00
epriestley
6219d30f6b Recommend dumping database backups with "--compress --output X" instead of "| gzip > X"
Summary:
Fixes T13304. Shell pipes and redirects do not have robust behavior when errors occur. We provide "--compress" and "--output" flags as robust alternatives, but do not currently recommend their use.

  - Recommend their use, since their error handling behavior is more robust in the face of issues like full disks.
  - If "--compress" is provided but won't work because the "zlib" extension is missing, raise an explicit error. I believe this extension is very common and this error should be rare. If that turns out to be untrue, we could take another look at this.
  - Also, verify some flag usage sooner so we can exit with an error faster if you mistype a "bin/storage dump" command.

Test Plan: Read documentation, hit affected error cases, did a dump and spot-checked that it came out sane looking.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13304

Differential Revision: https://secure.phabricator.com/D20572
2019-06-18 11:37:14 -07:00
epriestley
1dd62f79ce Fix more "msort()" vs "msortv()" callsites
Summary:
See <https://discourse.phabricator-community.org/t/unhandled-exception-when-logging-in-with-mfa/2828>. The recent changes to turn `msort()` on a vector an error have smoked out a few more of these mistakes.

These cases do not meaningfully rely on sort stability so there's no real bug being fixed, but we'd still prefer `msortv()`.

Test Plan: Viewed MFA and External Account settings panels. Did a `git grep 'msort(' | grep -i vector` for any more obvious callsites, but none turned up.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20587
2019-06-18 11:36:22 -07:00
epriestley
731b45d818 In "bin/repository reparse", continue on "PhabricatorWorkerPermanentFailureException"
Summary:
Fixes T13315. See that task for discussion.

Without `--background`, we currently treat this as a catastrophic failure, but it's relatively routine for some repository states. We can safely continue reparsing other steps.

Test Plan: Ran `bin/repository reparse --all X --message` with commits faked to all be unreachable. Got warnings instead of a hard failure on first problem.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13315

Differential Revision: https://secure.phabricator.com/D20588
2019-06-18 11:32:54 -07:00
epriestley
aba7c98bae Skip loading transaction handles in an old migration
Summary:
Ref T13305. See that task for discussion.

This old migration may indirectly cause search index worker tasks to queue by loading handles. They'll fail since we later added `dateCreated` to the worker task table.

Use `needHandles(false)` (since we don't use them) to disable loading handles and avoid the problem.

Test Plan:
  - Ran `bin/storage upgrade -f` on an older instance (late 2016) and hit this issue.
  - Applied the patch, got a clean migration to modernity.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13305

Differential Revision: https://secure.phabricator.com/D20570
2019-06-17 13:57:47 -07:00
epriestley
bab35f28e4 Respect repository identities when selecting author vs auditor actions
Summary:
Depends on D20580. Fixes T13311. When we choose which actions to show a user, we can either show them "auditor" actions (like "raise concern") or "author" actions (like "request verification").

Currently, we don't show "author" actions if you're the author of the commit via an identity mapping, but we should. Use identity mappings where they exist.

(Because I've implemented `getEffectiveAuthorPHID()` in a way that requires `$data` be attached, it's possible this will make something throw a "DataNotAttached" exception, but: probably it won't?; and that's easy to fix if it happens.)

Test Plan:
See D20580. As `@alice`, viewed the commit in the UI.

  - Before: got auditor actions presented to me.
  - After: got author actions.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13311

Differential Revision: https://secure.phabricator.com/D20581
2019-06-17 13:56:38 -07:00
epriestley
4450c90881 When triggering audits, respect committer identities when importing commits
Summary:
Ref T13311. We currently don't use committer identity mappings when triggering audits, so if a user is only associated with an identity via manual mapping we won't treat them as the author.

Instead, use the identity and manual mapping if they're available.

Test Plan:
  - Pushed a commit as `xyz <xyz@example.org>`, an address with no corresponding user.
  - In the UI, manually associated that identity with user `@alice`.
  - Ran `bin/repository reparse --publish <hash>` to trigger audits and publishing for the commit.
  - Before: observed the `$author_phid` was `null`.
  - After: observed the `$author_phid` is Alice.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13311

Differential Revision: https://secure.phabricator.com/D20580
2019-06-17 13:47:31 -07:00
epriestley
4af73a625f Don't require users be logged in to access the Logout controller, so users with no Spaces can log out
Summary:
Fixes T13310. Use cases in the form "users with no access to any spaces can not <do things>" are generally unsupported (that is, we consider this to mean that the install is misconfigured), but "log out" is a somewhat more reasonable sort of thing to do and easy to support.

Drop the requirement that users be logged in to access the Logout controller. This skips the check for access to any Spaces and allows users with no Spaces to log out.

For users who are already logged out, this just redirects home with no effect.

Test Plan:
  - As a user with access to no Spaces, logged out. (Before: error; after: worked).
  - As a logged-out user, logged out (was redirected).
  - As a normal user, logged out (normal logout).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13310

Differential Revision: https://secure.phabricator.com/D20578
2019-06-17 13:44:02 -07:00
epriestley
14b076578f In "Download Raw Diff", engage the chunk engine to handle 8MB+ changes
Summary:
Fixes T13313. The "Download Raw Diff" workflow in Differential currently uses an older way of interacting with Files that doesn't engage the chunk engine and can't handle 8MB+ files.

Update to `IteratorFileUploadSource` -- we're still passing in a single giant blob, but this approach can be chunked.

This will still break somewhere north of 8MB (it will break at 2GB with the PHP string limit if nowhere sooner, since we're putting the entire raw diff in `$raw_diff` rather than using a rope/stream) but will likely survive diffs in the hundreds-of-megabytes range for now.

Test Plan:
  - Added `str_repeat('x', 1024 * 1024 * 9)` to the `$raw_diff` to create a 9MB+ diff.
  - Configured file storage with no engine explicitly configured for >8MB chunks (i.e., "reasonably").
  - Clicked "Download Raw Diff".
  - Before: misleading file storage engine error ("no engine can store this file").
  - After: large, raw diff response.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13313

Differential Revision: https://secure.phabricator.com/D20579
2019-06-17 13:31:35 -07:00
epriestley
2bc045bab8 Fix another stray "msort()/msortv()" issue
Summary: Ref T13303. See B22967. This should be "msortv()" but didn't get updated properly.

Test Plan: The system works!

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13303

Differential Revision: https://secure.phabricator.com/D20585
2019-06-17 13:20:10 -07:00
epriestley
874282db75 Correct "msort()" vs "msortv()" to more fully stabilize transaction sorts after recent changes
Summary: Ref T13303. I upgraded this to a vector-based sort but forgot to type a "v", which means the sort has different stability under PHP 5.5. See D20582 for a root cause fix.

Test Plan: Locally, on PHP7, not much changes. I expect this to fix the odd selection of title stories in mail and notification stories on `secure`, which is running PHP 5.5.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13303

Differential Revision: https://secure.phabricator.com/D20583
2019-06-17 13:07:38 -07:00
epriestley
d3112392d1 Allow "Sign with MFA" to be applied as a comment action without requiring "CAN_EDIT"
Summary:
Fixes T13307. We currently require "CAN_EDIT" to sign actions, but it's fine to sign a comment with only "CAN_INTERACT".

Since the actions like "Accept Revision" already work like this, the fix is one line.

Test Plan: {F6488135}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13307

Differential Revision: https://secure.phabricator.com/D20574
2019-06-17 10:41:42 -07:00
epriestley
64a9500078 Add "bin/file migrate" options to support import of a local-disk backup for Phacility instances
Summary:
Ref T13306. Currently, there's no easy way to import a third-party local-disk file dump into a Phacility instance.

Add some more options to `bin/files migrate` to support this. In particular, this enables:

```
$ ./bin/files --from-engine local-disk --engine amazon-s3 --local-disk-source path/to/backup
```

...to import these files into S3 directly.

These are general-purpose options and theoretically useful in other use cases, although realistically those cases are probably very rare.

Test Plan: Used `bin/files` with the new options to move files in and out of local disk storage in an arbitrary backup directory. Got clean exports/imports.

Reviewers: amckinley

Maniphest Tasks: T13306

Differential Revision: https://secure.phabricator.com/D20571
2019-06-15 08:12:11 -07:00
epriestley
f1a588c771 Add a basic profiler to Herald transcripts
Summary:
Ref T13298. Add a simple profiler as a starting point to catch any egregiously expensive rules or conditions.

This doesn't profile rule actions, so if "Add subscriber" (or whatever) is outrageously expensive it won't show up on the profile. Right now, actions get evaluated inside the Adapter so they're hard to profile. A future change could likely dig them out without too much trouble. I generally expect actions to be less expensive than conditions.

This also can't pin down a //specific// condition being expensive, but if you see that `H123` takes 20s to evaluate you can probably guess that the giant complicated regex is the expensive part.

Test Plan: {F6473407}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13298

Differential Revision: https://secure.phabricator.com/D20566
2019-06-05 08:50:41 -07:00
epriestley
b7aacaa4d3 Differentiate Remarkup header sizes more clearly
Summary:
Ref PHI1275. Previously, see T591. See also T7963. Headers are currently very visually similar to one another, and similar to the text size:

{F6485441}

I think the design intent was to make it hard to make bad-looking documents, but all the headers end up being very samey.

Differentiate the sizes of the headers better so they're much more obvious (e.g., when scrolling through a document) and the different levels are more distinct.

This might be a little overboard, but we can always pull it back a bit if it's too much, and I think giving users more control in Remarkup (in cases where it doesn't create some weird syntax/parsing nightmare) is generally a good thing.

Test Plan: {F6485447}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20569
2019-06-04 16:03:32 -07:00
epriestley
d7890d08b8 Add "bin/herald rule ..." to modify Herald rules from the CLI
Summary:
Depends on D20566. Ref T13298. See PHI1280. Currently, there's no clean way to disable problematic personal rules. This comes up occasionally and sometimes isn't really the best approach to solving a problem, but is a generally reasonable capability to provide.

Allow Herald rules (including personal rules) to be disabled/enabled via `bin/herald rule ... --disable/--enable`.

Test Plan: Used the CLI to disable and enable a personal rule.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jmeador

Maniphest Tasks: T13298

Differential Revision: https://secure.phabricator.com/D20567
2019-06-04 07:12:15 -07:00
epriestley
760406762a When a revision is closed, always promote it out of draft
Summary:
Fixes T13300. Currently, if you create a revision and then immediately land it (either using `--draft` or just beating Harbormaster to the punch) it can be stuck in "Draft" forever.

Instead, count landing changes like this as a publishing action.

Test Plan:
  - Used `arc diff --hold` to create a revision, then pushed the commit immediately.
  - Before change: revision closed, but was stuck in draft.
  - After change: revision closed and was promoted out of draft.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13300

Differential Revision: https://secure.phabricator.com/D20565
2019-06-04 06:56:22 -07:00
epriestley
67f062b004 When triggering audits, count "Accepted" revisions as successfully reviewed
Summary:
See PHI1118. That issue may describe more than one bug, but the recent ordering changes to the import pipeline likely make this at least part of the problem.

Previously, commits would always close associated revisions before we made it to the "publish" step. This is no longer true, so we might be triggering audits on a commit before the associated revision actually closes.

Accommodate this by counting a revision in either "Accepted" or "Published (Was Previously Accepted)" as "reviewed".

Test Plan:
  - With commit C affecting paths in package P with "Audit Unreviewed Commits and Commits With No Owner Involvement", associated with revision R, with both R and C authored by the same user, and "R" in the state "Accepted", used `bin/repository reparse --publish <hash>` to republish the commit.
  - Before change: audit by package P triggered.
  - After change: audit by package P no longer triggered.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20564
2019-05-30 17:18:11 -07:00
epriestley
81134d7e7d After reloading transactions for the recipient while building transaction mail, put them in the input order
Summary:
Ref T13303. In D20525 I fixed an issue where transaction rendering could use cached values with the wrong viewer by reloading transactions.

However, reloading transactions may also reorder them as a side effect, since `withPHIDs(...)` does not imply an order. This can make transaction rendering order in mail wrong/inconsistent.

Instead, reorder the transactions before continuing so mail transaction order is consistent.

Test Plan: Applied a group of transactions to a task, saw a more consistent rendering order in mail after the change.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13303

Differential Revision: https://secure.phabricator.com/D20563
2019-05-30 17:14:54 -07:00
epriestley
9a32a563f0 Add a "View Task" button to HTML mail from Maniphest
Summary:
See downstream <https://phabricator.wikimedia.org/T1050>. Some time ago, we added a "View Revision" button to Differential mail. This hasn't created any problems and generally seems good / desirable.

It isn't trivial to just add everywhere since we need a translation string in each case, but at least add it to Maniphest for now. Going forward, we can fill in more applications as they come up.

Test Plan:
Used `bin/mail show-outbound --id <x> --dump-html`:

{F6470461}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20561
2019-05-30 15:24:22 -07:00
epriestley
fb5dec4c03 Require valid comments to contain at least one non-whitespace character
Summary:
See downstream <https://phabricator.wikimedia.org/T88655>. This is very marginal, but we currently allow comments consisting of //only// whitespace.

These are probably always mistakes, so treat them like completely empty comments.

(We intentionally do not trim leading or trailing whitespace from comments when posting them becuase leading spaces can be used to trigger codeblock formatting.)

Test Plan:
  - Posted empty, nonempty, and whitespace-only comments.
  - Whitespace-only comments now have the same behavior as truly empty comments (e.g., do not actually generate a transaction).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20562
2019-05-30 15:15:24 -07:00
epriestley
8747204807 Clean up "phabricator.timezone" configuration instructions a little bit
Summary: These instructions are fairly old and can be a little fancier and more clear in the context of modern Phabricator. Drop the reference to "HPHP", link the actual timezone list, wordsmith a little.

Test Plan: d( O_o )b

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20560
2019-05-30 15:08:11 -07:00