1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-05 21:26:14 +01:00
Commit graph

13360 commits

Author SHA1 Message Date
epriestley
b38449ce8f Implement an "Author's packages" Herald field for Differential
Summary: Ref T13480. Add an "Author's packages" field to Herald to support writing rules like "if affected packages include X, and author's packages do not include X, raise the alarm".

Test Plan: Wrote and executed rules with the field, saw a sensible field value in the transcript.

Maniphest Tasks: T13480

Differential Revision: https://secure.phabricator.com/D20947
2020-01-22 18:27:51 -08:00
epriestley
6c4500046f Add "Project tags added" and "Project tags removed" fields in Herald
Summary: Ref T13480. These fields don't serve a specific strong use case, but are broadly reasonable capabilities after "state" vs "change" actions were relaxed by T13283.

Test Plan: Wrote rules using the new fields, added and removed projects (and did neither) to fire them / not fire them. Inspected the transcripts to see the project PHIDs making it to the field values.

Maniphest Tasks: T13480

Differential Revision: https://secure.phabricator.com/D20946
2020-01-22 18:20:57 -08:00
epriestley
6ccb6a6463 Update "git rev-parse" invocation to work in Git 2.25.0
Summary:
Fixes T13479. The behavior of "git rev-parse --show-toplevel" has changed in Git 2.25.0, and it now fails in bare repositories.

Instead, use "git rev-parse --git-dir" to sanity-check the working copy. This appears to have more stable behavior across Git versions, although it's a little more complicated for our purposes.

Test Plan:
  - Ran `bin/repository update ...` on an observed, bare repository.
  - ...on an observed, non-bare ("legacy") repository.
  - ...on a hosted, bare repository.

Maniphest Tasks: T13479

Differential Revision: https://secure.phabricator.com/D20945
2020-01-16 11:39:23 -08:00
epriestley
f806528983 Allow the Herald Rule Editor to apply generic "Edge" transactions
Summary: Fixes T13469. Currently, "Mute" applies a generic edge transaction but the editor doesn't whitelist them.

Test Plan: Muted a Herald rule.

Maniphest Tasks: T13469

Differential Revision: https://secure.phabricator.com/D20943
2020-01-15 08:29:46 -08:00
epriestley
767528c0ed Move search query parser/compiler classes to Phabricator
Summary: Ref T13472. Ref T13395. These classes are only used by Phabricator and not likely to find much use in Arcanist.

Test Plan: Grepped libphutil and Arcanist for removed symbols.

Maniphest Tasks: T13472, T13395

Differential Revision: https://secure.phabricator.com/D20939
2020-01-14 11:49:49 -08:00
epriestley
54bcbdaba9 Fix an XSS issue with certain high-priority remarkup rules embedded inside lower-priority link rules
Summary:
See <https://hackerone.com/reports/758002>. The link rules don't test that their parameters are flat text before using them in unsafe contexts.

Since almost all rules are lower-priority than these link rules, this behavior isn't obvious. However, two rules have broadly higher priority (monospaced text, and one variation of link rules has higher priority than the other), and the latter can be used to perform an XSS attack with input in the general form `()[ [[ ... | ... ]] ]` so that the inner link rule is evaluated first, then the outer link rule uses non-flat text in an unsafe way.

Test Plan:
Tested examples in HackerOne report. A simple example of broken (but not unsafe) behavior is:

```
[[ `x` | `y` ]]
```

Differential Revision: https://secure.phabricator.com/D20937
2019-12-13 10:37:50 -08:00
Arturas Moskvinas
4cd333b33f Use same method to get object URI as used in DifferentialTransactionEditor and PhabricatorApplicationTransactionEditor
Summary: Maniphest object has `getURI` method, let's use it

Test Plan: Create event in task - URI generated as expected in email notification

Reviewers: epriestley, Pawka, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20935
2019-12-10 17:37:30 +02:00
epriestley
33c534f9b7 Extend Config to full-width
Summary:
Ref T13362. Some applications moved to fixed-width a while ago but I was generally unsatisfied with where they ended up and have been pushing them back to full-width.

Push Config back to full-width. Some of the subpages end up a little weird, but this provides more space to work with to make some improvements, like makign `maniphest.statuses` more legible in the UI>

Test Plan: Grepped for `setFixed(`, updated each page in `/config/`. Browsed each controller, saw workable full-width UIs.

Maniphest Tasks: T13362

Differential Revision: https://secure.phabricator.com/D20925
2019-11-25 16:18:41 -08:00
epriestley
1667acfa5d Implement "PolicyInterface" on "UserEmail" so "EmailQuery" can load them properly
Summary:
See PHI1558. Ref T11860. Ref T13444. I partly implemented PHIDs for "UserEmail" objects, but they can't load on their own so you can't directly `bin/remove destroy` them yet.

Allow them to actually load by implementing "PolicyInterface".

Addresses are viewable and editable by the associated user, unless they are a bot/list address, in which case they are viewable and editable by administrators (in preparation for T11860). This has no real impact on anything today.

Test Plan:
  - Used `bin/remove destroy <phid>` to destroy an individual email address.
  - Before: error while loading the object by PHID in the query policy layer.
  - After: clean load and destroy.

Maniphest Tasks: T13444, T11860

Differential Revision: https://secure.phabricator.com/D20927
2019-11-25 15:08:13 -08:00
epriestley
eb6df7a209 Remove "phlog()" of exeptions during Conduit calls
Summary: Fixes T13465. This "phlog()" made some degree of sense at one time, but is no longer useful or consistent. Get rid of it. See T13465 for discussion.

Test Plan: Made a conduit call that hit a policy error, no longer saw error in log.

Maniphest Tasks: T13465

Differential Revision: https://secure.phabricator.com/D20924
2019-11-22 12:03:08 -08:00
epriestley
2abf292821 Fix an issue where editing paths in Owners packages could raise an error: undefined index "display"
Summary:
Fixes T13464. Fully-realized paths have a "path" (normalized, effective path) and a "display" path (user-facing, un-normalized path).

During transaction validation we build ref keys for paths before we normalize the "display" values. A few different approaches could be taken to resolve this, but just default the "display" path to the raw "path" if it isn't present since that seems simplest.

Test Plan: Edited paths in an Owners package, no longer saw a warning in the logs.

Maniphest Tasks: T13464

Differential Revision: https://secure.phabricator.com/D20923
2019-11-21 11:29:04 -08:00
epriestley
374f8b10b3 Add a "--dry-run" flag to "bin/repository rebuild-identities"
Summary: Ref T13444. Allow the effects of performing an identity rebuild to be previewed without committing to any changes.

Test Plan: Ran "bin/repository rebuild-identities --all-identities" with and without "--dry-run".

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20922
2019-11-19 12:38:20 -08:00
epriestley
63d84e0b44 Improve use of keys when iterating over commits in "bin/audit delete" and "bin/repository rebuild-identities"
Summary:
Fixes T13457. Ref T13444. When we iterate over commits in a particular repository, the default iteration strategy can't effectively use the keys on the table.

Tweak the ordering so the "<repositoryID, epoch, [id]>" key can be used.

Test Plan:
  - Ran `bin/audit delete --repository X` and `bin/repository rebuild-identities --repository X` before and after changes.
    - With just the key changes, performance was slightly better. My local data isn't large enough to really emphasize the key changes.
    - With the page size changes, performance was a bit better (~30%, but on 1-3 second run durations).
  - Used `--trace` and ran `EXPLAIN ...` on the new queries, saw them select the "<repositoryID, epoch, [id]>" key and report a bare "Using index condition" in the "Extra" column.

Maniphest Tasks: T13457, T13444

Differential Revision: https://secure.phabricator.com/D20921
2019-11-19 10:18:55 -08:00
epriestley
a7aca500bc Update repository identities after all mutations to users and email addresses
Summary:
Ref T13444. Currently, many mutations to users and email addresses (particularly: user creation; and user and address destruction) do not propagate properly to repository identities.

Add hooks to all mutation workflows so repository identities get rebuilt properly when users are created, email addresses are removed, users or email addresses are destroyed, or email addresses are reassigned.

Test Plan:
- Added random email address to account, removed it.
- Added unassociated email address to account, saw identity update (and associate).
  - Removed it, saw identity update (and disassociate).
- Registered an account with an unassociated email address, saw identity update (and associate).
  - Destroyed the account, saw identity update (and disassociate).
- Added address X to account A, unverified.
  - Invited address X.
  - Clicked invite link as account B.
  - Confirmed desire to steal address.
  - Saw identity update and reassociate.

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20914
2019-11-19 09:41:59 -08:00
epriestley
89dcf9792a Give "PhabricatorUserEmail" a PHID
Summary:
Ref T13444. To interact meaningfully with "DestructionEngine", objects need a PHID. The "UserEmail" object currently does not have one (or a real "Query").

Provide basic PHID support so "DestructionEngine" can interact with the object more powerfully.

Test Plan:
  - Ran migrations, checked data in database, saw sensible PHIDs assigned.
  - Added a new email address to my account, saw it get a PHID.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20913
2019-11-19 09:41:29 -08:00
epriestley
d69a7360ea Use DestructionEngine to destroy UserEmail objects
Summary: Ref T13444. Prepare to hook identity updates when user email addreses are destroyed.

Test Plan:
  - Destroyed a user with `bin/remove destroy ... --trace`, saw email deleted.
  - Destroyed an email from the web UI, saw email deleted.

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20912
2019-11-19 09:40:58 -08:00
epriestley
18da346972 Add additional flags to "bin/repository rebuild-identities" to improve flexibility
Summary:
Ref T13444. Repository identities have, at a minimum, some bugs where they do not update relationships properly after many types of email address changes.

It is currently very difficult to fix this once the damage is done since there's no good way to inspect or rebuild them.

Take some steps toward improving observability and providing repair tools: allow `bin/repository rebuild-identities` to effect more repairs and operate on identities more surgically.

Test Plan: Ran `bin/repository rebuild-identities` with all new flags, saw what looked like reasonable rebuilds occur.

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20911
2019-11-19 09:39:48 -08:00
epriestley
0014d0404c Consolidate repository identity resolution and detection code
Summary: Ref T13444. Send all repository identity/detection through a new "DiffusionRepositoryIdentityEngine" which handles resolution and detection updates in one place.

Test Plan:
  - Ran `bin/repository reparse --message ...`, saw author/committer identity updates.
  - Added "goose@example.com" to my email addresses, ran daemons, saw the identity relationship get picked up.
  - Ran `bin/repository rebuild-identities ...`, saw sensible rebuilds.

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20910
2019-11-19 09:39:11 -08:00
epriestley
6afbb6102d Remove "PhabricatorEventType::TYPE_DIFFUSION_LOOKUPUSER" event
Summary: Ref T13444. This is an ancient event and part of the old event system. It is not likely to be in use anymore, and repository identities should generally replace it nowadays anyway.

Test Plan: Grepped for constant and related methods, no longer found any hits.

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20909
2019-11-19 09:38:03 -08:00
epriestley
a2b2c391a1 Distinguish between "Assigned" and "Effective" identity PHIDs more clearly and consistently
Summary:
Ref T13444. You can currently explicitly unassign an identity (useful if the matching algorithm is misfiring). However, this populates the "currentEffectiveUserPHID" with the "unassigned()" token, which mostly makes things more difficult.

When an identity is explicitly unassigned, convert that into an explicit `null` in the effective user PHID.

Then, realign "assigned" / "effective" language a bit. Previously, `withAssigneePHIDs(...)` actualy queried effective users, which was misleading. Finally, bulk up the list view a little bit to make testing slightly easier.

Test Plan:
  - Unassigned an identity, ran migration, saw `currentEffectiveUserPHID` become `NULL` for the identity.
  - Unassigned a fresh identity, saw NULL.
  - Queried for various identities under the modified constraints.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20908
2019-11-19 09:37:44 -08:00
epriestley
df0f5c6cee Make repository identity email address association case-insensitive
Summary:
Ref T13444. Currently, identities for a particular email address are queried with "LIKE" against a binary column, which makes the query case-sensitive.

  - Extract the email address into a separate "sort255" column.
  - Add a key for it.
  - Make the query a standard "IN (%Ls)" query.
  - Deal with weird cases where an email address is 10000 bytes long or full of binary junk.

Test Plan:
  - Ran migration, inspected database for general sanity.
  - Ran query script in T13444, saw it return the same hits for "git@" and "GIT@".

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20907
2019-11-19 09:37:26 -08:00
epriestley
d58eddcf0a When predicting project membership during edits, predict milestones will have parent membership
Summary:
Depends on D20919. Ref T13462. When editing milestones, we currently predict they will have no members for policy evaluation purposes. This isn't the right rule.

Instead, predict that their membership will be the same as the parent project's membership, and pass this hint to the policy layer.

See T13462 for additional context and discussion.

Test Plan:
  - Set project A's edit policy to "Project Members".
  - Joined project A.
  - Tried to create a milestone of project A.
    - Before: policy exception that the edit policy excludes me.
    - After: clean milestone creation.
  - As a non-member, tried to create a milestone. Received appropriate policy error.

Maniphest Tasks: T13462

Differential Revision: https://secure.phabricator.com/D20920
2019-11-18 23:07:05 -08:00
epriestley
959504a488 When predicting object policies for project milestones, adjust objects so they behave like milestones
Summary:
Ref T13462. Currently, when testing milestone edit policies during creation, the project object does not behave like a milestone:

  - it doesn't have a milestone number yet, so it doesn't try to access the parent project; and
  - the parent project isn't attached yet.

Instead: attach the parent project sooner (which "should" be harmless, although it's possible this has weird side effects); and give the adjusted policy object a dummy milestone number if it doesn't have one yet. This forces it to act like a milestone when emitting policies.

Test Plan:
  - Set "Projects" application default edit policy to "No One".
  - Created a milestone I had permission to create.
    - Before: failed with a policy error, because the project behaved like a non-milestone and returned "No One" as the effective edit policy.
    - After: worked properly, correctly evaluting the parent project edit policy as the effective edit policy.
  - Tried to create a milestone I did not have permission to create (no edit permission on parent project).
    - Got an appropriate edit policy error.

Maniphest Tasks: T13462

Differential Revision: https://secure.phabricator.com/D20919
2019-11-18 23:06:03 -08:00
epriestley
de66a8ece1 Remove "stronger/weaker" policy color hints from object headers
Summary:
Fixes T13461. Some applications provide hints about policy strength in the header, but these hints are inconsistent and somewhat confusing. They don't make much sense for modern objects with Custom Forms, which don't have a single "default" policy.

Remove this feature since it seems to be confusing things more than illuminating them.

Test Plan:
  - Viewed various objects, no longer saw colored policy hints.
  - Grepped for all removed symbols.

Maniphest Tasks: T13461

Differential Revision: https://secure.phabricator.com/D20918
2019-11-18 22:05:26 -08:00
epriestley
1996b0cd55 Update the "owner can always view/edit" policy exception rule
Summary: Fixes T13460. This rule vanished from the UI in D20165; update things so it returns to the UI.

Test Plan: {F7035134}

Maniphest Tasks: T13460

Differential Revision: https://secure.phabricator.com/D20917
2019-11-18 21:54:45 -08:00
epriestley
e86aae99de Surface edits to "Text" panels on dashboards as remarkup edits
Summary: Fixes T13456. These edits are remarkup edits and should attach files, trigger mentions, and so on.

Test Plan: Created a text panel, dropped a file in. After changes, saw the file attach properly.

Maniphest Tasks: T13456

Differential Revision: https://secure.phabricator.com/D20906
2019-11-13 10:35:09 -08:00
epriestley
2adc36ba0b Correctly identify more SSH private key problems as "formatting" or "passphrase" related
Summary:
Ref T13454. Fixes T13006. When a user provide us with an SSH private key and (possibly) a passphrase:

  # Try to verify that they're correct by extracting the public key.
  # If that fails, try to figure out why it didn't work.

Our success in step (2) will vary depending on what the problem is, and we may end up falling through to a very generic error, but the outcome should generally be better than the old approach.

Previously, we had a very unsophisticated test for the text "ENCRYPTED" in the key body and questionable handling of the results: for example, providing a passphrase when a key did not require one did not raise an error.

Test Plan:
Created and edited credentials with:

  - Valid, passphrase-free keys.
  - Valid, passphrased keys with the right passphrase.
  - Valid, passphrase-free keys with a passphrase ("surplus passphrase" error).
  - Valid, passphrased keys with no passphrase ("missing passphrase" error).
  - Valid, passphrased keys with an invalid passphrase ("invalid passphrase" error).
  - Invalid keys ("format" error).

The precision of these errors will vary depending on how helpful "ssh-keygen" is.

Maniphest Tasks: T13454, T13006

Differential Revision: https://secure.phabricator.com/D20905
2019-11-13 10:22:00 -08:00
epriestley
72f82abe07 Improve recovery from panel action rendering exceptions, and mark "Changeset" queries as not suitable for panel generation
Summary:
Fixes T13443. When a panel raises an exception during edit action generation, it currently escapes to top level. Instead, catch it more narrowly.

Additionally, mark "ChangesetSearchEngine" as not being a suitable search engine for use in query panels. There's no list view or search URI so it can't generate a sensible panel.

Test Plan:
  - Added a changeset panel to a dashboard.
  - Before: entire dashboard fataled.
  - After: panel fataled narrowly, menu fatals narrowly, dashboard no longer permits creation of another Changeset query panel.

Maniphest Tasks: T13443

Differential Revision: https://secure.phabricator.com/D20902
2019-11-08 17:15:21 -08:00
epriestley
a3f4cbd748 Correct rendering of workboard column move stories when a single transaction performs moves on multiple boards
Summary:
See <https://discourse.phabricator-community.org/t/unhandled-exception-rendering-maniphest-task/3234>.

If a single transaction performs column moves on multiple different boards (which is permitted in the API), the rendering logic currently fails. Make it render properly.

Test Plan: {F7011464}

Differential Revision: https://secure.phabricator.com/D20901
2019-11-08 16:57:35 -08:00
epriestley
b83b3224bb Add an "Advanced/Developer..." action item for viewing object handle details and hovercards
Summary:
Ref T13442. Ref T13157. There's a secret URI to look at an object's hovercard in a standalone view, but it's hard to remember and impossible to discover.

In developer mode, add an action to "View Hovercard". Also add "View Handle", which primarily shows the object PHID.

Test Plan: Viewed some objects, saw "Advanced/Developer...". Used "View Hovercard" to view hovercards and "View Handle" to view handles.

Maniphest Tasks: T13442, T13157

Differential Revision: https://secure.phabricator.com/D20887
2019-11-08 16:47:05 -08:00
epriestley
cd60a8aa56 Update various Asana odds-and-ends for "gid" API changes
Summary:
Ref T13453. Some of the Asana integrations also need API updates.

Depends on D20899.

Test Plan:
  - Viewed "asana.workspace-id" in Config, got a sensible GID list.
  - Created a revision, saw the associated Asana task get assigned.
  - Pasted an Asana link I could view into a revision description, saw it Doorkeeper in the metadata.

Maniphest Tasks: T13453

Differential Revision: https://secure.phabricator.com/D20900
2019-11-08 09:07:59 -08:00
epriestley
2223d6b914 Update Asana Auth adapter for "gid" API changes
Summary: Ref T13453. The Asana API has changed, replacing all "id" fields with "gid", including the "users/me" API call result.

Test Plan: Linked an Asana account. Before: error about missing 'id'. After: clean link.

Maniphest Tasks: T13453

Differential Revision: https://secure.phabricator.com/D20899
2019-11-08 09:02:15 -08:00
epriestley
d4491ddc22 Fix an issue with 1up diff block rendering for added or removed blocks
Summary: Ref T13425. When a change adds or removes a block (vs adding or removing a document, or changing a block), we currently try to render `null` as cell content in the unified view. Make this check broader to catch both "no opposite document" and "no opposite cell".

Test Plan: {F7008772}

Subscribers: artms

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20897
2019-11-08 07:37:06 -08:00
epriestley
502ca4767e When "Fetch Refs" are configured for a repository, highlight the "Branches" menu item in the Diffusion Management UI
Summary: Ref T13448. Minor UI issue: setting a "Fetch Refs" value does not highlight the associated menu item, but should.

Test Plan: Set only "Fetch Refs", now saw menu item highlighted.

Maniphest Tasks: T13448

Differential Revision: https://secure.phabricator.com/D20895
2019-11-07 16:28:33 -08:00
epriestley
1b40f7e540 Always initialize Git repositories with "git init", never with "git clone"
Summary:
Fixes T13448. We currently "git clone" to initialize repositories, but this will fetch too many refs if "Fetch Refs" is configured.

In modern Phabricator, there's no apparent reason to "git clone"; we can just "git init" instead. This workflow naturally falls through to an update, where we'll do a "git fetch" and pull in exactly the refs we want.

Test Plan:
  - Configured an observed repository with "Fetch Refs".
  - Destroyed the working copy.
  - Ran "bin/repository pull X --trace --verbose".
    - Before: saw "git clone" pull in the world.
    - After: saw "git init" create a bare empty working copy, then "git fetch" fill it surgically.

Both flows end up in the same place, this one is just simpler and does less work.

Maniphest Tasks: T13448

Differential Revision: https://secure.phabricator.com/D20894
2019-11-07 16:17:35 -08:00
epriestley
8dd57a1ed2 When fetching Git repositories, pass "--no-tags" to make explicit "Fetch Refs" operate more narrowly
Summary:
Ref T13448. The default behavior of "git fetch '+refs/heads/master:refs/heads/master'" is to follow and fetch associated tags.

We don't want this when we pass a narrow refspec because of "Fetch Refs" configuration. Tell Git that we only want the refs we explicitly passed.

Note that this doesn't prevent us from fetching tags (if the refspec specifies them), it just stops us from fetching extra tags that aren't part of the refspec.

Test Plan:
  - Ran "bin/repository pull X --trace --verbose" in a repository with a "Fetch Refs" configuration, saw only "master" get fetched (previously: "master" and reachable tags).
  - Ran "git fetch --no-tags '+refs/*:refs/*'", saw tags fetched normally ("--no-tags" does not disable fetching tags).

Maniphest Tasks: T13448

Differential Revision: https://secure.phabricator.com/D20893
2019-11-07 16:17:08 -08:00
epriestley
9dbde24dda Manually prune Git working copy refs instead of using "--prune", to improve "Fetch Refs" behavior
Summary:
Ref T13448. When "Fetch Refs" is configured:

  - We switch to a narrow mode when running "ls-remote" against the local working copy. This excludes surplus refs, so we'll incorrectly detect that the local and remote working copies are identical in cases where the local working copy really has surplus refs.
  - We rely on "--prune" to remove surplus local refs, but it only prunes refs matching the refspecs we pass "git fetch". Since these refspecs are very narrow under "Fetch Only", the pruning behavior is also very narrow.

Instead:

  - When listing local refs, always list everything. If we have too much stuff locally, we want to get rid of it.
  - When we identify surplus local refs, explicitly delete them instead of relying on "--prune". We can just do this in all cases so we don't have separate "--prune" and "manual" cases.

Test Plan:
  - Created a new repository, observed from a GitHub repository, with many tags/refs/branches. Pulled it.
  - Observed lots of refs in `git for-each-ref`.
  - Changed "Fetch Refs" to "refs/heads/master".
  - Ran `bin/repository pull X --trace --verbose`.

On deciding to do something:

  - Before: since "master" did not change, the pull declined to act.
  - After: the pull detected surplus refs and deleted them. Since the states then matched, it declined further action.

On pruning:

  - Before: if the pull was forced to act, it ran "fetch --prune" with a narrow refspec, which did not prune the working copy.
  - After: saw working copy pruned explicitly with "update-ref -d" commands.

Also, set "Fetch Refs" back to the default (empty) and pulled, saw everything pull.

Maniphest Tasks: T13448

Differential Revision: https://secure.phabricator.com/D20892
2019-11-07 16:15:46 -08:00
epriestley
f5f2a0bc56 Allow username changes which modify letter case to go through as valid
Summary:
Fixes T13446. Currently, the validation logic here rejects a rename like "alice" to "ALICE" (which changes only letter case) but this is a permissible rename.

Allow collisions that collide with the same user to permit this rename.

Also, fix an issue where an empty rename was treated improperly.

Test Plan:
  - Renamed "alice" to "ALICE".
    - Before: username collision error.
    - After: clean rename.
  - Renamed "alice" to "orange" (an existing user). Got an error.
  - Renamed "alice" to "", "!@#$", etc (invalid usernames). Got sensible errors.

Maniphest Tasks: T13446

Differential Revision: https://secure.phabricator.com/D20890
2019-11-06 08:10:36 -08:00
epriestley
6bada7db4c Change the Herald "do not include [any of]" condition label to "include none of"
Summary: Fixes T13445. Make the meaning of this condition more clear, since the current wording is ambiguous between "any of" and "all of".

Test Plan: Edited a Herald rule with a PHID list field ("Project tags"), saw text label say "include none of".

Maniphest Tasks: T13445

Differential Revision: https://secure.phabricator.com/D20889
2019-11-06 07:55:53 -08:00
epriestley
be2b8f4bcb Support querying projects by "Root Projects" in the UI, and "min/max depth" in the API
Summary:
Fixes T13441. Internally, projects can be queried by depth, but this is not exposed in the UI.

Add a "Is root project?" contraint in the UI, and "minDepth" / "maxDepth" constraints to the API.

Test Plan:
  - Used the UI to query root projects, got only root projects back.
  - Used "project.search" in the API to query combinations of root projects and projects at particular depths, got matching results.

Maniphest Tasks: T13441

Differential Revision: https://secure.phabricator.com/D20886
2019-10-31 12:56:33 -07:00
epriestley
e46e383bf2 Clean up "Revisions/Commits" table in Maniphest slightly
Summary: Ref T13440. Give the table more obvious visual structure and get rid of the largely useless header columns.

Test Plan: Viewed table, saw a slightly cleaner result.

Maniphest Tasks: T13440

Differential Revision: https://secure.phabricator.com/D20885
2019-10-31 12:29:53 -07:00
epriestley
c48f300eb1 Add support for rendering section dividers in tables; use section dividers for changes on tasks
Summary: Depends on D20883. Ref T13440. In most cases, all changes belong to the same repository, which makes the "Repository" column redundant and visually noisy. Show repository information in a section header.

Test Plan: {F6989932}

Maniphest Tasks: T13440

Differential Revision: https://secure.phabricator.com/D20884
2019-10-31 12:13:25 -07:00
epriestley
7bdfe5b46a Show commits and revisions on tasks in a tabular view instead of handle lists
Summary: Depends on D20882. Ref T13440. Instead of lists of "Differential Revisions" and "Commits", show all changes related to the task in a tabular view.

Test Plan: {F6989816}

Maniphest Tasks: T13440

Differential Revision: https://secure.phabricator.com/D20883
2019-10-31 12:10:09 -07:00
epriestley
b0d9f89c95 Remove "State Icons" from handles
Summary: Ref T13440. This feature is used in only one interface which I'm about to rewrite, so throw it away.

Test Plan: Grepped for all affected symbols, didn't find any hits anywhere.

Maniphest Tasks: T13440

Differential Revision: https://secure.phabricator.com/D20882
2019-10-31 12:04:43 -07:00
epriestley
97bed35085 Show repository information (and use repository identities) in commit hovercards
Summary:
Ref T12164. Ref T13439. Commit hovercards don't currently show the repository. Although this is sometimes obvious from context, it isn't at other times and it's clearly useful/important.

Also, use identities to render author/committer information and show committer if the committer differs from the author.

Test Plan: {F6989595}

Maniphest Tasks: T13439, T12164

Differential Revision: https://secure.phabricator.com/D20881
2019-10-31 09:58:20 -07:00
Arturas Moskvinas
bcf15abcd3 Return empty data if fact dimension is missing, not yet available
Summary:
On fresh installation which doesn't have yet any task closed you will not be able to open charts because of error below:
Fixes:
```
[Mon Oct 21 15:42:41 2019] [2019-10-21 15:42:41] EXCEPTION: (TypeError) Argument 1 passed to head_key() must be of the type array, null given, called in ..phabricator/src/applications/fact/chart/PhabricatorFactChartFunction.php on line 86 at [<phutil>/src/utils/utils.php:832]
[Mon Oct 21 15:42:41 2019]   #0 phlog(TypeError) called at [<phabricator>/src/aphront/handler/PhabricatorAjaxRequestExceptionHandler.php:27]
[Mon Oct 21 15:42:41 2019]   #1 PhabricatorAjaxRequestExceptionHandler::handleRequestThrowable(AphrontRequest, TypeError) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:797]
[Mon Oct 21 15:42:41 2019]   #2 AphrontApplicationConfiguration::handleThrowable(TypeError) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:345]
[Mon Oct 21 15:42:41 2019]   #3 AphrontApplicationConfiguration::processRequest(AphrontRequest, PhutilDeferredLog, AphrontPHPHTTPSink, MultimeterControl) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:214]
[Mon Oct 21 15:42:41 2019]   #4 AphrontApplicationConfiguration::runHTTPRequest(AphrontPHPHTTPSink) called at [<phabricator>/webroot/index.php:35
```

To fix issue - lets return empty data set instead

Test Plan:
1) Create fresh phabricator installation
2) Create fresh project
3) Try viewing charts

Reviewers: epriestley, Pawka, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, yelirekim

Differential Revision: https://secure.phabricator.com/D20861
2019-10-30 16:34:22 +02:00
epriestley
9d8cdce8e1 Make the top-level burndown chart in "Maniphest > Reports" show open tasks, not total tasks
Summary:
Ref T13279. See PHI1491. Currently, the top-level "Burnup Rate" chart in Maniphest shows total created tasks above the X-axis, without adjusting for closures.

This is unintended and not very useful. The filtered-by-project charts show the right value (cumulative open tasks, i.e. open minus close). Change the value to aggregate creation events and status change events.

Test Plan: Viewed top-level chart, saw the value no longer monotonically increasing.

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20879
2019-10-29 13:48:43 -07:00
epriestley
114166dd32 Roughly implement "harbormaster.artifact.search"
Summary: Ref T13438. This is a sort of minimal plausible implementation.

Test Plan: Used "harbormaster.artifact.search" to query information about artifacts.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13438

Differential Revision: https://secure.phabricator.com/D20878
2019-10-29 13:37:35 -07:00
epriestley
5d8457a07e In the repository URI index, store Phabricator's own URIs as tokens
Summary:
Fixes T13435. If you move Phabricator or copy data from one environment to another, the repository URI index currently still references the old URI, since it writes the URI as a plain string. This may make "arc which" and similar workflows have difficulty identifying repositories.

Instead, store the "phabricator.base-uri" domain and the "diffusion.ssh-host" domain as tokens, so lookups continue to work correctly even after these values change.

Test Plan:
  - Added unit tests to cover the normalization.
  - Ran migration, ran daemons, inspected `repository_uriindex` table, saw a mixture of sensible tokens (for local domains) and static domains (like "github.com").
  - Ran this thing:

```
$ echo '{"remoteURIs": ["ssh://git@local.phacility.com/diffusion/P"]}' | ./bin/conduit call --method repository.query --trace --input -
Reading input from stdin...
>>> [2] (+0) <conduit> repository.query()
>>> [3] (+3) <connect> local_repository
<<< [3] (+3) <connect> 555 us
>>> [4] (+5) <query> SELECT `r`.* FROM `repository` `r` LEFT JOIN `local_repository`.`repository_uriindex` uri ON r.phid = uri.repositoryPHID WHERE (uri.repositoryURI IN ('<base-uri>/diffusion/P')) GROUP BY `r`.phid ORDER BY `r`.`id` DESC LIMIT 101
<<< [4] (+5) <query> 596 us
<<< [2] (+6) <conduit> 6,108 us
{
  "result": [
    {
      "id": "1",
      "name": "Phabricator",
      "phid": "PHID-REPO-2psrynlauicce7d3q7g2",
      "callsign": "P",
      "monogram": "rP",
      "vcs": "git",
      "uri": "http://local.phacility.com/source/phabricator/",
      "remoteURI": "https://github.com/phacility/phabricator.git",
      "description": "asdf",
      "isActive": true,
      "isHosted": false,
      "isImporting": false,
      "encoding": "UTF-8",
      "staging": {
        "supported": true,
        "prefix": "phabricator",
        "uri": null
      }
    }
  ]
}
```

Note the `WHERE` clause in the query normalizes the URI into "<base-uri>", and the lookup succeeds.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13435

Differential Revision: https://secure.phabricator.com/D20872
2019-10-28 14:44:06 -07:00
epriestley
292f8fc612 Fix an issue where added or removed source files could incorrectly select a DocumentEngine
Summary:
Ref T13425. The changes in D20865 could incorrectly lead to selection of a DocumentEngine that can not generate document diffs if a file was added or removed (for example, when a source file is added).

Move the engine pruning code to be shared -- we should always discard engines which can't generate a diff, even if we don't have both documents.

Test Plan: Viewed an added source file, no more document ref error arising from document engine selection.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20866
2019-10-26 12:16:49 -07:00
epriestley
8ff0e3ab35 Support rich diff rendering with DocumentEngine for added/removed files
Summary: Ref T13425. When a file (like a Jupyter notebook) is added or removed, we can still render a useful line-by-line diff.

Test Plan:
  - Viewed add/modify/remove of Jupyter, source code, and images in 2up/1up mode, everything looked okay.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20865
2019-10-26 08:21:46 -07:00
epriestley
38694578e1 Improve project member list behaviors related to disabled users
Summary:
Fixes T13431.

Increase the "panel" version of project member lists to 10 users, hide disabled users, swap the buttons to "tail buttons".

Sort disabled users to the bottom of "full list" versions of member lists.

For UI consistency, render the remove "X" as disabled but visible if users don't have permission to remove members.

Test Plan:
  - Viewed a project with disabled members.
    - Saw only enabled members on the main project page.
    - Saw disabled members sorted to the bottom on the members page.
  - Clicked "View All" to jump from the panel to the members page.
  - As a user who could not edit a project, viewed the members page and saw a disabled "X" with a policy error when clicked.
    - Removed a member as before, as a normal user with permission to remove members.

Maniphest Tasks: T13431

Differential Revision: https://secure.phabricator.com/D20864
2019-10-24 18:41:33 -07:00
epriestley
633aa5288c Persist login instructions onto flow-specific login pages (username/password and LDAP)
Summary:
Fixes T13433. Currently, "Login Screen Instructions" in "Auth" are shown only on the main login screen. If you enter a bad password or bad LDAP credential set and move to the flow-specific login failure screen (for example, "invalid password"), the instructions vanish.

Instead, persist them. There are reasonable cases where this is highly useful and the cases which spring to mind where this is possibly misleading are fairly easy to fix by making the instructions more specific.

Test Plan:
  - Configured login instructions in "Auth".
  - Viewed main login screen, saw instructions.
  - Entered a bad username/password and a bad LDAP credential set, got kicked to workflow sub-pages and still saw instructions (previously: no instructions).
  - Grepped for other callers to `buildProviderPageResponse()` to look for anything weird, came up empty.

Maniphest Tasks: T13433

Differential Revision: https://secure.phabricator.com/D20863
2019-10-24 18:38:15 -07:00
epriestley
5b36f0c97a Add default branch, description, and metrics (commit count, recent commit) to "diffusion.repository.search"
Summary: Fixes T13430. Provide more information about repositories in "diffusion.repository.search".

Test Plan: Used API console to call method (with new "metrics" attachment), reviewed output. Saw new fields returned.

Maniphest Tasks: T13430

Differential Revision: https://secure.phabricator.com/D20862
2019-10-24 17:16:58 -07:00
epriestley
f497b93e43 Fix a fatal in the "Projects" curtain extension when a project edge connects an object to a non-project
Summary:
Ref T13429. It's currently possible to write "TYPE_EDGE" relationships for the "object has project" edge to PHIDs which may not actually be projects. Today, this fatals.

As a first step, unfatal it. T13429 discusses general improvements and greater context.

Test Plan:
Used "maniphest.edit" to write a "project" edge to a user PHID, viewed the task in the UI. Previously it fataled; now it renders unusually (the object is "tagged" with a user) but faithfully reflects database state.

{F6957606}

Maniphest Tasks: T13429

Differential Revision: https://secure.phabricator.com/D20860
2019-10-17 09:49:01 -07:00
epriestley
d34dfa3746 Fix an error message when calling "transaction.search" with a non-transactional object PHID as an "objectIdentifier"
Summary: See PHI1499. This error message doesn't provide parameters, and can be a little bit more helpful.

Test Plan: {F6957550}

Differential Revision: https://secure.phabricator.com/D20859
2019-10-17 09:19:54 -07:00
epriestley
5dafabd5b4 Fix deprecated argument order for "implode()"
Summary: Fixes T13428. In modern PHP, "implode()" should take the glue parameter first.

Test Plan:
Used the linter introduced in D20857 to identify affected callsites.

```
$ git grep -i implode | cut -d: -f1 | sort | uniq | xargs arc lint --output summary --never-apply-patches | grep -i glue
```

Maniphest Tasks: T13428

Differential Revision: https://secure.phabricator.com/D20858
2019-10-17 09:11:27 -07:00
epriestley
7badec23a7 Use "git log ... --stdin" instead of "git log ... --not ..." to avoid oversized command lines
Summary:
Depends on D20853. See PHI1474. If the list of "--not" refs is sufficiently long, we may exceed the maximum size of a command.

Use "--stdin" instead, and swap "--not" for the slightly less readable but functionally equivalent "^hash", which has the advantage of actually working with "--stdin".

Test Plan: Ran `bin/repository refs ...` with nothing to be done, and with something to be done.

Differential Revision: https://secure.phabricator.com/D20854
2019-10-02 14:08:15 -07:00
epriestley
6d74736a7e When a large number of commits need to be marked as published, issue the lookup query in chunks
Summary: See PHI1474. This query can become large enough to exceed reasonable packet limits. Chunk the query so it is split up if we have too many identifiers.

Test Plan: Ran `bin/repository refs ...` on a repository with no new commits and a repository with some new commits.

Differential Revision: https://secure.phabricator.com/D20853
2019-10-02 14:06:07 -07:00
Arturas Moskvinas
960c447aab Support more than 9 portals
Summary: If portal is created with id > 9 - then those portals are not reachable and always return 404.

Test Plan: Create at least 10 portals, 10th and rest of them will no longer return 404.

Reviewers: epriestley, Pawka, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20852
2019-10-02 22:35:59 +03:00
epriestley
344a2e39be In Jupyter notebooks, apply intraline diffing to source code lines
Summary: Ref T13425. When we render a diff between two source lines, highlight intraline changes.

Test Plan: Viewed a Jupyter notebook with a code diff in it, saw the changed subsequences in the line highlighted.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20851
2019-10-02 12:34:59 -07:00
epriestley
9f7aaa8ee4 Fix an issue where any diff which could possibly be rendered as Jupyter decided to render as Jupyter
Summary:
See PHI1468. Engine selection for diffs is currently too aggressive in trying to find a shared engine and will fall back a shared engine with a very low score, causing all ".json" files to render as Jupyter files.

Only pick an engine as a difference engine by default if it's the highest-scoring engine for the new file.

Test Plan: Viewed ".json" files and ".ipynb" files in a revision. Before, both rendered as Jupyter. Now, the former rendered as JSON and the latter rendered as Jupyter.

Differential Revision: https://secure.phabricator.com/D20850
2019-10-01 18:52:36 -07:00
epriestley
76d9912932 Force unified abstract block diffs into roughly usable shape
Summary: Depends on D20845. Ref T13425. In unified mode, render blocks diffs less-unreasonably.

Test Plan: {F6910436}

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20848
2019-09-30 10:44:07 -07:00
epriestley
d9515e82a3 Perform basic block interdiffs when diffing abstract blocks, and interdiff markdown in Jupyter notebooks
Summary:
Depends on D20844. Ref T13425. When we line up two blocks and they can be interdiffed (generally: they both have the same type of content), let the Engine interdiff them.

Then, make the Jupyter engine interdiff markdown.

Test Plan: {F6898583}

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20845
2019-09-30 10:43:15 -07:00
epriestley
5afdc620db Make basic Juypter notebook rendering improvements and roughly support folding unchanged context
Summary:
Depends on D20843. Ref T13425. Add very basic support for "Show Hidden Context", in the form of folding it behind an unclickable shield. This isn't ideal, but should be better than nothing.

Prepare for "intraline" diffs on content blocks.

Fix newline handling in Markdown sections in Jupyter notebooks.

Remove the word "visibile" from the codebase.

Test Plan: {F6898192}

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20844
2019-09-30 10:41:21 -07:00
epriestley
a7f3316aa3 Improve sequencing of various content/header checks in abstract block diffs
Summary:
Ref T13425. Some diff checks currently sequence incorrectly:

  - When we're rendering block lists, syntax highlighting isn't relevant.
  - The "large change" guard can prevent rendering of otherwise-renderable changes.
  - Actual errors in the document engine (like bad JSON in a ".ipynb" file) aren't surfaced properly.

Improve sequencing somewhat to resolve these issues.

Test Plan:
  - Viewed a notebook, no longer saw a "highlighting disabled" warning.
  - Forced a notebook to fail, got a useful inline error instead of a popup dialog error.
  - Forced a notebook to have a large number of differences, got a rendering out of it.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20843
2019-09-30 10:40:12 -07:00
epriestley
7a0090f4d0 Fix an issue where the "viewer" is not passed to Bulk Edit controls properly
Summary:
See PHI1442. If you have a bulk-editable datasource field with a composite datasource, it can currently fatal on the bulk edit workflow because the viewer is not passed correctly.

The error looks something like this:

> Argument 1 passed to PhabricatorDatasourceEngine::setViewer() must be an instance of PhabricatorUser, null given, called in /Users/epriestley/dev/core/lib/phabricator/src/applications/typeahead/datasource/PhabricatorTypeaheadCompositeDatasource.php on line 231

Test Plan: Configured a Maniphest custom field with a composite datasource, then tried a bulk edit. Things worked cleanly instead of fataling.

Differential Revision: https://secure.phabricator.com/D20841
2019-09-26 12:03:49 -07:00
epriestley
2c06815edb When rendering Jupyter notebook diffs, split code inputs into individual blocks
Summary:
Ref T13425. Currently, code inputs and all outputs are grouped into a single block. This is fine for display notebooks but not great for diffing notebooks.

Instead, split source code input into individual lines with one line per block, and each output into its own block.

This allows you to leave actual line-by-line inlines on source, and comment on outputs individually.

Test Plan: {F6888583}

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20840
2019-09-25 21:05:18 -07:00
epriestley
884cd74cc4 In prose diffs, use hash-and-diff for coarse "level 0" diffing to scale better
Summary: Depends on D20838. Fixes T13414. Instead of doing coarse diffing with "PhutilEditDistanceMatrix", use hash-and-diff with "DocumentEngine".

Test Plan:
  - On a large document (~3K top level blocks), saw a more sensible diff, instead of the whole thing falling back to "everything changed" mode.
  - On a small document, still saw a sensible granular diff.

{F6888249}

Maniphest Tasks: T13414

Differential Revision: https://secure.phabricator.com/D20839
2019-09-25 16:50:49 -07:00
epriestley
281598d65c Use a hash-and-diff strategy to produce a diff layout for block-based documents
Summary:
Depends on D20835. Ref T13425. Ref T13414. When a document has a list of content blocks, we may not be able to diff it directly, but we can hash each block and then diff the hashes (internally "diff" also does approximately the same thing).

We could do this ourselves with slightly fewer layers of indirection, but: diff already exists; we already use it; we already have a bunch of abstractions on top of it; and it's likely much faster on large inputs than the best we can do in PHP.

Test Plan: {F6888169}

Maniphest Tasks: T13425, T13414

Differential Revision: https://secure.phabricator.com/D20836
2019-09-25 16:40:53 -07:00
epriestley
932d829af3 Improve behavior of inline comment highlight reticle for block diffs
Summary:
Depends on D20834. Ref T13425. After the change from "th" to "td" for accessibility, the algorithm picks which cells it should highlight slightly improperly (it picks too many cells since it can no longer find the line numbers).

Ideally, it would probably highlight //only// the source content, but there isn't an easy way to do this right now. Settle for an incremental improvement for the moment.

Test Plan: Hovered over line numbers, saw a more accurate highlight area.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20835
2019-09-25 16:39:18 -07:00
epriestley
a09b298d85 Correct DOM node metadata to let inline comments work against block-based diffs
Summary: Depends on D20833. Ref T13425. This look like it "just works"?

Test Plan: Left inline comments on a Juptyer notebook. Nothing seemed broken? Confusing.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20834
2019-09-25 16:38:06 -07:00
epriestley
1c4450d39f Allow the Jupyter engine to elect to emit diffs, and emit Jupyter documents as blocks
Summary:
Depends on D20832. Ref T13425. Emit Jupyter notebooks as diffable blocks with block keys.

No diffing or proper inlines yet.

Test Plan: {F6888058}

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20833
2019-09-25 16:32:36 -07:00
epriestley
7ae711ed3e Add a "View as..." option to diff dropdowns for selecting between document engines
Summary:
Depends on D20831. Ref T13425. As an escape hatch to get out of future DocumentEngine rendering behavior, provide a "View As.." option.

Now I can break DocumentEngine real bad and no one can complain.

Test Plan: Used "View As" to swap document engines for image files.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20832
2019-09-25 16:29:21 -07:00
epriestley
bb71ef6ad6 Render image diffs as abstract blocks diffs via DocumentEngine
Summary:
Depends on D20830. Ref T13425. Have the image engine elect into block rendering, then emit blocks.

This is rough (the blocks aren't actually diffed yet) but image diffs were already pretty rough so this is approximately a net improvement.

Test Plan: Viewed image diffs, saw nothing worse than before.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20831
2019-09-25 16:25:06 -07:00
epriestley
a73f592d7d Allow DocumentEngine to elect into diff construction
Summary:
Ref T13425. Allow DocumentEngines to claim they can produce diffs. If both sides of a change can be diffed by the same document engine and it can produce diffs, have it diff them.

This has no impact on runtime behavior because no upstream engine elects into diff generation yet.

Test Plan: Loaded some revisions, nothing broke.

Maniphest Tasks: T13425

Differential Revision: https://secure.phabricator.com/D20830
2019-09-25 16:23:06 -07:00
epriestley
06edcf2709 Fix an issue where ancestors of permanent refs might not be published during import or if a branch is later made permanent
Summary:
Fixes T13284. See that task for substantial discussion. There are currently two cases where we'll skip over commits which we should publish:

  - if a branch is not permanent, then later made permanent; or
  - in some cases, the first time we examine branches in a repository.

In both cases, this error is one-shot and things work correctly going forward. The root cause is conflation between the states "this ref currently permanent" and "this ref was permanent the last time we updated refs".

Separate these pieces of state and cover all these cases. Also introduce a "--rebuild" flag to fix the state of bad commits.

Test Plan:
See T13284 for the three major cases:

  - initial import;
  - push changes to a nonpermanent branch, update, then make it permanent;
  - push chanegs to a nonpermanent branch, update, push more changes, then make it permanent.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13284

Differential Revision: https://secure.phabricator.com/D20829
2019-09-25 08:54:50 -07:00
epriestley
6af776f84a Allow installs to provide "Request a Username Change" instructions
Summary: Fixes T13420. Allow installs to provide username change instructions if there's someone you should contact to get this done.

Test Plan: {F6885027}

Maniphest Tasks: T13420

Differential Revision: https://secure.phabricator.com/D20828
2019-09-24 11:09:26 -07:00
epriestley
0c331458a8 When non-administrators click "Change Username", explain why they can't continue
Summary: Ref T13420. This workflow currently dead-ends for non-administrators. Instead, provide explanatory text.

Test Plan:
  - Clicked "Change Username" as an administrator, same workflow as always.
  - Clicked "Change Username" as a non-administrator, got explanatory text.

Maniphest Tasks: T13420

Differential Revision: https://secure.phabricator.com/D20827
2019-09-24 11:08:49 -07:00
epriestley
058b7ae945 Update "Change Username" instructions to be less foreboding
Summary:
Ref T13420. These warnings are currently more severe than they need to be; weaken them.

Among other cases, the upstream supports and encourages changing usernames when users change human names.

The login/password instructions are also out of date since sessions were decoupled from usernames about a year ago.

Test Plan: Hit dialog as an administrator

Maniphest Tasks: T13420

Differential Revision: https://secure.phabricator.com/D20826
2019-09-24 11:08:11 -07:00
epriestley
be2a0f8733 Fix a cursor paging issue in Conduit call logs
Summary: Fixes T13423. The "Query" class for conduit call logs is missing a "withIDs()" method.

Test Plan: Paged through Conduit call logs.

Maniphest Tasks: T13423

Differential Revision: https://secure.phabricator.com/D20823
2019-09-24 08:36:49 -07:00
epriestley
09d86c2d20 Unprototype "Facts" to clear Maniphest/chart fatals
Summary: Ref T13279. Facts is still fairly rough, but not broken/policy-violating, so it can be unprototyped to fix the issue where Maniphest reports (which are now driven by Facts) don't work if prototypes are disabled.

Test Plan: Viewed Maniphest reports and Project reports with prototypes on/off and Fact installed/uninstalled.

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20822
2019-09-23 13:01:18 -07:00
epriestley
16de9151c7 Give "Burndown" charts a more straightforward definition and move all the event stuff into "Activity" charts
Summary:
Depends on D20818. Ref T13279. The behavior of the "burndown" chart has wandered fairly far afield; make it look more like a burndown.

Move the other thing into an "Activity" chart.

Test Plan: {F6865207}

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20819
2019-09-17 13:30:29 -07:00
epriestley
c06dd4818b Support explicit stacking configuration in stacked area charts
Summary:
Ref T13279. Allow engines to choose how areas in a stacked area chart stack on top of one another.

This could also be accomplished by using multiple stacked area datasets, but datasets would still need to know if they're stacking "up" or "down" so it's probably about the same at the end of the day.

Test Plan: {F6865165}

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20818
2019-09-17 13:26:07 -07:00
epriestley
d4ed5d0428 Make various UX improvements to charts so they're closer to making visual sense
Summary: Ref T13279. Fix some tabular stuff, draw areas better, make the "compose()" API more consistent, unfatal the demo chart, unfatal the project burndown, make the project chart do something roughly physical.

Test Plan: Looked at charts, saw fewer obvious horrors.

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20817
2019-09-17 09:43:21 -07:00
epriestley
080e132aa7 Track chart datapoints from their sources and provide a tabular view of chart data
Summary: Depends on D20815. Ref T13279. Give datapoints "refs", which allow us to figure out where particular datapoints came from even after the point is transformed by functions. For now, show the raw points in a table below the chart.

Test Plan: Viewed chart data, saw reasonable-looking numbers.

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20816
2019-09-17 09:41:02 -07:00
epriestley
769e745a3f In charts, make "min" and "max" into pure functions and formally mark pure functions as pure
Summary:
Depends on D20814. Currently, "min()" and "max()" are still "min(f, n)". This is no longer consistent with the construction of functions a function-generators that are composed at top level.

Turn them into "min(n)" and "max(n)" (i.e., not higher-order functions).

Then, mark all the functions which are pure mathematical functions and not higher-order as "pure". These functions have no function parameters and do not reference external data. For now, this distinction has no immediate implications, but it will simplify the next change (which tracks where data came from when it originated from an external source -- these pure functions never have any source information, since they only apply pure mathematical transformations to data).

Test Plan: Loaded a burnup chart, nothing seemed obviously broken.

Subscribers: yelirekim

Differential Revision: https://secure.phabricator.com/D20815
2019-09-17 09:28:23 -07:00
epriestley
f529abf900 In stacked area charts, group nearby points so they don't overlap
Summary: Ref T13279. We currently draw a point on the chart for each datapoint, but this leads to many overlapping circles. Instead, aggregate the raw points into display points ("events") at the end.

Test Plan: Viewed a stacked area chart with many points, saw a more palatable number of drawn dots.

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20814
2019-09-17 09:26:54 -07:00
epriestley
3f66203362 Fix a straggling callsite to "renderApplicationPolicy()"
Summary: Ref T13411. This is a leftover from recent policy rendering changes.

Test Plan: Viewed feed with application policy stories, no more fatal.

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20811
2019-09-12 16:26:57 -07:00
epriestley
41f0b8b0a3 Allow subtypes to specify "mutations", to control the behavior of the "Change Subtype" action
Summary:
Fixes T13415. Provide a way for subtypes to customize the behavior of "Change Subtype" actions that appear above comment areas.

Subtypes may disable this action by specifying `"mutations": []`, or provide a list of subtypes.

The bulk editor and API can still perform any change.

Test Plan:
  - Tried to define an invalid "mutations" list with a bad subtype, got a sensible error.
  - Specified a limited mutations list and an empty mutations list, verified that corresponding tasks got corresponding actions.
  - Used the bulk editor to perform a freeform mutation.
  - Verified that tasks of a subtype with no "mutations" still work the same way they used to (allow mutation into any subtype).

Maniphest Tasks: T13415

Differential Revision: https://secure.phabricator.com/D20810
2019-09-12 16:17:02 -07:00
epriestley
3e60128037 Support "Subtype" in Herald
Summary: See PHI1434. For objects that support subtypes and have subtypes configured, allow Herald rules to act on subtypes.

Test Plan:
  - Configured task and project subtypes, wrote Herald rules, saw "Subtypes" as an option, saw appropriate typeahead values and detail page rendering.
  - Unconfigured project subtypes, saw field vanish from UI for new rules.
  - Wrote a "subtype"-depenent rule that added a comment, interacted with tasks of that subtype and a different subtype. Saw Herald act only on tasks with the correct subtype.

Differential Revision: https://secure.phabricator.com/D20809
2019-09-12 14:34:06 -07:00
epriestley
d60d4e6a05 Don't present users with Herald fields/actions for uninstalled applications, unless the rule already uses them
Summary:
Fixes T7961. Currently, we present Herald users with actions like "Require legalpad signatures" and "Run build plans" even if Legalpad and Harbormaster are not installed.

Instead, allow fields and actions to be made "unavailable", which means that we won't present them as options when adding to new or existing rules.

If you edit a rule which already uses one of these fields or actions, it isn't affected.

Test Plan:
  - Created a rule with a legalpad action, uninstalled legalpad, edited the rule. Action remained untouched.
  - Created a new rule, wasn't offered the legalpad action.
  - Reinstalled the application, saw the action again.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T7961

Differential Revision: https://secure.phabricator.com/D20808
2019-09-12 14:33:28 -07:00
epriestley
4f845d8f8c When users encounter a policy exception for a non-view capability with a custom policy, inline the policy rules
Summary:
Fixes T13411. This looks like the last case where you hit a policy explanation and have permission to see the policy, but we don't currently show you the policy rules.

This implementation is slightly clumsy, but likely harmless.

Test Plan: {F6856421}

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20807
2019-09-12 09:49:17 -07:00
epriestley
0c7ea8c942 When users fail a "CAN_SEE" check, give them an "opaque" policy explanation
Summary:
Ref T13411. Currently, if you hit a policy exception because you can't view an object, we disclose details about the view policy of the object, particularly which project's members can see the object for project policies.

Although there's a large amount of grey area here, this feels like a more substantial disclosure than we offer in other contexts. Instead, if you encounter a policy exception while testing "CAN_VIEW" or don't have "CAN_VIEW", present an "opaque" explanation which omits details that viewers who can't view the object shouldn't have access to. Today, this is the name of "Project" policies (and, implicitly, the rulesets of custom policies, which we now disclose in other similar contexts).

Test Plan:
  - Hit policy exceptions for "CAN_VIEW" on an object with a project view policy, saw an opaque explanation.
  - Hit policy exceptions for "CAN_EDIT" on an object with a project edit policy and a view policy I satisfied, saw a more detailed explanation.

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20806
2019-09-12 09:42:02 -07:00
epriestley
9a36e6931c Inline custom policy rules inside policy capability explanation dialogs
Summary: Ref T13411. When users click a link to explain a capability (like the policy header on many objects, or the link next to specific capabilities in "Applications", "Diffusion", etc), inline the full ruleset for the custom policy into the dialog if the object has a custom policy.

Test Plan: {F6856365}

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20805
2019-09-12 09:40:50 -07:00
epriestley
506f93b4a3 Give policy name rendering explicit "text name", "capability link", and "transaction link" pathways
Summary:
Ref T13411. This cleans up policy name rendering. We ultimately render into three contexts:

  - Plain text contexts, like `bin/policy show`.
  - Transaction contexts, where we're showing a policy change. In these cases, we link some policies (like project policies and custom policies) but the links go directly to the relevant object or a minimal explanation of the change. We don't link policies like "All Users".
  - Capability contexts, where we're describing a capability, like "Can Push" or cases in Applicaitons. In these cases, we link all policies to the full policy explanation flow.

Test Plan:
  - Used `bin/policy show` to examine the policy of an object with a project policy, no longer saw HTML.
  - Viewed the transaction logs of Applications (ModularTransactions) and Tasks (not ModularTransactions) with policy edits, including project and custom policies.
  - Clicked "Custom Policy" in both logs, got consistent dialogs.
  - Viewed application detail pages, saw all capabities linked to explanatory capability dialogs. The value of having this dialog is that the user can get a full explanation of special rules even if the policy is something mundane like "All Users".

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20804
2019-09-12 09:39:05 -07:00
epriestley
c9b0d107f0 Remove unused "icon" parameter from policy name rendering
Summary: Ref T13411. This pathway has an unused "icon" parameter with no callsites. Throw it away to ease refactoring.

Test Plan: Grepped for callsites, found none using this parameter.

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20803
2019-09-12 09:38:01 -07:00
epriestley
9c6969e810 Remove "Editable By" description fields in Passphrase, Phame, and Spaces
Summary:
Ref T13411. These three applications render an "Editable By: <policy>" field in their descriptions.

The pages that these appear on all have "Edit <thing>" actions which either tell you the policy or allow you to discover the policy, and this field is unusual (the vast majority of objects don't have it). I think it largely got copy/pasted or used as space-filler and doesn't offer much of value.

Remove it to simplify/standardize these pages and make changes to how this field works simpler to implement.

Test Plan: Viewed a Credential, Blog, and Space; no longer saw the "Editable By" field.

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20802
2019-09-12 09:36:50 -07:00
epriestley
a35d7c3c21 Update rendering of policy edit transactions in Applications
Summary:
Ref T13411. Since circa D19829, transactions have rendered policy changes in a modern way, notably making "Custom Policy" clickable to show the policy rules.

Edit transactions in Applications still use a separate, older approach to render policies. This produces policy renderings which don't use modern quoting rules and don't link in a modern way.

Make Applications use the same rendering code that other transactions (like normal edit/view edits) use.

Test Plan: Edited policies in Applications, saw more useful transactions in the log. Clicked "Custom Policy" in the transaction log and got a useful explanation of the policy.

Maniphest Tasks: T13411

Differential Revision: https://secure.phabricator.com/D20801
2019-09-12 09:32:52 -07:00
epriestley
a0ade503e1 Remove "Moved Document from ..." notice in Phriction
Summary:
Ref T13410. See PHI1431. Currently, when you move a document in Phriction, the target shows a "This document was moved from ..." banner until it is edited.

This banner isn't particularly useful, and it's distracting and it isn't obvious how to dismiss it, and making a trivial edit to dismiss it is awkward.

This information is also already available in the transaction log.

Just remove this banner since it doesn't really serve any clear purpose.

Test Plan:
  - Moved a page in Phriction, then loaded the destination page. Before change: header banner. After change: nothing.
  - Viewed a normal (non-moved) page, saw normal behavior.
  - Reviewed transactions, saw "Moved from ..." in the timeline.

Maniphest Tasks: T13410

Differential Revision: https://secure.phabricator.com/D20800
2019-09-12 09:32:26 -07:00
epriestley
d2e1c4163a When a project has a custom icon, use that icon to label the project policy in the policy dropown
Summary:
Fixes T8808. Currently, all project use the default ("Briefcase") project icon when they appear in a policy dropdown.

Since project policies are separated out into a "Members of Projects" section of the dropdown anyway, there is no reason not to use the actual project icon, which is often more clear.

Test Plan: {F6849927}

Maniphest Tasks: T8808

Differential Revision: https://secure.phabricator.com/D20799
2019-09-09 13:38:12 -07:00
epriestley
1d1a60fdda Improve rendering of Herald rules in "Another Herald rule..." field
Summary:
Fixes T9136.

  - Fix a bug where the name is rendered improperly.
  - Put disabled rules at the bottom.
  - Always show the rule monogram so you can distingiush between rules with the same name.

Test Plan: {F6849915}

Maniphest Tasks: T9136

Differential Revision: https://secure.phabricator.com/D20798
2019-09-09 13:29:49 -07:00
epriestley
7593a265d5 When Herald changes object subscribers, always hide the feed story
Summary:
Fixes T8952. These feed stories are not interesting and tend to be generated as collateral damage when a non-story update is made to an old task and someone has a "subscribe me" Herald rule.

Also clean up some of the Herald field/condition indexing behavior slightly.

Test Plan: Wrote a "Subscribe X" herald rule, made a trivial update to a task. Before: low-value feed story; after: no feed story.

Maniphest Tasks: T8952

Differential Revision: https://secure.phabricator.com/D20797
2019-09-09 13:17:36 -07:00
epriestley
4547714463 Add a "Remove flag" action to Herald
Summary: Fixes T13409. This is a companion to the existing "Mark with flag" rule.

Test Plan: Used a "remove flag" rule on an object with no flag (not removed), the right type of flag (removed), and a different type of flag (not removed).

Maniphest Tasks: T13409

Differential Revision: https://secure.phabricator.com/D20796
2019-09-09 13:15:52 -07:00
epriestley
d965d9a669 Index Herald fields, not just actions, when identifying objects related to a particular Herald rule
Summary:
Fixes T13408. Currently, when a package (or other object) appears in a field (rather than an action), it is not indexed.

Instead: index fields too, not just actions.

Test Plan:
  - Wrote a rule like "[ Affected packages include ] ...".
  - Updated the search index.
  - Saw rule appear on "Affected By Herald Rules" on the package detail page.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13408

Differential Revision: https://secure.phabricator.com/D20795
2019-09-09 12:50:43 -07:00
epriestley
aaaea57591 Fix fatal during redirection safety check for searching for Phabricator base-uri with no trailing slash
Summary: Fixes T13412. If you search for "https://phabricator.example.com" with no trailing slash, we currently redirect you to "", which is fouled by a safety check in the redirection flow.

Test Plan:
  - Searched for "https://local.phacility.com"; before: fatal in redirection; after: clean redirect.
  - Searched for other self-URIs, got normal redirects.

Maniphest Tasks: T13412

Differential Revision: https://secure.phabricator.com/D20794
2019-09-09 12:45:24 -07:00
epriestley
278092974f Don't offer personal saved queries in global "Search Scope" settings dropdown
Summary: Fixes T13405. We currently offer non-global custom saved queries here, but this doesn't make sense as a global default setting.

Test Plan: Saved a global search query, edited global search settings, no longer saw the non-global query as an option.

Maniphest Tasks: T13405

Differential Revision: https://secure.phabricator.com/D20793
2019-09-09 12:21:25 -07:00
epriestley
535c8e6bdc Remove the "ONLY_FULL_GROUP_BY" SQL mode setup warning and change the setup key for "STRICT_ALL_TABLES"
Summary:
Ref T13404. Except for one known issue in Multimeter, Phabricator appears to function properly in this mode. It is broadly desirable that we run in this mode; it's good on its own, and enabled by default in at least some recent MySQL.

Additionally, "ONLY_FULL_GROUP_BY" and "STRICT_ALL_TABLES" shared a setup key, so ignoring one would ignore both. Change the key so that existing ignores on "ONLY_FULL_GROUP_BY" do not mask "STRICT_ALL_TABLES" warnings.

Test Plan: Grepped for `ONLY_FULL_GROUP_BY`.

Maniphest Tasks: T13404

Differential Revision: https://secure.phabricator.com/D20791
2019-09-09 12:17:51 -07:00
epriestley
f16365ed07 Weaken the guidance recommending that installs enable "STRICT_ALL_TABLES"
Summary: Ref T13404. Enabling "STRICT_ALL_TABLES" is good, but if you don't want to bother it doesn't matter too much. All upstream development has been on "STRICT_ALL_TABLES" for a long time.

Test Plan: {F6847839}

Maniphest Tasks: T13404

Differential Revision: https://secure.phabricator.com/D20790
2019-09-09 12:17:05 -07:00
epriestley
caccbb69d2 When users try to log out with no providers configured, warn them of the consequences
Summary: Fixes T13406. On the logout screen, test for no configured providers and warn users they may be getting into more trouble than they expect.

Test Plan:
  - Logged out of a normal install and a fresh (unconfigured) install.

{F6847659}

Maniphest Tasks: T13406

Differential Revision: https://secure.phabricator.com/D20789
2019-09-08 12:27:29 -07:00
Aviv Eyal
318e8ebdac Allow bin/config to create config file
Summary:
See D20779, https://discourse.phabricator-community.org/t/3089. `bin/config set` complains about
missing config file as if it's un-writable.

Test Plan: run `bin/config set` with missing, writable, unwritable conf.json and parent dir.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20788
2019-09-08 00:16:19 +00:00
epriestley
7e2bec9280 Add a global setting for controlling the default main menu search scope
Summary: Fixes T13405. The default behavior of the global search bar isn't currently configurable, but can be made configurable fairly easily.

Test Plan: Changed setting as an administrator, saw setting reflected as a user with no previous preference. As a user with an existing preference, saw preference retained.

Maniphest Tasks: T13405

Differential Revision: https://secure.phabricator.com/D20787
2019-09-06 08:39:28 -07:00
epriestley
f7290bbbf2 Update a straggling "getAuthorities()" call in Fund
Summary: Ref T13366. The "authorities" mechanism was replaced, but I missed this callsite. Update it to use the request cache mechanism.

Test Plan: As a user without permission to view some initiatives, viewed a list of initiatives.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20783
2019-09-04 07:15:20 -07:00
epriestley
22b075df97 Fix "ONLY_FULL_GROUP_BY" issue in SystemAction queries
Summary: Ref T13404. This query is invalid under "sql_mode=ONLY_FULL_GROUP_BY". Rewrite it to avoid interacting with `actorIdentity` at all; this is a little more robust in the presence of weird data and not really more complicated.

Test Plan:
  - Enabled "ONLY_FULL_GROUP_BY".
  - Hit system actions (e.g., login).
    - Before: error.
    - After: clean login.
  - Tried to login with a bad password many times in a row, got properly limited by the system action rate limiter.

Maniphest Tasks: T13404

Differential Revision: https://secure.phabricator.com/D20782
2019-09-03 16:50:33 -07:00
epriestley
d9badba147 Give "bin/config" a friendlier error message if "local.json" is not writable
Summary: Ref T13403. We currently emit a useful error message, but it's not tailored and has a stack trace. Since this is a relatively routine error and on the first-time-setup path, tailor it so it's a bit nicer.

Test Plan:
  - Ran `bin/config set ...` with an unwritable "local.json".
  - Ran `bin/config set ...` normally.

Maniphest Tasks: T13403

Differential Revision: https://secure.phabricator.com/D20779
2019-09-03 12:47:06 -07:00
epriestley
8ff3a133c4 Generalize repository proxy retry logic to writes
Summary:
Ref T13286. The current (very safe / conservative) rules for retrying git reads generalize to git writes, so we can use the same ruleset in both cases.

Normally, writes converge rapidly to only having good nodes at the head of the list, so this has less impact than the similar change to reads, but it generally improves consistency and allows us to assert that writes which can be served will be served.

Test Plan:
  - In a cluster with an up node and a down node, pushed changes.
  - Saw a push to the down node fail, retry, and succeed.
  - Did some pulls, saw appropriate retries and success.
  - Note that once one write goes through, the node which received the write always ends up at the head of the writable list, so nodes need to be explicitly thawed to reproduce the failure/retry behavior.

Maniphest Tasks: T13286

Differential Revision: https://secure.phabricator.com/D20778
2019-09-03 12:34:10 -07:00
epriestley
ff3d1769b4 Instead of retrying safe reads 3 times, retry each eligible service once
Summary: Ref T13286. When retrying a read request, keep retrying as long as we have canididate services. Since we consume a service with each attempt, there's no real reason to abort early, and trying every service allows reads to always succeed even if (for example) 8 nodes of a 16-node cluster are dead because of a severed network link between datacenters.

Test Plan: Ran `git pull` in a clustered repository with an up node and a down node; saw retry count dynamically adjust to available node count.

Maniphest Tasks: T13286

Differential Revision: https://secure.phabricator.com/D20777
2019-09-03 10:43:33 -07:00
epriestley
95fb237ab3 On Git cluster read failure, retry safe requests
Summary:
Depends on D20775. Ref T13286. When a Git read request fails against a cluster and there are other nodes we could safely try, try more nodes.

We DO NOT retry the request if:

  - the client read anything;
  - the client wrote anything;
  - or we've already retried several times.

Although //some// requests where bytes went over the wire in either direction may be safe to retry, they're rare in practice under Git, and we'd need to puzzle out what state we can safely emit.

Since most types of failure result in an outright connection failure and this catches all of them, it's likely to almost always be sufficient in practice.

Test Plan:
  - Started a cluster with one up node and one down node, pulled it.
  - Half the time, hit the up node and got a clean pull.
  - Half the time, hit the down node and got a connection failure followed by a retry and a clean pull.
  - Forced `$err = 1` so even successful attempts would retry.
  - On hitting the up node, got a "failure" and a decline to retry (bytes already written).
  - On hitting the down node, got a failure and a real retry.
  - (Note that, in both cases, "git pull" exits "0" after the valid wire transaction takes place, even though the remote exited non-zero. If the server gave Git everything it asked for, it doesn't seem to care if the server then exited with an error code.)

Maniphest Tasks: T13286

Differential Revision: https://secure.phabricator.com/D20776
2019-09-03 10:08:43 -07:00
epriestley
b6420e0f0a Allow repository service lookups to return an ordered list of service refs
Summary:
Ref T13286. To support request retries, allow the service lookup method to return an ordered list of structured service references.

Existing callsites continue to immediately discard all but the first reference and pull a URI out of it.

Test Plan: Ran `git pull` in a clustered repository with an "up" node and a "down" node, saw 50% serivce failures and 50% clean pulls.

Maniphest Tasks: T13286

Differential Revision: https://secure.phabricator.com/D20775
2019-09-03 10:05:40 -07:00
epriestley
9316cbf7fd Move web application classes into "phabricator/"
Summary: Ref T13395. Companion change to D20773.

Test Plan: See D20773.

Maniphest Tasks: T13395

Differential Revision: https://secure.phabricator.com/D20774
2019-09-02 07:58:59 -07:00
epriestley
b2b17485b9 Clean up two straggling UI issues in Phortune
Ref T13401. The checkout UI didn't get fully updated to the new View objects,
and account handles are still manually building a URI that goes to the wrong
place.
2019-08-31 09:36:23 -07:00
epriestley
533a5535b6 Remove the "grant authority" mechanism from users
Summary:
Ref T13393. See some previous discussion in T13366.

Caching is hard and all approaches here have downsides, but the request cache likely has fewer practical downsides for this kind of policy check than other approaches. In particular, the grant approach (at least, as previously used in Phortune) has a major downside that "Query" classes can no longer fully enforce policies.

Since Phortune no longer depends on grants and they've now been removed from instances, drop the mechanism completely.

Test Plan: Grepped for callsites, found none.

Maniphest Tasks: T13393

Differential Revision: https://secure.phabricator.com/D20754
2019-08-30 09:26:08 -07:00
epriestley
3c26e38487 Provide a simple read-only maintenance mode for repositories
Summary:
Ref T13393. While doing a shard migration in the Phacility cluster, we'd like to stop writes to the migrating repository. It's safe to continue serving reads.

Add a simple maintenance mode for making repositories completely read-only during maintenance.

Test Plan: Put a repository into read-only mode, tried to write via HTTP + SSH. Viewed web UI. Took it back out of maintenance mode.

Maniphest Tasks: T13393

Differential Revision: https://secure.phabricator.com/D20748
2019-08-29 15:23:10 -07:00
epriestley
c6642213d5 Straighten out replication/cache behavior in "bin/storage dump"
Summary:
Fixes T13336.

  - Prevent `--no-indexes` from being combined with `--for-replica`, since combining these options can only lead to heartbreak.
  - In `--for-replica` mode, dump caches too. See discussion in T13336. It is probably "safe" to not dump these today, but fragile and not correct.
  - Mark the "MarkupCache" table as having "Cache" persistence, not "Data" persistence (no need to back it up, since it can be fully regenerated from other datasources).

Test Plan: Ran `bin/storage dump` with various combinations of flags.

Maniphest Tasks: T13336

Differential Revision: https://secure.phabricator.com/D20743
2019-08-28 08:25:40 -07:00
epriestley
0943561dcb Fix incorrect construction of subtype map when validating "subtype" transactions against non-subtypable objects
Summary:
Fixes T13389. Currently, we try to "newSubtypeMap()" unconditionally, even if the underlying object does not support subtypes.

  - Only try to build a subtype map if subtype transactions are actually being applied.
  - When subtype transactions are applied to a non-subtypable object, fail more explicitly.

Test Plan: Clicked "Make Editable" in a fresh Calendar transaction form, got an editable form instead of a fatal from "newSubtypeMap()". (Calendar events are not currently subtypable.)

Maniphest Tasks: T13389

Differential Revision: https://secure.phabricator.com/D20741
2019-08-28 06:57:04 -07:00
epriestley
97a4a59cf2 Give the Phortune external portal an order view
Summary:
Depends on D20739. Ref T13366. Slightly modularize/update components of order views, and make orders viewable from either an account context (existing view) or an external context (new view).

The new view is generally simpler so this mostly just reorganizes existing code.

Test Plan: Viewed orders as an account owner and an external user.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20740
2019-08-26 07:49:17 -07:00
epriestley
a0a3879712 In Phortune, send order email to account external addresses
Summary: Depends on D20738. Ref T13366. Fixes T8389. Now that the infrastructure is in place, actually send email to external addresses.

Test Plan: Used `bin/phortune invoice` to generate invoices and saw associated external accounts receive mail in `bin/mail list-outbound`.

Maniphest Tasks: T13366, T8389

Differential Revision: https://secure.phabricator.com/D20739
2019-08-26 07:48:27 -07:00
epriestley
4e13551e85 Add credential rotation and statuses (disabled, unsubscribed) to Phortune external email
Summary: Depends on D20737. Ref T13367. Allow external addresses to have their access key rotated. Account managers can disable them, and anyone with the link can permanently unsubscribe them.

Test Plan: Enabled/disabled addresses; permanently unsubscribed addresses.

Maniphest Tasks: T13367

Differential Revision: https://secure.phabricator.com/D20738
2019-08-26 07:47:44 -07:00
epriestley
8f6a1ab015 Roughly support external/email user views of Phortune recipts and invoices
Summary: Ref T13366. This gives each account email address an "external portal" section so they can access invoices and receipts without an account.

Test Plan: Viewed portal as user with authority and in an incognito window.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20737
2019-08-26 07:39:08 -07:00
epriestley
a39a37fc0e Update the Phortune cart/invoice workflow for policy changes
Summary:
Depends on D20734. Ref T13366. This makes the cart/order flow work under the new policy scheme with no "grantAuthority()" calls.

It prepares for a "Void Invoice" action, although the action doesn't actually do anything yet.

Test Plan: With and without merchant authority, viewed and paid invoices and went through the other invoice interaction workflows.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20735
2019-08-23 07:09:00 -07:00
epriestley
b3f8045b87 Make minor flavor updates
Summary: Refresh the 404 text since it hasn't been updated in a while, and swap the "Save Query" button back to grey since I never got used to blue.

Test Plan: Hit 404 page, saved a query.

Differential Revision: https://secure.phabricator.com/D20734
2019-08-23 07:08:09 -07:00
epriestley
9bcd683c08 Update Phortune Merchant UI to bring it in line with Account UI
Summary:
Depends on D20732. Ref T13366. This generally makes the "Merchant" UI look and work like the "Payment Account" UI.

This is mostly simpler since the permissions have largely been sorted out already and there's less going on here and less weirdness around view/edit policies.

Test Plan: Browsed all Merchant functions as a merchant member and non-member.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20733
2019-08-22 21:12:33 -07:00
epriestley
c93ac91dc6 Update Charge and Cart policies in Phortune, and make URIs more consistent
Summary:
Ref T13366. Depends on D20721. Continue applying UI and policy updates to the last two Phortune objects.

Charges aren't mutable and Carts are already transactional, so this is less involved than prior changes.

Test Plan: Viewed various charge/order interfaces as merchants and account members.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20732
2019-08-22 21:09:35 -07:00
epriestley
a542024b63 Update Phortune subscriptions for modern infrastructure
Summary:
Depends on D20720. Ref T13366.

  - Use modern policies and policy interfaces.
  - Use new merchant authority cache.
  - Add (some) transactions.
  - Move MFA from pre-upgrade-gate to post-one-shot-check.
  - Simplify the autopay workflow.
  - Use the "reloading arrows" icon for subscriptions more consistently.

Test Plan: As a merchant-authority and account-authority, viewed, edited, and changed autopay for subscriptions.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20721
2019-08-22 21:07:17 -07:00
epriestley
201634848e Make Phortune payment methods transaction-oriented and always support "Add Payment Method"
Summary:
Depends on D20718. Ref T13366. Ref T13367.

  - Phortune payment methods currently do not use transactions; update them.
  - Give them a proper view page with a transaction log.
  - Add an "Add Payment Method" button which always works.
  - Show which subscriptions a payment method is associated with.
  - Get rid of the "Active" status indicator since we now treat "disabled" as "removed", to align with user expectation/intent.
  - Swap out of some of the super weird div-form-button UI into the new "big, clickable" UI for choice dialogs among a small number of options on a single dimension.

Test Plan:
  - As a mechant-authority and account-authority, created payment methods from carts, subscriptions, and accounts. Edited and viewed payment methods.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13367, T13366

Differential Revision: https://secure.phabricator.com/D20719
2019-08-22 21:04:04 -07:00
epriestley
c4e0ac4d27 Update PhortunePaymentMethod for modern policy interfaces
Summary:
Depends on D20717. Ref T13366. Make PhortunePaymentMethod use an extended policy interface for consistency with modern approaches. Since Accounts have hard-coded policy behavior (and can't have object policies like "Subscribers") this should have no actual impact on program behavior.

This leaves one weird piece in the policy dialog UIs, see T13381.

Test Plan: Viewed and edited payment methods as a merchant and account member. Merchants can only view, not edit.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20718
2019-08-22 21:03:16 -07:00
epriestley
0cc7e8eeb8 Update Phortune payment account interfaces to handle merchant vs customer views
Summary: Depends on D20716. Ref T13366. This implements the new policy behavior cleanly in all top-level Phortune payment account interfaces.

Test Plan: As a merchant with an account relationship (not an account member) and an account member, browsed all account interfaces and attempted to perform edits. As a merchant, saw a reduced-strength view.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20717
2019-08-22 21:02:41 -07:00
epriestley
a3213ab20b In Phortune, use actual merchant authority (not authority grants) to control account visibility
Summary:
Depends on D20715. Ref T13366. See that task for discussion.

Replace the unreliable "grantAuthority()"-based check with an actual "can the viewer edit any merchant this account has a relationship with?" check.

This makes these objects easier to use from a policy perspective and makes it so that the `Query` alone can fully enforce permissions properly with no setup, so general infrastructure (like handles and transactions) works properly with Phortune objects.

Test Plan: Viewed merchants and accounts as users with no authority, direct authority on the account, and indirect authority via a merchant relationship.

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20716
2019-08-22 21:01:55 -07:00
epriestley
277bce5638 In Phortune, write relationships between payment accounts and merchants they interact with
Summary:
Depends on D20713. Ref T13366. When a payment account establishes a relationship with a merchant by creating a cart or subscription, create an edge to give the merchant access to view the payment account.

Also, migrate all existing subscriptions and carts to write these edges.

This aims at straightening out Phortune permissions, which are currently a bit wonky on a couple of dimensions. See T13366 for detailed discussion.

Test Plan:
  - Created and edited carts/subscriptions, saw edges write.
  - Ran migrations, saw edges write.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20715
2019-08-22 21:01:04 -07:00
epriestley
e3ba53078e Add scaffolding for ad-hoc email addresses associated with Phortune accounts
Summary: Depends on D20697. Ref T8389. Add support for adding "billing@enterprise.com" and similar to Phortune accounts.

Test Plan: Added and edited email addresses for a payment account.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T8389

Differential Revision: https://secure.phabricator.com/D20713
2019-08-22 20:57:35 -07:00
epriestley
719a7d82c5 Refactor the Phortune account detail page into a series of smaller, more focused sections
Summary:
Ref T13366. Some of the information architecture is a little muddy here, notably an item called "Billing / History" which contains payment methods.

Split things up a bit to prepare for adding support for "Email Addresses".

Test Plan: {F6676988}

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20697
2019-08-22 20:56:43 -07:00
epriestley
ecbc82da33 Expose "commits.add|set|remove" on "maniphest.edit" API calls
Summary: See PHI1396. Ideally this would be some kind of general-purpose tie-in to object relationships, but see D18456 for precedent.

Test Plan: Used `maniphest.edit` to edit associated commits for a task.

Differential Revision: https://secure.phabricator.com/D20731
2019-08-22 13:34:33 -07:00
epriestley
353155a203 Add "modifiedStart" and "modifiedEnd" constraints to "differential.revision.search"
Summary:
Fixes T13386. See PHI1391. These constraints largely exist already, but are not yet exposed to Conduit.

Also, tweak some keys to support the underlying query.

Test Plan: Ran `differential.revision.search` queries with the new constraints.

Maniphest Tasks: T13386

Differential Revision: https://secure.phabricator.com/D20730
2019-08-22 13:34:18 -07:00
epriestley
109d7dcaf1 Convert "Empower" from state-based MFA to one-shot MFA
Summary: Ref T13382. Currently, the "Make Administrator" action in the web UI does state-based MFA. Convert it to one-shot MFA.

Test Plan: Empowered and unempowered a user from the web UI, got one-shot MFA'd. Empowered a user from the CLI, no MFA issues.

Maniphest Tasks: T13382

Differential Revision: https://secure.phabricator.com/D20729
2019-08-22 08:34:46 -07:00
epriestley
5741514aeb When a client submits an overlong "sourcePath", truncate it and continue
Summary:
Ref T13385. Currently, if you run `arc diff` in a CWD with more than 255 characters, the workflow fatals against the length of the `sourcePath` database column.

In the long term, removing this property is likely desirable.

For now, truncate long values and continue. This only meaningfully impacts relatively obscure interactive SVN workflows negatively, and even there, "some arc commands are glitchy in very long working directories in SVN" is still better than "arc diff fatals".

Test Plan:
  - Modified `arc` to submit very long source paths.
  - Ran `arc diff`.
    - Before: Fatal when inserting >255 characters into `sourcePath`.
    - After: Path truncated at 255 bytes.

Maniphest Tasks: T13385

Differential Revision: https://secure.phabricator.com/D20727
2019-08-21 19:28:18 -07:00
epriestley
fc34554892 Replace "bin/people profileimage" with "bin/user enable|empower"
Summary:
Ref T13382.

  - Remove "bin/people profileimage" which previously generated profile image caches but now feels obsolete.
  - Replace it with "bin/user", with "enable" and "empower" flows. This command is now focused on regaining access to an install after you lock your keys inside.
  - Document the various ways to unlock objects and accounts from the CLI.

Test Plan:
  - Ran `bin/user enable` and `bin/user empower` with various flags.
  - Grepped for `people profileimage` and found no references.
  - Grepped for `bin/people` and found no references.
  - Read documentation.

Maniphest Tasks: T13382

Differential Revision: https://secure.phabricator.com/D20724
2019-08-20 17:51:14 -07:00
epriestley
721a86401f Implement "drydock.resource.search"
Summary: Fixes T13383. Provide a basic "drydock.resource.search". Also allow "drydock.lease.search" to be queried by resource PHID.

Test Plan: Called "drydock.resource.search" and "drydock.lease.search" with various constraints.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13383

Differential Revision: https://secure.phabricator.com/D20723
2019-08-20 13:07:07 -07:00
epriestley
d890c03ac3 Namespace all column references in ProjectQuery to fix ambiguity with Ferret constraints
Summary:
Fixes T13378. If we join Ferret tables and page, we can end up with an ambiguous `id` column here.

Explicitly refer to "project.x" in all cases that we're interacting with the project table.

Test Plan:
  - Changed page size to 3.
  - Issued a Projects query for "~e", matching more than 3 results.
  - Clicked "Next Page".
    - Before: ambiguous id column fatal.
    - After: next page.

Maniphest Tasks: T13378

Differential Revision: https://secure.phabricator.com/D20714
2019-08-15 12:21:54 -07:00
epriestley
82cf97ad65 When many commits are discovered at once, import them at lower priority
Summary:
Ref T13369. See that task for discussion.

When the discovery daemon finds more than 64 commits to import, demote the worker queue priority of the resulting tasks.

Test Plan:
  - Pushed one commit, ran `bin/repository discover --verbose --trace ...`, saw commit import with "at normal priority" message and priority 2500 ("PRIORITY_COMMIT").
  - Pushed 3 commits, set threshold to 3, ran `bin/repository discover ...`, saw commist import with "at lower priority" message and priority 4000 ("PRIORITY_IMPORT").

Maniphest Tasks: T13369

Differential Revision: https://secure.phabricator.com/D20712
2019-08-12 12:59:00 -07:00
epriestley
006cb659cb Make the success message from "bin/config" more clear
Summary:
Ref T13373. When you "bin/config set x ..." a value, the success message ("Set x ...") is somewhat ambiguous and can be interpreted as "First, you need to set x..." rather than "Success, wrote x...".

Make the messaging more explicit. Also make this string more translatable.

Test Plan: Ran `bin/config set ...` with various combinations of flags, saw more clear messaging.

Maniphest Tasks: T13373

Differential Revision: https://secure.phabricator.com/D20711
2019-08-12 12:50:03 -07:00
epriestley
c092492a53 Fix missing display cell in daemon summary table
Summary: Fixes T13374. The "Temporary Failures" row is missing a cell definiton from the addition of "Average Queue Time".

Test Plan: Viewed "/daemon/" with some temporary failures and and odd number of rows above the "Temporary Failures" row. Saw cell properly zebra-striped.

Maniphest Tasks: T13374

Differential Revision: https://secure.phabricator.com/D20710
2019-08-12 12:46:48 -07:00
epriestley
0a3c26998f When the feed query on project profile pages overheats, contain the damage
Summary:
Ref T13349. This is almost the same change as D20678, but for project profiles instead of user profiles.

The general reproduction case is "view a project where you can't see more than 50 of the 500 most recent feed stories".

Test Plan:
  - Forced all queries to overheat.
  - Viewed a project profile page.
  - Before: overheating fatal near top level.
  - After: damage contained to feed panel.

Maniphest Tasks: T13349

Differential Revision: https://secure.phabricator.com/D20704
2019-08-08 21:01:50 -07:00
epriestley
9bd74dfa6c Autofocus the "App Code" input on the TOTP prompt during MFA gates after login
Summary: See downstream <https://phabricator.wikimedia.org/T229757>. The "autofocus" attribute mostly just works, so add it to this input.

Test Plan: As a user with TOTP enabled, established a new session. Saw browser automatically focus the "App Code" input on the TOTP prompt screen.

Differential Revision: https://secure.phabricator.com/D20703
2019-08-08 12:54:22 -07:00
epriestley
46d9065bf1 Drop test for awardable badges on "Badges" tab of user profiles to avoid overheating
Summary:
Fixes T13370. We currently show an "Award Badge" button conditionally, based on whether the viewer can award any badges or not.

The query to test this may overheat and this pattern isn't consistent with other UI anyway. Stop doing this test.

Test Plan:
  - Created 12 badges.
  - As a user who could not edit any of the badges, viewed the "Badges" section of a user profile.

Maniphest Tasks: T13370

Differential Revision: https://secure.phabricator.com/D20702
2019-08-08 10:18:54 -07:00
epriestley
937edcdc58 Fix a warning in BoardLayoutEngine when no objects are being updated
Summary: Fixes T13368. Some workflows (like "Move tasks to...") execute board layout without objects to update. In these cases, we can hit a warning because `objectPHIDs` is not initialized to `array()`.

Test Plan: Went through the "Move tasks to..." workflow on a workboard, no longer saw a warning when trying to iterate over an empty `objectPHIDs` list.

Maniphest Tasks: T13368

Differential Revision: https://secure.phabricator.com/D20701
2019-08-08 10:17:59 -07:00
epriestley
0561043a1f In "Move task to..." workflow, separate visible and hidden columns in the dropdown
Summary:
Ref T13368. Currently, both visible and hidden columns are shown in the "Move tasks to..." dropdown on workflows from workboards.

When the dropdown contains hidden columns, move them to a separate section to make it clear that they're not likely targets.

Test Plan:
  - Used "Move tasks to project..." targeting a board with no hidden columns. Saw a single ungrouped dropdown.
  - Used "Move tasks to project..." targeting a board with hidden columns. Saw a dropdown grouped into "Visible" and "Hidden" columns.

Maniphest Tasks: T13368

Differential Revision: https://secure.phabricator.com/D20700
2019-08-07 09:23:53 -07:00
epriestley
6deac35659 Don't show proxy (subproject/milestone) columns as options in "Move tasks..." workflows from workboards
Summary:
Ref T13368. Proxy columns should not be selectable from this workflow. If you want to move tasks to milestone/subproject X, do "Move tasks to project..." and pick X as the project.

(This could be made to work some day.)

Test Plan: Went through a "Move tasks to project..." workflow targeting a project with subprojects. No longer saw subproject columns presented as dropdown options.

Maniphest Tasks: T13368

Differential Revision: https://secure.phabricator.com/D20699
2019-08-07 09:23:29 -07:00
epriestley
31254c5124 Correct column options presented in "Move tasks to project..." on workboards
Summary: Ref T13368. The column options presented to the user are currently incorrect because the wrong set of columns are drawn from.

Test Plan: On a workboard, used "Move tasks to project..." to target another board, saw that board's columns.

Maniphest Tasks: T13368

Differential Revision: https://secure.phabricator.com/D20698
2019-08-07 09:22:57 -07:00
epriestley
87f878ec8a Stop trying to CC merchants on invoices/receipts
Summary:
Fixes T13341. Currently, cart emails (invoices/receipts) are sent to members of the associated merchant account. This was just a simple way to keep an eye on things when this was first written.

The system works fine, and recent changes (almost certainly D20525) stopped these emails from working (presumably because of the slightly weird merchant permissions model).

This could be sorted out in more detail, but it looks like the path forward is to introduce a side channel for email anyway (via T8389), and that's a better way to implement this behavior since it means the normal recipients won't see a bunch of random staff/merchant email addresses on their receipts.

Test Plan: Grepped for `merchant` in this editor.

Maniphest Tasks: T13341

Differential Revision: https://secure.phabricator.com/D20696
2019-08-02 10:51:17 -07:00
epriestley
6c41508906 Fix an issue where lines with more than one pattern match highlighted improperly in Diffusion
Summary:
Ref T13339. If a search pattern matches more than once on a line, we currently render the line incorreclty, duplicating some of the text.

`substr()` is being called as though the third parameter was `end_offset`, but it's actually `length`. Correct the parameter.

Test Plan:
Before:

{F6676625}

After:

{F6676623}

Maniphest Tasks: T13339

Differential Revision: https://secure.phabricator.com/D20695
2019-08-02 09:44:59 -07:00
epriestley
1fe6311167 Modernize user and repository "delete" workflows and improve documentation
Summary:
Fixes T8830. Fixes T13364.

  - The inability to destroy objects from the web UI is intentional. Make this clear in the messaging, which is somewhat out of date and partly reflects an earlier era when things could be destroyed.
  - `bin/remove destroy` can't rewind time. Document expectations around the "put the cat back in the bag" use case.

Test Plan: Read documentation, clicked through both workflows.

Maniphest Tasks: T13364, T8830

Differential Revision: https://secure.phabricator.com/D20694
2019-08-02 09:30:50 -07:00
epriestley
f5c380bfc9 Add very basic support for generating PDF documents
Summary: Ref T13358. This is very minimal, but technically works. The eventual goal is to generate PDF invoices to make my life easier when I have to interact with Enterprise Vendor Procurement.

Test Plan: {F6672439}

Maniphest Tasks: T13358

Differential Revision: https://secure.phabricator.com/D20692
2019-08-01 10:50:24 -07:00
epriestley
b81c8380fb Document support for "limit" in tokenizer-based Custom Fields
Summary:
Fixes T13356. This option is supported and works fine, it just isn't documented.

Add documentation and fix the config option to actually link to it to make life a little easier.

Test Plan: Read documentation.

Maniphest Tasks: T13356

Differential Revision: https://secure.phabricator.com/D20691
2019-07-31 13:13:24 -07:00
epriestley
8e263a2f64 Support "date" custom fields in "*.edit" endpoints
Summary: Fixes T13355. This didn't appear to be a ton of extra work, we just didn't get it for free in the original implementation in D14635.

Test Plan:
  - Saw "date" custom fields appear in Conduit API documentation for "maniphest.edit".
  - Set custom "date" field to null and non-null values via the API.

{F6666582}

Maniphest Tasks: T13355

Differential Revision: https://secure.phabricator.com/D20690
2019-07-31 13:10:14 -07:00
epriestley
76cd181bf3 Don't try to emit project board update events if there are no projects to update
Summary: Ref T4900. We may execute a bad query here if the task has no projects at all.

Test Plan: Edited a task with no new or old projects. Instead of an exception, things worked.

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20689
2019-07-31 12:48:41 -07:00
epriestley
47d497aa60 When users visit a Phame post URI with an old blog ID, canonicalize the URI instead of 404'ing
Summary:
Fixes T13353. If you:

  - Visit a blog post and save the URI.
  - Move the blog post to a different blog.
  - Revisit the old URI.

...we currently 404. We know what you're trying to do and should just redirect you to the new URI instead. We already do this if you visit a URI with a noncanonical slug.

Test Plan:
  - Created post A.
  - Copied the live URI.
  - Moved it to a different blog.
  - Visited the saved URI from the earlier step.
  - Before: 404.
  - After: Redirect to the canonical URI.

Maniphest Tasks: T13353

Differential Revision: https://secure.phabricator.com/D20688
2019-07-31 11:44:25 -07:00
epriestley
2ec39afcd1 Deprecate ancient "slowvote.info" API method
Summary: Depends on D20686. Fixes T13350. Now that "slowvote.poll.search" exists, deprecate this old method.

Test Plan: Reviewed method description in Condiut API console in the web UI.

Maniphest Tasks: T13350

Differential Revision: https://secure.phabricator.com/D20687
2019-07-31 11:28:08 -07:00
epriestley
f92480fb77 Fix two minor display issues with the Conduit "*.search" API documentation
Summary:
Depends on D20685. Ref T13350. Currently:

  - When a SearchEngine parameter is marked as hidden from Conduit, we may still render a table of possible values. Instead, only render the table if the parameter is actually usable.
  - The table header is hard-coded to say `'statuses'`, which is just a silly mistake. (Most commonly, this table does have `statuses` constants.)

Test Plan: Viewed the Conduit API documentation for the new "slowvote.poll.search" API method, saw more sensible display behavior.

Maniphest Tasks: T13350

Differential Revision: https://secure.phabricator.com/D20686
2019-07-31 11:27:05 -07:00
epriestley
0b0ab1bd7c Add a "slowvote.poll.search" API method
Summary: Ref T13350. Add a modern "*.search" API method for Slowvote so "slowvote.info" can be deprecated with a reasonable replacement.

Test Plan: Used Conduit test console to call method, saw reasonable results.

Maniphest Tasks: T13350

Differential Revision: https://secure.phabricator.com/D20685
2019-07-31 11:26:41 -07:00
epriestley
d81d0c3ea0 Fix an issue where editing cards on a workboard with implicit column ordering could reorder cards improperly
Summary:
Depends on D20680. Ref T4900. The "BoardLayoutEngine" operates on PHIDs without knowledge of the underlying objects, but this means it has to be sensitive to PHID input order when falling back to a default layout order.

We use "default layout order" on workboards which are sorted by "Natual" order but which have one or more cards which no user has ever reordered. For example, if you add 10 tasks to a project, then create a board, there's no existing order for those tasks in the "Backlog" column. The layout engine uses the input order to place them in the column, with the expectation that input order is ID/creation order, so new cards will end up on top.

I think this code never really made an explicit effort to guarantee that the LayoutEngine received objects in ID order, and it just sort of happened to by coincidence and good fortune. Some recent change has disrupted this, so the edit operation can end up with the PHIDs arranged in arbitrary order.

Explicitly put them in ID order so we always get an implicit default layout order to fall back to. Also, update to `msortv()`.

Test Plan:
  - Tagged several tasks with project X, a project without a board yet.
  - Created the project X workboard.
  - (Did not drag any tasks around on the project X board!)
  - Viewed the board in "Natural" order.

This creates a view of the board where tasks are ordered by implicit/virtual/input order. The expectation, and "view" behavior of this board, is that this order is "newest on top".

  - Edited one of the cards on the board, changing the title (don't reorder it!)
  - Before: page state synchronized with cards in arbitrary/random/different order.
  - After: page state synchronized with cards in the same order as before ("newest on top").

Reviewers: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20681
2019-07-30 13:17:30 -07:00
epriestley
7d41535010 When a task card is edited, emit update events for old boards and parent boards
Summary:
Ref T4900. When a card is edited, we currently emit an update notification for all the projects the task is tagged with. This isn't quite the right set:

  - We want to emit notifications for projects the task //was previously// tagged with, so it can be removed from boards it should no longer be part of.
  - We want to emit notifications for ancestors of projects the task is or was tagged with, so parent project boards can be updated.
  - However, we don't need to emit notifications for projects that don't actually have workboards.

Adjust the notification set to align better to these rules.

Test Plan:
  - Removal of Parent Project: Edited a task on board "A > B", removing the "B" project tag. Saw board A update in another window.
  - Normal Update: Edited a task title on board X, saw board X update in another window.
  - Used `bin/aphlict debug` to inspect the notification set, saw generally sensible-seeming data going over the wire.

Reviewers: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20680
2019-07-30 13:16:33 -07:00
epriestley
7e09da3313 Fix policy behavior of "slowvote.info" API method
Summary: Ref T13350. This ancient API method is missing modern policy checks.

Test Plan:
  - Set visibility of vote X to "Only: epriestley".
  - Called "slowvote.info" as another user.
  - Before: retrieved poll title and author.
  - After: policy error.
  - Called "slowvote.info" on a visible poll, got information before and after.

Maniphest Tasks: T13350

Differential Revision: https://secure.phabricator.com/D20684
2019-07-30 11:55:55 -07:00
epriestley
f6621a5fdc Tailor "Restart All Builds" for the complex realities of modern build restart rules
Summary:
Fixes T13348. Currently, the Harbormaster UI shows "Restart All Builds", but it really means "Restart Restartable Builds", which is often fewer than "All" builds (because of autobuilds, permissions, and/or configuration).

Remove the misleading term "All" and make the workflow preview exactly which builds will and will not be affected, and why.

Test Plan:
{F6636313}

{F6636314}

{F6636315}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13348

Differential Revision: https://secure.phabricator.com/D20679
2019-07-24 09:25:46 -07:00
epriestley
99c864f5e6 Provide a basic detail view for user activity logs
Summary:
Depends on D20673. Ref T13343. Since we're now putting log IDs in email, make the UI a little better for working with log IDs.

Some day, this page might have actions like "report this as suspicious" or whatever, but I'm not planning to do any of that for now.

Test Plan: {F6608631}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20674
2019-07-24 07:14:07 -07:00
epriestley
60db658d52 Record account recovery email links in the user activity log and make the mail message reference the log
Summary:
Depends on D20672. Ref T13343. When a user requests an account access link via email:

  - log it in the activity log; and
  - reference the log in the mail.

This makes it easier to ban users misusing the feature, provided they're coming from a single remote address, and takes a few steps down the pathway toward a button in the mail that users can click to report the action, suspend account recovery for their account, etc.

Test Plan:
  - Requested an email recovery link.
  - Saw request appear in the user activity log.
  - Saw a reference to the log entry in the mail footer.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20673
2019-07-24 07:13:34 -07:00
epriestley
57799bc82b Give user log types a tokenizer and datasource instead of a page of checkboxes
Summary: Depends on D20671. Ref T13343. Now that log types are modular, provide a datasource/tokenizer for selecting them since we already have a lot (even after I purged a few in D20670) and I'm planning to add at least one more ("Request password reset").

Test Plan: {F6608534}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20672
2019-07-24 07:11:42 -07:00
epriestley
32dd13d434 Modularize user activity log message types
Summary:
Depends on D20670. Ref T13343. The user activity message log types are currently hard-coded, so only upstream code can really use the log construct.

Under the theory that we're going to keep this log around going forward (just focus it a little bit), modularize things so the log is extensible.

Test Plan:
Grepped for `UserLog::`, viewed activity logs in People and Settings.

(If I missed something here -- say, misspelled a constant -- the effect should just be that older logs don't get a human-readable label, so stakes are very low.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20671
2019-07-24 07:10:18 -07:00
epriestley
6831ed94fa Contain fallout from overheating feed queries on user profile pages
Summary: Fixes T13349. If the user profile page feed query overheats, it currently takes the whole page with it. Contain the blast to a smaller radius.

Test Plan: {F6633322}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13349

Differential Revision: https://secure.phabricator.com/D20678
2019-07-24 07:09:08 -07:00
Arturas Moskvinas
cd44925425 Allow users with no CAN_EDIT permissions to silence projects if they want to
Summary: Humble user cannot silence/mute project if he/she has no CAN_EDIT permissions in it. You can actually leave it but if project is locked - then you're scr*wed.

Test Plan:
1. On a testing phabricator instance created a dummy project
2. Changed that project permissions CAN_EDIT to be by admin only
3. Added poor soul with no CAN_EDIT permissions
4. Logged it in with poor soul
5. Tried to silence the project
6. The Project is successfully silenced
7. User is happy :)

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, Pawka

Differential Revision: https://secure.phabricator.com/D20675
2019-07-23 13:13:54 +03:00
epriestley
4fd473e7ed Remove explicit administrative actions from the user activity log
Summary:
Depends on D20669. Ref T13343. Currently, the user activity log includes a number of explicit administrative actions which some administrator (not a normal user or a suspicious remote address) takes. In most/all cases, these changes are present in the user profile transaction log too, and that's //generally// a better place for them (for example, it doesn't get GC'd after a couple months).

Some of these are so old that they have no writers (like DELETE and EDIT). I'd generally like to modernize this a bit so we can reference it in email (see T13343) and I'd like to modularize the event types as part of that -- partly, cleaning this up makes that modularization easier.

There's maybe some hand-wavey argument that administrative vs non-administrative events could be related and might be useful to see in a single log, but I can't recall a time when that was actually true, and we could always build that kind of view later by just merging the two log sources, or by restoring double-writes for some subset of events. In practice, I've used this log mostly to look for obvious red flags when users report authentication difficulty (e.g., many unauthorized login attempts), and removing administrative actions from the log is only helpful in that use case.

Test Plan: Grepped for all the affected constants, no more hits in the codebase.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20670
2019-07-19 15:46:20 -07:00
epriestley
2ee5e71029 Simplify implementation of "SysetemAction->getSystemActionConstant()"
Summary: Depends on D20668. Ref T13343. Just an easy cleanup/simplification while I'm here.

Test Plan: `grep` for `getActionConstant()`

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20669
2019-07-19 15:45:37 -07:00
epriestley
a75766c0e5 Replace old rate limiting in password login flow with "SystemAction" rate limiting
Summary:
Depends on D20667. Ref T13343. Password auth currently uses an older rate limiting mechanism, upgrade it to the modern "SystemAction" mechanism.

This mostly just improves consistency, although there are some tangential/theoretical benefits:

  - it's not obvious that making the user log GC very quickly could disable rate limiting;
  - if we let you configure action limits in the future, which we might, this would become configurable for free.

Test Plan:
  - With CAPTCHAs off, made a bunch of invalid login attempts. Got rate limited.
  - With CAPTCHAs on, made a bunch of invalid login attempts. Got downgraded to CAPTCHAs after a few.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20668
2019-07-19 15:44:52 -07:00
epriestley
e090b32c75 Add a rate limit to requesting account recovery links from a given remote address
Summary:
Depends on D20666. Ref T13343. In D20666, I limited the rate at which a given user account can be sent account recovery links.

Here, add a companion limit to the rate at which a given remote address may request recovery of any account. This limit is a little more forgiving since reasonable users may plausibly try multiple variations of several email addresses, make typos, etc. The goal is just to hinder attackers from fishing for every address under the sun on installs with no CAPTCHA configured and no broad-spectrum VPN-style access controls.

Test Plan: {F6607846}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20667
2019-07-19 15:42:53 -07:00
epriestley
80294e7a4a Add a rate limit to generating new account recovery links for a given account
Summary:
Depends on D20665. Ref T13343. We support CAPTCHAs on the "Forgot password?" flow, but not everyone configures them (or necessarily should, since ReCAPTCHA is a huge external dependency run by Google that requires you allow Google to execute JS on your domain) and the rate at which any reasonable user needs to take this action is very low.

Put a limit on the rate at which account recovery links may be generated for a particular account, so the worst case is a trickle of annoyance rather than a flood of nonsense.

Test Plan: {F6607794}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20666
2019-07-19 15:42:21 -07:00
epriestley
ced416cc73 Allow Auth messages to have detailed descriptions and default values, then give "Email Login" both
Summary:
Depends on D20664. Ref T13343. There's a reasonable value for the default "Email Login" auth message (generic "you reset your password" text) that installs may reasonably want to replace. Add support for a default value.

Also, since it isn't completely obvious where this message shows up, add support for an extended description and explain what's going on in more detail.

Test Plan:
  - Viewed message detail page, saw more detailed information.
  - Sent mail (got default), overrode message and sent mail (got custom message), deleted message (got default again).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20665
2019-07-19 15:39:21 -07:00
epriestley
38d30af362 Give "Auth Messages" a view/detail state before users customize them
Summary:
Depends on D20663. Ref T13343. Currently, if an Auth message hasn't been customized yet, clicking the message type takes you straight to an edit screen to create a message.

If an auth message has already been customized, you go to a detail screen instead.

Since there's no detail screen on the "create for the first time" flow, we don't have anywhere to put a more detailed description or a preview of a default value.

Add a view screen that works if a message is "empty" so we can add this stuff.

(The only reason we don't already have this is that it took a little work to build; this also generally improves the consistency and predictability of this interface.)

Test Plan: {F6607665}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20664
2019-07-19 15:38:45 -07:00
epriestley
a0c9f9f90c Allow installs to customize mail body guidance in the "Email Login" and "Set Password" emails
Summary:
Depends on D20662. Ref T13343. Installs may reasonably want to change the guidance users receive in "Email Login"/"Forgot Password" email.

(In an upcoming change I plan to supply a piece of default guidance, but Auth Messages need a few tweaks for this.)

There's probably little reason to provide guidance on the "Set Password" flow, but any guidance one might issue on the "Email Login" flow probably doesn't make sense on the "Set Password" flow, so I've included it mostly to make it clear that this is a different flow from a user perspective.

Test Plan:
  - Set custom "Email Login" and "Set Password" messages.
  - Generated "Email Login" mail by using the "Login via email" link on the login screen.
  - Generated "Set Password" email by trying to set a password on an account with no password yet.
  - Saw my custom messages in the resulting mail bodies.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20663
2019-07-19 15:37:43 -07:00
epriestley
5dd4895001 Move "Password Reset" email to "PeopleMailEngine"
Summary:
Ref T13343. This makes "Password Reset" email a little more consistent with other modern types of email. My expectation is that this patch has no functional changes, just organizes code a little more consistently.

The new `setRecipientAddress()` mechanism deals with the case where the user types a secondary (but still verified) address.

Test Plan:
  - Sent a normal "login with email" email.
  - Sent a "login with email to set password" email by trying to set a password on an account with no password yet.
  - Tried to email reset a bot account (no dice: they can't do web logins so this operation isn't valid).
  - Tested existing "PeopleMailEngine" subclasses:
    - Created a new user and sent a "welcome" email.
    - Renamed a user and sent a "username changed" email.
  - Reviewed all generated mail with `bin/mail list-outbound`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13343

Differential Revision: https://secure.phabricator.com/D20662
2019-07-19 15:30:34 -07:00
epriestley
f55aac49f4 Rename "pastebin" database to "paste"
Summary:
See D20650. Long ago, this got added as "pastebin", but that's the name of another product/company, not a generic term for paste storage.

Rename the database to `phabricator_paste`.

(An alternate version of this patch would rename `phabricator_search` to `phabricator_bing`, `phabricator_countdown` to `phabricator_spacex`, `phabricator_pholio` to `phabricator_adobe_photoshop`, etc.)

Test Plan:
  - Grepped for `pastebin`, now only found references in old patches.
  - Applied patches.
  - Browsed around Paste in the UI without encountering issues.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20661
2019-07-19 05:53:26 -07:00
epriestley
17caecdda3 Make workboard real-time updates mostly work
Summary:
Depends on D20654. Ref T4900. When a task is edited, emit a "workboards" event for all boards it appears on (in a future change, this should also include all boards it //previously// appeared on, and all parents of both sets of boards -- but I'm just getting things working for now).

When we receive a "workboards" event, check if the visible board should be updated.

Aphlict has a complicated intra-window leader/follower election system which could let us process this update event exactly once no matter how many windows a user has open with the same workboard. I'm not trying to do any of this since it seems fairly rare. It makes sense for events like "you have new notifications" where we don't want to generate 100 Ajax calls if the user has 100 windows open, but very few users seem likely to have 100 copies of the same workboard open.

Test Plan:
  - Ran `bin/aphlict debug`.
  - Opened workboard A in two windows, X and Y.
  - Edited and moved tasks in window X.
  - Saw "workboards" messages in the Aphlict log.
  - Saw window Y update in nearly-real-time (locally, this is fast enough that it feels instantaneous).

Then:

  - Stopped the Aphlcit server.
  - Edited a task.
  - Started the Aphlict server.
  - Saw window Y update after a few moments (i.e., update in response to a reconnect).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20656
2019-07-18 10:00:17 -07:00
epriestley
9ab5f59ca2 Export "date" and "remarkup" custom fields to Excel + "zip" extension check
Summary:
Fixes T13342. This does a few different things, although all of them seem small enough that I didn't bother splitting it up:

  - Support export of "remarkup" custom fields as text. There's some argument here to export them in some kind of structure if the target is JSON, but it's hard for me to really imagine we'll live in a world some day where we really regret just exporting them as text.
  - Support export of "date" custom fields as dates. This is easy except that I added `null` support.
  - If you built PHP from source without "--enable-zip", as I did, you can hit the TODO in Excel exports about "ZipArchive". Since I had a reproduction case, test for "ZipArchive" and give the user a better error if it's missing.
  - Add a setup check for the "zip" extension to try to avoid getting there in the first place. This is normally part of PHP so I believe users generally won't hit it, I just hit it because I built from source. See also T13232.

Test Plan:
  - Added a custom "date" field. On tasks A and B, set it to null and some non-null value. Exported both tasks to Excel/JSON/text, saw null and a date, respectively.
  - Added a custom "remarkup" field, exported some values, saw the values in Excel.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13342

Differential Revision: https://secure.phabricator.com/D20658
2019-07-18 09:59:20 -07:00
epriestley
d02beaf816 Make reloading workboards with "R" respect workboard ordering
Summary:
Depends on D20653. Ref T4900. Pass ordering details to the reload endpoint so it can give the client accurate ordering/header information in the response.

The removed comment mentions this, but here's why this is a difficult mess:

  - In window A, view a board with "Group by: Owner" and no tasks owned by "Alice". Since "Alice" owns no tasks, this means the columns do not have an "Assigned to: Alice" header!
  - In window B, edit task T and assign it to Alice.
  - In window A, press "R".

Window A now not only needs to update to properly reflect the state of task T, it actually needs to draw a new "Assigned to: Alice" header in every column.

Fortunately, the "group by" code anticipates this being a big mess, is fairly careful about handling it, and the client can handle this state change and the actual code change here isn't too involved. This is just causing a lot of not-very-obvious indirect effects in the pipeline to handle these situations that need complex redraws.

Test Plan:
  - After making various normal edits/creates/moves in window A, pressed "R" in window B. Saw ordering reflected correctly after sync.
  - Went through the whole "Group by: Owner" + assign to unrepresented owner flow above. After pressing "R", saw "Assigned to: Alice" appear on the board.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20654
2019-07-17 13:17:00 -07:00
epriestley
8669c3c0d2 When updating a workboard with "R", send the client visible set with version numbers
Summary:
Depends on D20652. Ref T4900. When the user presses "R", send a list of cards currently visible on the client and their version numbers.

On the server:

  - Compare the client verisons to the server versions so we can skip updates for objects which have not changed. (For now, the client version is always "1" and the server version is always "2", so this doesn't do anything meaningful, and every card is always updated.)
  - Compare the client visible set to the server visible set and "remove" any cards which have been removed from the board.

I believe this means that "R" always puts the board into the right state (except for some issues with client orderings not being fully handled yet). It's not tremendously efficient, but we can make versioning better (using the largest object transaction ID) to improve that and loading the page in the first place doesn't take all that long so even sending down the full visible set shouldn't be a huge problem.

Test Plan:
  - In window A, removed a card from a board.
  - In window B, pressed "R" and saw the removal reflected on the client.
  - (Also added cards, edited cards, etc., and didn't catch anything exploding.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20653
2019-07-17 13:16:03 -07:00
epriestley
1ee6ecf397 Move "BoardResponseEngine" toward a more comprehensive update model
Summary:
Depends on D20639. Ref T4900. Currently, "BoardResponseEngine" has a `setObjectPHID()` method. This is called after edit operations to mean "we just edited object X, so we know it needs to be updated".

Move toward `setUpdatePHIDs(...)` in all cases, with `setUpdatePHIDs(array(the-object-we-just-edited))` as a special case of that. After this change, callers pass:

  - An optional list of PHIDs they know need to be updated on the client. Today, this is always be a card we just edited (on edit/move flows), or a sort of made-up list of PHIDs for the moment (when you press "R"). In the future, the "R" endpoint will do a better job of figuring out a more realistic update set.
  - An optional list of PHIDs currently visible on the client. This is used to update ordering details and mark cards for removal. This is currently passed by edit/move, but not by pressing "R" (it will be in the future).
  - An optional list of objects. The "R" workflow has to load these anyway, so we can save a couple queries by letting callers pass them. For now, the edit/move flows still rely on the engine to figure out what it needs to load.

This does very little to actually change client behavior, it mostly just paves the way for the next update to the "R" workflow to make it handle add/remove cases properly.

Test Plan:
  - Edited and moved cards on a workboard.
  - Pressed "R" to reload a workboard.

Neither of these operations seem any worse off than they were before. They still don't fully work:

  - When you edit a card and delete the current workboard project from it, it remains visible. This is also the behavior on `master`. This is sort of intentional since we don't necessarily want to make these cards suddenly disappear? Ideally, we would probably have some kind of "tombstone" state where the card can still be edited but can't be dragged, and the next explicit user interaction would clean up old tombstones. This interaction is very rare and I don't think it's particularly important to specialize.
  - When a card is removed from the board, "R" can't currently figure out that it should be removed from the client. This is because the client does not yet pass a "visiblePHIDs" state. It will in an upcoming change.
  - The "R" flow always sends a full set of card updates, and can not yet detect that some cards have not changed.
  - There's a TODO, but some ordering stuff isn't handled yet.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20652
2019-07-17 13:13:15 -07:00
epriestley
db69686927 Make pressing "R" on your keyboard reload the card state on workboards
Summary:
Depends on D20638. Ref T4900. This is an incremental step toward proper workboard updates.

Currently, the client can mostly update its view because we do updates when you edit or move a card, and the client and server know how to send lists of card updates, so a lot of the work is already done.

However, the code assumes we're only updating/redrawing one card at a time. Make the client accept and process multiple card updates.

In future changes, I'll add versioning (so we only update cards that have actually changed), fix the "TODO" around ordering, and move toward actual Aphlict-based real-time updates.

Test Plan:
  - Opened the same workboard in two windows.
  - Edited cards in one window, pressed "R" (capital letter, with no modifier keys) to reload the second window.
  - Saw edits and moves reflected accurately after sync, except for some special cases of header/order interaction (see "TODO").

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20639
2019-07-17 13:11:26 -07:00
Austin McKinley
97c1699756 Fix transaction title rendering for AuthenticationConfigs
Summary: I was poking around in `PhabricatorAuthProviderViewController` and noticed that none of the subclass-specific rendering was working. Figured out that no one ever calls `PhabricatorAuthProviderConfigTransaction->setProvider()`, so instead of adding all those calls, just pull the provider out of the config object.

Test Plan:
Before: {F6598145}
After: {F6598147}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20655
2019-07-17 12:41:00 -07:00
Austin McKinley
2f313a0e0d Remove "unstable" status and T2784-specific warning message
Summary: Ref T2784. These are lookin' pretty stable. Subclasses like `DiffusionGetLintMessagesConduitAPIMethod` have their warnings about unstable methods, so just remove this warning in the base class.

Test Plan: Loaded `/conduit`, observed lack of unstable warnings. Only unstable methods are now `diffusion.getlintmessages`, `diffusion.looksoon`, and `diffusion.updatecoverage`.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T2784

Differential Revision: https://secure.phabricator.com/D20651
2019-07-15 14:05:26 -07:00
Austin McKinley
7852adb84b Actually enforce auth.lock-config
Summary: Forgot to post this after D20394. Fixes T7667.

Test Plan:
    * Edited some providers with the config locked and unlocked.
    * Opened the edit form with the config unlocked, locked the config, then saved, and got a sensible error: {F6576023}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T7667

Differential Revision: https://secure.phabricator.com/D20645
2019-07-15 11:52:55 -07:00
epriestley
d2935fd7bd Fix a bad call to "writeInfo()" in "bin/phd stop" with no PHABRICATOR_INSTANCE defined
Summary: See <https://discourse.phabricator-community.org/t/phd-status-calls-to-undefined-method-when-theres-no-instance/2918>. This call should be `logInfo()`.

Test Plan:
  - Purged `PHABRICATOR_INSTANCE` from my environment. In a Phacility development environment, it comes from loading `services/`.
  - Ran `bin/phd stop` with all daemons already stopped.
    - Before: bad call.
    - After: helpful error.
  - Ran some other `bin/phd start`, `bin/phd status`, etc., to kick the tires.
  - Grepped for remaining `writeInfo()` calls (found none).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20649
2019-07-12 09:15:17 -07:00
epriestley
41ea204144 Update one straggling "CAN_INTERACT" check in comment removal
Summary: See rPaacc62463d61. D20551 added some `CAN_INTERACT` checks, but `CAN_INTERACT` needs to be checked with `canInteract()` to fall back to `CAN_VIEW` properly. D20558 cleaned up most of this but missed one callsite; fix that up too.

Test Plan: Removed a comment on a commit.

Reviewers: amckinley, 20after4

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20648
2019-07-11 16:09:52 -07:00
epriestley
099919366b Fix "add more metadata" fatal in Pholio
Summary: Ref T13332. This fix isn't terribly satisfying, but resolves the issue: this behavior may attempt to build HTML blocks with metadata after Javascript footer rendering has started. Use `hsprintf()` to flatten the markup earlier.

Test Plan: Put a `T123` reference in the description of a Pholio image, then loaded a mock.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13332

Differential Revision: https://secure.phabricator.com/D20647
2019-07-11 15:55:56 -07:00
Austin McKinley
2c435433e0 Start fleshing out PhabricatorAuthProviderViewController
Summary:
Ref D20645. Start making this view a little more useful:

{F6573605}

Test Plan: Mk. 1 eyeball

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20646
2019-07-10 10:45:31 -07:00
Austin McKinley
3c43222525 Fix paging fatal with flagged objects
Summary: Fixes T13331. Just adds a generic `withIDs()` method to `PhabricatorFlagQuery`.

Test Plan: Flagged > 100 objects, observed fatal attempting to page. Next page loads as expected after fix.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T13331

Differential Revision: https://secure.phabricator.com/D20642
2019-07-05 14:25:06 -07:00
epriestley
45f4211541 Fix URI escaping, which should actually be "%s", not "%p"
See <https://discourse.phabricator-community.org/t/projects-workboards-links-issue/2896>.

The documentation on `urisprintf()` isn't very clear here, I'll update it in
a followup. "%p" is for cases like encoding a branch name (which may contain
slashes) as a single path component in a URI.
2019-07-04 11:00:04 -07:00
epriestley
2de8a23adb Remove remnants of clumsy old URI state handling from workboards
Summary:
Depends on D20636. Ref T4900. Previously, some workflows didn't know how to identify the default state for the board, so they needed explicit ("force") parameters.

Everything uses the same state management code now so we can rip out the old stuff.

Test Plan: Changed board filters, selected a custom filter, edited a custom filter.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20637
2019-07-03 10:05:34 -07:00
epriestley
58e2fa0d47 Differentiate between "Move Tasks to Column..." and "Move Tasks to Project..." in the workboard UI
Summary:
Depends on D20635. Ref T4900. Fixes T13316.

Currently, "Move Tasks to Column..." first prompts you to select a project, then prompts you for a column. The first step is prefilled with the current project, so the common case (moving to another column on the same board) requires you to confirm that you aren't doing an off-project move by clicking "Continue", then you can select a column.

This isn't a huge inconvenience and the workflow isn't terribly common, but it's surprising enough that it has come up a few times as a stumbling block. Particularly, we're suggesting to users that they're about to pick a column, then we're asking them to pick a project. The prompt also says "Project: XYZ", not "Project: Keep in current project" or something like that.

Smooth this out by splitting the action into two better-cued flows:

  - "Move Tasks to Project..." is the current flow: pick a project, then pick a column.
    - The project selection no longer defaults to the current project, since we now expect you to usually use this flow to move tasks to a different project.
  - "Move Tasks to Column..." prompts you to select a column on the same board.
    - This just skips step 1 of the workflow.
    - This now defaults to the current column, which isn't a useful selection, but is more clear.

In both cases, the action cue ("Move tasks to X...") now matches what the dialog actually asks you for ("Pick an X").

Test Plan:
  - Moved tasks across projects and columns within the same project.
  - Hit all (I think?) the error cases and got sensible error and recovery behavior.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13316, T4900

Differential Revision: https://secure.phabricator.com/D20636
2019-07-03 10:02:44 -07:00
epriestley
24b466cd62 Move workboard "Move Tasks to Column..." workflow to a separate controller
Summary: Depends on D20634. Ref T4900. Ref T13316. I'm planning to do a bit of additional cleanup here in followups, but this separates the main workflow out of the common controller.

Test Plan:
  - Used "Move Tasks to Column..." to move some tasks on a board.
  - Tried to move an empty column, hit an error.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13316, T4900

Differential Revision: https://secure.phabricator.com/D20635
2019-07-02 15:16:38 -07:00
epriestley
ec352b1b31 Move workboard "Bulk Edit Tasks" workflow to a separate controller
Summary: Depends on D20633. Ref T4900. Separate the "Bulk Edit Tasks..." flow out of the main workboard controller.

Test Plan:
  - Used "Bulk Edit Tasks" on a column with some tasks, got an appropraite edit operation.
  - Used "Bulk Edit Tasks" on an empty column, got an error.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20634
2019-07-02 15:04:38 -07:00
epriestley
9ea7227f0f Move workboard "View as Query" workflow to a separate controller
Summary:
Depends on D20632. Ref T4900. As with other workflows on the board controller, this one is currently in the giant main "do everything" method. Move it to a separate controller.

This makes one material improvement: previously, we built the full board and did layout on all the cards before building the query. However, we do not actually need to do this: we don't need the cards. Instead, just do layout without handing over any card PHIDs. This is slightly faster, particularly on large boards.

Test Plan:
  - Clicked "View as Query" on a board, got a query page for the column.
  - Applied a custom filter, then clicked "View as Query" on a board. Got a query page merging the two filters.
  - Applied a custom filter, then clicked "Veiw as Query" on a board, in a subproject column. Got a query page merging the two filters, respecting the project-ness of the column.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20633
2019-07-02 15:00:36 -07:00
epriestley
577020aea9 Move workboard "filter" workflow to a separate controller
Summary:
Depends on D20629. Ref T4900. Currently, the "Advanced Filter..." workflow on workboards (where you build a custom query) is inline in the main board controller.

This is because the filter flow depends on some of the board view state: we want to start with the current filter applied to the board, and preserve other state after you change the filter.

Now that `ViewState` can handle state management, we can separate this stuff out pretty easily.

Test Plan:
  - Changed filters on a board.
  - Applied a custom filter to a board.
  - Changed the ordering of a board, then applied a custom filter. Verified "Cancel" and "Apply Filter" both preserve the order state.
  - Changed the ordering of a board, then applied a custom filter, intentionally making a mistake in configuring the filter by entering an invalid date. Saw a dialog with an error. After correcting the error, saw state preserved properly.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20632
2019-07-02 14:39:34 -07:00
epriestley
9e096cd274 Give the workboard "default" workflows more modern state handling
Summary:
Depends on D20628. Ref T4900. Currently, the "Save Current Order/Filter As Default" flows on workboards duplicate some state construction, and require parameters to be passed to them explicitly.

Now that state management is separate, they can reuse a bit more code and be made to look more like other similar controllers.

Test Plan:
  - Changed the default order of a workboard.
  - Changed the default filter of a workboard.
  - Changed the order of a board to something non-default, then changed the filter, then saved the new filter as the default. Saw the modified order preserved and the modified filter removed, so I ended up in the right ("most correct") place: on the board, with my custom order in a URI parameter, and no filter URI parameter so I could see my new default filter behavior. This is an edge case that's not terribly important to get right, but we do get it right.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20629
2019-07-02 14:36:15 -07:00
epriestley
9c190d68ed Separate workboard view state (ordering, filtering, hidden columns) from the View controller
Summary:
Depends on D20627. Ref T4900. If a user orders a board by "Sort by Title", then toggles the visibility of hidden columns, we want to keep the board sorted by title. To accomplish this, we pass the board state around to all the workflows here.

Pull the "bag of state properties" code out of the View controller. This class basically:

  - reads state from a request (order, hidden, filter);
  - manages defaults;
  - provides the application with the current settings; and
  - generates URIs with "?order=X&hidden=Y&filter=Z" to preserve state.

This is still a little questionable/transitional since some of the controllers need more cleanup.

Test Plan:
Toggled state, order, filters, clicked around various workflows and saw the filters preserved.

A lot of these workflows are pretty serious edge cases. For example, here's a feature this implements:

  - Changed workboard order to "Title".
  - Selected "Bulk Edit Tasks..." in an empty column and command-clicked it to open the link in a new window.
  - Hovered over "Cancel".
  - Saw the link properly generate with "?order=title", preserving the order.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20628
2019-07-02 14:34:33 -07:00
epriestley
0ae9e2c75d Remove property "id" from Workboard View controller
Summary: Depends on D20626. Ref T4900. On this controller, "id" is a separate property, but serves little purpose and complicates separating state management. Remove it.

Test Plan: Bulk edited a column, managed filters, did show/hide on columns, edited a column.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20627
2019-07-02 05:17:01 -07:00
epriestley
df29b82ad6 Remove unused property "slug" from Workboard View controller
Summary:
Ref T4900. The Workboard view controller currently has a lot of different responsibilities (it's ~1,500 lines long) because it has to manage the board filter/sort state.

I'd like to split it up and make it easier to move some workboard features (like "move all tasks in column...") to other Controllers, so we can have smaller controllers implementing specific workflows.

I think the state handling isn't really all that bad, it just needs to be separated a little better than it currently is.

To start with, remove the unused "slug" property.

Test Plan: Searched for "slug", got no hits. This class is final and the property is private, so this is certainly unused.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T4900

Differential Revision: https://secure.phabricator.com/D20626
2019-07-02 05:16:34 -07:00
epriestley
d22f6d219b Lightly modernize OAuth server application view pages
Summary: Depends on D20624. Fixes T13330. The OAuth client pages are using some out-of-date rendering conventions; update them to modern conventions.

Test Plan:
Viewed a page, saw a modern header layout + curtain:

{F6534135}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13330

Differential Revision: https://secure.phabricator.com/D20625
2019-07-02 05:15:37 -07:00
epriestley
0e2cb6e7c4 Fix missing URI for "OAuthServerClient" object handles, causing dialog with no button
Summary:
Ref T13330. Handles for "OAuthServerClient" objects currently do not have a URI, which causes some obscure fallout like a missing "Close" button when examining their transactions.

Add a URI.

Test Plan:
  - Viewed an OAuth server client detail page.
  - Edited a policy, changing it to a custom policy.
  - Clicked "Custom Policy" in the resulting transaction to view a dialog explaining the changes.
  - Before change: dialog has no close button.
  - After change: dialog has a close button.

{F6534121}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13330

Differential Revision: https://secure.phabricator.com/D20624
2019-07-02 05:14:14 -07:00
epriestley
159fd44203 Correct transaction strengths after inconsitent scaling by 100 vs 1000
Summary:
See D20540. I mistakenly multiplied some strenghts by 100 and others by 1000 when converting them to integers for `PhutilSortVector`.

Multiply them all by 100 (that is, divide the ones which were multiplied by 1000 by 10) to put things back the way they were.

Test Plan: quick mafs

Reviewers: amckinley, richardvanvelzen

Reviewed By: richardvanvelzen

Differential Revision: https://secure.phabricator.com/D20622
2019-06-26 07:19:33 -07:00
epriestley
da0dfc057d Make "bin/files" parsing of working set arguments more consistent
Summary:
Fixes T13326. In D20571, I slightly generalized construction of an iterator over a set of files, but missed some code in other "bin/files ..." commands which was also affected.

Today, basically all of these workflows define their own "--all" and "names" flags. Pull these definitions up and implement them more consistently.

Test Plan: Ran multiple different `bin/files` commands with different combinations of arguments, saw consistent handling of iterator construction.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13326

Differential Revision: https://secure.phabricator.com/D20614
2019-06-24 16:02:39 -07:00
epriestley
a3397fb876 Consider "all account members are disabled" to be a permanent failure when billing a Phortune subscription
Summary:
Fixes T13327. Currently, when we try to bill an account and all members are disabled, we fail temporarily and the task retries forever.

At least for now, just treat this as a permanent failure.

Test Plan:
  - Used `bin/phortune invoice` to generate a normal invoice for a regular subscription.
  - Disabled all the account members, then tried again. Got a helpful permanent failure:

```
$ ./bin/phortune invoice --subscription PHID-PSUB-kbedwt5cyepoc6tohjq5 --auto-range
Set current time to Mon, Jun 24, 2:47 PM.
Preparing to invoice subscription "localb.phacility.com" from Fri, May 31, 10:14 AM to Sun, Jun 30, 10:14 AM.
 WARNING
Manually invoicing will double bill payment accounts if the range overlaps an
existing or future invoice. This script is intended for testing and
development, and should not be part of routine billing operations. If you
continue, you may incorrectly overcharge customers.

    Really invoice this subscription? [y/N] y

[2019-06-24 14:47:57] EXCEPTION: (PhabricatorWorkerPermanentFailureException) All members of the account ("PHID-ACNT-qp54y3unedoaxgkkjpj4") for this subscription ("PHID-PSUB-kbedwt5cyepoc6tohjq5") are disabled. at [<phabricator>/src/applications/phortune/worker/PhortuneSubscriptionWorker.php:88]
arcanist(head=experimental, ref.master=d92fa96366c0, ref.experimental=db4cd55d4673), corgi(head=master, ref.master=6371578c9d32), instances(head=stable, ref.master=ba9e4a19df1c, ref.stable=37fb1f4917c7), libcore(), phabricator(head=master, ref.master=65bc481c91de, custom=11), phutil(head=master, ref.master=7adfe4e4f4a3), services(head=master, ref.master=5424383159ac)
  #0 PhortuneSubscriptionWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:124]
  #1 PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:163]
  #2 PhabricatorWorker::scheduleTask(string, array, array) called at [<phabricator>/src/applications/phortune/management/PhabricatorPhortuneManagementInvoiceWorkflow.php:169]
  #3 PhabricatorPhortuneManagementInvoiceWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:457]
  #4 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:349]
  #5 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/setup/manage_phortune.php:21]
$
```

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13327

Differential Revision: https://secure.phabricator.com/D20613
2019-06-24 15:29:31 -07:00
epriestley
65bc481c91 Remove "phd.pid-directory" configuration and stop passing "piddir" to daemons
Summary: Ref T13321. The daemons no longer write PID files, so we no longer need to pass any of this stuff to them.

Test Plan: Grepped for affected symbols.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20608
2019-06-24 11:28:58 -07:00
epriestley
2498e373b9 Make "phd start" and "phd reload" use the process list, not PID files
Summary:
Ref T13321. This gets rid of the last pidfile readers in Phabricator; we just use the process list instead.

These commands always only work on the current instance since they don't make much sense otherwise.

Test Plan: Ran `bin/phd start` and `bin/phd reload` with and without daemons running.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20606
2019-06-24 11:27:25 -07:00
epriestley
08b9e70bea Make "bin/phd status" report local daemons from the process list, not a mess of local/remote information
Summary:
Ref T13321. Fixes T11037. Realign "bin/phd status" to just mean "show daemon processes on this host".

The value of `bin/phd status` as a mixed remote/local command isn't clear, and the current output is a confusing mess (see T11037).

This also continues letting us move away from PID files.

Test Plan: Ran `bin/phd status`, saw sensible local process status.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321, T11037

Differential Revision: https://secure.phabricator.com/D20604
2019-06-24 11:25:53 -07:00
epriestley
b99c240aa3 Deprecate "bin/phd ... --gently" and update documentation
Summary:
Ref T13321. Previously, the behavior was:

  - `bin/phd stop --gently`: Stop all daemons with PID files that belong to the current instance.
  - `bin/phd stop`: Stop all daemons with PID files that belong to the current instance. Complain if there are more processes.
  - `bin/phd stop --force`: Stop all processes that look like daemons, ignoring instances.

The new behavior is:

  - `bin/phd stop`: Stop all processes that look like daemons and belong to the current instance.
  - `bin/phd stop --force`: Stop all processes that look like daemons, period.

Test Plan: Grep / documentation only.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20602
2019-06-24 11:16:00 -07:00
epriestley
d98bf8ef8e Drive "phd stop" entirely from the process list, not PID files on disk
Summary:
Ref T13321. Depends on D20600. Make `bin/phd stop` mean:

  - `bin/phd stop`: Stop all processes which have daemon process titles. If we're instanced, only stop daemons for the current instance.
  - `bin/phd stop --force`: Stop all processes which have deamon process titles for any instance.

We no longer read or care about PID files on disk, and this moves us away from PID files.

This makes unusual flag `--gently` do nothing. A followup will update the documentation and flags to reflect actual usage/behavior.

This also removes the ability to stop specific PIDs. This was somewhat useful long, long ago when you might explicitly run different copies of the `PullLocal` daemon with flags to control which repositories they updated, but with the advent of clustering it's no longer valid to run custom daemon loadouts.

Test Plan: Ran `bin/phd start`, then `bin/phd stop`. Saw instance daemons stop. Ran `bin/phd stop --force`, saw all daemons stop.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20601
2019-06-24 11:05:56 -07:00
epriestley
75c3598359 Require commit identities when editing commits to resolve an issue with audit actions not applying properly
Summary:
See <https://discourse.phabricator-community.org/t/cannot-audit-a-git-commit/2848>. In D20581, I made some audit behavior dependent upon identities, but the actual edit flow doesn't load them. This can cause us to raise an "attach identities first" exception in the bowels of the edit workflow and trigger unexpected behavior at top level.

Load identities when editing a commit so that the transaction flows have access to identity information and can use it to figure out if a user is an author, etc.

Test Plan:
  - As an auditor, applied an "Accept Commit" action to an open audit after D20581.
  - Before patch: accept no-ops internally since the preconditions throw.
  - After patch: accept works properly.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20612
2019-06-24 10:50:23 -07:00
epriestley
ca56e8590a Don't handle JIRA/Asana URIs with anchors or query parameters in a special way (with Doorkeeper)
Summary:
Ref T13291. See PHI1312. Currently, if you link to a JIRA or Asana issue with an anchor (`#asdf`) or query parameters (`?a=b`), we:

  - treat the link as an external object reference and attempt a lookup on it;
  - if the lookup succeeds, we discard the fragment or parameters when re-rendering the rich link (with the issue/task title).

Particularly, the re-rendering part uses the canonical URI of the object, and can discard these parameters/fragments, which is broken/bad.

As a first pass at improving this, just don't apply special behavior for links with anchors or parameters -- simply treat them as links.

In some future change, we could specialize this behavior and permit certain known parameters or anchors or something, but these use cases are likely fairly marginal.

Test Plan:
Before:

{F6516392}

After:

{F6516393}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291

Differential Revision: https://secure.phabricator.com/D20592
2019-06-21 06:34:44 -07:00
Austin McKinley
6b9f4a918b Modularize PhabricatorEditEngineConfigurationTransaction
Summary: Ref T13319. Ref PHI1302. Migrate `PhabricatorEditEngineConfigurationTransaction` to modular transactions and add some additional transaction rendering to make these edits less opaque.

Test Plan: Hit all the form edit controllers, viewed resulting transaction timeline.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T13319

Differential Revision: https://secure.phabricator.com/D20595
2019-06-20 16:25:21 -07:00
epriestley
c0dc411d23 Update "phabricator/" for "topological" API changes
Summary: Ref T13325.

Test Plan:
  - Grepped for `topograph`.
  - Viewed a task graph since that's easy, looked fine.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13325

Differential Revision: https://secure.phabricator.com/D20599
2019-06-20 16:11:56 -07:00
epriestley
53f8ad14fa Fix an issue in Owners where a transaction change could show too many effects
Summary:
Fixes T13324. Ref PHI1288. Currently, if you edit an Owners package that has some paths with no trailing slashes (like `README.md`) so their internal names and display names differ (`/README.md` display, vs `/README.md/` internal), the "Show Details" in the transaction log shows the path as re-normalized even if you didn't touch it.

Instead, be more careful about handling display paths vs internal paths.

(This code on the whole is significantly less clear than it probably could be, but this issue is so minor that I'm hesitant to start ripping things out.)

Test Plan:
  - In a package with some paths like `/src/` and some paths like `/src`:
  - Added new paths.
  - Removed paths.
  - Changed paths from `/src/` to `/src`.
  - Changed paths from `/src` to `/src/`.

In all cases, the "paths" list and the transaction record identically reflected the edit in the way I expected them to.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13324

Differential Revision: https://secure.phabricator.com/D20596
2019-06-20 16:08:31 -07:00
epriestley
37e26f1b45 Improve rendering of "default value changed" custom form transactions to at least have all the information
Summary:
Ref T13319. Currently, transactions about changes to a default form value use a raw internal key for the affected field and don't show the actual value change.

An ideal implementation will likely require us to specialize a great deal of rendering, but we can do much better than we currently do without too much work:

  - Try to pull the actual `EditField` object for the key so we can `getLabel()` it and get a human-readable label (like `Visible To` instead of `policy.view`).
  - Add a "(Show Changes)" action that dumps the raw values as more-or-less JSON, so you can at least figure out what happened if you're sophisticated enough.

Test Plan:
Before:

{F6516640}

After:

{F6516642}

The quality of "Show Details" varies a lot. For some fields, like "Description", it's pretty good:

{F6516645}

For others, like "Assigned To", it's better than nothing but pretty technical:

{F6516647}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13319

Differential Revision: https://secure.phabricator.com/D20594
2019-06-19 13:47:07 -07:00
epriestley
caaa1394ef Don't count "Cc: x@y.com" as a legitimate recipient if the user who has "x@y.com" attached to their account has not verified the address
Summary:
Fixes T13317. On `admin.phacility.com`, an enterprising user added `noreply@admin.phacility.com` to their account. This caused them to become CC'd on several support issues over the last year, because we send mail "From" this address and it can get CC'd via reply/reply all/whatever else.

The original driving goal here is that if I reply to a task email and CC you on my reply, that should count as a CC in Phabricator, since this aligns with user intent and keeps them in the loop.

This misfire on `noreply@` is ultimately harmless (being CC'd does not grant the user access permission, see T4411), but confusing and undesirable. Instead:

  - Don't allow reserved addresses ("noreply@", "ssladmin@", etc) to trigger this subscribe-via-CC behavior.
  - Only count verified addresses as legitimate user recipients.

Test Plan:
  - Added a `bin/mail receive-test --cc ...` flag to make this easier to test.
  - Sent mail as `bin/mail receive-test --to X --as alice --cc bailey@verified.com`. Bailey was CC'd both before and after the change.
  - Sent mail as `bin/mail receive-test --to X --as alice --cc unverified@imaginary.com`, an address which Bailey has added to her account but not verified.
    - Before change: Bailey was CC'd on the task anyway.
    - After change: Bailey is not CC'd on the task.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13317

Differential Revision: https://secure.phabricator.com/D20593
2019-06-19 12:51:00 -07:00
epriestley
5d8ee504d6 Replace weird, redundant list of branches in Diffusion "Manage" UI with a link to the main branch list
Summary:
Fixes T13312. Currently, {nav Manage > Branches} has a list of branches on the same page. This has a few minor issues:

  - Pager is at the top (see T13312), which is weird.
  - "Default" icon is mystery meat.
  - Table is kind of pointless/redundant in general?

Previously, this table had more information about technical status of each branch (autoclose/track/publish) but most of these details have been simplified/eliminated, and the main "Branches" view now has more information than it did before.

Get rid of this and just link to the main view.

Test Plan: Viewed "Branches" in UI, saw a link to the main view instead of a weird table.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13312

Differential Revision: https://secure.phabricator.com/D20584
2019-06-18 15:19:53 -07:00
epriestley
dda5c13ef5 Parse "shallow" frames in the Git "upload-pack" wire protocol parser
Summary:
Fixes T13309. If you void the warranty on a repository on disk and turn it into a shallow clone, Phabricator currently can't serve it.

We don't support hosting shallow working copies, but we should still parse and proxy the protocol rather than breaking in a mysterious way.

Test Plan:
  - Created a shallow working copy with `mv X X.full; git clone --depth Y file://.../X.full X` in the storage directory on disk.
  - Cloned it with `git clone <uri>`.
  - Deleted all the refs inside it so the wire only has "shallow" frames; cloned it.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13309

Differential Revision: https://secure.phabricator.com/D20577
2019-06-18 15:14:52 -07:00
epriestley
7538286499 Fix missing link targets for "View Object" header buttons in HTML email
Summary:
See <https://discourse.phabricator-community.org/t/view-task-from-maniphest-e-mail-doesnt-have-url/2827>.

I added "View Task" / "View Commit" buttons recently but the logic for generating URIs isn't quite right. Fix it up.

Test Plan:
  - Commented on a task.
  - Used `bin/mail show-outbound --id ... --dump-html > out.html` to dump the HTML.
  - Previewed the HTML in a browser.
  - This time, actually clicked the button to go to the task.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20586
2019-06-18 13:20:56 -07:00
epriestley
1dd62f79ce Fix more "msort()" vs "msortv()" callsites
Summary:
See <https://discourse.phabricator-community.org/t/unhandled-exception-when-logging-in-with-mfa/2828>. The recent changes to turn `msort()` on a vector an error have smoked out a few more of these mistakes.

These cases do not meaningfully rely on sort stability so there's no real bug being fixed, but we'd still prefer `msortv()`.

Test Plan: Viewed MFA and External Account settings panels. Did a `git grep 'msort(' | grep -i vector` for any more obvious callsites, but none turned up.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20587
2019-06-18 11:36:22 -07:00
epriestley
731b45d818 In "bin/repository reparse", continue on "PhabricatorWorkerPermanentFailureException"
Summary:
Fixes T13315. See that task for discussion.

Without `--background`, we currently treat this as a catastrophic failure, but it's relatively routine for some repository states. We can safely continue reparsing other steps.

Test Plan: Ran `bin/repository reparse --all X --message` with commits faked to all be unreachable. Got warnings instead of a hard failure on first problem.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13315

Differential Revision: https://secure.phabricator.com/D20588
2019-06-18 11:32:54 -07:00
epriestley
bab35f28e4 Respect repository identities when selecting author vs auditor actions
Summary:
Depends on D20580. Fixes T13311. When we choose which actions to show a user, we can either show them "auditor" actions (like "raise concern") or "author" actions (like "request verification").

Currently, we don't show "author" actions if you're the author of the commit via an identity mapping, but we should. Use identity mappings where they exist.

(Because I've implemented `getEffectiveAuthorPHID()` in a way that requires `$data` be attached, it's possible this will make something throw a "DataNotAttached" exception, but: probably it won't?; and that's easy to fix if it happens.)

Test Plan:
See D20580. As `@alice`, viewed the commit in the UI.

  - Before: got auditor actions presented to me.
  - After: got author actions.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13311

Differential Revision: https://secure.phabricator.com/D20581
2019-06-17 13:56:38 -07:00
epriestley
4450c90881 When triggering audits, respect committer identities when importing commits
Summary:
Ref T13311. We currently don't use committer identity mappings when triggering audits, so if a user is only associated with an identity via manual mapping we won't treat them as the author.

Instead, use the identity and manual mapping if they're available.

Test Plan:
  - Pushed a commit as `xyz <xyz@example.org>`, an address with no corresponding user.
  - In the UI, manually associated that identity with user `@alice`.
  - Ran `bin/repository reparse --publish <hash>` to trigger audits and publishing for the commit.
  - Before: observed the `$author_phid` was `null`.
  - After: observed the `$author_phid` is Alice.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13311

Differential Revision: https://secure.phabricator.com/D20580
2019-06-17 13:47:31 -07:00
epriestley
4af73a625f Don't require users be logged in to access the Logout controller, so users with no Spaces can log out
Summary:
Fixes T13310. Use cases in the form "users with no access to any spaces can not <do things>" are generally unsupported (that is, we consider this to mean that the install is misconfigured), but "log out" is a somewhat more reasonable sort of thing to do and easy to support.

Drop the requirement that users be logged in to access the Logout controller. This skips the check for access to any Spaces and allows users with no Spaces to log out.

For users who are already logged out, this just redirects home with no effect.

Test Plan:
  - As a user with access to no Spaces, logged out. (Before: error; after: worked).
  - As a logged-out user, logged out (was redirected).
  - As a normal user, logged out (normal logout).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13310

Differential Revision: https://secure.phabricator.com/D20578
2019-06-17 13:44:02 -07:00
epriestley
14b076578f In "Download Raw Diff", engage the chunk engine to handle 8MB+ changes
Summary:
Fixes T13313. The "Download Raw Diff" workflow in Differential currently uses an older way of interacting with Files that doesn't engage the chunk engine and can't handle 8MB+ files.

Update to `IteratorFileUploadSource` -- we're still passing in a single giant blob, but this approach can be chunked.

This will still break somewhere north of 8MB (it will break at 2GB with the PHP string limit if nowhere sooner, since we're putting the entire raw diff in `$raw_diff` rather than using a rope/stream) but will likely survive diffs in the hundreds-of-megabytes range for now.

Test Plan:
  - Added `str_repeat('x', 1024 * 1024 * 9)` to the `$raw_diff` to create a 9MB+ diff.
  - Configured file storage with no engine explicitly configured for >8MB chunks (i.e., "reasonably").
  - Clicked "Download Raw Diff".
  - Before: misleading file storage engine error ("no engine can store this file").
  - After: large, raw diff response.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13313

Differential Revision: https://secure.phabricator.com/D20579
2019-06-17 13:31:35 -07:00
epriestley
2bc045bab8 Fix another stray "msort()/msortv()" issue
Summary: Ref T13303. See B22967. This should be "msortv()" but didn't get updated properly.

Test Plan: The system works!

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13303

Differential Revision: https://secure.phabricator.com/D20585
2019-06-17 13:20:10 -07:00
epriestley
874282db75 Correct "msort()" vs "msortv()" to more fully stabilize transaction sorts after recent changes
Summary: Ref T13303. I upgraded this to a vector-based sort but forgot to type a "v", which means the sort has different stability under PHP 5.5. See D20582 for a root cause fix.

Test Plan: Locally, on PHP7, not much changes. I expect this to fix the odd selection of title stories in mail and notification stories on `secure`, which is running PHP 5.5.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13303

Differential Revision: https://secure.phabricator.com/D20583
2019-06-17 13:07:38 -07:00
epriestley
d3112392d1 Allow "Sign with MFA" to be applied as a comment action without requiring "CAN_EDIT"
Summary:
Fixes T13307. We currently require "CAN_EDIT" to sign actions, but it's fine to sign a comment with only "CAN_INTERACT".

Since the actions like "Accept Revision" already work like this, the fix is one line.

Test Plan: {F6488135}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13307

Differential Revision: https://secure.phabricator.com/D20574
2019-06-17 10:41:42 -07:00
epriestley
64a9500078 Add "bin/file migrate" options to support import of a local-disk backup for Phacility instances
Summary:
Ref T13306. Currently, there's no easy way to import a third-party local-disk file dump into a Phacility instance.

Add some more options to `bin/files migrate` to support this. In particular, this enables:

```
$ ./bin/files --from-engine local-disk --engine amazon-s3 --local-disk-source path/to/backup
```

...to import these files into S3 directly.

These are general-purpose options and theoretically useful in other use cases, although realistically those cases are probably very rare.

Test Plan: Used `bin/files` with the new options to move files in and out of local disk storage in an arbitrary backup directory. Got clean exports/imports.

Reviewers: amckinley

Maniphest Tasks: T13306

Differential Revision: https://secure.phabricator.com/D20571
2019-06-15 08:12:11 -07:00
epriestley
f1a588c771 Add a basic profiler to Herald transcripts
Summary:
Ref T13298. Add a simple profiler as a starting point to catch any egregiously expensive rules or conditions.

This doesn't profile rule actions, so if "Add subscriber" (or whatever) is outrageously expensive it won't show up on the profile. Right now, actions get evaluated inside the Adapter so they're hard to profile. A future change could likely dig them out without too much trouble. I generally expect actions to be less expensive than conditions.

This also can't pin down a //specific// condition being expensive, but if you see that `H123` takes 20s to evaluate you can probably guess that the giant complicated regex is the expensive part.

Test Plan: {F6473407}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13298

Differential Revision: https://secure.phabricator.com/D20566
2019-06-05 08:50:41 -07:00
epriestley
d7890d08b8 Add "bin/herald rule ..." to modify Herald rules from the CLI
Summary:
Depends on D20566. Ref T13298. See PHI1280. Currently, there's no clean way to disable problematic personal rules. This comes up occasionally and sometimes isn't really the best approach to solving a problem, but is a generally reasonable capability to provide.

Allow Herald rules (including personal rules) to be disabled/enabled via `bin/herald rule ... --disable/--enable`.

Test Plan: Used the CLI to disable and enable a personal rule.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jmeador

Maniphest Tasks: T13298

Differential Revision: https://secure.phabricator.com/D20567
2019-06-04 07:12:15 -07:00
epriestley
760406762a When a revision is closed, always promote it out of draft
Summary:
Fixes T13300. Currently, if you create a revision and then immediately land it (either using `--draft` or just beating Harbormaster to the punch) it can be stuck in "Draft" forever.

Instead, count landing changes like this as a publishing action.

Test Plan:
  - Used `arc diff --hold` to create a revision, then pushed the commit immediately.
  - Before change: revision closed, but was stuck in draft.
  - After change: revision closed and was promoted out of draft.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13300

Differential Revision: https://secure.phabricator.com/D20565
2019-06-04 06:56:22 -07:00
epriestley
67f062b004 When triggering audits, count "Accepted" revisions as successfully reviewed
Summary:
See PHI1118. That issue may describe more than one bug, but the recent ordering changes to the import pipeline likely make this at least part of the problem.

Previously, commits would always close associated revisions before we made it to the "publish" step. This is no longer true, so we might be triggering audits on a commit before the associated revision actually closes.

Accommodate this by counting a revision in either "Accepted" or "Published (Was Previously Accepted)" as "reviewed".

Test Plan:
  - With commit C affecting paths in package P with "Audit Unreviewed Commits and Commits With No Owner Involvement", associated with revision R, with both R and C authored by the same user, and "R" in the state "Accepted", used `bin/repository reparse --publish <hash>` to republish the commit.
  - Before change: audit by package P triggered.
  - After change: audit by package P no longer triggered.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20564
2019-05-30 17:18:11 -07:00
epriestley
81134d7e7d After reloading transactions for the recipient while building transaction mail, put them in the input order
Summary:
Ref T13303. In D20525 I fixed an issue where transaction rendering could use cached values with the wrong viewer by reloading transactions.

However, reloading transactions may also reorder them as a side effect, since `withPHIDs(...)` does not imply an order. This can make transaction rendering order in mail wrong/inconsistent.

Instead, reorder the transactions before continuing so mail transaction order is consistent.

Test Plan: Applied a group of transactions to a task, saw a more consistent rendering order in mail after the change.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13303

Differential Revision: https://secure.phabricator.com/D20563
2019-05-30 17:14:54 -07:00
epriestley
9a32a563f0 Add a "View Task" button to HTML mail from Maniphest
Summary:
See downstream <https://phabricator.wikimedia.org/T1050>. Some time ago, we added a "View Revision" button to Differential mail. This hasn't created any problems and generally seems good / desirable.

It isn't trivial to just add everywhere since we need a translation string in each case, but at least add it to Maniphest for now. Going forward, we can fill in more applications as they come up.

Test Plan:
Used `bin/mail show-outbound --id <x> --dump-html`:

{F6470461}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20561
2019-05-30 15:24:22 -07:00
epriestley
fb5dec4c03 Require valid comments to contain at least one non-whitespace character
Summary:
See downstream <https://phabricator.wikimedia.org/T88655>. This is very marginal, but we currently allow comments consisting of //only// whitespace.

These are probably always mistakes, so treat them like completely empty comments.

(We intentionally do not trim leading or trailing whitespace from comments when posting them becuase leading spaces can be used to trigger codeblock formatting.)

Test Plan:
  - Posted empty, nonempty, and whitespace-only comments.
  - Whitespace-only comments now have the same behavior as truly empty comments (e.g., do not actually generate a transaction).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20562
2019-05-30 15:15:24 -07:00
epriestley
8747204807 Clean up "phabricator.timezone" configuration instructions a little bit
Summary: These instructions are fairly old and can be a little fancier and more clear in the context of modern Phabricator. Drop the reference to "HPHP", link the actual timezone list, wordsmith a little.

Test Plan: d( O_o )b

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20560
2019-05-30 15:08:11 -07:00
epriestley
e5a1681903 Render timezone names more readably, with spaces rather than underscores ("America/Los Angeles", not "America/Los_Angeles").
Summary:
See downstream <https://phabricator.wikimedia.org/T902>. Currently, timezones are rendered with their raw internal names (like `America/Los_Angeles`) which include underscores.

Replacing underscores with spaces is a more human-readable (and perhaps meaningfully better for things like screen readers, although this is pure speculation).

There's some vague argument against this, like "administrators may need to set a raw internal value in `phabricator.timezone` and this could mislead them", but we already give a pretty good error message if you do this and could improve hinting if necessary.

Test Plan: Viewed timezone list in {nav Settings} and the timezone "reconcile" dialog, saw a more-readable "Los Angeles".

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20559
2019-05-30 15:03:11 -07:00
epriestley
53b9acfb7d Test for "CAN_INTERACT" on comment edits in a way that survives objects which only implement "CAN_VIEW"
Summary:
Ref T13289. See D20551. In D20551, I implemented some "CAN_INTERACT" checks against certain edits, but these checks end up testing "CAN_INTERACT" against objects like Conpherence threads which do not support a distinct "CAN_INTERACT" permission. I misrembered how the "CAN_INTERACT" fallback to "CAN_VIEW" actually works: it's not fully automatic, and needs some explicit "interact, or view if interact is not available" checks.

Use the "interact" wrappers to test these policies so they fall back to "CAN_VIEW" if an object does not support "CAN_INTERACT". Generally, objects which have a "locked" state have a separate "CAN_INTERACT" permission; objects which don't have a "locked" state do not.

Test Plan: Created and edited comments in Conpherence (or most applications other than Maniphest).

Reviewers: amckinley

Maniphest Tasks: T13289

Differential Revision: https://secure.phabricator.com/D20558
2019-05-28 10:14:43 -07:00
epriestley
ce6fc5be90 Fix a looping workflow when trying to submit a partially-effectless transaction group
Summary:
Ref T13289. If you do this:

  - Subscribe to a task (so we don't generate a subscribe side-effect later).
  - Prepare a transaction group: sign with MFA, change projects (don't make any changes), add a comment.
  - Submit the transaction group.

...you'll get prompted "Some actions don't have any effect (the non-change to projects), apply remaining effects?".

If you confirm, you get MFA'd, but the MFA flow loses the "continue" confirmation, so you get trapped in a workflow loop of confirming and MFA'ing.

Instead, retain the "continue" bit through the MFA.

Also, don't show "You can't sign an empty transaction group" if there's a comment.

See also T13295, since the amount of magic here can probably be reduced. There's likely little reason for "continue" or "hisec" to be magic nowadays.

Test Plan:
  - Went through the workflow above.
  - Before: looping workflow.
  - After: "Continue" carries through the MFA gate.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13289

Differential Revision: https://secure.phabricator.com/D20552
2019-05-23 19:16:17 -07:00
epriestley
719dd6d3f4 Remove the "search_documentfield" table
Summary: Ref T11741. See PHI1276. After the switch to "Ferret", this table has no remaining readers or writers.

Test Plan:
  - Ran `bin/storage upgrade -f`, no warnings.
  - Grepped for class name, table name, `stemmedCorpus` column; got no relevant hits.
  - Did a fulltext search.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T11741

Differential Revision: https://secure.phabricator.com/D20549
2019-05-23 19:11:38 -07:00
epriestley
aacc62463d Prevent editing and deleting comments in locked conversations
Summary:
Ref T13289. This tightens up a couple of corner cases around locked threads.

Locking is primarily motivated by two use cases: stopping nonproductive conversations on open source installs (similar to GitHub's feature); and freezing object state for audit/record-keeping purposes.

Currently, you can edit or remove comments on a locked thread, but neither use case is well-served by allowing this. Require "CAN_INTERACT" to edit or remove a comment.

Administrators can still remove comments from a locked thread to serve "lock a flamewar, then clean it up", since "Remove Comment" on a comment you don't own is fairly unambiguously an administrative action.

Test Plan:
  - On a locked task, tried to edit and remove my comments as a non-administrator. Saw appropriate disabled UI state and error dialogs (actions were disallowed).
  - On a locked task, tried to remove another user's comments as an administrator. This works.
  - On a normal task, edited comments normally.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13289

Differential Revision: https://secure.phabricator.com/D20551
2019-05-23 19:04:55 -07:00
epriestley
f838ad1827 Fix two straggling pagination issues in Drydock
Summary:
Ref T13289. See <https://discourse.phabricator-community.org/t/fatal-error-in-pagination-in-drydock-resources-host-logs-all-logs/2735>.

`bin/drydock lease` and the web UI for reviewing all object logs when there is more than one page of logs didn't get fully updated to the new cursors.

  - Use a cursor pager in `bin/drydock lease`.
  - Implement `withIDs()` in `LeaseQuery` so the default paging works properly.

Test Plan:
  - Ran `bin/drydock lease`, got a lease with log output along the way.
  - Set page size to 2, viewed host logs with multiple pages, paged to page 2.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13289

Differential Revision: https://secure.phabricator.com/D20553
2019-05-23 18:50:06 -07:00
epriestley
2e2dc47f07 In the Herald test console, don't consider transactions that Herald rules applied
Summary:
Depends on D20546. Ref T13283. Currently, if you do something (transactions "A", "B") and Herald does some things in response (transaction "C"), Herald acts only on the things you did ("A", "B") since the thing it did ("C") didn't exist yet, until it ran.

However, if you use the test console to test rules against the object we'll pick up all three transactions since they're all part of the same group. This isn't ideal.

To fix this, skip transactions which Herald applied, since it obviously didn't consider them when it was evaluating.

Test Plan:
  - Created a revision, in the presence of a Herald rule that adds reviewers.
  - Then, ran the revision through the test console.
  - Before: saw the "Herald added reviewers: ..." transaction in the transaction group Herald evaluated.
  - After: saw only authentic human transactions.

{F6464064}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13283

Differential Revision: https://secure.phabricator.com/D20547
2019-05-22 16:34:34 -07:00
epriestley
31e623afcc Use the same transaction group ID for transactions applied indirectly by a sub-editor
Summary:
Ref T13283. Currently, each Editor sets its own group ID, so if you create a revision and then Herald does some stuff, the two groups of transactions get different group IDs.

This means the test console is slightly misleading (it will only pick up the Herald transactions). It's going to be misleading anyway (Herald obviously can't evaluate Herald transactions) but this is at least a little closer to reality and stops Herald actions from masking non-Herald actions.

Test Plan:
  - Created a revision. Herald applied one transaction.
  - Used the test console.
  - Before: The test console only picked up the single most recent Herald transaction.
  - After: The test console picked up the whole transaction group.

{F6464059}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13283

Differential Revision: https://secure.phabricator.com/D20546
2019-05-22 16:33:03 -07:00
epriestley
f6af1c4374 When creating a Phriction document, mark initial transactions as "create" transactions to fix weird email
Summary:
Ref T13289. When you create a Phriction document, you currently get an email with the whole new content as a "diff".

You also get extra transactions in the email and on the page.

This is because Phriction isn't on EditEngine and doesn't mark "create" transactions in a modern way. Get them marked properly to fix these obviously-broken behaviors. This can all go away once Phriction switches to EditEngine, although I don't have any particular plans to do that in the immediate future.

Test Plan:
  - Created a new document, viewed email, no longer saw redundant "edited content" transaction or "CHANGES TO CONTENT" diff.
  - Updated a document, viewed email, got interdiff.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13289

Differential Revision: https://secure.phabricator.com/D20548
2019-05-22 16:28:25 -07:00
epriestley
b95bf722d5 Drop the "update revision with commit diff" transaction if the revision is already closed
Summary:
Ref T13290. Prior to recent changes, if we parsed some commit C which was associated with a revision R, but R was already closed, we'd skip the whole set of updates because the "close the revision" transaction would fail and we'd throw because we did not `setContinueOnNoEffect()`.

We now continue on no effect so we can get the edge ("commit has revision" / "revision has commit"), since we want it in all cases, but this means we may also apply an extra "Updated revision to reflect committed changes" transaction and new diff. This can happen even if we're careful about not trying to apply this transaction to closed revisions, since two workers may race. (Today, we aren't too careful about this.)

To fix this, just make this transaction no-op itself if the revision is already closed by the time it tries to apply.

This happened on D20451 because a merge commit with the same hash as the last diff was pushed, but it's easiest to reproduce by just running `bin/repository reparse --message <commit>`, which updates related revisions with a new diff every time.

Test Plan:
  - Ran `bin/repository reparse --messsage <commit>` several times, on a commit with an associated revision.
  - Before: each run attached a new diff and created a new "updated to reflect committed changes" transaction.
  - After: repeated runs had no effects.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13290

Differential Revision: https://secure.phabricator.com/D20545
2019-05-22 16:26:45 -07:00
epriestley
1eff4fdca3 Prevent "Differential Revision: ..." from counting as a mention in commit messages
Summary:
Ref T13290. Ref T13291. Now that a full URI is a "mention", the full URI in "Differential Revision: ..." also triggers a mention.

Stop it from doing that, since these mentions are silly/redundant/unintended.

The API here is also slightly odd; simplify it a little bit to get rid of doing "append" with "get + append + set".

Test Plan: Used `bin/repository reparse --publish` to republish commits with "Differential Revision: ..." and verified that the revision PHID was properly dropped from the mention list.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291, T13290

Differential Revision: https://secure.phabricator.com/D20544
2019-05-22 16:22:01 -07:00
epriestley
fa4dcaa3aa Stabilize sorting of feed stories with similar strength
Summary:
See PHI1222. When we publish several transactions to feed at once, we sort them by "action strength" to figure out which one gets to be the title story.

This sort currently uses `msort()`, which uses `asort()`, which is not a stable sort and has inconsistent behavior across PHP versions:

{F6463721}

Switch to `msortv()`, which is a stable sort. Previously, see also T6861.

If all transactions have the same strength, we'll now consistently pick the first one.

This probably (?) does not impact anything in the upstream, but is good from a consistency point of view.

Test Plan:
Top story was published after this change and uses the chronologically first transaction as the title story.

Bottom story was published before this change and uses the chronologically second transaction as the title story.

Both stories have two transactions with the same strength ("create" + "add reviewer").

{F6463722}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20540
2019-05-22 15:50:59 -07:00
epriestley
f91bef64f1 Stack chart functions in a more physical way
Summary:
Ref T13279. See that task for some discussion.

The accumulations of some of the datasets may be negative (e.g., if more tasks are moved out of a project than into it) which can lead to negative area in the stacked chart.

Introduce `min(...)` and `max(...)` to separate a function into points above or below some line, then mangle the areas to pick the negative and positive regions apart so they at least have a plausible physical interpretation and none of the areas are negative.

This is presumably not a final version, I'm just trying to produce a chart that isn't a sequence of overlapping regions with negative areas that is "technically" correct but not really possible to interpret.

Test Plan: {F6439195}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20506
2019-05-22 05:40:39 -07:00
epriestley
f190c42bcd Store charts earlier and build them out a little later
Summary:
Ref T13279. Currently, we store a fairly low-level description of functions and datasets in a chart. This will create problems with (for example) translating function labels.

If you view a chart someone links you, it should say "El Charto" if you speak Spanish, not "The Chart" if the original viewer speaks English.

To support this, store a slightly higher level version of the chart: the chart engine key, plus configuration parameters. This is very similar to how SearchEngine works.

For example, the burndown chart now stores a list of project PHIDs, instead of a list of `[accumulate [sum [fact task.open <project-phid>]]]` functions.

(This leaves some serialization code with no callsites, but we may eventually have a "CustomChartEngine" which stores raw functions, so I'm leaving it for now.)

As a result, function labels provided by the chart engine are now translatable.

(Note that the actual chart is meaningless since the underlying facts can't be stacked like they're being stacked, as some are negative in some areas of their accumulation.)

Test Plan: {F6439121}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20504
2019-05-22 05:39:32 -07:00
epriestley
493a6b72c1 Automatically select the range for charts in a general way
Summary:
Ref T13279. Replace the hard-coded default range with a range computed by examining the chart data.

Instead of having a "Dataset" return a blob of wire data, "Dataset" now returns a structure with raw wire data plus a range. I expect to add more structured data here in future changes (tooltip/hover event data, maybe function labels).

Test Plan: {F6439101}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20503
2019-05-22 05:36:58 -07:00
epriestley
e90360c289 Wrap "<min, max>" chart domain pairs in an "Interval" class
Summary: Ref T13279. Slightly simplify domain handling by putting all the "[x, y]" stuff in an Interval class. I'm planning to do something similar for ranges next, so this should make that easierr.

Test Plan: Viewed chart, saw same chart as before.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20502
2019-05-22 05:28:49 -07:00
epriestley
a80426b339 Provide chart function labels over the wire instead of making them up
Summary: Ref T13279. Makes charts incrementally more useful by allowing the server to provide labels and colors for functions.

Test Plan: {F6438872}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20501
2019-05-22 05:22:59 -07:00
epriestley
c6052b41a6 Label important data on charts
Summary:
Ref T13279. Adds client-side support for rendering function labels on charts, then labels every function as important data.

Works okay on mobile, although I'm not planning to target mobile terribly heavily for v0.

Test Plan: {F6438860}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20500
2019-05-22 05:21:26 -07:00
epriestley
81456db559 Roughly support stacked area charts
Summary:
Ref T13279. This adds support for:

  - Datasets can have types, like "stacked area".
  - Datasets can have multiple functions.
  - Charts can store dataset types and datasets with multiple functions.
  - Adds a "stacked area" dataset.
  - Makes D3 actually draw a stacked area chart.

Lots of rough edges here still, but the result looks slightly more like it's supposed to look.

D3 can do some of this logic itself, like adding up the area stacks on top of one another with `d3.stack()`. I'm doing it in PHP instead because I think it's a bit easier to debug, and it gives us more options for things like caching or "export to CSV" or "export to API" or rendering a data table under the chart or whatever.

Test Plan: {F6427780}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20498
2019-05-22 05:19:41 -07:00
epriestley
5c1b91ab45 Consolidate burndown logic into a "BurndownChartEngine"
Summary:
Ref T13279. For now, we need to render burndowns from both Maniphest (legacy) and Projects (new prototype).

Consolidate this logic into a "BurndownChartEngine". I plan to expand this to work a bit like a "SearchEngine", and serve as a UI layer on top of the raw chart features.

The old "ChartEngine" is now "ChartRenderingEngine".

Test Plan:
  - Viewed burndowns ("burnups") in Maniphest.
  - Viewed burndowns in Projects.
  - Saw the same chart.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20496
2019-05-22 05:10:42 -07:00
epriestley
0aee3da19e Add a "Reports" menu item to Projects
Summary:
Ref T13279. Since the use cases that have made it upstream are all for relatively complex charts (e.g., requiring aggregation and composition of multiple data series in nontrivial ways) I'm currently looking at an overall approach like this:

  - At least for now, Charts provides a low-level internal-only API for composing charts from raw datasets.
  - This is exposed to users through pre-built `SearchEngine`-like interfaces that provide a small number of more manageable controls (show chart from date X to date Y, show projects A, B, C), but not the full set of composition features (`compose(scale(2), cos())` and such).
  - Eventually, we may put more UI on the raw chart composition stuff and let you build your own fully custom charts by gluing together datasets and functions.
  - Or we may add this stuff in piecemeal to the higher-level UI as tools like "add goal line" or "add trend line" or whatever.

This will let the low-level API mature/evolve a bit before users get hold of it directly, if they ever do. Most requests today are likely satisfiable with a small number of chart engines plus raw API data access, so maybe UI access to flexible charting is far away.

Step toward this by adding a "Reports" section to projects. For now, this just renders a basic burnup for the current project. Followups will add an "Engine" layer above this and make the chart it produces more useful.

Test Plan: {F6426984}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20495
2019-05-22 05:08:55 -07:00
epriestley
f87c1ac362 Start the fact daemon in "bin/phd start"
Summary:
Depends on D20488. Ref T13279. When installs run `bin/phd start`, start the fact daemon alongside other daemons.

Since "Reports" in Maniphest now relies on Facts data, populate it.

Test Plan: Ran `bin/phd start`, saw the Fact daemon start.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20489
2019-05-22 04:57:55 -07:00
epriestley
10afe1f2b5 Fix handling of "null" domain values in Charts
Summary: Depends on D20487. If you `min(1, 2, null)`, you get `null`. We want `1`.

Test Plan: Viewed a "burnup for project X" chart where one dataseries had no datapoints. Saw a sensible domain selected automatically.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Differential Revision: https://secure.phabricator.com/D20488
2019-05-22 04:51:22 -07:00
epriestley
f8ebc71b8f Replace the chart in Maniphest Reports with a chart driven by Facts
Summary:
Depends on D20485. Ref T13279. This removes the ad-hoc charting in Maniphest and replaces it with a Facts-based chart.

(To do this, we build a dashboard panel inline and render it.)

Test Plan: {F6412720}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20486
2019-05-22 04:44:10 -07:00
epriestley
ff6b13872c Add a rough "Chart" Dashboard Panel
Summary:
Depends on D20484. Ref T13279. Allows a chart to render as a panel.

Configuring these is currently quite low-level (you have to manually copy/paste a chart key in), but works well enough.

Test Plan: {F6412708}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20485
2019-05-22 04:36:09 -07:00
epriestley
c458b50b85 Render charts from storage instead of just one ad-hoc hard-coded chart
Summary:
Ref T13279. This changes the chart controller:

  - if we have no arguments, build a demo chart and redirect to it;
  - otherwise, load the specified chart from storage and render it.

This mostly prepares for "Chart" panels on dashboards.

Test Plan: Visited `/fact/chart/`, got redirected to a chart from storage.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20483
2019-05-22 04:31:48 -07:00
epriestley
4180b337cf Add a "{src ...}" Remarkup rule to provide a more flexible way to reference source files in Diffusion
Summary: Depends on D20538. Ref T13291. We now recognize full source URIs, but encoding full URIs isn't super human-friendly and we can't do stuff like relative links with them. Add `{src ...}` as a way to get to this behavior that supports options and more flexible syntax.

Test Plan: {F6463607}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291

Differential Revision: https://secure.phabricator.com/D20539
2019-05-21 13:12:28 -07:00
epriestley
56e7bde68d Recognize self-URI links to Diffusion files and give them special rendering behavior
Summary:
Depends on D20530. Ref T13291. When users paste links to files in Diffusion into remarkup contexts, identify them and specialize the rendering.

When the URIs are embedded with `{...}`, parse them in more detail.

This is a lead-up to a `{src ...}` rule which will use the same `View` but give users more options to customize presentation.

Test Plan: {F6463580}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291

Differential Revision: https://secure.phabricator.com/D20538
2019-05-21 13:07:37 -07:00
epriestley
7c1f6519e0 Support "none()" in Differential to find revisions with no (un-resigned) reviewers
Summary:
Ref T13289. In Maniphest, you can currently search for "Owner: none()" to find tasks with no owner, but there's no way to search for "Reviewers: none()" in Differential right now.

Add support for this, since it's consistent and reasonable and doesn't seem too weird or niche.

Test Plan: Searched for "Reviewers: none()", found revisions with no reviewers. Searched for "Reviewers: alice, none()", "Reviewers: alice", and "Reviewers: <no constraint>" and got sensible results.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13289

Differential Revision: https://secure.phabricator.com/D20537
2019-05-21 12:56:59 -07:00
epriestley
2f38695768 Support export of feed transactions to CSV/Excel/etc
Summary: Depends on D20534. Ref T13294. Add export support so you can dump these out, print them on paper, notarize them, and store them in a box under a tree or whatever.

Test Plan: Exported transactions to a flat file, read the file.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13294

Differential Revision: https://secure.phabricator.com/D20535
2019-05-21 12:47:46 -07:00
epriestley
16537f7b32 Support filtering feed transactions by object type
Summary: Depends on D20533. Allow querying for transactions of a specific object type, so you can run queries like "Show all edits to Herald rules between date X and Y".

Test Plan: {F6463478}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20534
2019-05-21 12:39:10 -07:00
epriestley
2e5b1885e7 Add support for querying feed transactions by author and date range
Summary: Depends on D20531. Ref T13294. Enable finding raw transactions in particular date ranges or with particular authors.

Test Plan: {F6463471}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13294

Differential Revision: https://secure.phabricator.com/D20533
2019-05-21 12:37:50 -07:00
epriestley
642113708a Build a rough transaction-level view of Feed
Summary:
Ref T13294. An install is interested in a way to easily answer audit-focused questions like "what edits were made to any Herald rule in Q1 2019?".

We can answer this kind of question with a more granular version of feed that focuses on being exhaustive rather than being human-readable.

This starts a rough version of it and deals with the two major tricky pieces: transactions are in a lot of different tables; and paging across them is not trivial.

To solve "lots of tables", we just query every table. There's a little bit of sleight-of-hand to get this working, but nothing too awful.

To solve "paging is hard", we order by "<dateCreated, phid>". The "phid" part of this order doesn't have much meaning, but it lets us put every transaction in a single, stable, global order and identify a place in that ordering given only one transaction PHID.

Test Plan: {F6463076}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13294

Differential Revision: https://secure.phabricator.com/D20531
2019-05-21 12:28:00 -07:00
epriestley
5305ebddda To improve wrapping behavior of rendered README files, don't use "PHUIDocumentView" in Diffusion
Summary:
See PHI1268. We currently do some weird width handling when rendering Diffusion readmes in a document directory view.

I think this came from D12330, which used `PHUIDocumentViewPro` to change the font, but we later reverted the font and were left with the `DocumentView`. Other changes after that modified `DocumentView` to have fixed-width behavior, but it doesn't make much sense here since the content panel is clearly rendered full-width.

Today, the `DocumentView` is a more structural element with methods like `setCurtain()`. Just get rid of it to simplify things, at least as a first step.

Test Plan:
Before:

{F6463493}

After:

{F6463492}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20536
2019-05-21 12:18:18 -07:00
epriestley
2f0e655a9b Add a 15-second timeout to external service calls to fill Doorkeeper link tags
Summary:
Depends on D20528. Ref T13291. Ref T13285. Currently, we don't put a timeout on external service calls when enriching URIs for external Asana/JIRA tasks.

Add a 15-second timeout so we'll do something reasonable-ish in the face of a downed service provider. Later, I plan to healthcheck Asana/JIRA providers in a generic way (see T13287) so we can stop making calls if they time out / fail too frequently.

Test Plan:
  - Linked to JIRA and Asana tasks in comments.
  - Set timeout to 0.0001 seconds, saw requests time out.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291, T13285

Differential Revision: https://secure.phabricator.com/D20530
2019-05-21 12:17:12 -07:00
epriestley
f420159380 Implement Asana and JIRA external links via HyperlinkEngineExtension, not separate Remarkup rules
Summary:
Depends on D20527. Ref T13291. Now that we have more flexible support for URI rewriting, use it for Doorkeeper URIs.

These are used when you set up Asana or JIRA and include the URI to an Asana task or a JIRA issue in a comment.

Test Plan:
  - Linked up to Asana and JIRA.
  - Put Asana and JIRA URIs in comments.
  - Saw the UI update to pull task titles from Asana / JIRA using my OAuth credentials.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291

Differential Revision: https://secure.phabricator.com/D20528
2019-05-21 12:15:39 -07:00
epriestley
33688c8a41 When an object is referenced by URI, treat it like a mention
Summary:
Ref T13291. Currently, `T123` is a mention and adds an "alice mentioned this on Txxx." to `T123`, but `https://install.com/T123` is not a mention.

Make the full URI a mention.

Test Plan: Commented a full URI, saw the target object get a mention story in its timeline.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13291

Differential Revision: https://secure.phabricator.com/D20527
2019-05-21 12:13:03 -07:00
epriestley
b8f6248e07 Fix an issue where handles could load with the incorrect viewer when building mail about changes to related objects
Summary:
See <https://phabricator.wikimedia.org/T179591>. Some time ago, all handle rendering preloaded handles: things emitted a list of PHIDs they'd need handles for, then later used only those PHIDs.

Later, we introduced `HandlePool` and lazy/on-demand handle loading. Modern transactions mostly use this to render object PHIDs.

When we build mail, many newer transactions use an on-demand load to fetch handles to render transactions. This on-demand load may use the original viewer (the acting user) instead of the correct viewer (the mail recipient): we fetch and reset handles using the correct viewer, but do not overwrite the active viewer for on-demand loading. This could cause mail to leak the titles of related objects to users who don't have permission to see them.

Instead, just reload the transactions with the correct viewer when building mail instead of playing a bunch of `setViewer()` and `clone` games. Until we're 100% on modular transactions, several pieces of the stack cache viewer or state information.

Test Plan:
  - Created task A (public) with subtask B (private).
  - Closed subtask B as a user with access to it.
  - Viewed mail sent to subscribers of task A who can not see subtask B.
    - Before change: mail discloses title of subtask B.
    - After change: mail properly labels subtask B as "Restricted Task".

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20525
2019-05-21 12:12:13 -07:00
epriestley
06778ea550 Separate the "configuration" and "evaluation" phases of chart functions
Summary:
Depends on D20446. Currently, chart functions are both configured through arguments and evaluated through arguments. This sort of conflates things and makes some logic more difficult than it should be.

Instead:

  - Function arguments are used to configure function behavior. For example, `scale(2)` configures a function which does `f(x) => 2 * x`.
  - Evaluation is now separate, after configuration.

We can get rid of "sourceFunction" (which was basically marking one argument as "this is the thing that gets piped in" in a weird magical way) and "canEvaluate()" and "impulse".

Sequences of functions are achieved with `compose(u, v, w)`, which configures a function `f(x) => w(v(u(x)))` (note order is left-to right, like piping `x | u | v | w` to produce `y`).

The new flow is:

  - Every chartable function is `compose(...)` at top level, and composes one or more functions. `compose(x)` is longhand for `id(x)`. This just gives us a root/anchor node.
  - Figure out a domain, through various means.
  - Ask the function for a list of good input X values in that domain. This lets function chains which include a "fact" with distinct datapoints tell us that we should evaluate those datapoints.
  - Pipe those X values through the function.
  - We get Y values out.
  - Draw those points.

Also:

  - Adds `accumluate()`.
  - Adds `sum()`, which is now easy to implement.
  - Adds `compose()`.
  - All functions can now always evaluate everywhere, they just return `null` if they are not defined at a given X.
  - Adds repeatable arguments for `compose(f, g, ...)` and `sum(f, g, ...)`.

Test Plan: {F6409890}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Differential Revision: https://secure.phabricator.com/D20454
2019-05-19 16:54:53 -07:00
epriestley
a76e91ea9e Remove obsolete Dashboard panel methods with no callsites
Summary: Ref T13272. Since the move to EditEngine, these methods have no callsites.

Test Plan: `grep`

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20484
2019-05-17 12:09:02 -07:00
epriestley
167f06d3eb Label transaction groups with a "group ID" so Herald can reconstruct them faithfully
Summary:
Ref T13283. See PHI1202. See D20519. When we apply a group of transactions, label all of them with the same "group ID".

This allows other things, notably Herald, to figure out which transactions applied together in a faithful way rather than by guessing, even though the guess was probably pretty good most of the time.

Also expose this to `transaction.search` in case callers want to do something similar. They get a list of transaction IDs from webhooks already anyway, but some callers use `transaction.search` outside of webhooks and this information may be useful.

Test Plan:
  - Ran Herald Test Console, saw faithful selection of recent transactions.
  - Changed hard limit from 1000 to 1, saw exception. Users should be very hard-pressed to hit this normally (they'd have to add 990-ish custom fields, then edit every field at once, I think) so I'm just fataling rather than processing some subset of the transaction set.
  - Called `transaction.search`, saw group ID information available.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13283

Differential Revision: https://secure.phabricator.com/D20524
2019-05-17 12:07:37 -07:00
epriestley
6bff2cee22 Improve the performance of tab replacement in common cases
Summary:
See PHI1210. For certain large inputs, we spend more time than we need to replacing tabs with spaces. Add some fast paths:

  - When a line only has tabs at the beginning of the line, we don't need to do as much work parsing the rest of the line.
  - When a line has no unicode characters, we don't need to vectorize it to get the right result.

Test Plan:
  - Added test coverage.
  - Profiled this, got a ~60x performance increase on a 36,000 line 3MB text file.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20477
2019-05-16 12:28:39 -07:00
epriestley
c5ecc388a2 Make branch status more clear on Diffusion branches view
Summary:
See PHI1225. Ref T13277. In Diffusion, show "default", "permanent", or "not permanent" when looking at branches.

For repositories with 100 or fewer branches, put default and permanent branches on top.

Test Plan: {F6426814}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: leoluk

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20493
2019-05-16 12:24:39 -07:00
epriestley
e059997e53 Expose subscribers transaction data via Conduit "transaction.search"
Summary:
Depends on D20507. See PHI1232. Previously, see T13255 and D20209.

Since nothing seems to have exploded after "projects" was exposed, give "subscribers" the same treatment.

Test Plan: Added, removed, and modified subscribers. Queried transactions with "transaction.search", saw sensible "type" and data.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20508
2019-05-16 12:19:45 -07:00
epriestley
23bfe0c0f6 Specialize rendering of self-URIs in the form "/X123"
Summary:
Depends on D20510. Ref T5378. When remarkup includes a hyperlink to the current install in the form "/X123" (which is common), load the corresponding object and specialize the rendering.

This doesn't cover everything (notably, no handling for Diffusion paths yet), but does cover a lot of the most common cases.

The "uri" form preserves the URI as written, but adds an icon, tag, and hovercard.

The "{uri}" form is more similar to `{T123}` and shows the object name.

Test Plan: {F6440367}

Reviewers: amckinley, joshuaspence

Reviewed By: joshuaspence

Subscribers: joshuaspence

Maniphest Tasks: T5378

Differential Revision: https://secure.phabricator.com/D20512
2019-05-16 12:16:38 -07:00
epriestley
706826bf02 When a user pastes a Phabricator URI into the search box, redirect to the URI
Summary:
Depends on D20509. See PHI1224. Ref T5378. With some frequency, I paste URIs into the global search input (I am dumb).

When I do this dumb thing, redirect to the URI as though the global search was a URI bar.

Maybe only I am dumb like this, but I don't think it'll hurt anything.

Test Plan: pasted a URI and hit return; tried to eat a rock

Reviewers: amckinley, joshuaspence

Reviewed By: joshuaspence

Maniphest Tasks: T5378

Differential Revision: https://secure.phabricator.com/D20510
2019-05-16 11:19:52 -07:00
epriestley
f2feb0378f Remove ancient "PhabricatorQuickSearchEngineExtension" compatibility class
Summary: Ref T5378. This class was renamed more than a year ago, in D19087. Remove the leftover compatiblity layer.

Test Plan: `grep`

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5378

Differential Revision: https://secure.phabricator.com/D20509
2019-05-16 11:14:49 -07:00
epriestley
4a3481a493 Enrich "draft", "summary", and "testPlan" transactions from Differential in "transaction.search"
Summary:
See PHI1232, which describes a reasonable use case for wanting information about the "draft" ("Hold as Draft / Do Not Auto-Promote") flag.

Also, flesh out "testPlan" and "summary". It's possible these "blob of remarkup" fields might have metadata some day (e.g., a rendered version or a list of PHIDs or something), but we could add more keys, and we already have some other transactions which work like this.

Test Plan: Used "transaction.search" to fetch these transaction types, saw type information and metadata.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20507
2019-05-16 10:50:28 -07:00
epriestley
18f0f8b029 Add support for custom "Wait for Approval" instructions
Summary:
See PHI1229. An install has a somewhat duct-taped registration flow which can dump users on the "Wait for Approval" screen without clear guidance. The desired guidance is something like "this is totally normal, just wait a bit for a bot to approve you".

Adding guidance here is generally reasonable and consistent with the intent of this feature.

Test Plan: {F6426583}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: kylec

Differential Revision: https://secure.phabricator.com/D20492
2019-05-16 10:41:52 -07:00
epriestley
e5fe4dffe1 Add a "Published document changed" rule to Herald for Phriction documents
Summary: Depends on D20519. Ref T13283. See PHI1202. Add a new rule which triggers when the current/most-recent transaction group includes a "content" or "publish" transaction, which means the published document content has changed.

Test Plan:
  - Wrote a Herald rule using this field.
  - Created a document (rule matched).
  - Edited a document (rule matched).
  - Edited a document, saving as a draft (no match).
  - Edited a draft, updating it (no match).
  - Published a draft docuemnt (rule matched).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13283

Differential Revision: https://secure.phabricator.com/D20520
2019-05-16 10:40:52 -07:00
epriestley
bf2f3991d2 When using the Herald test console on a transactional object, guess a reasonable set of transactions to simulate
Summary:
Depends on D20518. Ref T13283. When you use the "Test Console" in Herald to test rules, pass the most recent group of transactions to the Adapter.

This will make it easier to test rules that depend on edit state, since you can make the type of edit you're trying to test and then use the Test Console to actually test if it matches in the way you expect.

The transactions we select may not be exactly the group of transactions that most recently applied. For example, if you make edits A, B, and C in rapid succession and then run the Test Console on the object, it may select "A + B + C" as a transaction group. But we'll show you what we selected and this is basically sane/reasonable and should be fine.

(Eventually, we could include a separate "transaction group ID" on transactions if we want to get this selection to match exactly.)

Test Plan: Ran the Test Console on various objects, saw sensible transaction lists in the transcripts.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13283

Differential Revision: https://secure.phabricator.com/D20519
2019-05-16 10:32:54 -07:00
epriestley
046888ac2e In Herald, save applied transaction PHIDs in the transcript and display them in the UI
Summary:
Ref T13283. Since D14575, we already pass applied transactions to Herald, but they exist only as a backwards compatibility layer and have no upstream callsites.

Save the applied transaction PHIDs as part of the object transcript, and show them in the UI.

Test Plan:
  - Viewed a modern transcript, saw a list of transactions.
  - Viewed an older transcript, saw nothing (since there were no transactions in the transcript).

{F6456431}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13283

Differential Revision: https://secure.phabricator.com/D20518
2019-05-16 09:58:16 -07:00
epriestley
a1f08de283 When viewing a workboard, only link to parent workboards in crumbs if parents have workboards
Summary:
Depends on D20516. See PHI1247. In D20331, I made the crumbs on workboards point at ancestor workboards.

However, this isn't a great destination if an ancestor doesn't actually have a workboard. In this case, point at the normal profile URI instead.

Test Plan:
  - Viewed a milestone workboard with a parent that had no workboard. Saw a profile link instead of a workboard link (new behavior).
  - Viewed a milestone workboard with a parent that also had a workboard. Saw a workboard link (existing old behavior still works).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20517
2019-05-15 07:08:54 -07:00
epriestley
104eefaa10 Hide the "added a commit/revision" stories from feed and mail
Summary:
Ref T13276. Previously, these edges were added directly with an `EdgeEditor`, so they did not generate transaction stories.

Now, they're added properly, but they aren't terribly useful in feed/mail. Hide them in those contexts, like we already do with other types of similar edges.

Test Plan: Will verify behavior on `secure`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20491
2019-05-14 09:39:26 -07:00
epriestley
3e0391c574 Don't mark subscribes/mentions as "Lock Overridden"
Summary:
See PHI1209. When a task is in "Hard Lock" mode, it's still possible to apply some changes to it. Notably:

  - You can subscribe/unsubscribe.
  - You can mention it on another object.
  - You can add a relationship from some other object to it (e.g., select it as a "Parent Task" for some other task).

Currently, these types of edits will show a "Lock Overridden" timeline emblem icon. However, they should not: you didn't override a lock to make these changes, they just bypass locks.

For now, special case these cases (self subscribe/unsubscribe + inverse edge edits) so they don't get the little icon, since I think this list is exhaustive today.

Some day we should modularize this, but we'd need code like this anyway (since TYPE_SUBSCRIBE is not modular yet), and this seems unlikely to cause problems even if it's a bit rough.

Test Plan:
  - Hard-locked a task.
    - Subscribed/unsubscribed, mentioned, relationship'd it as a non-author. No timeline emblems.
  - Soft-locked a task.
    - Subscribed/unsubscribed, mentioned, relationship'd it, no timeline emblems.
    - Clicked "Edit", answered "yes" to the override prompt, edited it. Got a timeline emblem.
  - Added some comments and stuff to a normal non-locked task, no emblems.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20513
2019-05-14 09:27:24 -07:00
epriestley
ee89c2fad9 Fix a fatal when navigating to a Workboard after removing the Workboard menu item
Summary:
See PHI1247. If you remove the Workboard from a project profile menu, then navigate to the URI, we currently fatal when trying to select the "Workboard" item.

Instead, only try to select the item if some matching item is present.

Test Plan:
  - Disabled the workboard on a project, navigated to `/board/`, got a sensible page with no navigation item selected instead of a fatal.
  - Viewed a normal workboard, saw the correct selection.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20516
2019-05-14 09:26:02 -07:00
epriestley
1d58f14469 Remove WePay support from Phortune, and Restful/Httpful dependencies
Summary: Ref PHI1166. I'm documenting our dependencies, and we have approximately 5,000 lines of external code to support WePay as a Phortune provider. We don't use it, I'm almost certain it doesn't work, and we have no plans to use it in the near future. If we did pursue it, I'd probably just wrap the API in a 100-line `WePayFuture` anyway since 5K lines of dependencies to make a couple method calls is ridiculous.

Test Plan: Grepped for `wepay`, `httpful`, `restful`.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: aurelijus

Differential Revision: https://secure.phabricator.com/D20521
2019-05-14 09:14:53 -07:00
epriestley
7a8d489ebd Fix a HunkParser typo, "identcal" -> "identical"
Summary: See rP3f8eccdaec8f85933bfea6ce5348bde25ced0489.

Test Plan: Read carefully.

Reviewers: amckinley, avivey

Reviewed By: avivey

Subscribers: avivey

Differential Revision: https://secure.phabricator.com/D20499
2019-05-07 11:16:55 -07:00
epriestley
5dc4e76eb9 Fix the direction of the commit/revision edge
Summary: Ref T13276. This edge is pointed the wrong way. Point it the right way.

Test Plan: Will verify production works better.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20490
2019-05-02 07:28:49 -07:00
epriestley
c43618a3a8 Add "Move Left" and "Move Right" to dashboard tab panels
Summary: Depends on D20474. Ref T13272. Provide an easy way to rearrange tabs on a tab panel, by moving them left or right from the context menu.

Test Plan: Moved tabs left and right. Tried to move them off the end of the tab list, no luck.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20475
2019-05-01 15:35:53 -07:00
epriestley
9d716379a3 Add a "Customize Query" action to query panels to make it easier to make minor query adjustments
Summary:
Depends on D20473. Ref T13272. Fixes T7216. If you want to tweak the query a panel uses, you currently have to complete 7 Great Labors.

Instead, add a "Customize Query" action which lets you update the query inline.

Test Plan: {F6402171}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T7216

Differential Revision: https://secure.phabricator.com/D20474
2019-05-01 11:01:09 -07:00
epriestley
a15b0063df Add "Revision has passing builds" Herald rules for commit content (pushes) and commits (discovery)
Summary:
Depends on D20469. Ref T13276. See PHI1159. See PHI953. See PHI901.

Allow Herald to detect when "arc land" would (or did) warn users about failed or ongoing builds. This respects the "Warn on Landing" build plan behavior.

To accomplish this:

  - When we close a revision, set a "wrong build state" flag if it lands in the wrong build state.
  - If the revision is closed when we hit Herald, look for the flag.
  - If not (common for push rules, can happen for commit rules if we race against the revision update worker), hit Harbormaster ourselves and check the current state.

Test Plan:

  - Wrote a "Require Green" rule.
  - Ran it against various commits with various build states (good, not good).
  - Fiddled with "Warn on Landing" and saw the effect in rule evaluation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20470
2019-05-01 10:02:47 -07:00
epriestley
a78f141b3f Unify code for parsing "Reverts X" magic, and when something "reverts <hash>", also revert associated revisions
Summary:
Depends on D20468. Ref T13276. See PHI1008.

When a commit or revision "reverts <hash>", and that's the hash of a commit which has a revision, also write a "reverts" edge to the revision.

Test Plan:
Created "reverts X" objects for:

  - a revision;
  - a commit;
  - a commit with a revision (both got marked properly).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20469
2019-05-01 09:47:40 -07:00
epriestley
56f5c2443b Consolidate readers of "differential.revisionID" property
Summary:
Depends on D20467. Ref T13277. Currently, the "MessageParserWorker" writes this property on commits, then Herald and Audit both read it.

Make them share code so this property has one writer and one reader. This property isn't great, but at least now the badness is hidden.

Currently, we can't just use edges because they may not have been written yet. I am likely to just do this, soon:

  - Just write the edges (in "MessageParserWorker").
  - Hide the edges from mail.

However, we'll sort-of lose the "revisionMatchData" explanation thing if I do that. Maybe this is fine? But when commits match because hashes match, it legitimately isn't obvious.

For now, just reduce the amount of harm/badness here.

Test Plan:
  - Ran `bin/repository reparse --publish ...`.
  - Ran a Herald "Audit" rule using the "Accepted Differential revision" field.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20468
2019-05-01 09:46:17 -07:00
epriestley
45f01dc716 Restore "Limit" to dashboard Query panels
Summary: See PHI1220. Ref T13272. I accidentally left the ability to set a query limit behind when updating this.

Test Plan: Edited a query panel, set/removed the limit, tried to set an invalid limit.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20472
2019-05-01 08:41:43 -07:00
epriestley
dfbc4c1cd3 Fix an issue where editing a dashboard panel from a dashboard could duplicate the panel
Summary:
Depends on D20472. Ref T13272. Currently, when you edit a panel from a dashboard, we try to add the panel to the dashboard. This always works since dashboards no longer enforce panel uniqueness, and you can end up with duplicate panels.

Instead, only add panels if we're creating them.

Test Plan:
  - Edited an existing panel, no duplication.
  - Created a new panel, saw it added to the dashboard.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20473
2019-04-30 08:27:20 -07:00
epriestley
648d5ce190 Set a URI on Auth Messages, so the "Change Details" dialog from the transaction record has a cancel button
Summary:
If you edit an auth message in Auth > Customize Messages, then click "Show Details" in the transaction record, the resulting dialog uses the object's handle's URI to generate a "cancel" button.

Since these handles currently have no URI, the dialog currently has no cancel/done button to close it.

Test Plan: Edited an auth message, clicked "Show Details", was now able to click "Done" to close the dialog.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20471
2019-04-30 06:59:04 -07:00
epriestley
6eae75d8f2 When building a synthetic diff in Differential, adjust diff properties correctly
Summary:
See PHI1218. When rendering "A vs B", we currently show the properties of diff A without modification.

Instead, take properties from the same place we're taking change details.

See T12664 for a followup.

Test Plan:
  - In diff A, removed "+x" from a file.
  - In diff B, changed the file but did not remove "+x" from it.
  - Diffed B vs A.
    - Before change: UI incorrectly shows "+x" removed (both sides incorrect, just showing the change from diff A).
    - After change: UI shows 100644 -> null, which is half right.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20478
2019-04-30 06:51:40 -07:00
epriestley
0d9362355b In ApplicationTransactionEditor, determine new objects with "getID()" instead of "getPHID()"
Summary:
See <https://discourse.phabricator-community.org/t/unable-to-reload-object-that-hasnt-been-loaded/2677>.

When editing "Config" objects, they currently get a PHID set outside of the TransactionEditor. They probably should not, but fixing that is likely an involved change.

This causes us to incorrectly fail to detect `$is_new` correctly and try to `reload()` and object with no ID.

To work around this, test for new objects with `getID()` instead of `getPHID()`.

Test Plan: Edited any config value with the web UI.

Reviewers: amckinley

Differential Revision: https://secure.phabricator.com/D20482
2019-04-29 07:37:03 -07:00
epriestley
c0a4d1de13 Merge the "Herald" and "Owners" daemon workers into a single "Publish" worker
Summary:
Depends on D20466. Ref T13277. Currently:

  - The "Owners" worker writes ownership relationships (e.g., commit X affects package Y, because it touches a path in package Y) -- these are just edges.
  - It also triggers audits.
  - Then it queues a "Herald" worker.
  - This formally publishes the commit and triggers Herald.

These aren't really separate steps and can happen more easily in one shot. Merge them.

Test Plan:
  - Ran `bin/repository reparse --publish` to republish various commits, got sensible behavior.
  - Grepped for "IMPORTED_OWNERS", "IMPORTED_HERALD", "--herald", "--owners", and "--force-local" flags.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20467
2019-04-25 09:45:59 -07:00
epriestley
0fab41ff3c Show "hold reasons" on commit page, not on "Edit" page
Summary:
Depends on D20465. Ref T13277. Currently, when a commit is unpublished, we put a single line about it on the "Edit Commit" page. This is pretty much impossible to find.

Move it to the main page. This treatment is more big/bold than I'd probably like to end up, but we should probably overshoot on the explanatory text until users get used to this behavior.

Also, allow searching for only published / unpublished commits.

Test Plan: {F6395705}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20466
2019-04-25 09:22:49 -07:00
epriestley
8f43c773b8 Remove nearly all remaining references to "Autoclose"
Summary:
Depends on D20464. Ref T13277. Broadly:

  - Move all the "should publish X" and "why aren't we publishing X" stuff to a separate class (`PhabricatorRepositoryPublisher`).
  - Rename things to be more consistent with modern terminology ("Publish", "Permanent Refs").

Test Plan:
This could use some trial-by-fire on `secure`, but:

  - Grepped for all symbols.
  - Viewed various commits.
  - Reparsed commits.
  - Here's a commit with an explanation:

{F6394569}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20465
2019-04-24 08:29:41 -07:00
epriestley
45b9859f02 Remove "--force-autoclose" from "bin/repository reparse"
Summary: Depends on D20463. Ref T13277. This flag was added some time before 2015 and I don't think I've ever used it. Just get rid of it.

Test Plan: Grepped for `force-autoclose`, `forceAutoclose`, `AUTOCLOSE_FORCED`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20464
2019-04-24 08:24:06 -07:00
epriestley
a2d3d8edeb Move "update related object after commit" to a separate worker in the task queue
Summary:
Depends on D20462. Ref T13276. Currently, the "Message" parser also updates related tasks and revisions when a commit is published.

For PHI1165, which ran into a race with message parsing, I originally believed we needed to separate this logic and lock + yield to avoid the race. D20462 provides what is probably a better approach for avoiding the race.

Still, I think separating these "update related revisions" and "updated related tasks" chunks into separate workers is a net improvement. There may still be some value in doing lock + yield in the future to deal with other issues, and when we occasionally run into problems with pulling a diff out of the repository to update the revision (usually because the diff is too big) this isolates the problem better and allows the commit to import.

I think the only thing to watch out for here is that Herald may now run before the revision and commit are attached to one another. This is fine for all current Herald rules, we just need to be mindful in implementing new rules.

Test Plan: Used `bin/repository reparse --message` on various commits, including commits that close revisions and close tasks.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20463
2019-04-24 06:32:27 -07:00
epriestley
b3b1cc64bd When applying transactions, acquire a read lock sooner
Summary:
Depends on D20461. Ref T13276. Ref T13054.

Currently, we acquire the transaction read lock after populating "old values" in transactions and filtering transactions with no effect.

This isn't early enough to prevent all weird chaotic races: if two processes try to apply a "close revision" transaction at the same time, this can happen:

```
PROCESS A             PROCESS B
Old Value = Open      Old Value = Open
Transaction OK: Yes   Transaction OK: Yes
Acquire Read Lock     Acquire Read Lock
Got Read Lock!        Wait...
Apply Transactions    Wait...
New Value = Closed    Wait...
Release Lock          Wait...
                      Got Read Lock!
                      Apply Transactions
                      New Value = Closed
                      Release Lock
```

That's not great: both processes apply an "Open -> Closed" transaction since this was a valid transaction from the viewpoint of each process when it did the checks.

We actually want this:

```
PROCESS A             PROCESS B
Acquire Read Lock     Acquire Read Lock
Got Read Lock!        Wait...
Old Value = Open      Wait...
Transaction OK: Yes   Wait...
Apply Transactions    Wait...
New Value = Closed    Wait...
Release Lock          Wait...
                      Got Read Lock!
                      >>> Old Value = Closed
                      >>> Transaction Has No Effect!
                      >>> Do Nothing / Abort
                      Release Lock
```

Move the "lock" part up so we do that.

This may cause some kind of weird second-order effects, but T13054 went through pretty cleanly and we have to do this to get correct behavior, so we can survive those if/when they arise.

Test Plan:
  - Added a `sleep(10)` before the lock.
  - Ran `bin/repository message --reparse X` in two console windows, where X is a commit that closes revision Y and Y is open.
    - Before patch: both windows closed the revision and added duplicate transactions.
    - After patch: only one of the processes had an effect.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jmeador

Maniphest Tasks: T13276, T13054

Differential Revision: https://secure.phabricator.com/D20462
2019-04-24 05:57:29 -07:00
epriestley
42c02557e4 Remove all remaining readers and writers for TABLE_COMMIT
Summary: Depends on D20459. Ref T13276. I'll file a followup to actually destroy the table.

Test Plan:
- Grepped for `TABLE_COMMIT`.
- Ran `bin/storage upgrade -f`, got a clean bill of health.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20461
2019-04-24 05:55:23 -07:00
epriestley
ec0085fd0c Align "RevisionQuery->needCommitPHIDs()" to use Edges, not the legacy table
Summary:
Depends on D20458. Ref T13276. Although I'm not thrilled about "needCommitPHIDs()", it has a few callers, including custom fields. Allow "need + attach + get" to survive for now since they're reasonably modern, at least.

However, use edges instead of "TABLE_COMMIT" and require `need...()` + `get...()`, removing the direct `load...()`.

Also remove `RevisionQuery->withCommitPHIDs(...)`, which has no callers.

Test Plan:
- Grepped for `loadCommitPHIDs` (only two hits, the private `RevisionQuery` method).
- Called "differential.getrevision", got commits.
- Viewed a revision, saw "Commits: ...".
- Grepped for `withCommitPHIDs()`, no callers on `RevisionQuery` (some other query classes have methods with this name).
- Called "differential.query", got commits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20459
2019-04-24 05:54:38 -07:00
epriestley
7e8dc0742b Remove all callers to "DifferentialRevision->loadIDsByCommitPHIDs()"
Summary: Depends on D20457. Ref T13276. Kill all remaining callers to this method and delete it.

Test Plan:
- Grepped for `loadIDsByCommitPHIDs`.
- Viewed blame again to make sure I didn't break it.
- Viewed "History" view for commits with revisions.
- Viewed "Graph" view for commits with revisions.
- Viewed "Merged Commits" table for commits with revisions.
- Viewed "Compare" table for commits with revisions.
- Viewed "Repository" main page history table for commits with revisions.
- Grepped for `linkRevision`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20458
2019-04-24 05:53:35 -07:00
epriestley
fc92cf4382 Move BlameController away from ancient "TABLE_COMMIT"
Summary:
Ref T13276. Differential has a pre-edge "TABLE_COMMIT" with about a half-dozen weird callers I'd like to get rid of.

Move blame to use edges instead. (Bonus: this makes blame respect edge edits in the UI.)

Since there are some more callers to clean up this code may move into some "RelatedObjectQueryThing" class or something, but I'm taking it one step at a time for now.

Test Plan: {F6394106}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20457
2019-04-24 05:44:40 -07:00
epriestley
70c643c685 Simplify implementation of "pure" Chart functions
Summary:
Depends on D20445. Ref T13279. I'm not sure what the class tree of functions actually looks like, and I suspect it isn't really a tree, so I'm hesitant to start subclassing. Instead, try adding some `isSomethingSomething()` methods.

We have some different types of functions:

  # Some functions can be evaluated anywhere, like "constant(3)", which always evaluates to 3.
  # Some functions can't be evaluated anywhere, but have values everywhere in some domain. This is most interesting functions, like "number of open tasks". These functions also usually have a distinct set of interesting points, and are constant between those points (any count of anything, like "open points in project" or "tasks closed by alice", etc).
  # Some functions can be evaluated almost nowhere and have only discrete values. This is most of the data we actually store, which is just "+1" when a task is opened and "-1" when a task is closed.

Soon, I'd like to be able to show ("all tasks" - "open tasks") and draw a chart of closed tasks. This is somewhat tricky because the two datasets are of the second class of function (straight lines connecting dots) but their "interesting" x values won't be the same (users don't open and close tasks every second, or at the same time).

The "subtract X Y" function will need to be able to know that `subtract "all tasks" 3` and `subtract "all tasks" "closed tasks"` evaluate slightly differently.

To make this worse, the data we actually //store// is of the third class of function (just the "derivative" of the line chart), then we accumulate it in the application after we pull it out of the database. So the code will need to know that `subtract "derivative of all tasks" "derivative of closed tasks"` is meaningless, or the UI needs to make that clear, or it needs to interpret it to mean "accumulate the derivative into a line first".

Anyway, I'll sort that out in future changes. For now, simplify the easy case of functions in class (1), where they're just actual functions.

Add "shift(function, number)" and "scale(function, number)". These are probably like "mul" and "add" but they can't take two functions -- the second value must always be a constant. Maybe these will go away in the future and become `add(function, constant(3))` or something?

Test Plan: {F6382885}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20446
2019-04-23 12:29:45 -07:00
epriestley
edaf17f3fe Make chart function argument parsing modular/flexible with 900 pages of error messages
Summary:
Depends on D20444. Ref T13279. Instead of ad-hoc parsing and messages, formalize chart function arguments.

Also, add a whole lot of extra type checking.

Test Plan: Built and charted various functions with various valid and invalid argument lists, got sensible-seeming errors and results.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20445
2019-04-23 12:26:32 -07:00
epriestley
7b8ac020b5 Change "Revision Close" story to use commit identities only with no fallback to commit data
Summary:
See PHI1213. If we don't have identities for "revision X closed by commit Y" stories, just do the plain non-attribution rendering rather than trying to fall back. Falling back won't work since we don't load the data, which should be OK now since identities seem like they're in generally good shape.

(We could probably just throw out the fallback behavior instead, but we can always clean things up later.)

Test Plan: Forced no commit identity on a revision, loaded it, saw reasonable story rendering.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20460
2019-04-23 11:57:33 -07:00
epriestley
b0b8926c75 Fix "before/after" cursor paging for API call "feed.query"
Summary:
Ref T13266. See <https://discourse.phabricator-community.org/t/undefined-method-setafterid-when-calling-feed-query/2653>.

This older API call needs an update to the newer paging/cursor API.

Test Plan: Called `feed.query` with an "after" parameter.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: Itms

Maniphest Tasks: T13266

Differential Revision: https://secure.phabricator.com/D20456
2019-04-23 11:51:16 -07:00
epriestley
a4a1143b18 Fix an issue where internal paging of notifications could fail if some notifications are not visible
Summary:
Ref T13266. See <https://discourse.phabricator-community.org/t/notification-page-throws-unrecoverable-fatal-error/2651/>.

The "notifications" query currently uses offset paging for no apparent reason (just a legacy issue?), so some of the paging code is only reachable internally.

  - Stop it from using offset paging, since modern cursor paging is fine here (and Feed has used cursor paging for a long time).
  - Fix the non-offset paging to work like Feed.

Also:

  - Remove a couple of stub methods with no callsites after cursor refactoring.

Test Plan:
  - Set things up so I had more than 100 notifications and some in the first 100 were policy filtered, to reproduce the issue (I just made `FeedStory` return `NO_ONE` as a visibility policy).
  - Applied the patch, notifications now page cleanly.
  - Verified that "Next Page" took me to the right place in the result list.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: hskiba

Maniphest Tasks: T13266

Differential Revision: https://secure.phabricator.com/D20455
2019-04-23 11:45:04 -07:00
epriestley
a82a2b8459 Simplify non-bare working copy rules for the new "git fetch" strategy
Summary:
Ref T13280. In D20421, I changed our observe strategy to `git fetch <uri>` in all cases.

This doesn't work in an ancient, non-bare repository if `master` is checked out and `master` is also fetch: `git` refuses to overwrite the local ref unless we pass `--update-head-ok`. Pass this flag.

Also, remove some code which examines branches and tags in a special way for non-bare working copies. The old `git fetch <origin>` code without explicit revsets meant that `refs/remotes/orgin/heads/xyz` got updated instead of `refs/heads/xyz`. We now update our local refs in all cases (bare and non-bare) so we can throw away this special casing.

Test Plan:
  - Replaced a modern bare working copy with a non-bare working copy by explicitly using `git clone` without `--bare`.
  - Ran `bin/repository update`, hit the `--update-head-ok` error. Applied the patch, got a clean update.
  - Used the "repository.branchquery" API method...
    - ...with "contains" to trigger the "git branch" case. Got identical results after removing the special casing.
    - ...without "contains" to trigger the "low level ref" case. Got identical results after removing the special casing.
  - Grepped for `isWorkingCopyBare()`. The only remaining callsites deal with hook paths, and genuinely need to be specialized.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13280

Differential Revision: https://secure.phabricator.com/D20450
2019-04-19 07:06:10 -07:00
epriestley
e1076528ef Copy the "line-chart" behavior to "line-chart-legacy" to keep "Maniphest > Reports" working
Summary:
Ref T13279. Charting changes alter how the "line-chart" behavior works, but the "Burnup Chart" still relies on the old behavior.

Although I'm intending to remove "Maniphest > Reports" once Facts is a minimally sufficient replacement, copy this behavior to keep it working until we're ready to pull the trigger.

Also fix a leftover typo from D20435.

Test Plan: Viewed a legacy Maniphest burnup rate report.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20449
2019-04-19 07:05:37 -07:00
Austin McKinley
de74332639 Improve timeline rendering for old macros
Summary: This always bugs me when I'm going through `secure` looking for the spiciest macros.

Test Plan: Forced a date to be null, saw reasonable text.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20453
2019-04-18 18:25:17 -07:00
epriestley
d8dba26a08 Add a "sin()" function to charts
Summary:
Depends on D20443. Ref T13279. This is probably not terribly useful on its own, but is mostly a function which takes another function as an argument, and a step toward more useful functions like arithmetic and drawing a picture of an owl.

The only structural change here is that functions now read data parameters (domain, sample limit) using a more tailored "ChartDataQuery" instead of reading the actual axis. Mostly, I want a more cohesive representation of query state that can be easily passed to sub-functions, as here.

Test Plan: {F6382432}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20444
2019-04-18 13:19:02 -07:00
epriestley
c4e4448ee6 Add chart functions "x()" and "constant(3)"
Summary:
Depends on D20442. Ref T13279. Add basic support for drawing chart functions that are not based on Facts first-party ETL datasets. Some general goals:

  - This might be useful to draw a line like "goal" or "profitability".
  - This might be useful to pull data from an external source.
  - For composable functions like "add" or "subtract", which are useful in manipulating ETL datasets, these value functions will make testing easier.

Test Plan:
Added a `constant(256)` function:

{F6382408}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20443
2019-04-18 13:16:42 -07:00
epriestley
cd2215fd4a Don't warn that workboard columns need a name when editing milestone columns
Summary: See <https://discourse.phabricator-community.org/t/columns-must-have-a-name-while-editong-point-limit-on-milestone-column/2650>. This check doesn't make sense for proxy columns, including milestone columns.

Test Plan: Added a point limit to a milestone column.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20448
2019-04-18 13:13:10 -07:00
epriestley
23d8f000f7 Select the domain (X-axis range) for charts before pulling data
Summary:
Depends on D20441. Ref T13279. Currently, we pull all the data, then decide what the X-axis should look like.

Since users will reasonably want to do stuff like "show me march-april 2018" in the future, we need to move toward flipping this around so that we can support cases where the domain is specified by the user.

For actual chart functions (like "constant(3)" or "cos(x)"), we must also know the domain before we pull data, since there are an infinite number of places where we can evaluate the function "constant(3)".

See note in T13279 about continunity.

Test Plan: {F6382356}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20442
2019-04-18 13:11:17 -07:00
epriestley
954831f533 Separate chart functions into a class tree
Summary:
Depends on D20440. Ref T13279. Create a class to represent a chartable function: something we can get some data points out of.

Then, make the chart chart two functions.

For now, the only supported function is "fact(key)", which pulls data from the Facts ETL pipeline, identified by "key", and takes no other arguments.

In future changes, I plan to support things like "fact(tasks.open.project, PHID-PROJ-xyz)", "constant(1000)" (e.g. to draw a goal line), "sum(fact(...), fact(...))" (to combine data from several projects), and so on.

The UI may not expose this level of power for a while (or maybe ever) but until we get close enough to the UI that these features are a ton of extra work I'm going to try to keep things fairly flexible/modular.

Test Plan: {F6382286}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20441
2019-04-18 13:05:52 -07:00
epriestley
45b3c23148 Fetch chart data via async request and redraw charts when the window is resized
Summary:
Depends on D20439. Ref T13279. Some day, charts will probably need to reload themselves or do a bunch of defer/request-shaping magic when they're on a dashboard with 900 other charts.

Give the controller separate "HTML placeholder" and "actual data" modes, and make the placeholder fetch the data in a separate request.

Then, make the chart redraw if you resize the window instead of staying at whatever size it started as.

Test Plan:
  - Loaded a chart, saw it load data asynchronously.
  - Resized the window, saw the chart resize.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20440
2019-04-18 07:10:05 -07:00
epriestley
044c6fbc19 Support drawing multiple functions on the same chart
Summary: Depends on D20438. Ref T13279. Widgets produced vs widgets sold, etc.

Test Plan: {F6381609}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20439
2019-04-18 07:07:33 -07:00
epriestley
02981f0add Fix negative chart values, add storage for charts
Summary:
Ref T13279. I think I'm going to fling some stuff at the wall for a bit here and hope most of it sticks, so this series of changes may not be terribly cohesive or focused. Here:

The range of the chart is locked to "[0, 105% of max]". This is trying to make a pleasing extra margin above the maximum value, but currently just breaks charts with negative values. Later:

    - I'll probably let users customize this.
    - We should likely select 0 as the automatic minimum for charts with no negative values.
    - For charts with positive values, it would be nice to automatically pick a pleasantly round number (25, 100, 1000) as a maximum by default.

We don't have any storage for charts yet. Add some. This works like queries, where every possible configuration gets a short URL slug. Nothing writes or reads this yet.

Rename `fn()` to `css_function()`. This builds CSS functions for D3. The JS is likely to get substantial structural rewrites later on, `fn()` was just particularly offensive.

Test Plan: Viewed a fact series with negative values. Ran `bin/storage upgrade`.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20438
2019-04-18 06:59:41 -07:00
epriestley
faf0a311ec In Git repositories, use "git symbolic-ref HEAD ..." to select the default branch
Summary:
Depends on D20434. Fixes T5963. Broadly, the issue here is that when:

  - You create a new, empty repository.
  - Then, you work on some branch other than `master`, without ever creating `master`.

...you get a warning on `git clone`:

> warning: remote HEAD refers to nonexistent ref, unable to checkout

To fix this, point the symbolic-ref HEAD at `refs/heads/<default-branch>` after installing commit hooks.

This fixes the warning, and also means that `git clone` will check out the repository default branch by default, which is nice.

There are a few caveats about this behavior (see T5963 for discussion) but nothing too substantial.

The only real issue is that Git prevents deletion of the default branch without a config setting. Just set that settting.

Test Plan:
See T5963.

In a repository, set `HEAD` to point somewhere invalid. Ran `bin/repository update ...`. Saw HEAD pointed back at the repository default branch.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5963

Differential Revision: https://secure.phabricator.com/D20435
2019-04-18 05:46:40 -07:00
epriestley
7be671fb07 Update "Autoclose" documentation to focus on "Permanent Refs" instead
Summary:
Depends on D20433. Ref T13277. Since "Autoclose" no longer exists, update the documentation.

Currently, this documentation focuses a lot on troubleshooting because users historically had a lot of trouble with figuring out why things were or were not autoclosing. I haven't seen any real confusion about this in years, so I suspect we may have improved the import pipeline and/or UI to make this less of a problem.

It's also possible that this document "fixed" the problem, but usually I expect a documentation fix to not affect the frequency of reports, just make them easier to resolve, so I doubt it.

If unclear things remain //and// documentation really did fix it, maybe we can fix the issues. Or we can just put the troubleshooting documentation back.

Test Plan: Read documentation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20434
2019-04-18 05:43:15 -07:00
epriestley
c33f544e74 Deprecate "Track Only" in the Diffusion UI
Summary:
Depends on D20432. Ref T13277. Fixes T12967. Removes some "Track Only" hints and warns that the feature is deprecated in favor of "Permanent Refs" and "Fetch Only".

(This "fixes" T12967 by mooting it.)

Test Plan: Viewed "branches" sectino of the manage UI, edited "braches" section of a repository.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277, T12967

Differential Revision: https://secure.phabricator.com/D20433
2019-04-18 05:40:02 -07:00
epriestley
6449a0ecb2 Rename some internal "Autoclose" mentions to "Permanent Refs"
Summary: Depends on D20428. Ref T13277.

Test Plan: Grep / reading.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20432
2019-04-18 05:38:09 -07:00
epriestley
d1223ac577 When a commit appears as an ancestor of a permanent ref for the first time, run all import steps
Summary:
Depends on D20427. Ref T13277. As an optimization, when we discover that a commit which was previously only on a non-permanent ref ("tmp-epriestley-123") is now reachable from a permanent ref ("master"), we currently queue only a new "message" parse step.

This is an optimization because these commits previously got the full treatment (feed, publish, audit, etc) as soon as they were discovered. Now, those steps only happen once a commit is reachable from a permanent ref, so we need to run everything.

Test Plan:
  - Pushed local "tmp-123" branch to remote tag "tmp-123".
  - Updated repository with "bin/repository update", saw commit import as a "not on any permanent ref" commit, with no herald/audit/etc.
  - Merged "tmp-123" tag into "master".
  - Pushed new "master".
  - Updated repository with "bin/repository refs ... --trace --verbose", saw commit detected as now reachable from a permament ref.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20428
2019-04-18 05:37:15 -07:00
epriestley
9107c2e262 Deprecate the "Commit is on autoclose/permanent branch" Herald "Commit" field
Summary:
Depends on D20426. Ref T13277. The new behavior is to fire Herald only once a commit becomes reachable from a permanent ref (previously, an "Autoclose" branch).

That means that every "Commit" Herald rule implicitly has this field as a condition, and it no longer does anything.

Test Plan: Wrote a Herald rule, saw this as an option in the "Deprecated" section.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20427
2019-04-18 05:32:45 -07:00
epriestley
aed755e1d8 Expose repository ref rules via "diffusion.repository.search"
Summary:
Depends on D20425. Ref T13277. See PHI1067. There's currently no way to retrieve branch/ref rules over the API, which makes some management operations against a large number of repositories difficult.

Expose these rules to the API.

Test Plan: Called `diffusion.repository.search`, got rules in the result set.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20426
2019-04-18 05:27:37 -07:00
epriestley
7a4ef2bad8 Move the repository "Publishing" option to the "Basics" panel in repository management
Summary:
Depends on D20424. Ref T13277. Now that the "Actions" panel only has one item ("Publishing"), just move it to the "Basics" panel.

Update the UI to show active/publishing status more clearly and relate them to one another and importing state.

Test Plan: {F6378087}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20425
2019-04-18 05:24:27 -07:00
epriestley
ec9237fe13 In repository settings, fold "Autoclose On/Off" into "Publishing On/Off"
Summary:
Depends on D20423. Ref T13277. Repositories currently have separate toggles for "Autoclose" and "Publishing".

Merge the "Autoclose" toggle into the "Publishing" toggle. I'm unaware of any valid use case for enabling one but not the other.

(This doesn't fix all the documentation, yet.)

Test Plan: Edited a repository, saw only one publishing option.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20424
2019-04-18 05:16:59 -07:00
epriestley
c7b2553ca0 Rename most user-facing "Autoclose" strings to "Permanent Refs"
Summary:
Depends on D20422. Ref T13277. Currently, "track only", "publish", and "autoclose" are three separate ideas. I'd like to generally merge them into a more natural idea called "permanent refs".

Since "Autoclose" effectively now controls both "autoclose" and "publish", rename it.

This doesn't rename all the methods or internals, and the documentation needs an update, but it renames most of the UI-facing stuff.

(You also can only specify branches as "Permanent Refs" today, but we may let you specify tags and other arbitrary refs in the future.)

Test Plan: Grepped, poked around the UI, saw UI show "Permanent" / "Permanent Refs" more often and "Autoclose" less.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20423
2019-04-18 05:14:36 -07:00
epriestley
e910c76e65 Add "Fetch Rules" to observed Git repositories
Summary:
Depends on D20421. Ref T13277. I'd generally like to move away from "Track Only".

Some of the use cases for "Track Only" (or adjacent to "Track Only") are better resolved with "Fetch Rules" -- basically, rules to fetch only some subset of refs from the observed remote.

Add configurable "Fetch Rules" for Git repositories. For example, if you only want to fetch `master`, you can now speify:

```
refs/heads/master
```

If you only want to fetch branches and tags, you can use:

```
refs/heads/*
refs/tags/*
```

In theory, this is slightly less powerful in the general case than "Track Only", but gives us better behavior in some cases (e.g., when the remote has 50K random temporary branches). In practice, I think this and a better "Autoclose Only" will let us move away from "Track Only", get default behavior which is better aligned with what users actually expect, and dodge all the "track tags/refs" questions.

Test Plan: Configured repositories with "Fetch Refs" rules, used `bin/repository pull --verbose --trace ...` to run pulls, saw expected pull/fetch behavior.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20422
2019-04-18 05:09:36 -07:00
epriestley
78aab6d4a5 When observing a repository in Git, just "fetch <url>" without worrying about the "origin" remote
Summary:
Depends on D20420. Ref T13277. We currently spend substantial effort trying to detect and correct the URL of the "origin" remote in Git repositories.

I believe this is unnecessary, and we can always `git fetch <url> ...` to get the desired result instead of `git muck-with-origin + git fetch origin ...`. We already do this in the more recent parts of the codebase (e.g., intracluster sync) and it works correctly in every case I'm aware of.

Test Plan:

  - Grepped for `origin`, ` origin `.
  - Ran `bin/repository update ...` to fetch a mirrored repository.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20421
2019-04-18 05:05:43 -07:00
epriestley
1cda1402c7 Do not publish/notify about commits which are not reachable from any "Autoclose" ref
Summary:
Depends on D20418. Ref T13277. Fixes T11314.

Currently, when you push commits to some arbitrary ref or tag (like `refs/pull/123` on GitHub, `refs/tags/phabricator/diff/123` on Phabricator, or `refs/changes/whatever` on Gerrit), we do not "autoclose" related objects. This means that we don't process `Ref T123` to create links to tasks, and don't process `Differential Revision: xyz` to close revisions.

However, we //do// still publish these commits. "Publish" means: trigger audits, publish feed stories, and run Herald rules.

  - Stop publishing these commits.
  - In the UI, show these commits as "Not Permanent" with a note that they are "Not [on any permanent branch]."

These commits will publish and autoclose if they ever become reachable from an "autoclose" ref (most commonly, if they are later merged to "master").

Test Plan:
  - Pushed a commit to `refs/tags/quack`.
  - Before: got a feed story.
  - After: no feed story, UI shows commit as "Not Permanent".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277, T11314

Differential Revision: https://secure.phabricator.com/D20419
2019-04-18 05:05:00 -07:00
epriestley
904dbf0db6 Make the "git upload-pack" proxy more robust
Summary:
Depends on D20381. Ref T8093. This makes minor improvements to the protocol proxy to handle cases where we add, remove, or replace refs and may need to move the "capabilities" section.

Rather than invoking a callback on every ref: parse the whole ref list into a data structure, mutate it if necessary (in a future diff), then dump it back into wire format.

This allows us to shift the capabilities data (which needs to be coupled with the first ref) around if we modify the first ref, and reorder the reflist alphabetically like git does.

When the server has no refs, Git sends no capabilities data. This is easy to emulate, just surprising.

Test Plan:
Tested the cases not covered by D20381:

  - Fetching where the fetch actually fetches data.
  - `ls-remote` when we hide the first ref (capabilities data properly moves to the first visible ref).
  - `ls-remote` when the remote is empty (we just drop the capabilities frame completely).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T8093

Differential Revision: https://secure.phabricator.com/D20436
2019-04-18 05:04:05 -07:00
epriestley
e08ba99dd3 Proxy the "git upload-pack" wire protocol
Summary:
Depends on D20380. Ref T8093. When prototypes are enabled, inject a (hopefully?) no-op proxy into the Git wire protocol.

This proxy decodes "git upload-pack" and allows the list of references to be rewritten, in a similar way to how we already proxy the Subversion protocol to rewrite URIs and proxy the Mercurial protocol to distinguish between read and write operations.

The piece we care about comes at the beginning, and looks like this:

```
<frame-length><ref-hash> <ref-name>\0<server-capabilities>\n
<frame-length><ref-hash> <ref-name>\n
<frame-length><ref-hash> <ref-name>\n
...
<0000>
```

We can add, remove, or modify this section to make it appear that the server has different refs than the refs that exist on disk.

Things I have tried:

  - `git ls-remote`
  - `git ls-remote` where the server hides some refs.
  - `git fetch` where the fetch is a no-op.

Things I have not tried:

  - `git fetch` where the fetch is not a no-op.
  - Tricking things into doing protocol v2. Or: I tried this, I wasn't successful. In v2, additional "\0" tricks are used to hide data in the capabilities, I think?
  - `git ls-remote` where we rewrite/hide the first ref in the list, and need to move the capabilities frame elsewhere.
  - `git ls-remote` where the server has no refs at all, or we remove every ref.

So the "interesting" piece of this works, but it almost certainly needs some cleanup to survive interaction with the real world.

Test Plan: See above.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T8093

Differential Revision: https://secure.phabricator.com/D20381
2019-04-18 04:57:51 -07:00
epriestley
35539a019c Add an optional protocol log to git SSH workflows
Summary:
Ref T8093. Support dumping the protocol bytes to a side channel logfile, as a precursor to parsing the protocol and rewriting protocol frames to virtualize refs.

The protocol itself is mostly ASCII text so the raw protocol bytes are pretty comprehensible.

Test Plan:
{F6363221}

{F6363222}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T8093

Differential Revision: https://secure.phabricator.com/D20380
2019-04-18 04:57:04 -07:00
epriestley
c648077625 Don't warn about a locked database value after users run "bin/auth lock"
Summary: Ref T7667. Also one text fix.

Test Plan: Ran `bin/auth lock`, didn't get a replacement setup issue.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T7667

Differential Revision: https://secure.phabricator.com/D20447
2019-04-17 19:54:11 -07:00
epriestley
02cfcfa079 Hide revision dependency stories from feed/notifications
Summary:
See PHI1134. Generally, "alice added a dependent revision: ..." isn't a very interesting story. This relationship itself is valuable, but the creation of the relationship is usually pretty obvious from context.

In the specific case of PHI1134, various scripts are racing one another, but I don't think this story is of much value in the general case anyway.

Test Plan: Edited parent/child revisions, no more feed stories.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20437
2019-04-17 17:08:14 -07:00
epriestley
382e2c32fc When an invalid "constraints" parameter is provided to a "*.search" method, raise a more tailored error
Summary:
See <https://discourse.phabricator-community.org/t/constraints-parameter-type-handling-in-maniphest-search/2636>.

(The caller provided `constraints={...}` via cURL, but you can't mix JSON and HTTP parameters like that.)

Test Plan: Called `curl 'http://local.phacility.com/api/maniphest.search?api.token=...&constraints=X'` (with a valid API token), got a more sensible error.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20429
2019-04-17 13:17:07 -07:00
epriestley
23b86bae6c Accept pushes with arbitrary Git refs
Summary:
Depends on D20419. Ref T13277. Fixes T8936. Fixes T9383. Fixes T12300. When you push arbitrary refs to Phabricator, the push log currently complains if those refs are not tags or branches.

Upstream Git now features "notes", and there's no reason to prevent writes to arbitrary refs, particularly beause we plan to start using them soon (see T13278).

Allow these writes as affecting raw refs.

Test Plan:
  - Pushed an arbitrary ref.
  - Pushed some Git notes.
  - Wrote a Herald ref rule, saw "ref" in the dropdown.

{F6376492}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277, T8936, T9383, T12300

Differential Revision: https://secure.phabricator.com/D20420
2019-04-17 12:43:20 -07:00
epriestley
870b01f2d0 Distinguish between "bad record format" and "bad record value" when validating Trigger rules
Summary:
Depends on D20416. Ref T13269. See D20329.

If you try to save an "Assign to" rule with no assignee, we currently replace the control with an "InvalidRule" control that isn't editable. We'd prefer to give you an empty field back and let you pick a different value.

Differentiate between "bad record format" (i.e., we can't really do anything with this) and "bad record value" (i.e., everything is structurally fine, you just typed the wrong thing). In the latter case, we still build a properly typed rule for the UI, we just refuse to update storage until you fix the problem.

Test Plan:
First, hit the original issue and got a nicer UI with a more consistent control width (note full-width error):

{F6374205}

Then, applied the rest of the patch and got a normal "fix the issue" form state instead of a dead-end:

{F6374211}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13269

Differential Revision: https://secure.phabricator.com/D20417
2019-04-17 12:40:55 -07:00
epriestley
5b6d6c4fb3 Use repository identities, not denormalized strings, to identify authors for "Revision closed by commit X" stories
Summary:
Ref T12164. Ref T13276. Currently, the parsing pipeline copies the author and committer names and PHIDs into the transcaction record as metadata. They are then rendered directly from the metadata.

This makes planned changes to the parsing pipeline (to prevent races when multiple commits matching a single revision are pushed simultaneously) more difficult, and generally won't work with repository identities.

Instead, load the commit and use its identities.

Test Plan: Loaded a revision, saw the same story rendering for a "Closed by commit" story.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276, T12164

Differential Revision: https://secure.phabricator.com/D20418
2019-04-17 12:24:49 -07:00
epriestley
9532bfbb32 Improve trigger editor behavior when switching to/from tokenizers
Summary:
Ref T13269. See D20329. When we switch trigger rule control types, reset the rule value.

Also, pick slightly nicer defaults for status/priority.

Test Plan:
  - Created a "Change Status To: X" rule.
  - Saved it.
  - Edited it.
  - Selected "Assign to" for the existing action's dropdown.
  - Before: tokenizer filled with nonsense.
  - After: tokenizer cleared.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13269

Differential Revision: https://secure.phabricator.com/D20416
2019-04-17 12:24:23 -07:00
epriestley
b8551bb5f9 Reduce drag-and-drop jank on dashboards
Summary:
Depends on D20414. Ref T13272. Several minor things here:

  - Currently, you can drag panels underneath the invisible "there are no items in this column" div and the "Create Panel / Add Existing Panel" buttons. This is silly; stop it.
  - Currently, when viewing a tab panel on a dashboard, you can drag the panels inside it. This is extremely silly. Make "movable" off by default and pass it through the async flow only when we actually need it.
  - Make the whole "Add Tab..." virtual tab clickable to open the dropdown. This removes the rare exception/todo combo I added earlier. {key F}
  - Add or remove some icons or something.

Test Plan: Moved panels around on dashboards. Tried to drag panels inside tab panels. Added tab. Things were less obviously broken.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20415
2019-04-17 12:20:44 -07:00
epriestley
e99c1974b0 Fix the "Add Query to Dashboard..." flow from "Use Results" on search result pages
Summary:
Depends on D20413. Ref T13272. When you search for stuff, you can "Use Results > Add to Dashboard" to generate a query panel.

This needs some updating after the recent refactoring. All the changes are pretty straightforward. Swaps a giant `<select />` for a tokenizer with a datasource.

Test Plan: Used the "Use Results > Add to Dashboard" flow to create a panel on a dashboard using a query.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20414
2019-04-17 12:18:52 -07:00
epriestley
4b8a67ccde Index and show Owners packages affected by Herald rules
Summary:
Depends on D20412. See PHI1147.

  - Index the targets of "Add Reviewer", "Add Blocking Reviewer", "Add Auditor", "Add Subscriber", and "Remove Subscriber" Herald rules. My major goal is to get Owners packages. This will also hit projects/users, but we just don't read those edges (for now, at least).
  - Add a "Related Herald Rules" panel to Owners Package pages.
  - Add a migration to reindex Herald rules for the recent build plan stuff and this, now that such a migration is easy to write.

Test Plan:
Ran migration, verified all rules reindexed.

{F6372695}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20413
2019-04-17 12:17:30 -07:00
epriestley
4eab3c4c8d Reindex dashboards and panels (allow migrations to queue a job to queue other indexing jobs)
Summary:
Depends on D20411. Ref T13272. Dashboards and panels have new indexes (Ferret and usage edges) that need a rebuild.

For large datasets like commits we have the "activity" flow in T11932, but realistically these rebuilds won't take more than a few minutes on any realistic install so we should be able to just queue them up as migrations.

Let migrations insert a job to basically run `bin/search index --type SomeObjectType`, then do that for dashboards and panels.

(I'll do Herald rules in a followup too, but I want to tweak one indexing thing there.)

Test Plan: Ran the migration, ran `bin/phd debug task`, saw everything get indexed with no manual intervention.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20412
2019-04-17 12:05:49 -07:00
Austin McKinley
0583f6dc50 Some formatting changes for showing auth provider config guidance
Summary:
Ref T7667. On the road to locking the auth config, also clean up some minor UI issues:

* Only show the warning about not Phacility instance auth if the user isn't a manager (see next diff).
* When rendering more than one warning in the guidance, add bullets.
* I didn't like the text in the `auth.config-lock` config setting.

Test Plan: Loaded the page, saw more reasonable-looking guidance: {F6369405}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T7667

Differential Revision: https://secure.phabricator.com/D20400
2019-04-17 11:08:16 -07:00
epriestley
e7a31832bf Show a warning when "git" is too old to support filesize limits
Summary: See <https://discourse.phabricator-community.org/t/git-push-failed-with-filesize-limit-on/2635/2>

Test Plan: {F6378519}

Reviewers: amckinley, avivey

Reviewed By: avivey

Differential Revision: https://secure.phabricator.com/D20431
2019-04-16 05:33:29 -07:00
epriestley
5a2d0f0437 Update documentation for "uri.allowed-protocols"
Summary: See <https://discourse.phabricator-community.org/t/download-tasks-and-others-as-excel-throw-exception/>.

Test Plan: Read config.

Reviewers: amckinley, avivey

Reviewed By: avivey

Subscribers: avivey

Differential Revision: https://secure.phabricator.com/D20430
2019-04-16 05:23:13 -07:00
epriestley
f13709b13b Update search indexes for Dashboards and Panels to Ferret, plus various minor fixes
Summary:
Depends on D20410. Ref T13272. Dashboards/Panels currently use older "ngram" indexing, which is a less-powerful precursor to Ferret. Throw away the ngram index and provide a Ferret index instead. Also:

  - Remove the NUX state, which links to the wrong place now and doesn't seem terribly important.
  - Add project tags to the search result list.
  - Make the "No Tags" tag a little less conspicious.

Test Plan:
  - Indexed dashboards and panels.
  - Searched for dashboards and panels via SearchEngine using Ferret "query" field.
  - Searched for panels via "Add Existing Panel" datasource typeahead.
  - Searched for dashboards via "Add Menu Item > Dashboard" on a ProfileMenu via typeahead.
  - Viewed dashboard NUX state (no special state, but no more bad link to "/create/").
  - Viewed dashboard list, saw project tags.
  - Viewed dashboards with no project tags ("No Tags" is now displayed but less visible).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20411
2019-04-14 10:28:19 -07:00
epriestley
80b7274e0b Remove legacy "DashboardInstall" table
Summary: Depends on D20409. Ref T13272. Before "ProfileMenu", dashboards were installed on specific objects using this table. Installs are now handled via ProfileMenu and this table no longer has any meaningful readers. Remove references to the table and destroy it.

Test Plan: Grepped for `DashboardInstall`.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20410
2019-04-14 10:27:52 -07:00
epriestley
fb994909cf Make "Move Panel" on dashboards use the new storage and transactions
Summary: Depends on D20408. Ref T13272. The actual JS is still a little bit iffy, but this makes the server side "move" operation work correctly by updating it to use the same code as everything else.

Test Plan: Moved panels around on single-column and multi-column dashboards, saw them move to reasonable places and stay there when I reloaded the page.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20409
2019-04-14 10:25:22 -07:00
epriestley
82c46f4b93 Update "add panel" and "remove panel" Dashboard flows to the new panel storage format
Summary:
Depends on D20407. Ref T13272. This updates the "add panel" (which has two flavors: "add existing" and "create new") and "remove panel" flows to work with the new duplicate-friendly storage format.

  - We now modify panels by "panelKey", not by panel PHID, so one dashboard may have multiple copies of the same panel and we can still figure out what's going on.
  - We now work with "contextPHID", not "dashboardID", to make some flows with tab panels (or other nested panels in the future) easier.

The only major remaining flow is the Javascript "move panels around with drag-and-drop" flow.

Test Plan:
  - Added panels to a dashboard with "Create New Panel".
  - Added panels to a dashboard with "Add Existing Panel".
  - Removed panels from a dashboard.
  - Added and removed duplicate panels, got a correctly-functioning dashboard that didn't care about duplicates.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20408
2019-04-14 10:24:58 -07:00
epriestley
a3c43c473b Convert dashboard read/display pathways to the new panel storage format
Summary:
Depends on D20406. Ref T13272. This gets about half of Dashboards working with the new "duplicate panel friendly" storage format. Followups will fix the write pathways.

Collateral damage here includes:

  - Remove the old Dashboard/Panel edge type. We have a new, more general edge type for "container X uses panel Y", and we don't need this edge type for anything else.
  - Remove "attachPanels()" from Dashboard. Only rendering actually needs this, and it can just load the panels.
  - Remove "attachPanelPHIDs()" from Dashboard. We can look at the panel refs to figure this out.
  - Remove "attachProjects()" from Dashboard. Nothing uses this and it's not a very modern approach.
  - `getPanelPHIDs()` just looks at the config now.
  - Deleted some `LayoutConfig`-related code which is broken/obsolete.

Test Plan:
  - Viewed various dashboards which were created before the changes, saw them render correctly.
  - Viewed a dashboard with two of the same panel! AMAZING!

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20407
2019-04-14 10:23:42 -07:00
epriestley
0381eed6e6 Modularize dashboard layout modes (one column, two columns, etc)
Summary:
Depends on D20405. Ref T13272. Currently, the `PhabricatorDashboardLayoutConfig` class uses a lot of `switch()` statements to define layout modes.

Although I'm not planning to add thousands of new layout modes, this (and upcoming changes) can be made substantially cleaner by using a standard modular approach.

(This doesn't fully remove `PhabricatorDashboardLayoutConfig` yet, but that will happen soon.)

Test Plan: Edited a dashboard, saw the same layout modes as before.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20406
2019-04-14 10:22:59 -07:00
epriestley
2ae7d612f2 Fix a missing pht() in Ponder
Summary: See <https://discourse.phabricator-community.org/t/translation-in-ponder/2630>.

Test Plan: Viewed a Ponder question page with no answers yet.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20404
2019-04-12 12:18:56 -07:00
epriestley
ce723f999d Move Dashboards main edit flow to EditEngine
Summary:
Depends on D20402. Ref T13272. Replaces an old-school hard-coded "EditController" with a more modern one.

The actual panel stuff is still using a weird mix of legacy manual `save()` calls, but that's up next.

Test Plan: Created Dashboards, edited all dashboard fields via "Edit Dashboard".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20403
2019-04-12 06:15:15 -07:00
epriestley
6fac904463 Modularize Dashboard transactions
Summary:
Depends on D20399. Ref T13272. I'm moving toward fixing all the "moving panels around on Dashboards breaks the entire world" problems.

On the way there, modularize Dashboard transactions.

Test Plan:
  - Created a new Dashboard.
  - Edited all fiedls of a dashboard.
  - Archived/restored a dashboard.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20402
2019-04-12 06:14:51 -07:00
epriestley
51f2ed498d On panel pages, show where panels are used
Summary:
Depends on D20398. Ref T13272. Fixes T6018. Previously, panels showed "used on dashboards: x, y", but this did not include cases where a panel was used by another container panel (today, a tab panel).

Do edge indexing when a dashboard or panel is saved, then pull the edges on the Panel page so we can provide a full list of uses.

Test Plan: {F6369289}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T6018

Differential Revision: https://secure.phabricator.com/D20399
2019-04-12 06:14:21 -07:00
epriestley
d62f4dbfc9 Index and surface usage sites for Dashboards
Summary:
Depends on D20397. Ref T13272. Similar to the recent "where are Herald rules used" stuff, show which menus Dashboards are installed in.

This is mostly straightforward, except that I pulled some of the Herald logic into a parent class so it could be shared.

Test Plan: {F6369164}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20398
2019-04-12 06:13:44 -07:00
epriestley
cbe13b3065 When editing a tab panel from a dashboard, redirect back to the dashboard
Summary:
Depends on D20396. Ref T13272. Currently, using the dropdowns to edit a tab panel from a dashboard redirects you to the tab panel page.

Instead, redirect back to the context page (usually, a dashboard -- but theoretically a containing tab panel, since you can put tab panels inside tab panels).

Also, fix some JS issues with non-integer panel keys. I've moved panel keys from "0, 1, 2, ..." to having arbitrary keys to make some operations less flimsy/error-prone, but this needs some JS tweaks.

Test Plan: Edited a tab panel from a dashboard, got sent sensibly back to the dashboard.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20397
2019-04-12 06:13:12 -07:00
epriestley
9ad9ac9be6 On Dashboard tab panels in edit mode, make the "Tab Name" and the "Dropdown Edit Caret" into different links
Summary:
Ref T13272. In edit mode, tab panels now have a dropdown menu. However, this sort of overrlaps with the actual action of clicking the tab to select it.

Separate these into different click targets so that "select tab X" and "open dropdown menu for X" are different operations.

This is more work than it appears because:

  - We have an "action icon" already, used when you put a dashboard on a portal/home to create an "Edit" link. It makes sense to attach dropdowns to this, but it has some hard-coded stuff.
  - In applications with a "Create <thing>" in the crumbs (like Maniphest), we may use a dropdown menu if there are multiple create forms available. However, this menu renders in a weird way by reading all the properties out of an actual "View" object and building something else.
  - The "list of tabs" stuff shares code with different "list of tabs" navigation used by Diffusion and Instances.

..but I think I fixed everything and didn't break anything.

Test Plan:
  - Clicked "select tab" and "open dropdown menu" as separate actions.
  - Viewed Diffusion, Maniphest with multiple create forms, Instances.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20396
2019-04-12 06:08:32 -07:00
epriestley
d02256df3c Remove very old "vsDiff" data from commit update / diff extraction pipeline
Summary:
Ref T13276. This was introduced in D2586 to power a "trigger audits when the committed change does not match the reviewed change" feature. It was removed without ceremony in D15939. Broadly, rebases mean that this sort of feature can't really work like this and this approach is inherently unreliable; see also T182.

This property no longer has readers, and is unlikely to get any in the future since my planned pathway for "committed code must match reviewed code, modulo an automated rebase" is automating the rebase via "Land Revision", not comparing the diff text.

Remove this to simplify the flow of data here so that things in T13276 can be fixed more easily.

Test Plan: Grepped for `vsDiff`, no hits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20395
2019-04-11 08:55:23 -07:00
Austin McKinley
0f9776fb58 Add a workflow and a new config option for locking authentication providers
Summary:
Ref T7667. Adds new flows `bin/auth lock` and `bin/auth unlock` to prevent compromised administrator accounts from doing additional damage by altering the authentication provider configuration.

Note that this currently doesn't actually do anything because we aren't checking this config key in any of the edit controllers yet.

Test Plan: Ran `lock` and `unlock`, checked for correct DB state, observed expected setup warning.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T7667

Differential Revision: https://secure.phabricator.com/D20394
2019-04-10 16:14:48 -07:00
epriestley
89d038f53e Make Portals indexable with Ferret
Summary:
Ref T13275. Add portals to the search index so that:

  - they show up in fulltext global search; and
  - the typeahead actually uses an index.

Also make them taggable with projects as an organizational aid.

Test Plan: Indexed portals with `bin/serach index`, searched for a portal with "Query", with fulltext search in main menu, with typehead on "Install Dashboard...", changed the name of a portal and searched again to check that the index updates properly.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20389
2019-04-10 13:33:54 -07:00
Austin McKinley
55d64d0fab Add trigger rule to remove projects from tasks
Summary: Ref T13269. Same as D20379 with the polarity reversed.

Test Plan: Added some triggers, removed some projects, observed expected results.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T13269

Differential Revision: https://secure.phabricator.com/D20390
2019-04-10 12:27:01 -07:00
Austin McKinley
8b475898ee Add trigger rule for adding projects to a task
Summary: Ref T13269. This is mostly copying code from the similar Herald implementation. Note that the drop effect preview always renders because we don't have the infrastructure to compare lists of edge targets.

Test Plan: Created some triggers, dragged some tasks around, checked that tasks that already had project membership didn't write additional edges.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T13269

Differential Revision: https://secure.phabricator.com/D20379
2019-04-10 12:10:08 -07:00
epriestley
1a52c8f713 Surface custom form instructions as a "Remarkup" field for the transaction layer
Summary: Ref T13263. See <https://discourse.phabricator-community.org/t/image-uploads-for-forms-too-restricted-by-default/2594>. Currently, when you add instructions to a custom form, we don't expose them as remarkup, so `{Fxxx}` references don't get attached correctly and won't get proper permissions.

Test Plan:
Dragged-and-dropped an image into form instructions, saw the file attach to the form:

{F6367710}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13263

Differential Revision: https://secure.phabricator.com/D20387
2019-04-10 11:13:03 -07:00
epriestley
9e65e6c503 Support "JIRA Issue URIs" as a Herald field for revisions
Summary:
See PHI1175. An install would like to trigger some reminders/guidance if users don't link revisions to JIRA issues.

Expose "JIRA Issue URIs" as a field so Herald can act on the presence or absence of issues.

I'm exposing "JIRA Issue URIs", not a field like "[ Has Jira Issue ][ is true ]", since it's a bit more flexible: you can use a regexp to test against particular `PROJ-123` project prefixes in JIRA, for example.

Test Plan: {F6367696}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20386
2019-04-10 11:12:01 -07:00
epriestley
a35fda2019 Rebuild Dashboards on EditEngine: v1 Major Jank Edition
Summary:
Depends on D20383. Ref T13272. Fixes T12363. See PHI997. This gets the edit flows for tab panels functional again. They aren't //nice//, and a lot of the workflows are fairly janky: for example, most of them end up with you on the tab panel's page, which isn't useful if you started on a dashboard page.

However, these flows were extremely janky before anyway (see T12363) and I suspect this is a net improvement even though it's a bit of a mess. I anticipate cleaning this up bit-by-bit in future diffs.

Test Plan: {F6366372}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T12363

Differential Revision: https://secure.phabricator.com/D20384
2019-04-10 08:59:32 -07:00
epriestley
fb19310631 Fix some overlooked profile menu construction callsites
Summary: Depends on D20384. Ref T13275. A bunch of this code got converted but I missed some callsites that aren't reached directly from the menu.

Test Plan:
  - Visited each controller, saw actual pages instead of menu construction fatals.
  - Grepped for `getProfileMenu()`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20385
2019-04-09 14:47:44 -07:00
epriestley
41afc7c7e6 Rebuild query panels on top of EditEngine
Summary: Depends on D20377. Ref T13272. In D20372, I temporarily removed the controls for actually editing Query panels. Restore them.

Test Plan:
  - Viewed existing Query panels, saw them working like they did before.
  - Created and edited Query panels.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20382
2019-04-09 14:08:41 -07:00
epriestley
8d24e3a21a When a dashboard has inconsistent panel policies: keep doing nothing
Summary:
Depends on D20376. Ref T8033. It's possible to put a bunch of secret panels on a public dashboard, and not obvious that the dashboard won't be very useful.

This was more of an issue long ago (when the dashboard broke or all the panels completely vanished or something?). Nowadays, the panels render "You don't have permission to view this" so it's likely easy to explain/fix. Still, we can warn about this.

But, for now, don't, since a lot of this works better now and it's not really clear that this is particularly valuable. We can revisit this after all the connected changes have more of a chance to settle.

Test Plan:
(Earlier behavior, not how things look in the final version.)

{F6335008}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T8033

Differential Revision: https://secure.phabricator.com/D20377
2019-04-09 14:04:15 -07:00
epriestley
2bd1bb52e4 On Dashboards, distinguish between invalid panels and restricted panels
Summary:
Depends on D20374. Panels may not be visible if they are restricted (no permission) or if they are invalid (e.g., the panel was deleted).

Render these as two separate states instead of one big combined state.

Test Plan: {F6334756}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20376
2019-04-09 14:00:13 -07:00
epriestley
4d0904ef95 Make "Favorites" work more like other customizable menus
Summary:
Depends on D20373. Ref T13272. When you put Dashboards in Favorites, the render without a navigation menu, which is kind of weird.

Instead, make Favorites display a navigation menu. This effectively turns it into a weird cross between the home page and a portal, but so be it.

Also change the icon from "star" to "bookmark" since I think that a clearer hint about how it works.

Test Plan: Viewed dashboards in favorites, got a navigation menu. Edited items from portals, home, favorites.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20374
2019-04-09 13:59:35 -07:00
epriestley
a1a89589b1 Make the Dashboard dropshadow a little lighter and turn panel management into a menu
Summary:
Depends on D20372. Ref T13272.

  - There's a very heavy dropshadow on panels right now that looks out of place. Reduce it a bit.
  - Panels currently have unlabeled pencil and trash icons. Turn this into a menu. I'm likely planning to add options like "Change Query..." to this menu to make managing some types of panels easier.

Test Plan: {F6332838}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20373
2019-04-09 13:58:38 -07:00
epriestley
b1e2d3cd29 Use EditEngine, not CustomFields, to define Dashboard Panel edit behavior
Summary:
Depends on D20371. Ref T13272. Dashboard panels use CustomField to specify editable panel behavior. This is an older approach which was largely or entirely obsoleted by EditEngine.

Throw away all the CustomField edit stuff. Convert the "text" panel to EditEngine to prove this at least mostly works.

This breaks "query" panels and "tab" panels (they'll still work fine, but they can't be meaningfully edited). I'll restore those in a future change.

Test Plan: Created and edited a "text" panel.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20372
2019-04-09 13:57:48 -07:00
epriestley
68d969094f Mostly replace the old panel "Edit" controller with the new "Editpro" controller
Summary:
Depends on D20370. Ref T13272. This tries to get panel editing fully on the newer "Modular Transactions" + "EditEngine" flow.

This breaks tab panels a bit, but I'll fix that in a followup. And they weren't exactly in great shape before.

Also makes the flow prettier. :3

Test Plan: {F6332746}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20371
2019-04-09 13:56:17 -07:00
epriestley
81b58dba8f Modularize Dashboard Panel transactionns
Summary: Depends on D20369. Ref T13272. Move toward a world where we can edit panels with just one controller, instead of separate "Edit" and "Editpro" controllers.

Test Plan: Created and edited panels. This will get vetted more thoroughly after additional changes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20370
2019-04-09 13:37:27 -07:00
epriestley
597ef60d7e Move Dashboard panel controllers into a "panel/" subdirectory
Summary:
Depends on D20368. Ref T13272. Dashboards have a lot of controllers, try to organize them a little better.

Note "EditController" vs "EditproController". Yikes.

Test Plan: Loaded dashboards. No code changes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20369
2019-04-09 13:36:17 -07:00
epriestley
f504f40ee5 Put "workflow" on Dashboard edit links when they're disabled
Summary:
Depends on D20367. Ref T13272. When an edit action is disabled, we add "workflow" so that the "You can't do this" message renders in a dialog instead of a separate page.

These actions are implemented in a nonstandard way; standardize them.

Test Plan: Clicked both actions as a user who could take them (got normal behavior); and as a user who could not (got permissions dialog errors).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20368
2019-04-09 13:35:37 -07:00
epriestley
5b38d2142c Remove the dashboard "template" workflow
Summary:
Depends on D20364. Ref T13272. When you create a dashboard, we currently give you a modal choice between an empty dashboard and a "simple template" dashboard.

Remove this choice, and create an empty dashboard in all cases instead.

I think this template dashboard flow isn't terribly useful, and is partly covering over other deficiencies in the workflow. I'm fixing many of those and suspect we can get away without this now. Users on this flow also may not really know what they want. This also contributes to having a lot of extra unmoored panels floating around.

If we did rebuild this, I'd like to address more specific use cases and probably build it as "Add a Template Panel..." or similar, as an action you can use to quickly update an existing workboard. This would be a lot more flexible than create-a-whole-template-board.

Test Plan: Created a new board, no more "template: yes or no?" gate.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20367
2019-04-09 13:34:51 -07:00
epriestley
12b9224387 Make the "Install Dashboard" flow smoother
Summary:
Depends on D20362. Ref T13272. Currently, Dashboards have an "Install Dashboard" flow which is pretty janky and only allows you to install things to the home page.

Instead, allow users to install things to any valid target (home, favorites, portals, projects). This also provides URIs like `dashboard/install/1/home/personal/` which allow you to link users to an "install a dashboard" page; this may or may not get used.

Test Plan: Installed dashboards on home, favorites, projects, and portals.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20364
2019-04-09 13:34:09 -07:00
epriestley
eea093bec8 Combine the "View", "Arrange", and "Manage" modes of Dashboards into a single mode
Summary:
Depends on D20361. Ref T13272. Currently, Dashboards have three separate modes: view, arrange, manage.

With the advent of Portals, I think we can simplify this, and make the dashboard view a combined view/edit/manage page. To view it in a cleaner standalone way, you can add it to a portal/home/project. I'll also improve the "Install" workflow.

Test Plan:
Viewed a dashboard page, clicked through all the actions, grepped for affected URIs.

{F6327027}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20362
2019-04-09 13:32:36 -07:00
Austin McKinley
7e1743a959 Add a trigger rule to reassign a task
Summary:
Ref T13269. Workboard triggers can now reassign tasks on column drop. Also sprinkles some `setViewer()` calls in places that needed them.

This mostly works, but a few issues:

* To set the owner to unassigned, you must explicitly put the "No Owner" token in the typeahead. Maybe this should just figure out you've put nothing in that field and set it for you?
* I'm pretty sure this was already broken, but if you change the rule type from a tokenizer to a different type, the default for the field doesn't populate correctly: {F6312227}

Also adds a new hook for trigger rules: `getValueForField($value)` which allows you to transform a value stored in the DB into a form suitable for setting on a form control.

Test Plan: Dragged tasks between columns and observed new owners as expected. Didn't try to get fancy to assign tasks to deleted users, users that the viewer can't see, bot users, etc etc. I'm relying on the underlying transaction to hopefully do the right thing.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T13269

Differential Revision: https://secure.phabricator.com/D20329
2019-04-05 09:17:20 -07:00
epriestley
e45ccdd892 Fix rendering of offset-paged query panels including "Notifications"
Summary:
See <https://discourse.phabricator-community.org/t/call-to-undefined-method-phuipagerview-gethasmoreresults-in-2019-week-13/2586/>.

A small number of queries (including "Notifications" and (global) "Search") use offset-based pagers which have a slightly different API `PHUIPagerView` instead of `AphrontCursorPagerView`. This leads to a fatal in the new code for the "View All Results" buttons.

To fix this, just do an `instanceof` test. Some day we can unify the pagers.

Test Plan: Added a notifications panel, rendered it, saw it work instead of fataling on "getHasMoreResults()". Also rendered some normal panels.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20366
2019-04-04 06:21:38 -07:00
epriestley
248d79f36d Fix "Actions" button on Phame standalone/live pages (bonus: JX.sprintf())
Summary:
See <https://discourse.phabricator-community.org/t/non-functional-actions-menu-on-live-phame-views/2593>. Several layers here:

The "Actions" button is broken because a menu behavior is failing, since we aren't rendering the menu.

When a behavior fails to initialize, catch and log the exception and continue. Previously, we stopped initializing behaviors if any failed, but behaviors are usually independent and continuing with an explicit exception seems reasonable.

Give "JX.log()" some "sprintf()" semantics to make logging the behavior failure easier. We can probably afford these extra 200 bytes now in 2019.

This fixes the button and gives us explicit errors in the log. So far, so good.

Then, when a page won't render chrome, don't try to render the main menu. This fixes the actual errors (we no longer try to initialize menu behaviors for nodes which don't exist).

Completely hide the "Actions" and "Comment" flows if the viewer isn't logged in. Although this isn't completely consistent with other applications, I think it's more appropriate for Phame. In applications like Maniphest, we show a full set of controls (but disable them) so that users who are not currently logged in have a clear path to interact with the content, under the assumption that this is a relatively common workflow. This is probably less common for Phame, where we expect most anonymous viewers not to log in or interact.

Finally, parametrize a one-off border color and add a border under the crumbs at the top of the page.

Test Plan:
  - Viewed a "Live" Phame blog post page, clicked "Actions", got a dropdown.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20378
2019-04-04 06:10:14 -07:00
epriestley
18732a0d2f Make Portals reachable without knowing the URI
Summary:
Depends on D20360. Ref T13275. This makes the "Dashboards" application start on a Drydock-like console page where you pick portals, dashboards, or panels.

Probably the "Dashboards" application should either be renamed to "IntelliknowledgePro" or Portals should be split off into a separate application eventually, but let's see how things go like this for now, since restructuring probably breaks some URIs at least a little bit so I'd like more confidence that we're headed in the right direction before we do it.

Test Plan:
  - Visited Dashboards via typeahead, got options for Dashboards/Portals/Panels.
  - Visited Portals pages, got simplified crumbs.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20361
2019-04-02 15:23:36 -07:00
epriestley
c9d3fb2ac5 Fix the incorrect link target for "Create Revision" as a Menu Item
Summary:
Depends on D20359. Fixes T12098. When you add a new "Form" item and pick "Create Revision", you currently get a bad link. This is because Differential is kind of special and the form isn't usable directly, even though Differential does use EditEngine.

Allow EditEngine to specify a different create URI, then specify the web UI paste-a-diff flow to fix this.

Test Plan:
  - Added "Create Revision" to a portal, clicked it, was sensibly put on the diff flow.
  - Grepped for `getCreateURI()`, the only other real use case is to render the "Create X" dropdowns in the upper right.
    - Clicked one of those, still worked great.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12098

Differential Revision: https://secure.phabricator.com/D20360
2019-04-02 15:21:59 -07:00
epriestley
dfe47157d3 When picking a default menu item to render, don't pick disabled items
Summary:
Depends on D20358. Fixes T12871. After refactoring, we can now tell when a "storage" menu item generated only disabled "display" menu items, and not pick any of them as the default rendering.

This means that if you're looking at a portal/menu with several dashboards, but can't see some at the top, you'll get the first one you can see.

Also clean up a lot of minor issues with less-common states.

Test Plan:
  - Created a portal with two private dashboards and a public dashboard.
  - Viewed it as another user, saw the default view show the dashboard I can actually see.
  - Minor fix: Disabled and enabled the hard-coded "Home" item, now worked cleanly with the right menu state.
  - Minor fix: added a motivator panel.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12871

Differential Revision: https://secure.phabricator.com/D20359
2019-04-02 15:21:27 -07:00
epriestley
5192ae4750 Update all existing ProfileMenuItems for the more-structured API
Summary:
Depends on D20357. Ref T13275. Now that there's a stronger layer between "stuff in the database" and "stuff on the screen", these subclasses all need to emit intermediate objects instead of raw, HTML-producing view objects.

This update is mostly mechanical.

Test Plan:
  - Viewed Home, Favorites, Portals, User Profiles, Project Profiles.
  - Clicked each item on each menu/profile type.
  - Added every (I think?) type of item to a menu and clicked them all.
  - Grepped for obsolete symbols (`newNavigationMenuItems`, `willBuildNavigationItems`).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20358
2019-04-02 15:20:39 -07:00
epriestley
950e9d085b In ProfileMenu, put more structure between "stored/configured items" and "display items"
Summary:
Depends on D20356. Ref T13275. See also T12871 and T12949.

Currently, the whole "ProfileMenu" API operates around //stored// items. However, stored items are allowed to produce zero or more //display// items, and we sometimes want to highlight display item X but render stored item Y (as is the case with "Link" items pointing at `?filter=xyz` on Workboards).

For the most part, this either: doesn't work; or works by chance; or is kind of glued together with hope and prayer (as in D20353).

Put an actual structural layer in place between "stored/configured item" and "display item" that can link them together more clearly. Now:

  - The list of `ItemConfiguration` objects (stored/configured items) is used to build an `ItemViewList`.
  - This handles the selection/highlighting/default state, and knows which display items are related to which stored items.
  - When we're all done figuring out what we're going to select and what we're going to highlight, it pops out an actual View which can build the HTML.

This requires API changes which are not included in this change, see next change.

This doesn't really do anything on its own, but builds toward a more satisfying fix for T12871. I'd hoped to avoid doing this for now, but wasn't able to get a patch I felt good about for T12871 built without fixing this first.

Test Plan: See next change.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20357
2019-04-02 15:19:59 -07:00
epriestley
971a272bf6 Automatically build mobile menus from navigation, and clean up external ProfileMenu API
Summary:
Depends on D20355. Ref T13275. Ref T13247. Currently, "Hamburger" menus are not automatically built from navigation menus. However, this is (I'm almost completely sure?) a reasonable and appropriate default behavior, and saves us some code around profile menus.

With this rule in place, we can remove `setApplicationMenu()` and `getApplicationMenu()` from `StandardPageView`, since they have no callers.

This also updates a lot of profile menu callsites to a new API which is added in the next change.

Test Plan: See the next two changes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275, T13247

Differential Revision: https://secure.phabricator.com/D20356
2019-04-02 15:17:44 -07:00
epriestley
47bf382435 Allow profile menu items to be locked to the top or bottom of the menu
Summary:
Depends on D20353. Ref T13275. This is just some small quality-of-life fixes:

  - When you add items to menus, they currently go below the "Edit Menu/Manage Menu" links by default. This isn't a very good place for them. Instead, lock "edit" items to the bottom of the menu.
  - Lock profile pictures to the top of the menu. This just simplifies things a little.
  - Show more iconography hints on the "edit menu items" UI.
  - Add a "drag stuff to do things" hint if some stuff can be dragged.

Test Plan:
  - Added new items to a Portal, they didn't go to the very bottom. Instead, they went above the "Edit/Manage" links; a sensible place for them.
  - Viewed the "edit menu items" screen, saw more hints and visual richness.
  - Viewed/edited Home, Projects, Portals, Favorites

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20355
2019-04-02 15:08:20 -07:00
epriestley
36a8b4ea17 When a ProfileMenu has a link item that adds URI parameters, highlight it when clicked
Summary:
Depends on D20352. Fixes T12949. If a user adds a link (for example, to a workboard) that takes you to the same page but with some URI parameters, we'd prefer to highlight the "link" item instead of the default "workboard" item when you click it.

For example, you add a `/query/assigned/` "link" item to a workboard, called "Click This To Show Tasks Assigned To Me On This Workboard", i.e. filter the current view.

This is a pretty reasonable thing to want to do. When you click it, we'd like to highlight that item to show that you've activated the "Assigned to Me" filter you added.

However, we currently highlight the thing actually serving the content, i.e. the "Workboard" item.

Instead:

  - When picking what to highlight, look through all the items for one with a link to the current request URI.
  - If we find one or more, pick the one that would be the default.
  - Otherwise, pick the first one.

This means that you can have several items like "?a=1", "?a=2", etc., and we will highlight them correctly when you click them.

This actual patch has some questionable bits (see some discussion in T13275), but I'd like to wait for stronger motivation to refactor it more extensively.

Test Plan:
  - On a portal, added a `?a=1` link. Saw it highlight properly when clikced.
  - On a workboard, added a link to the board itself with a different filter. Saw it highlight appropriately when clicked.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12949

Differential Revision: https://secure.phabricator.com/D20353
2019-04-02 14:45:33 -07:00
epriestley
408cbd633c On portals, make the "selected" / "default" logic more straightforward
Summary:
Depends on D20349. Ref T13275. Currently, a default item is selected as a side effect of generating the full list of items, for absolutely no reason.

The logic to pick the currently selected item can also be separated out pretty easily.

(And fix a bug in with a weird edge case in projects.)

This doesn't really change anything, but it will probably make T12949 a bit easier to fix.

Test Plan: Viewed Home / projects / portals, clicked various links, got same default/selection behavior as before.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20352
2019-04-02 14:44:26 -07:00
epriestley
d0d49d1efd Allow Portals to be edited, and improve empty/blank states
Summary:
Depends on D20348. Ref T13275. Portals are mostly just a "ProfileMenuEngine" menu, and that code is already relatively modular/flexible, so set that up to start with.

The stuff it gets wrong right now is mostly around empty/no-permission states, since the original use cases (project menus) didn't have any of these states: it's not possible to have a project menu with no content.

Let the engine render an "empty" state (when there are no items that can render a content page) and try to make some of the empty behavior a little more user-friendly.

This mostly makes portals work, more or less.

Test Plan: {F6322284}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20349
2019-04-02 14:43:05 -07:00
epriestley
90df4b2bd1 Add skeleton for Portals, a collection of dashboards and other resources
Summary:
Ref T13275. Today, you can build a custom page on the home page, on project pages, and in your favorites menu.

PHI374 would approximately like to build a completely standalone custom page, and this generally seems like a reasonable capability which we should support, and which should be easy to support if the "custom menu" stuff is built right.

In the near future, I'm planning to shore up some of the outstanding issues with profile menus and then build charts (which will have a big dashboard/panel component), so adding Portals now should let me double up on a lot of the testing and maybe make some of it a bit easier.

Test Plan:
Viewed the list of portals, created a new portal. Everything is currently a pure skeleton with no unique behavior.

Here's a glorious portal page:

{F6321846}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20348
2019-04-02 14:42:26 -07:00
Austin McKinley
3e05ff2e99 Improve Conpherence behavior for logged out users.
Summary:
There are two issues here I was trying to fix:

* Viewing `/conpherence` by logged out users on `secure` would generate an overheated query on `ConpherenceThreadQuery` `secure` has a ton of wacky threads with bogus names.
* When a user views a specific thread that they don't have permission to see, we attempt to fetch the thread's transactions before applying policy filtering. If the thread has more than 1000 comments, that query will also overheat instead of returning a policy exception.

I fixed the first problem, but started trying to fix the second by moving the transaction fetch to `didFilterPage` but it broke in strange ways so I gave up.

Also fix a dangling `qsprintf` update.

Test Plan: Loaded threads and the Conpherence homepage with and without logged in users.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20375
2019-04-02 14:03:08 -07:00
epriestley
cddbe306f9 Correct a case where a single-hunk diff may incorrectly be identified as multi-hunk by the Scope engine
Summary: See PHI985. The layers above this may return `array()` to mean "one hunk with a line-1 offset". Accept either `array()` or `array(1 => ...)` to engage the scope engine.

Test Plan: See PHI985.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20363
2019-04-01 14:55:11 -07:00
epriestley
02f94cd7d2 Fix an issue with Duo not live-updating properly on login gates
Summary:
See <https://discourse.phabricator-community.org/t/duo-broken-in-2019-week-12/2580/>.

The "live update Duo status" endpoint currently requires full sessions, and doesn't work from the session upgrade gate on login.

Don't require a full session to check the status of an MFA challenge.

Test Plan: Went through Duo gate in a new session, got a live update.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20347
2019-03-29 11:00:38 -07:00
epriestley
c4856c37e7 Fix content overflow in user hovercards
Summary:
Fixes T13273. This element is a bit weird, but I think I fixed it without breaking anything.

The CSS is used by project hovercards and user hovercards, but they each have a class which builds mostly-shared-but-not-really-identical CSS, instead of having a single `View` class with modes. So I'm not 100% sure I didn't break something obscure, but I couldn't find anything this breaks.

The major issue is that all the text content has "position: absolute". Instead, make the image "absolute" and the text actual positioned content. Then fix all the margins/padding/spacing/layout and add overflow. Seems to work?

Plus: hide availability for disabled users, for consistency with D20342.

Test Plan:
Before:

{F6320155}

After:

{F6320156}

I think this is pixel-exact except for the overflow behavior.

Also:

  - Viewed some other user hovercards, including a disabled user. They all looked unchanged.
  - Viewed some project hovercards. They all looked good, too.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13273

Differential Revision: https://secure.phabricator.com/D20344
2019-03-28 21:10:09 -07:00
epriestley
953a449305 Hide "Availability" and "Calendar" on user profiles for disabled users
Summary:
See downstream <https://phabricator.wikimedia.org/T138723>. That suggestion is a little light on details, but I basically agree that showing "Availability: Available" on disabled user profiles is kind of questionable/misleading.

Just hide event information on disabled profiles, since this doesn't seem worth building a special "Availability: Who Knows, They Are Disabled, Good Luck" disabled state for.

Test Plan: Looked at disabled and non-disabled user profiles, saw Calendar stuff only on the former.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20342
2019-03-28 17:24:35 -07:00
epriestley
f1a7eb66da Fix a straggling issue with cursor changes impacting Conpherence thread indexing
Summary: Ref T13266. Caught one more of these "directly setting afterID" issues in the logs.

Test Plan: Ran `bin/search index --type ConpherenceThread` before and after changes. Before: fatal about a direct call. After: clean index rebuild.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13266

Differential Revision: https://secure.phabricator.com/D20341
2019-03-28 17:23:53 -07:00
epriestley
15cc475cbd When a comment was signed with MFA, require MFA to edit it
Summary:
Ref PHI1173. Currently, you can edit an MFA'd comment without redoing MFA. This is inconsistent with the intent of the MFA badge, since it means an un-MFA'd comment may have an "MFA" badge on it.

Instead, implement these rules:

  - If a comment was signed with MFA, you MUST MFA to edit it.
  - When removing a comment, add an extra MFA prompt if the user has MFA. This one isn't strictly required, this action is just very hard to undo and seems reasonable to MFA.

Test Plan:
  - Made normal comments and MFA comments.
  - Edited normal comments and MFA comments (got prompted).
  - Removed normal comments and MFA comments (prompted in both cases).
  - Tried to edit an MFA comment without MFA on my account, got a hard "MFA absolutely required" failure.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20340
2019-03-28 15:55:14 -07:00
epriestley
e69b349b1b Prevent users from removing task titles with "Bulk Edit"
Summary:
See downstream <https://phabricator.wikimedia.org/T209449>.

The "Bulk Edit" flow works with `setContinueOnMissingFields(true)`, so `newRequiredError()` errors are ignored. This allows you to apply a transaction which changes the title to `""` (the empty string) without actually hitting any errors which the workflow respects.

(Normally, `setContinueOnMissingFields(...)` workflows only edit properties that can't be missing, like the status of an object, so this is an unusual flow.)

Instead, validate more narrowly:

  - Transactions which would remove the title get an "invalid" error, which is respected even under "setContinueOnMissingFields()".
  - Then, we try to raise a "missing/required" error if everything otherwise looks okay.

Test Plan:
  - Edited a task title normally.
  - Edited a task to remove the title (got an error).
  - Created a task with no title (disallowed: got an error).
  - Bulk edited a task to remove its title.
    - Before change: allowed.
    - After change: disallowed.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20339
2019-03-28 09:06:56 -07:00
epriestley
6bb9d3ac67 Allow users to add "ProfileMenu" items on mobile
Summary:
Depends on D20337. Fixes T12167. Ref T13272. On this page ("Favorites > Edit Favorites > Personal", for example) the curtain actions aren't available on mobile.

Normally, curtains are built with `Controller->newCurtainView()`, which sets an ID on the action list, which populates the header button. This curtain is built directly because there's no `Controller` handy.

To fix the issue, just set an ID. This could probably be cleaner, but that's likely a much more involved change.

Test Plan: Edited my favorites, narrowed the window, saw an "Actions" button.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T12167

Differential Revision: https://secure.phabricator.com/D20338
2019-03-27 16:05:55 -07:00
epriestley
6648942bc8 Don't allow "Conpherence" menu items to be added to editable menus if Conpherence is not installed
Summary: Depends on D20336. Ref T13272. Fixes T12745. If you uninstall Conpherence, "Edit Favorites" and other editable menu interfaces still allow you to add "Conpherence" items. Prevent this.

Test Plan:
  - Installed Conpherence: Saw option to add a "Conpherence" link when editing favorites menu.
  - Uninstalled Conpherence: No more option.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T12745

Differential Revision: https://secure.phabricator.com/D20337
2019-03-27 16:01:16 -07:00
epriestley
73feac47c7 When query panels have more results, show a "View All Results" button at the bottom
Summary:
Depends on D20335. Ref T13263. Ref T13272. See PHI854. Ref T9903.

Currently, we don't provide a clear indicator that a query panel is showing a partial result set (UI looks the same whether there are more results or not).

We also don't provide any way to get to the full result set (regardless of whether it is the same as the visible set or not) on tab panels, since they don't inherit the header buttons.

To (mostly) fix these problems, add a "View All Results" button at the bottom of the list if the panel shows only a subset of results.

Test Plan:
{F6314560}

{F6314562}

{F6314564}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T13263, T9903

Differential Revision: https://secure.phabricator.com/D20336
2019-03-27 15:55:48 -07:00
epriestley
2c184bd4cd When query panels overheat, degrade them and show a warning
Summary:
Depends on D20334. Ref T13272. After recent changes to make overheating queries throw by default, dashboard panels now fail into an error state when they overheat.

This is a big step up from the hard-coded homepage panels removed by D20333, but can be improved. Let these panels render partial results when they overheat and show a human-readable warning.

Test Plan: {F6314114}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20335
2019-03-27 15:54:28 -07:00
epriestley
6b069f26c0 Give Dashboard query panels a more obvious "View All" button
Summary:
Depends on D20333. Ref T13272. Currently, dashboard query panels have an aesthetic but hard-to-see icon to view results, with no text label.

Instead, provide an easier-to-see button with a text label.

Test Plan: {F6314091}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20334
2019-03-27 15:53:51 -07:00
epriestley
a68b6cfe65 Use real Dashboard Panels to render the default hard-coded homepage, not hacky fake panels
Summary:
Ref T13272. Currently, the hard-coded default homepage looks like a dashboard but is actually rendered completely manually.

This means that various panel rendering improvements we'd like to make (including better "Show More" behavior and better handling of overheated queries) won't work on the home page naturally: we'd have to make these changes twice, once for dashboards and once for the home page.

Instead, build the home page out of real panels. This turns out to be significantly simpler (I think the backend part of panels/dashboards is mostly on solid footing, the frontend just needs some work).

Test Plan:
Loaded the default home page, saw a thing which looked the same as the old thing. Changes I know about / expect:

  - The headers for these panels are no longer linked, but they weren't colorized before so the links were hard to find. I plan to improve panel behavior for "find/more" in a followup.
  - I've removed the "follow us on twitter" default NUX if feed is empty, since this seems like unnecessary incidental complexity.
  - (Internal exception behavior should be better, now.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20333
2019-03-27 15:21:36 -07:00
epriestley
ee54e71ba9 On workboards, link ancestor project breadcrumbs to their workboards
Summary:
Ref T13269. Currently, if you're on a milestone workboard like this:

> Projects > Parent > Milestone > Workboard

The "Parent" link goes to the parent profile. More often, I want it to go to the parent workboard. Try doing that? This is kind of one-off but I suspect it's a better rule.

Also, consolidate one billion manual constructions of "/board/" URIs.

Test Plan: Viewed a milestone workboard, clicked the parent link, ended up on the parent workboard.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13269

Differential Revision: https://secure.phabricator.com/D20331
2019-03-27 14:42:57 -07:00
epriestley
4485482fd4 Fix task hovercards showing a "Not Editable" state
Summary:
Ref T13269. These cards really have three states:

  - Editable: shows a pencil icon edit button.
  - You Do Not Have Permission To Edit This: shows a "no editing" icon in red.
  - Hovecard: shouldn't show anything.

However, the "hovercard" and "no permission" states are currently the same state, so when I made the "no permission" state more obvious that made the hovercard go all weird.

Make these states explicitly separate states.

Test Plan:
Looked at a...

  - Editable card on workboard: edit state.
  - No permission card on workboard: no permission state.
  - Any hovercard: "not editable, this is a hovercard" state.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13269

Differential Revision: https://secure.phabricator.com/D20330
2019-03-26 15:56:09 -07:00
epriestley
b328af0a1b Raise a more tailored exception if transform/thumbnail support is missing for cover images
Summary: If "GD" doesn't support a particular image type, applying a cover image currently goes through but no-ops. Fail it earlier in the process with a more specific error.

Test Plan: Without PNG support locally, dropped a PNG onto a card on a workboard. Got a more useful error.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20328
2019-03-26 15:16:31 -07:00
Austin McKinley
d347b102a1 Add workboard trigger rule for changing task priority
Summary: This is a copy/paste/find-and-replace-all of the status rule added by D20288.

Test Plan: Made some triggers, moved some tasks, edited some triggers. Grepped for the word "status" in the new file.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20325
2019-03-26 11:39:49 -07:00
epriestley
71c89bd057 Pass all adjacent card PHIDs from the client to the server when moving a card
Summary:
Depends on D20321. Fixes T12175. Ref T13074. Now that before/after PHIDs are suggestions, we can give the server a more complete view of what the client is trying to do so we're more likely to get a good outcome if the client view is out of date.

Instead of passing only the one directly adjacent card PHID, pass all the card PHIDs that the client thinks are in the same group.

(For gigantic columns with tens of thousands of tasks this might need some tweaking -- like, slice both lists down to 10 items -- but we can cross that bridge when we come to it.)

Test Plan:
  - Dragged some cards around to top/bottom/middle positions, saw good positioning in all cases.
  - In two windows, dragged stuff around on the same board. At least at first glance, conflicting simultaneous edits seemed to do reasonable things.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13074, T12175

Differential Revision: https://secure.phabricator.com/D20322
2019-03-26 07:46:05 -07:00
epriestley
6138e50962 When moving cards on workboards, treat before/after cards as optional hints, not strict requirements
Summary:
Depends on D20320. Ref T12175. Ref T13074. Currently, when you move a card between columns, the internal transaction takes exactly one `afterPHID` or `beforePHID` and moves the card before or after the specified card.

This is a fairly strict interpretation and causes a number of practical issues, mostly because the user/client view of the board may be out of date and the card they're dragging before or after may no longer exist: another user might have moved or hidden it between the last client update and the current time.

In T13074, we also run into a more subtle issue where a card that incorrectly appears in multiple columns fatals when dropped before or after itself.

In all cases, a better behavior is just to complete the move and accept that the position may not end up exactly like the user specified. We could prompt the user instead:

> You tried to drop this card after card X, but that card has moved since you last loaded the board. Reload the board and try again.

...but this is pretty hostile and probably rarely/never what the user wants.

Instead, accept a list of before/after PHIDs and just try them until we find one that works, or accept a default position if none work. In essentially all cases, this means that the move "just works" like users expect it to instead of fataling in a confusing/disruptive/undesirable (but "technically correct") way.

(A followup will make the client JS send more beforePHIDs/afterPHIDs so this works more often.)

We could eventually add a "strict" mode in the API or something if there's some bot/API use case for precise behavior here, but I suspect none exist today or are (ever?) likely to exist in the future.

Test Plan:
  - (T13074) Inserted two conflicting rows to put a card on two columns on the same board. Dropped one version of it underneath the other version. Before: confusing fatal. After: cards merge sensibly into one consistent card.
  - (T12175) Opened two views of a board. Moved card A to a different column on the first view. On the second view, dropped card B under card A (still showing in the old column). Before: confusing fatal. After: card ended up in the right column in approximately the right place, very reasonably.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13074, T12175

Differential Revision: https://secure.phabricator.com/D20321
2019-03-26 07:45:24 -07:00
epriestley
6182193cf5 Give the "Code" tab in Diffusion more consistent (path-retaining) behavior
Summary:
Fixes T13270. In Diffusion, the "Code" tab is linked in a weird way that isn't consistent with the other tabs.

Particularly, if you navigate to `x/y/z/` and toggle between the "Branches" and "History" tabs (or other tabs), you keep your path. If you click "Code", you lose your path.

Instead, retain the path, so you can navigate somewhere and then toggle to/from the "Code" tab to get different views of the same path.

Test Plan: Browed into a repository, clicked "History", clicked "Code", ended up back in the place I started.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13270

Differential Revision: https://secure.phabricator.com/D20323
2019-03-25 14:45:02 -07:00
epriestley
c53ed72e4c Provide a clearer UI for "view all results" in partial result panels
Summary:
In some cases, we show a limited number of one type of object somewhere else, like "Recent Such-And-Such" or "Herald Rules Which Use This" or whatever.

We don't do a very good job of communicating that these are partial lists, or how to see all the results. Usually there's a button in the upper right, which is fine, but this could be better.

Add an explicit "more stuff" button that shows up where a pager would appear and makes it clear that (a) the list is partial; and (b) you can click the button to see everything.

Test Plan: {F6302793}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20315
2019-03-25 14:35:08 -07:00
epriestley
47856dc93f Track how many columns use a particular trigger
Summary:
Ref T5474. In 99% of cases, a separate "archived/active" status for triggers probably doesn't make much sense: there's not much reason to ever disable/archive a trigger explcitly, and the archival rule is really just "is this trigger used by anything?".

(The one reason I can think of to disable a trigger manually is because you want to put something in a column and skip trigger rules, but you can already do this from the task detail page anyway, and disabling the trigger globally is a bad way to accomplish this if it's in use by other columns.)

Instead of adding a separate "status", just track how many columns a trigger is used by and consider it "inactive" if it is not used by any active columns.

Test Plan: This is slightly hard to test exhaustively since you can't share a trigger across multiple columns right now, but: rebuild indexes, poked around the trigger list and trigger details, added/removed triggers.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20308
2019-03-25 14:04:55 -07:00
epriestley
bfa5ffe8a1 Add a "Play Sound" workboard trigger rule
Summary:
Ref T5474. Allow columns to play a sound when tasks are dropped.

This is a little tricky because Safari has changed somewhat recently to require some gymnastics to play sounds when the user didn't explicitly click something. Preloading the sound on the first mouse interaction, then playing and immediately pausing it seems to work, though.

Test Plan: Added a trigger with 5 sounds. In Safari, Chrome, and Firefox, dropped a card into the column. In all browsers, heard a nice sequence of 5 sounds played one after the other.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20306
2019-03-25 14:03:57 -07:00
epriestley
1277db9452 When users hover over a column trigger menu, show a "preview" with the rules instead of a tooltip
Summary:
Ref T5474. The first rough cut of triggers showed some of the trigger rules in a tooltip when you hover over the "add/remove" trigger menu.

This isn't great since we don't have much room and it's a bit finnicky / hard to read.

Since we have a better way to show effects now in the drop preview, just use that instead. When you hover over the trigger menu, preview the trigger in the "drop effect" element, with a "Trigger: such-and-such" header.

Test Plan:
  - This is pretty tough to screenshot.
  - Hovered over menu, got a sensible preview of the trigger effects.
  - Dragged a card over the menu, no preview.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20304
2019-03-25 14:02:10 -07:00
epriestley
614f39b806 Show a trigger rule summary on the rule view page
Summary: Ref T5474. When you view the main page for a rule, show what the rule does before you actually edit it.

Test Plan:
Viewed a real trigger, then faked invalid/unknown rules:

{F6300211}

{F6300212}

{F6300213}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20303
2019-03-25 13:29:12 -07:00
epriestley
ff128e1b32 Write workboard trigger rules to the database
Summary: Ref T5474. Read and write trigger rules so users can actually edit them.

Test Plan: Added, modified, and removed trigger rules. Saved changes, used "Show Details" to review edits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20302
2019-03-25 13:26:21 -07:00
epriestley
567dea5449 Mostly make the editor UI for triggers work
Summary:
Ref T5474. This provides a Herald-like UI for editing workboard trigger rules.

This probably has some missing pieces and doesn't actually save anything to the database yet, but the basics at least roughly work.

Test Plan: {F6299886}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20301
2019-03-25 13:25:14 -07:00
epriestley
a5b3e33e3c Don't show workboard action previews if the action won't have any effect
Summary:
Ref T10335. When you (for example) drag a "Resolved" task into a column with "Trigger: change status to resolved.", don't show a hint that the action will "Change status to resolved." since this isn't helpful and is somewhat confusing.

For now, the only visibility operator is "!=" since all current actions are simple field comparisons, but some actions in the future (like "add subscriber" or "remove project") might need other conditions.

Test Plan:
Dragged cards in ways that previously provided useless hints: move from column A to column B on a "Group by Priority" board; drag a resolved task to a "Trigger: change status to as resolved" column. Saw a more accurate preview in both cases.

Drags which actually cause effects still show the effects correctly.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10335

Differential Revision: https://secure.phabricator.com/D20300
2019-03-25 13:24:01 -07:00
epriestley
5dca1569b5 Preview the effects of a drag-and-drop operation on workboards
Summary:
Ref T10335. Ref T5474. When you drag-and-drop a card on a workboard, show a UI hint which lists all the things that the operation will do.

This shows: column moves; changes because of dragging a card to a different header; and changes which will be caused by triggers.

Not implemented here:

  - Actions are currently shown even if they have no effect. For example, if you drag a "Normal" task to a different column, it says "Change priority to Normal.". I plan to hide actions which have no effect, but figuring this out is a little bit tricky.
  - I'd like to make "trigger effects" vs "non-trigger effects" a little more clear in the future, probably.

Test Plan:
Dragged stuff between columns and headers, and into columns with triggers. Got appropriate preview text hints previewing what the action would do in the UI.

(This is tricky to take a screenshot of since it only shows up while the mouse cursor is down.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10335, T5474

Differential Revision: https://secure.phabricator.com/D20299
2019-03-25 13:22:56 -07:00
epriestley
149f8cc959 Hard code a "close task" action on every column Trigger
Summary: Depends on D20287. Ref T5474. This hard-codes a storage value for every trigger, with a "Change status to <default closed status>" rule and two bogus rules. Rules may now apply transactions when cards are dropped.

Test Plan: Dragged cards to a column with a trigger, saw them close.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20288
2019-03-25 13:21:55 -07:00
epriestley
916bf1a8f9 Allow triggers to be attached to and removed from workboard columns
Summary:
Depends on D20286. Ref T5474. Attaches triggers to columns and makes "Remove Trigger" work.

(There's no "pick an existing named trigger from a list" UI yet, but I plan to add that at some point.)

Test Plan: Attached and removed triggers, saw column UI update appropriately.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20287
2019-03-25 13:21:26 -07:00
epriestley
0204489a52 Modularize workboard column transactions
Summary: Depends on D20279. Ref T5474. Modernize these transactions before I add a new "TriggerTransaction" for setting triggers.

Test Plan: Created a column. Edited a column name and point limit. Hid and un-hid a column. Grepped for removed symbols.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20286
2019-03-25 13:14:25 -07:00
epriestley
252b6f2260 Provide basic scaffolding for workboard column triggers
Summary:
Depends on D20278. Ref T5474. This change creates some new empty objects that do nothing, and some new views for looking at those objects. There's no actual useful behavior yet.

The "Edit" controller is custom instead of being driven by "EditEngine" because I expect it to be a Herald-style "add new rules" UI, and EditEngine isn't a clean match for those today (although maybe I'll try to move it over).

The general idea here is:

  - Triggers are "real" objects with a real PHID.
  - Each trigger has a name and a collection of rules, like "Change status to: X" or "Play sound: Y".
  - Each column may be bound to a trigger.
  - Multiple columns may share the same trigger.
  - Later UI refinements will make the cases around "copy trigger" vs "reference the same trigger" vs "create a new ad-hoc trigger" more clear.
  - Triggers have their own edit policy.
  - Triggers are always world-visible, like Herald rules.

Test Plan: Poked around, created some empty trigger objects, and nothing exploded. This doesn't actually do anything useful yet since triggers can't have any rule behavior and columns can't actually be bound to triggers.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20279
2019-03-25 13:13:58 -07:00
epriestley
b081053e26 Don't show a "Manage" button in Legalpad if the user is signing a TOS document
Summary:
When a TOS-like Legalpad document is marked "Require this document to use Phabricator", the login prompt shows a "Manage" button, but that button doesn't work.

When we're presenting a document as a session gate, don't show "Manage".

Test Plan: Viewed a required document during a session gate (no "Manage" button) and normally (saw "Manage" button).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20312
2019-03-25 11:30:46 -07:00
epriestley
930cc7a6dd Generalize Legalpad validation logic for "Require Signature"
Summary:
See downstream <https://phabricator.wikimedia.org/T208254>.

I can't actually reproduce any issue here (we only show this field when creating a document, and only if the viewer is an administrator), so maybe this relied on some changes or was originally reported against older code.

Regardless, the validation isn't quite right: it requires administrator privileges to apply this transaction at all, but should only require administrator privileges to change the value.

Test Plan:
Edited Legalpad documents as an administrator and non-administrator before and after the change, with and without signatures being required.

Couldn't reproduce the original issue, but this version is generally more correct/robust.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20311
2019-03-25 11:26:13 -07:00
epriestley
e15b3dd3c6 When a repository is inactive, mark its handle as "closed"
Summary: See downstream <https://phabricator.wikimedia.org/T182232>. We currently don't mark repository handles as closed.

Test Plan: Mentioned two repositories with `R1` (active) and `R2` (inactive). After patch, saw `R2` visually indicated as closed/inactive.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20309
2019-03-25 11:25:37 -07:00
epriestley
79e36dc7fa Explain the relationship between "Runnable" and "Restartable" more clearly in Build Plans
Summary: See PHI1153. The "Runnable" and "Restartable" behaviors interact (to click "restart", you must be able to run the build AND it must be restartable). Make this more clear.

Test Plan: {F6301739}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20307
2019-03-25 11:24:04 -07:00
epriestley
c3563ca156 Correct use of the paging API in Phame
Summary: Ref T13266. This callsite is using the older API; swap it to use pagers.

Test Plan: Viewed a Phame blog post with siblings, saw the previous/next posts linked.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: nicolast

Maniphest Tasks: T13263, T13266

Differential Revision: https://secure.phabricator.com/D20319
2019-03-25 10:55:21 -07:00
epriestley
1d73ae3b50 Fix two minor timezone display issues
Summary:
Ref T13263. Two minor issues:

  - The "reconcile" dialog shows the wrong sign because JS signs differ from normal signs (for example, PST or PDT or whatever we're in right now is shown as "UTC+7", but should be "UTC-7").
  - The big dropdown of possible timezones lumps "UTC+X:30" timezones into "UTC+X".

Test Plan:
  - Reconciled "America/Nome", saw negative UTC offsets for "America/Nome" and "America/Los_Angeles" (previously: improperly positive).
  - Viewed the big timzone list, saw ":30" and ":45" timezones grouped/labeled more accurately.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13263

Differential Revision: https://secure.phabricator.com/D20314
2019-03-25 10:53:36 -07:00
epriestley
2ebe675ae5 Fix an unusual internal cursor in Conpherence
Summary:
See <https://discourse.phabricator-community.org/t/error-when-sending-a-message-chat-room/2548>.

Conpherence calls `setAfterID()` and `setBeforeID()` directly on a subquery, but these methods no longer exist.

Use a pager instead. This code probably shouldn't exist (we should use some other approach to fetch this data in most cases) but that's a larger change.

Test Plan: Sent messages in a Conpherence thread. Before: fatal; after: success. Viewed the Conphrence menu, loaded threads, etc.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20318
2019-03-25 10:52:58 -07:00
epriestley
f047b90d93 Don't draw the task graph line image on devices by default
Summary:
See downstream <https://phabricator.wikimedia.org/T210482>.

On mobile, the task graph can take up most of the screen. Hide it on devices. Keep it on the standalone view if you're really dedicated and willing to rotate your phone or whatever to see the lines.

Test Plan: Dragged window real narrow, saw graph hide.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20313
2019-03-25 10:52:11 -07:00
epriestley
b825570734 Fix transaction queries failing on "withIDs()" after clicking "Show Older"
Summary:
See <https://discourse.phabricator-community.org/t/unhandled-exception-on-show-older-changes/2545/>.

Before T13266, this query got away without having real paging because it used simple ID paging only and results are never actually hidden (today, you can always see all transactions on an object).

Provide `withIDs()` so the new, slightly stricter paging works.

Test Plan: On an object with "Show Older" in the transaction record, clicked the link. Before: exception in paging code (see Discourse link above). After: transactions loaded cleanly.

Reviewers: amckinley, avivey

Reviewed By: avivey

Differential Revision: https://secure.phabricator.com/D20317
2019-03-24 12:25:29 -07:00
epriestley
aae7bdbbdd Reference raw "title" row in Phriction paging, not "c.title"
Trivial mistake -- the query "SELECT x.y ..." selects a column named "y".
The other calls to "getRawRowProperty()" get this right.
2019-03-23 07:27:42 -07:00
epriestley
c16a8c7ab2 Fix a content join condition in Phriction
After the cursor changes, we may fatal on pages with a large number of children
because "c.title" is not a selected column. We currently join the "content"
table if "updated" is part of the order vector, but not if "title" is part of
the order vector. This isn't right: "updated" is on the primary table, and only
"content" is on the joined table.
2019-03-23 07:18:01 -07:00
epriestley
3940c8e1f4 Make the UI when you use an invalid cursor ("?after=19874189471232892") a little nicer
Summary:
Ref T13259. Currently, visiting a page that executes a query with an invalid cursor raises a bare exception that escapes to top level.

Catch this a little sooner and tailor the page a bit.

Test Plan: Visited `/maniphest/?after=335234234223`, saw a nicer exception page.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20295
2019-03-19 13:04:07 -07:00
epriestley
18b444e427 When queries overheat, raise an exception
Summary:
Ref T13259. Currently, queries set a flag and return a partial result set when they overheat. This is mostly okay:

  - It's very unusual for queries to overheat if they don't have a real viewer.
  - Overheating is rare in general.
  - In most cases where queries can overheat, the context is a SearchEngine UI, which handles this properly.

In T13259, we hit a case where a query with an omnipotent viewer can overheat: if you have more than 1,000 consecutive commits in the database with invalid `repositoryID` values, we'll overheat and bail out. This is pretty bad, since we don't process everything.

Change this beahvior:

  - Throw by default, so this stuff doesn't slip through the cracks.
  - Handle the SearchEngine case explicitly ("it's okay to overheat, we'll handle it").
  - Make `QueryIterator` disable overheating behavior: if we're iterating over all objects, we want to hit the whole table even if most of it is garbage.

There are some cases where this might cause new exception behavior that we don't necessarily want. For example, in Owners, each package shows "recent commits in this package". If you can't see the first 1,000 recent commits, you'd previously get a slow page with no results. Now you'll probably get an exception.

If these crop up, I think the best approach for now is to deal with them on a case-by-case basis and see how far we get. In the "Owners" case, it might be good to query by repositories you can see first, then query by commits in the package in those repositories. That should give us a better outcome than any generic behavior we could implement.

Test Plan:
  - Added 100000 to all repositoryID values for commits on my local install.
  - Before making changes, ran `bin/repository rebuild-identities --all --trace`. Saw the script process 1,000 rows and exit silently.
  - Applied the first part ("throw by default") and ran `bin/repository rebuild-identities`. Saw the script process 1,000 rows, then raise an exception.
  - Applied the second part ("disable for queryiterator") and ran the script again. Saw the script process all 15,000 rows without issues (although none are valid and none actually load).
  - Viewed Diffusion, saw appropriate NUX / "overheated" UIs.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20294
2019-03-19 13:02:59 -07:00
epriestley
8449c1793a Convert complex query subclasses to use internal cursors
Summary:
Depends on D20292. Ref T13259. This converts the rest of the `getPagingValueMap()` callsites to operate on internal cursors instead.

These are pretty one-off for the most part, so I'll annotate them inline.

Test Plan:
  - Grouped tasks by project, sorted by title, paged through them, saw consistent outcomes.
  - Queried edges with "edge.search", paged through them using the "after" cursor.
  - Poked around the other stuff without catching any brokenness.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20293
2019-03-19 13:02:16 -07:00
epriestley
d4847c3eeb Convert simple query subclasses to use internal cursors
Summary:
Depends on D20291. Ref T13259. Move all the simple cases (where paging depends only on the partial object and does not depend on keys) to a simple wrapper.

This leaves a smaller set of more complex cases where we care about external data or which keys were requested that I'll convert in followups.

Test Plan: Poked at things, but a lot of stuff is still broken until everything is converted.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20292
2019-03-19 13:00:27 -07:00
epriestley
a6fd8f0479 When performing complex edits, pause sub-editors before they publish to propagate "Must Encrypt" and other state
Summary:
See PHI1134. Previously, see T13082 and D19969 for some sort-of-related stuff.

Currently, edits work roughly like this:

  - Suppose we're editing object X, and we're also going to edit some other object, Y, because X mentioned Y or the edit is making X a child or parent of Y, or unblocking Y.
  - Do the actual edit to X, including inverse edits ("alice mentioned Y on X.", "alice added a child revision: X", etc) which apply to Y.
  - Run Herald rules on X.
  - Publish the edit to X.

The "inverse edits" currently do this whole process inline, in a sub-editor. So the flow expands like this:

  - Begin editing X.
  - Update properties on X.
    - Begin inverse-edge editing Y.
    - Update properties on Y.
    - Run (actually, skip) Herald rules on Y.
    - Publish edits to Y.
  - Run Herald rules on X.
  - Publish edits to X.

Notably, the "Y" stuff publishes before the "X" Herald rules run. This creates potential problems:

  - Herald rules may change the name or visibility policy of "X", but we'll publish mail about it via the edits to Y before those edits apply. This is a problem only in theory, we don't ship any upstream rules like this today.
  - Herald rules may "Require Secure Mail", but we won't know that at the time we're building mail about the indirect change to "Y". This is a problem in practice.

Instead, switch to this new flow, where we stop the sub-editors before they publish, then publish everything at the very end once all the edits are complete:

  - Begin editing X.
  - Update properties on X.
    - Begin inverse-edge editing Y.
    - Update properties on Y.
    - Skip Herald on Y.
  - Run Herald rules on X.
  - Publish X.
    - Publish all child-editors of X.
      - Publish Y.

Test Plan:
  - Created "Must Encrypt" Herald rules for Tasks and Revisions.
  - Edited object "A", an object which the rules applied to directly, and set object "B" (a different object which the rules did not hit) as its parent/child and/or unblocked it.
  - In `bin/mail list-outbound`, saw:
    - Mail about object "A" all flagged as "Must Encrypt".
    - Normal mail from object B not flagged "Must Encrypt".
    - Mail from object B about changing relationships to object A flagged as "Must Encrypt".

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20283
2019-03-18 15:20:45 -07:00
epriestley
b469a5134d Allow "SMTP" and "Sendmail" mailers to have "Message-ID" behavior configured in "cluster.mailers"
Summary:
Fixes T13265. See that task for discussion. Briefly:

  - For mailers that use other mailers (SMTP, Sendmail), optionally let administrators set `"message-id": false` to improve threading behavior if their local Postfix is ultimately sending through SES or some other mailer which will replace the "Message-ID" header.

Also:

  - Postmark is currently marked as supporting "Message-ID", but it does not actually support "Message-ID" on `secure.phabricator.com` (mail arrives with a non-Phabricator message ID). I suspect this was just an oversight in building or refactoring the adapter; correct it.
  - Remove the "encoding" parameter from "sendmail". It think this was just missed in the cleanup a couple months ago; it is no longer used or documented.

Test Plan: Added and ran unit tests. (These feel like overkill, but this is super hard to test on real code.) See T13265 for evidence that this overall approach improves behavior.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13265

Differential Revision: https://secure.phabricator.com/D20285
2019-03-16 12:35:55 -07:00
epriestley
492b03628f Fix a typo in Drydock "Land" operations
Summary: Misspelling.

Test Plan: Careful reading.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20290
2019-03-15 14:53:05 -07:00
epriestley
df53d72e79 Allow "Move Tasks to Column..." to prompt for MFA
Summary:
Ref T13074. Currently, if you "Move Tasks to Column..." on a board and some of the tasks require MFA to edit, the workflow fatals out.

After this change, it works properly. You still have to answer a separate MFA prompt for //each// task, which is a little ridiculous, but at least doable. A reasonable future refinement would be to batch these MFA prompts, but this is currently the only use case for that.

Test Plan: Set a task to a "Require MFA" status, bulk-moved it with other tasks on a workboard. Was prompted, answered MFA prompt, got a move.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13074

Differential Revision: https://secure.phabricator.com/D20282
2019-03-13 16:30:17 -07:00
epriestley
04f9e72cbd Don't subscribe bots implicitly when they act on objects, or when they are mentioned
Summary:
See PHI1098. When users comment on objects, they are automatically subscribed. And when `@alice` mentions `@bailey` on a task, that usually subscribes `@bailey`.

These rules make less sense if the user is a bot. There's generally no reason for a bot to automatically subscribe to objects it acts on (it's not going to read email and follow up later), and it can always subscribe itself pretty easily if it wants (since everything is `*.edit` now and supports subscribe transactions).

Also, don't subscribe bots when they're mentioned for similar reasons. If users really want to subscribe bots, they can do so explicitly.

These rules seem slightly like "bad implicit magic" since it's not immediately obvious why `@abc` subscribes that user but `@xyz` may not, but some of these rules are fairly complicated already (e.g., `@xyz` doesn't subscribe them if they unsubscribed or are implicitly subscribed) and this feels like it gets the right/desired result almost-always.

Test Plan:
On a fresh task:

  - Mentioned a bot in a comment with `@bot`.
    - Before patch: bot got CC'd.
    - After patch: no CC.
  - Called `maniphest.edit` via the API to add a comment as a bot.
    - Before patch: bot got CC'd.
    - After patch: no CC.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20284
2019-03-13 16:28:40 -07:00
epriestley
a6e17fb702 Improve workboard "Owner" grouping, add "Author" grouping and "Title" sort
Summary:
Depends on D20277. Ref T10333.

  - Put profile icons on "Group by Owner".
  - Add a similar "Group by Author". Probably not terribly useful, but cheap to implement now.
  - Add "Sort by Title". Very likely not terribly useful, but cheap to implement and sort of flexible?

Test Plan: {F6265396}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20278
2019-03-12 14:30:38 -07:00
epriestley
8d74492875 Make workboard sort-order inversions more clear by using "-1 * ..." instead of "(int)-(int)"
Summary: Depends on D20279. See D20269. Agreed that explicit `-1` is probably more clear.

Test Plan: Viewed boards in each sort/group order.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20281
2019-03-12 14:06:18 -07:00
epriestley
a400d82932 Add "Group by Status" to Workboards
Summary:
Depends on D20276. Ref T10333. This one is a little bit rough/experimental, and I'm sort of curious what feedback we get about it. Weird stuff:

  - All statuses are always shown, even if the filter prevents tasks in that status from appearing (which is the default, since views are "Open Tasks" by default).
    - Pro: you can close tasks by dragging them to a closed status.
    - Con: lots of empty groups.
  - The "Duplicate" status is shown.
    - Pro: Shows closed duplicate tasks.
    - Con: Dragging tasks to "Duplicate" works, but is silly.
  - Since boards show "open tasks" by default, dragging stuff to a closed status and then reloading the board causes it to vanish. This is kind of how everything works, but more obvious/defaulted on "Status".

These issues might overwhelm its usefulness, but there isn't much cost to nuking it in the future if feedback is mostly negative/confused.

Test Plan: Grouped a workboard by status, dragged stuff around.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20277
2019-03-12 13:55:54 -07:00
epriestley
03b7aca019 Implement "Sort by Points" on workboards
Summary: Depends on D20275. Fixes T10578. This is a static sorting (like "By Date Created") where you can't change point values by dragging. You can still drag cards between columns, or use the "Edit" icon to change point values.

Test Plan: {F6265191}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10578

Differential Revision: https://secure.phabricator.com/D20276
2019-03-12 13:54:18 -07:00
epriestley
c020f027bb Add an "Sort by Creation Date" filter to workboards and modularize remaining order behaviors
Summary:
Depends on D20274. Ref T10578. This is en route to an ordering by points, it's just a simpler half-step on the way there.

Allow columns to be sorted by creation date, so the newest tasks rise to the top.

In this ordering you can never reposition cards, since editing a creation date by dragging makes no sense. This will be true of the "points" ordering too (although we could imagine doing something like prompting the user, some day).

Test Plan: Viewed boards by "natural" (allows reordering both when dragging within and between columns), "priority" (reorder only within columns), and "creation date" (reorder never). Dragged cards around between and within columns, got apparently sensible behavior.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10578

Differential Revision: https://secure.phabricator.com/D20275
2019-03-12 13:46:49 -07:00
epriestley
804be81f5d Provide better UI feedback about cards that can't be dragged or edited
Summary:
Depends on D20273. Fixes T10722. Currently, we don't make it very clear when a card can't be edited. Long ago, some code made a weak attempt to do this (by hiding the "grip" on the card), but later UI changes hid the "grip" unconditionally so that mooted things.

Instead:

  - Replace the edit pencil with a red lock.
  - Provide cursor hints for grabbable / not grabbable.
  - Don't let users pick up cards they can't edit.

Test Plan: On a workboard with a mixture of editable and not-editable cards, hovered over the different cards and was able to figure out which ones I could drag or not drag pretty easily. Picked up cards I could pick up, wasn't able to drag cards I can't edit.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10722

Differential Revision: https://secure.phabricator.com/D20274
2019-03-12 13:43:46 -07:00
epriestley
74de153e59 Allow MFA task edits to go through on workboards
Summary: Depends on D20272. Ref T13074. When a task requires MFA to edit, you currently get a fatal. Provide a cancel URI so the prompt works and the edit can go through.

Test Plan:
  - Locked a task, dragged it on a workboard.
  - Before: fatal trying to build an MFA gate.
  - After: got MFA gated, answered prompt, action went through.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13074

Differential Revision: https://secure.phabricator.com/D20273
2019-03-12 13:42:33 -07:00
epriestley
21dd79b35a When creating or editing a card on a sorted/grouped workboard, adjust headers appropriately
Summary:
Depends on D20270. Ref T10333. If you create a task with a new owner, or edit a task and change the priority/owner, we want to move it (and possibly create a new header) when the response comes back.

Make sure the response includes the appropriate details about the object's header and position.

Test Plan:
  - Grouped by Owner.
  - Created a new task with a new owner, saw the header appear.
  - Edited a task and changed it to give it a new owner, saw the header appear.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20271
2019-03-12 13:28:31 -07:00
epriestley
2fdab434fa Implement "Group by Owner" on Workboards
Summary: Depends on D20269. Ref T10333. Now that orderings are modularized, this is fairly easy to implement. This isn't super fancy for now (e.g., no profile images) but I'll touch it up in a general polish followup.

Test Plan: {F6264596}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20270
2019-03-12 13:25:55 -07:00
epriestley
9a8019d4a9 Modularize workboard column orders
Summary:
Depends on D20267. Depends on D20268. Ref T10333. Currently, we support "Natural" and "Priority" orders, but a lot of the particulars are pretty hard-coded, including some logic in `ManiphestTask`.

Although it's not clear that we'll ever put other types of objects on workboards, it seems generally bad that you need to modify `ManiphestTask` to get a new ordering.

Pull the ordering logic out into a `ProjectColumnOrder` hierarchy instead, and let each ordering define the things it needs to work (name, icon, what headers look like, how different objects are sorted, and how to apply an edit when you drop an object under a header).

Then move the existing "Natural" and "Priority" orders into this new hierarchy.

This has a minor bug where using the "Edit" workflow to change a card's priority on a priority-ordered board doesn't fully refresh card/header order since the response isn't ordering-aware. I'll fix that in an upcoming change.

Test Plan: Grouped workboards by "Natural" and "Priority", dragged stuff around within and between columns, grepped for all touched symbols.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20269
2019-03-12 13:07:50 -07:00
epriestley
4bad6bc42a Remove all readers/writers for task "subpriority"
Summary:
Depends on D20265. Ref T10333. Now that neither task lists nor workboards use subpriority, we can remove all the readers and writers.

I'm not actually getting rid of the column data yet, but anticipate doing that in a future change.

Note that the subpriority algorithm (removed here) is possibly better than the "natural order" algorithm still in use. It's a bit more clever, and likely performs far fewer writes. I might make the "natural order" code use an algorithm more similar to the "subpriority" algorithm in the future.

Test Plan: Grepped for `subpriority`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20266
2019-03-12 12:57:04 -07:00
epriestley
46ed8d4a5e On Workboards, sort groups by "natural order", not subpriority
Summary:
Depends on D20263. Ref T10333. I want to add groups like "Assignee" to workboards. This means you may have several tasks grouped under, say, "Alice".

When you drag the bottom-most task under "Alice" to the top, what does that mean?

Today, the only grouping is "Priority", and it means "change the task's secret/hidden global subpriority". However, this seems to generally be a somewhat-bad answer, and is quite complex. It also doesn't make much sense for an author grouping, since one task can't really be "more assigned" to Alice than another task.

Users likely intend this operation to mean "move it, visually, with no other effects" -- that is, user intent is to shuffle sticky notes around on a board, not edit anything substantive. The meaning is probably something like "this is similar to other nearby tasks" or "maybe this is a good place to start", which we can't really capture with any top-level attribute.

We could extend "subpriority" and give tasks a secret/hidden "sub-assignment strength" and so on, but this seems like a bad road to walk down. We'll also run into trouble later when subproject columns may appear on the board, and a user could want to put a task in different positions on different subprojects, conceivably.

In the "Natural" order view, we already have what is probably a generally better approach for this: a task display order particular to the column, that just remembers where you put the sticky notes.

Move away from "subpriority", and toward a world where we mostly keep sticky notes where you stuck them and move them around only when we have to. With no grouping, we still sort by "natural" order, as before. With priority grouping, we now sort by `<priority, natural>`. When you drag stuff around inside a priority group, we update the natural order.

This means that moving cards around on a "priority" board will also move them around on a "natural" board, at least somewhat. I think this is okay. If it's not intuitive, we could give every ordering its own separate "natural" view, so we remember where you stuck stuff on the "priority" board but that doesn't affect the "Natural" board. But I suspect we won't need to.

Test Plan:
  - Viewed and dragged a natural board.
  - Viewed and dragged a priority board.
  - Dragged within and between groups of 0, 1, and multiple items.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20265
2019-03-12 12:48:12 -07:00
epriestley
00543f0620 Remove the ability to drag tasks up and down on (non-Workboard) priority list views
Summary:
Ref T13074. Today, in normal task list views in Maniphest (not workboards), you can (sometimes) reorder tasks if the view is priority-sorted.

I suspect no one ever does this, few users know it's supported, and that it was basically rendered obsolete the day we shipped workboards.

This also means that we need to maintain a global "subpriority" for tasks, which distinguishes between different tasks at the same priority level (e.g., "High") and maintains a consistent ordering on workboards.

As we move toward making workboards more flexible (e.g., group by author / owner / custom fields), I'd like to try moving away from "subpriority" and possibly removing it entirely, in favor of "natural order", which basically means "we kind of remember where you put the card and it works a bit like a sticky note".

Currently, the "natural order" and "subpriority" systems are sort of similar but also sort of in conflict, and the "subpriority" system can't really be extended while the "natural order / column position" system can.

The only real reason to have a global "subpriority" is to support the list-view drag-and-drop.

It's possible I'm wrong about this and a bunch of users love this feature, but we can re-evaluate if we get feedback in this vein.

(This just removes UI, the actual subpriority system is still intact and still used on workboards.)

Test Plan: Viewed task lists, was no longer able to drag stuff. Grepped for affected symbols. Dragged stuff in remaining grippable lists, like "Edit Forms" in EditEngine config.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13074

Differential Revision: https://secure.phabricator.com/D20263
2019-03-12 12:47:36 -07:00
epriestley
46ab71f834 Fix "abou" typo of "about" in SearchEngine API methods
Summary: See <https://discourse.phabricator-community.org/t/minor-typo-abou-in-conduit-help/2498/2>.

Test Plan: Read carefully.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20280
2019-03-12 12:02:05 -07:00
epriestley
40af472ff5 Make drag-and-drop on workboards interact with priority column headers
Summary:
Ref T10333. Ref T8135. Depends on D20247. Allow users to drag-and-drop cards on a priority-sorted workboard under headers, even if the header has no other cards.

As of D20247, headers show up but they aren't really interactive. Now, you can drag cards directly underneath a header (instead of only between other cards). For example, if a column has only one "Wishlist" task, you may drag it under the "High", "Normal", or "Low" priority headers to select a specific priority.

(Some of this code still feels a little rough, but I think it will generalize once other types of sorting are available.)

Test Plan: Dragged cards within and between priority groups, saw appropriate priority edits applied in every case I could come up with.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333, T8135

Differential Revision: https://secure.phabricator.com/D20248
2019-03-09 10:33:26 -08:00
epriestley
14a433c773 Add priority group headers to workboard columns (display only)
Summary:
Ref T10333. When workboards are ordered (for example, by priority), add headers to the various groups. Major goals are:

  - Allow users to drag-and-drop to set values that no cards currently have: for example, you can change a card priority to "normal" by dragging it under the "normal" header, even if no other cards in the column are currently "Normal".
  - Make future orderings more useful, particularly "order by assignee". We don't really have room to put the username on every card and it would create a fair amount of clutter, but we can put usernames in these headers and then reference them with just the profile picture. This also allows you to assign to users who are not currently assigned anything in a given column.
  - Make the drag-and-drop behavior more obvious by showing what it will do more clearly (see T8135).
  - Make things a little easier to scan in general: because space on cards is limited, some information isn't conveyed very clearly (for example, priority information is currently conveyed //only// through color, which can be hard to pick out visually and is probably not functional for users who need vision accommodations).
  - Maybe do "swimlanes": this is pretty much a "swimlanes" UI if we add whitespace at the bottom of each group so that the headers line up across all the columns (e.g., "Normal" is at the same y-axis position in every column as you scroll down the page). Not sold on this being useful, but it's just a UI adjustment if we do want to try it.

NOTE: This only makes these headers work for display.

They aren't yet recognized as targets by the drag list UI, so you can't drag cards into an empty group. I'll tackle that in a followup.

Test Plan: {F6257686}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10333

Differential Revision: https://secure.phabricator.com/D20247
2019-03-09 10:32:55 -08:00
epriestley
c1bff3b801 Add an "Restartable: If Failed" behavior to Harbormaster build plans
Summary:
Ref T13249. Ref T13258. In some cases, builds are not idempotent and should not be restarted casually.

If the scary part is at the very end (deploy / provision / whatever), it could be okay to restart them if they previously failed.

Also, make the "reasons why you can't restart" and "explanations of why you can't restart" logic a little more cohesive.

Test Plan:
  - Tried to restart builds in various states (failed/not failed, restartable always/if failed/never, already restarted), got appropriate errors or restarts.
  - (I'm not sure the "Autoplan" error is normally reachable, since you can't edit autoplans to configure things to let you try to restart them.)

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258, T13249

Differential Revision: https://secure.phabricator.com/D20252
2019-03-07 16:47:57 -08:00
epriestley
1d4f6bd444 Index "Call Webhook" in Herald, and show calling rules on the Webhook page
Summary:
Depends on D20259. Now that we can index Herald rules to affected objects, show callers on the "Webhooks" UI.

A few other rule types could get indexes too ("Sign Legalpad Documents", "Add Reviewers", "Add Subscribers"), but I think they're less likely to be useful since those triggers are usually more obvious (the transaction timeline makes it clearer what happened/why). We could revisit this in the future now that it's a possibility.

Test Plan: {F6260106}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20260
2019-03-07 14:48:14 -08:00
epriestley
9913754a2a Improve utilization of "AuthTemporaryToken" table keys in LFS authentication queries
Summary:
See PHI1123. The key on this table is `<resource, type, code>` but we currently query for only `<type, code>`. This can't use the key.

Constrain the query to the resource we expect (the repository) so it can use the key.

Test Plan: Pushed files using LFS. See PHI1123 for more, likely.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20261
2019-03-07 14:02:41 -08:00
epriestley
950a7bbb19 On Harbormaster build plans, show which Herald rules trigger builds
Summary:
Ref T13258. Provide an easy way to find rules which trigger a particular build plan from the build plan page.

The implementation here ends up a little messy: we can't just search for `actionType = 'build' AND targetPHID = '<build plan PHID>'` since the field is a blob of JSON.

Instead, make rules indexable and write a "build plan is affected by rule actions" edge when indexing rules, then search on that edge.

For now, only "Run Build Plan: ..." rules actually write this edge, since I think (?) that it doesn't really have meaningful values for other edge types today. Maybe "Call Webhooks", and you could get a link from a hook to rules that trigger it? Reasonable to do in the future.

Things end up a little bit rough overall, but I think this panel is pretty useful to add to the Build Plan page.

This index needs to be rebuilt with `bin/search index --type HeraldRule`. I'll call this out in the changelog but I'm not planning to explicitly migrate or add an activity, since this is only really important for larger installs and they probably (?) read the changelog. As rules are edited over time, this will converge to the right behavior even if you don't rebuild the index.

Test Plan: {F6260095}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20259
2019-03-07 13:51:40 -08:00
epriestley
77221bee72 Allow objects to specify custom policy unlocking behavior, and tasks to have owners unlocked
Summary: Depends on D20256. Ref T13249. See PHI1115. This primarily makes `bin/policy unlock --owner epriestley T123` work. This is important for "Edit Locked" tasks, since changing the edit policy doesn't really do anything.

Test Plan: Hard-locked a task as "alice", reassigned it to myself with `bin/policy unlock --owner epriestley`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20257
2019-03-07 12:27:11 -08:00
epriestley
c86dca3ffc Update "bin/policy unlock" to be more surgical, flexible, modular, and modern
Summary:
See PHI1115. Ref T13249. Currently, you can `bin/policy unlock` objects which have become inaccessible through some sort of policy mistake.

This script uses a very blunt mechanism to perform unlocks: just manually calling `setXPolicy()` and then trying to `save()` the object. Improve things a bit:

  - More surgical: allow selection of which policies you want to adjust with "--view", "--edit", and "--owner" (potentially important for some objects like Herald rules which don't have policies, and "edit-locked" tasks which basically ignore the edit policy).
  - More flexible: Instead of unlocking into "All Users" (which could be bad for stuff like Passphrase credentials, since you create a short window where anyone can access them), take a username as a parameter and set the policy to "just that user". Normally, you'd run this as `bin/policy unlock --view myself --edit myself` or similar, now.
  - More modular: We can't do "owner" transactions in a generic way, but lay the groundwork for letting applications support providing an owner reassignment mechanism.
  - More modern: Use transactions, not raw `set()` + `save()`.

This previously had some hard-coded logic around unlocking applications. I've removed it, and the new generic stuff doesn't actually work. It probably should be made to work at some point, but I believe it's exceptionally difficult to lock yourself out of applications, and you can unlock them with `bin/config set phabricator.application-settings ...` anyway so I'm not too worried about this. It's also hard to figure out the PHID of an application and no one has ever asked about this so I'd guess the reasonable use rate of `bin/policy unlock` to unlock applications in the wild may be zero.

Test Plan:
  - Used `bin/policy unlock` to unlock some objects, saw sensible transactions.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20256
2019-03-07 12:24:25 -08:00
epriestley
bacf1f44e0 Modularize HeraldRule transactions
Summary:
Ref T13249. See PHI1115. I initially wanted to make `bin/policy unlock --owner <user> H123` work to transfer ownership of a Herald rule, although I'm no longer really sure this makes much sense.

In any case, this makes things a little better and more modern.

I removed the storage table for rule comments. Adding comments to Herald rules doesn't work and probably doesn't make much sense.

Test Plan: Created and edited Herald rules, grepped for all the transaction type constants.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20258
2019-03-07 11:55:31 -08:00
epriestley
9918ea1fb7 Fix an exception with user cache generation in "bin/conduit call --as <user>"
Summary:
Ref T13249. Using "--as" to call some Conduit methods as a user can currently fatal when trying to access settings/preferences.

Allow inline regeneration of user caches.

Test Plan: Called `project.edit` to add a member. Before: constructing a policy field tried to access the user's preferences and failed. After: Smooth sailing.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20255
2019-03-07 11:48:24 -08:00
epriestley
a3ebaac0f0 Tweak the visual style of the ">>" / "<<" depth change indicators slightly
Summary:
Ref T13249.

  - When a line has only increased in indent depth, don't red-fill highlight the left side of the diff. Since reading a diff //mostly// involves focusing on the right side, indent depth changes are generally visible enough without this extra hint. The extra hint can become distracting in cases where there is a large block of indent depth changes.
  - Move the markers slightly to the left, to align them with the gutter.
  - Make them slightly opaque so they're a little less prominent.

Test Plan: See screenshots.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20251
2019-03-07 11:46:26 -08:00
epriestley
7e46812344 Add a warning to revision timelines when changes land with ongoing or failed builds
Summary:
Ref T13258. The general idea here is "if arc land prompted you and you hit 'y', you get a warning about it on the timeline".

This is similar to the existing warning about landing revisions in the wrong state and hitting "y" to get through that. See D18808, previously.

These warnings make it easier to catch process issues at a glance, especially because the overall build status is now more complicated (and may legally include some failures on tests which are marked as unimportant).

The transaction stores which builds had problems, but I'm not doing anything to render that for now. I think you can usually figure it out from the UI already; if not, we could refine this.

Test Plan:
  - Used `bin/differential attach-commit` to trigger extraction/attachment.
  - Attached a commit to a revision with various build states, and various build plan "Warn When Landing" flags.
  - Got sensible warnings and non-warnings based on "Warn When Landing" setting.

{F6251631}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20239
2019-03-06 06:32:12 -08:00
epriestley
f97df9ebea Implement Build Plan behavior "Affects Buildable"
Summary: Ref T13258. Make the "Affects Buildable" option actually work.

Test Plan:
  - As in previous change, created a "wait for HTTP request" build plan and had it always run against every revision.
  - Created revisions, waited a bit, then sent the build a "Fail" message, with different values of "Affects Buildable":
  - "Always": Same behavior as today. Buildable waited for the build, then failed when it failed.
  - "While Building": Buildable waited for the build, but passed even though it failed (buildable has green checkmark even though build is red):

{F6250359}

  - "Never": Buildable passed immediately (buildable has green checkmark even though build is still running):

{F6250360}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20233
2019-03-06 06:30:09 -08:00
epriestley
718cdc2447 Implement Build Plan "Hold Drafts" behavior
Summary: Ref T13258. Makes the new "Hold Drafts" behavior actually work.

Test Plan:
  - Created a build plan which does "Make HTTP Request" somewhere random and then waits for a message.
  - Created a Herald rule which "Always" runs this plan.
  - Created revisions, loaded them, then sent their build targets a "fail" message a short time later.
    - With "Always": Current behavior. Revision was held as a draft while building, and returned to me for changes when the build failed.
    - With "If Building": Revision was held as a draft while building, but promoted once the build failed.
    - With "Never": Revision promoted immediately, ignoring the build completely.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20232
2019-03-06 06:27:49 -08:00
epriestley
578de333df Make the new Build Plan behavior "Restartable" work
Summary:
Ref T13258. Implements the "Restartable" behavior, to control whether a build may be restarted or not.

This is fairly straightforward because there are already other existing reasons that a build may not be able to be restarted.

Test Plan: Restarted a build. Marked it as not restartable, saw "Restart" action become disabled. Tried to restart it anyway, got a useful error message.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20230
2019-03-06 06:25:54 -08:00
epriestley
ee0ad4703e Make the new Build Plan "Runnable" behavior work
Summary:
Ref T13258. Fixes T11415. This makes "Runnable" actually do something:

  - With "Runnable" set to "If Editable" (default): to manually run, pause, resume, abort, or restart a build, you must normally be able to edit the associated build plan.
  - If you toggle "Runnable" to "If Viewable", anyone who can view the build plan may take these actions.

This is pretty straightforward since T9614 already got us pretty close to this ruleset a while ago.

Test Plan:
  - Created a Build Plan, set "Can Edit" to just me, toggled "Runnable" to "If Viewable"/"If Editable", tried to take actions as another user.
  - With "If Editable", unable to run, pause, resume, abort, or restart as another user.
  - With "If Viewable", those actions work.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258, T11415

Differential Revision: https://secure.phabricator.com/D20229
2019-03-06 06:01:02 -08:00
epriestley
983cf885e7 Expose Build Plan behaviors via "harbormaster.buildplan.search"
Summary: Ref T13258. This will support changing behaviors in "arc land".

Test Plan: Called "harbormaster.buildplan.search", saw behavior information in results.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20228
2019-03-06 05:47:35 -08:00
epriestley
d36d0efc35 Add behaviors to Build Plans: hold drafts, affect buildables, warn on landing, restartable, runnable
Summary:
Depends on D20219. Ref T13258. Ref T11415. Installs sometimes have long-running builds or unimportant builds which they may not want to hold up drafts, affect buildable status, or warn during `arc land`.

Some builds have side effects (like deployment or merging) and are not idempotent. They can cause problems if restarted.

In other cases, builds are isolated and idempotent and generally safe, and it's okay for marketing interns to restart them.

To address these cases, add "Behaviors" to Build Plans:

  - Hold Drafts: Controls how the build affects revision promotion from "Draft".
  - Warn on Land: Controls the "arc land" warning.
  - Affects Buildable: Controls whether we care about this build when figuring out if a buildable passed or failed overall.
  - Restartable: Controls whether this build may restart or not.
  - Runnable: Allows you to weaken the requirements to run the build if you're confident it's safe to run it on arbitrary old versions of things.

NOTE: This only implements UI, none of these options actually do anything yet.

Test Plan:
Mostly poked around the UI. I'll actually implement these behaviors next, and vet them more thoroughly.

{F6244828}

{F6244830}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258, T11415

Differential Revision: https://secure.phabricator.com/D20220
2019-03-06 05:40:06 -08:00
epriestley
ea6c0c9bde Refine the "Mangled Webserver Response" setup check
Summary:
Ref T13259. In some configurations, making a request to ourselves may return a VPN/Auth response from some LB/appliance layer.

If this response begins or ends with whitespace, we currently detect it as "extra whitespace" instead of "bad response".

Instead, require that the response be nearly correct (valid JSON with some extra whitespace, instead of literally anything with some extra whitespace) to hit this specialized check. If we don't hit the specialized case, use the generic "mangled" response error, which prints the actual body so you can figure out that it's just your LB/auth thing doing what it's supposed to do.

Test Plan:
  - Rigged responses to add extra whitespace, got "Extra Whitespace" (same as before).
  - Rigged responses to add extra non-whitespace, got "Mangled Junk" (same as before).
  - Rigged responses to add extra whitespace and extra non-whitespace, got "Mangled Junk" with a sample of the document body instead of "Extra Whitespace" (improvement).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20235
2019-03-05 12:58:32 -08:00
epriestley
c116deef63 Remove "Effective User" attachment from Repository Identities
Summary:
See <https://discourse.phabricator-community.org/t/phabricatorrepositoryidentity-attacheffectiveuser-must-be-an-instance-of-phabricatoruser-null-given/1820/3>.

It's possible for an Idenitity to be bound to user X, then for that user to be deleted, e.g. with `bin/remove destroy X`. In this case, we'll fail to load/attach the user to the identity.

Currently, we don't actually use this anywhere, so just stop loading it. Once we start using it (if we ever do), we could figure out what an appropriate policy is in this case.

Test Plan: Browsed around Diffusion, grepped for affected "effective user" methods and found no callsites.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20224
2019-03-05 11:35:12 -08:00
epriestley
cc8dda6299 Recognize the official "Go" magic regexp for generated code as generated
Summary: See PHI1112. See T784. Although some more general/flexible solution is arriving eventually, adding this rule seems reasonable for now, since it's not a big deal if we remove it later to replace this with some fancier system.

Test Plan: Created a diff with the official Go generated marker, saw the changeset marked as generated.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20237
2019-03-05 11:34:11 -08:00
epriestley
920ab13cfb Correct a possible fatal in the non-CSRF Duo MFA workflow
Summary:
Ref T13259. If we miss the separate CSRF step in Duo and proceed directly to prompting, we may fail to build a response which turns into a real control and fatal on `null->setLabel()`.

Instead, let MFA providers customize their "bare prompt dialog" response, then make Duo use the same "you have an outstanding request" response for the CSRF and no-CSRF workflows.

Test Plan: Hit Duo auth on a non-CSRF workflow (e.g., edit an MFA provider with Duo enabled). Previously: `setLabel()` fatal. After patch: smooth sailing.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20234
2019-03-05 11:33:25 -08:00
epriestley
34e90d8f51 Clean up a few "%Q" stragglers in SVN repository browsing code
Summary: See <https://discourse.phabricator-community.org/t/unsafe-raw-string-warning-for-svn-in-diffusion/2469>.

Test Plan: Browed a Subversion repository in Diffusion. These are all reachable from the main landing page if the repository has commits/files, I think. Before change: errors in log; after change: no issues.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20244
2019-03-05 11:30:55 -08:00
epriestley
e15fff00a6 Use "LogLevel=ERROR" to try to improve "ssh" hostkey behavior without doing anything extreme/hacky
Summary:
Ref T13121. When you connect to a host with SSH, don't already know the host key, and don't have strict host key checking, it prints "Permanently adding host X to known hosts". This is super un-useful.

In a perfect world, we'd probably always have strict host key checking, but this is a significant barrier to configuration/setup and I think not hugely important (MITM attacks against SSH hosts are hard/rare and probably not hugely valuable). I'd imagine a more realistic long term approach is likely optional host key checking.

For now, try using `LogLevel=ERROR` instead of `LogLevel=quiet` to suppress this error. This should be strictly better (since at least some messages we want to see are ERROR or better), although it may not be perfect (there may be other INFO messages we would still like to see).

Test Plan:
  - Ran `ssh -o LogLevel=... -o 'StrictHostKeyChecking=no' -o 'UserKnownHostsFile=/dev/null'` with bad credentials, for "ERROR", "quiet", and default ("INFO") log levels.
  - With `INFO`, got a warning about adding the key, then an error about bad credentials (bad: don't want the key warning).
  - With `quiet`, got nothing (bad: we want the credential error).
  - With `ERROR`, got no warning but did get an error (good!).

Not sure this always gives us exactly what we want, but it seems like an improvement over "quiet".

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13121

Differential Revision: https://secure.phabricator.com/D20240
2019-03-04 09:38:00 -08:00
epriestley
75dfae1011 Don't require any special capabilities to apply a "closed a subtask" transaction to a parent task
Summary:
See PHI1059. If you close a task, we apply an "alice closed a subtask: X" transaction to its parents.

This transaction is purely informative, but currently requires `CAN_EDIT` permission after T13186. However, we'd prefer to post this transaction anyway, even if: the parent is locked; or the parent is not editable by the acting user.

Replace the implicit `CAN_EDIT` requirement with no requirement.

(This transaction is only applied internally (by closing a subtask) and can't be applied via the API or any other channel, so this doesn't let attackers spam a bunch of bogus subtask closures all over the place or anything.)

Test Plan:
  - Created a parent task A with subtask B.
  - Put task A into an "Edits Locked" status.
  - As a user other than the owner of A, closed B.

Then:

  - Before: Policy exception when trying to apply the "alice closed a subtask: B" transaction to A.
  - After: B closed, A got a transaction despite being locked.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20223
2019-02-28 19:48:28 -08:00
epriestley
4cc556b576 Clean up a PhutilURI "alter()" callsite in Diffusion blame
Summary: See <https://discourse.phabricator-community.org/t/exception-when-viewing-previous-revision-from-blame/2454/2>.

Test Plan: Viewed blame, clicked "Skip Past This Commit". Got jumped to the right place instead of a URI exception.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20222
2019-02-28 19:47:46 -08:00
epriestley
620047bcfe Add a "Recent Builds" element to the Build Plan UI and tighten up a few odds and ends
Summary:
Depends on D20218. Ref T13258. It's somewhat cumbersome to get from build plans to related builds but this is a reasonable thing to want to do, so make it a little easier.

Also clean up / standardize / hint a few things a little better.

Test Plan: {F6244116}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20219
2019-02-28 07:38:37 -08:00
epriestley
8338f94057 Provide "harbormaster.buildplan.edit" in the API
Summary: Depends on D20217. Ref T13258. Mostly for completeness. You can't edit build steps so this may not be terribly useful, but you can do bulk policy edits or whatever?

Test Plan: Edited a build plan via API.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20218
2019-02-28 07:36:31 -08:00
epriestley
f6ed873f17 Move Harbormaster Build Plans to modular transactions
Summary: Depends on D20216. Ref T13258. Bland infrastructure update to prepare for bigger things.

Test Plan: Created and edited a build plan.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13258

Differential Revision: https://secure.phabricator.com/D20217
2019-02-28 07:32:13 -08:00
epriestley
41c03bab39 Remove unusual "Created" element from Build Plan curtain UI
Summary: Ref T13088. Build Plans currently have a "Created" date in the right-hand "Curtain" UI, but this is unusual and the creation date is evident from the timeline. It's also not obvious why anyone would care. Remove it for simplicity/consistency. I think this may have just been a placeholder during initial implementation.

Test Plan: Viewed a build plan, no more "Created" element.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D20216
2019-02-28 07:31:21 -08:00
epriestley
b28b05342b Simplify one "array_keys/range" -> "phutil_is_natural_list()" in "phabricator/"
Summary: Depends on D20213. Simplify this idiom.

Test Plan: Squinted hard; `grep array_keys | grep range`.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20215
2019-02-28 07:29:36 -08:00
epriestley
814e6d2de9 Add more type checking to transactions queued by Herald
Summary:
See PHI1096. Depends on D20213. An install is reporting a hard-to-reproduce issue where a non-transaction gets queued by Herald somehow. This might be in third-party code.

Sprinkle the relevant parts of the code with `final` and type checking to try to catch the problem before it causes a fatal we can't pull a stack trace out of.

Test Plan: Poked around locally (e.g., edited revisions to cause Herald to trigger), but hard to know if this will do what it's supposed to or not without deploying and seeing if it catches anything.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20214
2019-02-28 07:10:56 -08:00
epriestley
dc9aaa0fc2 Fix a stray "%Q" warning when hiding/showing inline comments
Summary: See PHI1095.

Test Plan: Viewed a revision, toggled hide/show on inline comments. Before: warning in logs; after: smooth sailing.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20212
2019-02-25 13:51:09 -08:00
epriestley
d1546209c5 Expand documentation for "transaction.search"
Summary: Depends on D20209. Ref T13255. It would probably be nice to make this into a "real" `*.search` API method some day, but at least document the features for now.

Test Plan: Read documentation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13255

Differential Revision: https://secure.phabricator.com/D20211
2019-02-25 10:52:29 -08:00
epriestley
83aba7b01c Enrich the "change project tags" transaction in "transaction.search"
Summary:
Depends on D20208. Ref T13255. See that task for some long-winded discussion and rationale. Short version:

  - This is a list of operations instead of a list of old/new PHIDs because of scalability issues for large lists (T13056).
  - This is a fairly verbose list (instead of, for example, the more concise internal map we sometimes use with "+" and "-" as keys) to try to make the structure obvious and extensible in the future.
  - The "add" and "remove" echo the `*.edit` operations.

Test Plan: Called `transaction.search` on an object with project tag changes, saw them in the results.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13255

Differential Revision: https://secure.phabricator.com/D20209
2019-02-25 10:50:04 -08:00
epriestley
f61e825905 Make the Diffusion warning about "svnlook" and PATH more clear
Summary:
See <https://discourse.phabricator-community.org/t/display-error-on-the-status-page-for-svn-repos/2443> for discussion.

The UI currently shows a misleading warning that looks like "found svnlook; can't find svnlook".

It actually means "found svnlook, but when Subversion wipes PATH before executing commit hooks, we will no longer be able to find it".

Test Plan: {F6240967}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20210
2019-02-25 10:48:12 -08:00
epriestley
767afd1780 Support an "authorPHIDs" constraint for "transaction.search"
Summary: Ref T13255. The "transaction.search" API method currently does not support author constraints, but this is a reasonable thing to support.

Test Plan: Queried transactions by author, hit the error cases.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13255

Differential Revision: https://secure.phabricator.com/D20208
2019-02-25 06:33:04 -08:00
epriestley
66161feb13 Fix a URI construction exception when filtering the Maniphest Burnup chart by project
Summary: See <https://discourse.phabricator-community.org/t/filtering-burnup-rate-by-project-produces-exception/2442>.

Test Plan: Viewed Maniphest burnup chart, filtered by project: no more URI construction exception.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20207
2019-02-25 05:36:31 -08:00
Ariel Yang
01d0fc443a Fix a typo
Summary: Fix a type

Test Plan: No need.

Reviewers: #blessed_reviewers, amckinley

Reviewed By: #blessed_reviewers, amckinley

Subscribers: amckinley, epriestley

Differential Revision: https://secure.phabricator.com/D20203
2019-02-24 13:37:14 +00:00
epriestley
701a9bc339 Fix Facebook login on mobile violating CSP after form redirect
Summary: Fixes T13254. See that task for details.

Test Plan: Used iOS Simulator to do a login locally, didn't get blocked. Verified CSP includes "m.facebook.com".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13254

Differential Revision: https://secure.phabricator.com/D20206
2019-02-23 05:25:09 -08:00
epriestley
90064a350a Fix URI construction of typeahead browse "more" pager
Summary: Ref T13251. See <https://phabricator.wikimedia.org/T216849>.

Test Plan:
  - With more than 100 projects (or, set `$limit = 3`)...
  - Edit a task, then click the "Browse" magnifying glass icon next to the "Tags" typeahead.
  - Before change: fatal on 'q' being null.
  - After change: no fatal.

Reviewers: amckinley, 20after4

Reviewed By: amckinley

Maniphest Tasks: T13251

Differential Revision: https://secure.phabricator.com/D20204
2019-02-22 18:09:16 -08:00
Austin McKinley
abc26aa96f Track total time from task creation to task archival
Summary:
Ref T5401. Depends on D20201. Add timestamps to worker tasks to track task creation, and pass that through to archive tasks. This lets us measure the total time the task spent in the queue, not just the duration it was actually running.

Also displays this information in the daemon status console; see screenshot: {F6225726}

Test Plan:
Stopped daemons, ran `bin/search index --all --background` to create lots of tasks, restarted daemons, observed expected values for `dateCreated` and `epochArchived` in the archive worker table.

Also tested the changes to `unarchiveTask` by forcing a search task to permanently fail and then `bin/worker retry`ing it.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T5401

Differential Revision: https://secure.phabricator.com/D20200
2019-02-20 14:44:29 -08:00
epriestley
1b83256421 Don't enable the "ScopeEngine" or try to identify scope context for diffs without context
Summary:
Depends on D20197. Ref T13161. We currently try to build a "ScopeEngine" even for diffs with no context (e.g., `git diff` instead of `git diff -U9999`).

Since we don't have any context, we won't really be able to figure out anything useful about scopes. Also, since ScopeEngine is pretty strict about what it accepts, we crash.

In these cases, just don't build a ScopeEngine.

Test Plan: Viewed a diff I copy/pasted with `git diff` instead of an `arc diff` / `git diff -U99999`, got a sensible diff with no context instead of a fatal.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161

Differential Revision: https://secure.phabricator.com/D20198
2019-02-20 10:16:20 -08:00
epriestley
f1a035d5c2 In Differential, give the "moved/copied from" gutter a more clear visual look
Summary:
Depends on D20196. See PHI985. When empty, the "moved/copied" gutter currently renders with the same background color as the rest of the line. This can be misleading because it makes code look more indented than it is, especially if you're unfamiliar with the tool:

{F6225179}

If we remove this misleading coloration, we get a white gap. This is more clear, but looks a little odd:

{F6225181}

Instead, give this gutter a subtle background fill in all casses, to make it more clear that it's a separate gutter region, not a part of the text diff:

{F6225183}

Test Plan: See screenshots. Copied text from a diff, added/removed inlines, etc.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20197
2019-02-20 10:12:16 -08:00
epriestley
a33409991c Remove an old Differential selection behavior
Summary:
Ref T12822. Ref PHI878. This is some leftover code from the old selection behavior that prevented visual selection of the left side of a diff if the user clicked on the right -- basically, a much simpler attack on what ultimately landed in D20191.

I think the change from `th` to `td` "broke" it so it didn't interfere with the other behavior, which is why I didn't have to remove it earlier. It's no longer necessary, in any case.

Test Plan: Grepped for behavior name, selected stuff on both sides of a diff.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12822

Differential Revision: https://secure.phabricator.com/D20196
2019-02-20 10:09:34 -08:00
epriestley
c10b283b92 Remove some ancient daemon log code
Summary:
Ref T13253. Long ago, daemon logs were visible in the web UI. They were removed because access to logs generally does not conform to policy rules, and may leak the existence (and sometimes contents) of hidden objects, occasionally leak credentials in certain error messages, etc.

These bits and pieces were missed.

Test Plan: Grepped for removed symbols.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13253

Differential Revision: https://secure.phabricator.com/D20199
2019-02-20 09:52:11 -08:00
epriestley
cf048f4402 Tweak some display behaviors for indent indicators
Summary:
Ref T13161.

  - Don't show ">>" when the line indentation changed but the text also changed, this is just "the line changed".
  - The indicator seems a little cleaner if we just reuse the existing "bright" colors, which already have colorblind colors anyway.

Test Plan: Got slightly better rendering for some diffs locally.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161

Differential Revision: https://secure.phabricator.com/D20195
2019-02-19 15:34:30 -08:00
epriestley
fe7047d12d Display some invisible/nonprintable characters in diffs by default
Summary:
Ref T12822. Ref T2495. This is the good version of D20193.

Currently, we display various nonprintable characters (ZWS, nonbreaking space, various control characters) as themselves, so they're generally invisible.

In T12822, one user reports that all their engineers frequently type ZWS characters into source somehow? I don't really believe this (??), and this should be fixed in lint.

That said, the only real reason not to show these weird characters in a special way was that it would break copy/paste: if we render ZWS as "🐑", and a user copy-pastes the line including the ZWS, they'll get a sheep.

At least, they would have, until D20191. Now that this whole thing is end-to-end Javascript magic, we can copy whatever we want.

In particular, we can render any character `X` as `<span data-copy-text="Y">X</span>`, and then copy "Y" instead of "X" when the user copies the node. Limitations:

  - If users select only "X", they'll get "X" on their clipboard. This seems fine. If you're selecting our ZWS marker *only*, you probably want to copy it?
  - If "X" is more than one character long, users will get the full "Y" if they select any part of "X". At least here, this only matters when "X" is several spaces and "Y" is a tab. This also seems fine.
  - We have to be kind of careful because this approach involves editing an HTML blob directly. However, we already do that elsewhere and this isn't really too hard to get right.

With those tools in hand:

  - Replace "\t" (raw text / what gets copied) with the number of spaces to the next tab stop for display.
  - Replace ZWS and NBSP (raw text) with a special marker for display.
  - Replace control characters 0x00-0x19 and 0x7F, except for "\t", "\r", and "\n", with the special unicode "control character pictures" reserved for this purpose.

Test Plan:
- Generated and viewed a file like this one:

{F6220422}

- Copied text out of it, got authentic raw original source text instead of displayed text.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12822, T2495

Differential Revision: https://secure.phabricator.com/D20194
2019-02-19 15:21:44 -08:00
epriestley
efccd75ae3 Correct various minor diff copy behaviors
Summary:
Ref T12822. Fixes a few things:

  - Firefox selection of weird ranges with an inline between the start and end of the range now works correctly.
  - "Show More Context" rows now render, highlight, and select properly.
  - Prepares for nodes to have copy-text which is different from display-text.
  - Don't do anything too fancy in 1-up/unified mode. We don't copy line numbers after the `content: attr(data-n)` change, but that's as far as we go, because trying to do more than that is kind of weird and not terribly intuitive.

Test Plan:
  - Selected and copied weird ranges in Firefox.
  - Kept an eye on "Show More Context" rows across select and copy operations.
  - Generally poked around in Safari/Firefox/Chrome.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12822

Differential Revision: https://secure.phabricator.com/D20192
2019-02-19 15:18:45 -08:00
epriestley
37f12a05ea Behold! Copy text from either side of a diff!
Summary:
Ref T12822. Ref T13161. By default, when users select text from a diff and copy it to the clipboard, they get both sides of the diff and all the line numbers. This is usually not what they intended to copy.

As of D20188, we use `content: attr(...)` to render line numbers. No browser copies this text, so that fixes line numbers.

We can use "user-select" CSS to visually prevent selection of line numbers and other stuff we don't want to copy. In Firefox and Chrome, "user-select" also applies to copied text, so getting "user-select" on the right nodes is largely good enough to do what we want.

In Safari, "user-select" is only visual, so we always need to crawl the DOM to figure out what text to pull out of it anyway.

In all browsers, we likely want to crawl the DOM anyway because this will let us show one piece of text and copy a different piece of text. We probably want to do this in the future to preserve "\t" tabs, and possibly to let us render certain character codes in one way but copy their original values. For example, we could render "\x07" as "␇".

Finally, we have to figure out which side of the diff we're copying from. The rule here is:

  - If you start the selection by clicking somewhere on the left or right side of the diff, that's what you're copying.
  - Otherwise, use normal document copy rules.

So the overall flow here is:

  - Listen for clicks.
  - When the user clicks the left or right side of the diff, store what they clicked.
  - When a selection starts, and something is actually selected, check if it was initiated by clicking a diff. If it was, apply a visual effect to get "user-select" where it needs to go and show the user what we think they're doing and what we're going to copy.
  - (Then, try to handle a bunch of degenerate cases where you start a selection and then click inside that selection.)
  - When a user clicks elsewhere or ends the selection with nothing selected, clear the selection mode.
  - When a user copies text, if we have an active selection mode, pull all the selected nodes out of the DOM and filter out the ones we don't want to copy, then stitch the text back together. Although I believe this didn't work well in ~2010, it appears to work well today.

Test Plan: This mostly seems to work in Safari, Chrome, and Firefox. T12822 has some errata. I haven't tested touch events but am satisfied if the touch event story is anything better than "permanently destroys data".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161, T12822

Differential Revision: https://secure.phabricator.com/D20191
2019-02-19 15:17:07 -08:00
epriestley
d4b96bcf6b Remove hidden zero-width spaces affecting copy behavior
Summary:
Ref T13161. Ref T12822. Today, we use invisible Zero-Width Spaces to try to improve copy/paste behavior from Differential.

After D20188, we no longer need ZWS characters to avoid copying line numbers. Get rid of these secret invisible semantic ZWS characters completely.

This means that both the left-hand and right-hand side of diffs become copyable, which isn't desired. I'll fix that with a hundred thousand lines of Javascript in the next change: this is a step toward everything working better, but doesn't fix everything yet.

Test Plan:
  - Grepped for `zws`, `grep -i zero | grep -i width`.
  - Copied text out of Differential: got both sides of the diff (not ideal).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161, T12822

Differential Revision: https://secure.phabricator.com/D20189
2019-02-19 15:10:04 -08:00
epriestley
98fe8fae4a Use <td class="n" data-n="3"> instead of <th>3</th> for line numbers
Summary:
Ref T13161. Ref T12822. See PHI870. Long ago, the web was simple. You could leave your doors unlocked, you knew all your neighbors, crime hadn't been invented yet, and `<th>3</th>` was a perfectly fine way to render a line number cell containing the number "3".

But times have changed!

  - In PHI870, this isn't good for screenreaders. We can't do much about this, so switch to `<td>`.
  - In D19349 / T13105 and elsewhere, this `::after { content: attr(data-n); }` approach seems like the least bad general-purpose approach for preventing line numbers from being copied. Although Differential needs even more magic beyond this in the two-up view, this is likely good enough for the one-up view, and is consistent with other views (paste, harbormaster logs, general source display) where this technique is sufficient on its own.

The chance this breaks //something// is pretty much 100%, but we've got a week to figure out what it breaks. I couldn't find any issues immediately.

Test Plan:
  - Created, edited, deleted inlines in 1-up and 2-up views.
  - Replied, keyboard-navigated, keyboard-replied, drag-selected, poked and prodded everything.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161, T12822

Differential Revision: https://secure.phabricator.com/D20188
2019-02-19 15:07:02 -08:00
epriestley
3f8eccdaec Put some whitespace behaviors back, but only for "diff alignment", not display
Summary:
Depends on D20185. Ref T13161. Fixes T6791.

See some discusison in T13161. I want to move to a world where:

  - whitespace changes are always shown, so users writing YAML and Python are happy without adjusting settings;
  - the visual impact of indentation-only whitespace changes is significanlty reduced, so indentation changes are easy to read and users writing Javascript or other flavors of Javascript are happy.

D20181 needs a little more work, but generally tackles these visual changes and lets us always show whitespace changes, but show them in a very low-impact way when they're relatively unimportant.

However, a second aspect to this is how the diff is "aligned". If this file:

```
A
```

..is changed to this file:

```
X
A
Y
Z
```

...diff tools will generally produce this diff:

```
+ X
  A
+ Y
+ Z
```

This is good, and easy to read, and what humans expect, and it will "align" in two-up like this:

```
       1 X
1 A    2 A
       3 Y
       4 Z
```

However, if the new file looks like this instead:

```
X
A'
Y
Z
```

...we get a diff like this:

```
- A
+ X
+ A'
+ Y
+ Z
```

This one aligns like this:

```
1 A
        1 X
        2 A'
        3 Y
        4 Z
```

This is correct if `A` and `A'` are totally different lines. However, if `A'` is pretty much the same as `A` and it just had a whitespace change, human viewers would prefer this alignment:

```
        1 X
1 A     2 A'
        3 Y
        4 Z
```

Note that `A` and `A'` are different, but we've aligned them on the same line. `diff`, `git diff`, etc., won't do this automatically, and a `.diff` doesn't have a way to say "these lines are more or less the same even though they're different", although some other visual diff tools will do this.

Although `diff` can't do this for us, we can do it ourselves, and already have the code to do it, because we already nearly did this in the changes removed in D20185: in "Ignore All" or "Ignore Most" mode, we pretty much did this already.

This mostly just restores a bit of the code from D20185, with some adjustments/simplifications. Here's how it works:

  - Rebuild the text of the old and new files from the diff we got out of `arc`, `git diff`, etc.
  - Normalize the files (for example, by removing whitespace from each line).
  - Diff the normalized files to produce a second diff.
  - Parse that diff.
  - Take the "alignment" from the normalized diff (whitespace removed) and the actual text from the original diff (whitespace preserved) to build a new diff with the correct text, but also better diff alignment.

Originally, we normalized with `diff -bw`. I've replaced that with `preg_replace()` here mostly just so that we have more control over things. I believe the two behaviors are pretty much identical, but this way lets us see more of the pipeline and possibly add more behaviors in the future to improve diff quality (e.g., normalize case? normalize text encoding?).

Test Plan:
{F6217133}

(Also, fix a unit test.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161, T6791

Differential Revision: https://secure.phabricator.com/D20187
2019-02-19 13:11:50 -08:00
epriestley
5310f1cdd9 Remove all whitespace options/configuration everywhere
Summary:
Depends on D20181. Depends on D20182. Fixes T3498. Ref T13161. My claim, at least, is that D20181 can be tweaked to be good enough to throw away this "feature" completely.

I think this feature was sort of a mistake, where the ease of access to `diff -bw` shaped behavior a very long time ago and then the train just ran a long way down the tracks in the same direction.

Test Plan: Grepped for `whitespace`, deleted almost everything. Poked around the UI a bit. I'm expecting the whitespace changes to get some more iteration this week so I not being hugely pedantic about testing this stuff exhaustively.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161, T3498

Differential Revision: https://secure.phabricator.com/D20185
2019-02-19 13:09:29 -08:00
epriestley
661c758ff9 Render indent depth changes more clearly
Summary:
Ref T13161. See PHI723. Our whitespace handling is based on whitespace flags like `diff -bw`, mostly just for historical reasons: long ago, the easiest way to minimize the visual impact of indentation changes was to literally use `diff -bw`.

However, this approach is very coarse and has a lot of problems, like detecting `"ab" -> "a b"` as "only a whitespace change" even though this is always semantic. It also causes problems in YAML, Python, etc. Over time, we've added a lot of stuff to mitigate the downsides to this approach.

We also no longer get any benefits from this approach being simple: we need faithful diffs as the authoritative source, and have to completely rebuild the diff to `diff -bw` it. In the UI, we have a "whitespace mode" flag. We have the "whitespace matters" configuration.

I think ReviewBoard generally has a better approach to indent depth changes than we do (see T13161) where it detects them and renders them in a minimal way with low visual impact. This is ultimately what we want: reduce visual clutter for depth-only changes, but preserve whitespace changes in strings, etc.

Move toward detecting and rendering indent depth changes. Followup work:

  - These should get colorblind colors and the design can probably use a little more tweaking.
  - The OneUp mode is okay, but could be improved.
  - Whitespace mode can now be removed completely.
  - I'm trying to handle tabs correctly, but since we currently mangle them into spaces today, it's hard to be sure I actually got it right.

Test Plan: {F6214084}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161

Differential Revision: https://secure.phabricator.com/D20181
2019-02-19 12:40:05 -08:00
epriestley
deea2f01f5 Allow unit tests to have arbitrarily long names (>255 characters)
Summary:
Depends on D20179. Ref T13088. See PHI351. See PHI1018. In various cases, unit tests names are 19 paths mashed together.

This is probably not an ideal name, and the test harness should probably pick a better name, but if users are fine with it and don't want to do the work to summarize on their own, accept them. We may summarize with "..." in some cases depending on how this fares in the UI.

The actual implementation is a separate "strings" table which is just `<hash-of-string, full-string>`. The unit message table can end up being mostly strings, so this should reduce storage requirements a bit.

For now, I'm not forcing a migration: new writes use the new table, existing rows retain the data. I plan to provide a migration tool, recommend migration, then force migration eventually.

Prior to that, I'm likely to move at least some other columns to use this table (e.g., lint names), since we have a lot of similar data (arbitrarily long user string constants that we are unlikely to need to search or filter).

Test Plan: {F6213819}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D20180
2019-02-19 11:21:42 -08:00