1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-23 23:32:40 +01:00
Commit graph

12358 commits

Author SHA1 Message Date
epriestley
92abe3c8fb Extract scope line selection logic from the diff rendering engine so it can reasonably be iterated on
Summary:
Ref T13249. Ref T11738. See PHI985. Currently, we have a crude heuristic for guessing what line in a source file provides the best context.

We get it wrong in a lot of cases, sometimes selecting very silly lines like "{". Although we can't always pick the same line a human would pick, we //can// pile on heuristics until this is less frequently completely wrong and perhaps eventually get it to work fairly well most of the time.

Pull the logic for this into a separate standalone class and make it testable to prepare for adding heuristics.

Test Plan: Ran unit tests, browsed various files in the web UI and saw as-good-or-better context selection.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249, T11738

Differential Revision: https://secure.phabricator.com/D20171
2019-02-19 10:55:10 -08:00
epriestley
8d348e2eeb Clean up a couple of %Q issues in "Has Parents" task queries
Summary: Stragglers from the great "%Q" migration.

Test Plan: Ran a query for tasks with parent tasks.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20183
2019-02-19 10:54:23 -08:00
epriestley
e44b40ca4d Make "Subscribe/Unsubscribe" require only "CAN_VIEW", not "CAN_INTERACT"
Summary:
Ref T13249. See PHI1059. Currently, Subscribe/Unsubscribe require CAN_INTERACT via the web UI and no permissions (i.e., effectively CAN_VIEW) via the API.

Weaken the requirements from the web UI so that you do not need "CAN_INTERACT". This is a product change to the effect that it's okay to subscribe/unsubscribe from anything you can see, even hard-locked tasks. This generally seems reasonable.

Increase the requirements for the actual transaction, which mostly applies to API changes:

  - To remove subscribers other than yourself, require CAN_EDIT.
  - To add subscribers other than yourself, require CAN_EDIT or CAN_INTERACT. You may have CAN_EDIT but not CAN_INTERACT on "soft locked" tasks. It's okay to click "Edit" on these, click "Yes, override lock", then remove subscribers other than yourself.

This technically plugs some weird, mostly theoretical holes in the API where "attackers" could sometimes make more subscription changes than they should have been able to. Now that we send you email when you're unsubscribed this could only really be used to be mildly mischievous, but no harm in making the policy enforcement more correct.

Test Plan: Against normal, soft-locked, and hard-locked tasks: subscribed, unsubscribed, added and removed subscribers, overrode locks, edited via API. Everything worked like it should and I couldn't find any combination of lock state, policy state, and edit pathway that did anything suspicious.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20174
2019-02-19 10:52:34 -08:00
epriestley
8cf6c68c95 Fix a PhutilURI issue in workboards
Ref PHI1082.
2019-02-19 09:01:10 -08:00
epriestley
adab702403 Fix a PhutilURI issue in Multimeter
Fished this out of the `secure` error logs.
2019-02-17 17:39:34 -08:00
epriestley
3058cae4b8 Allow task statuses to specify that either "comments" or "edits" are "locked"
Summary:
Ref T13249. See PHI1059. This allows "locked" in `maniphest.statuses` to specify that either "comments" are locked (current behavior, advisory, overridable by users with edit permission, e.g. for calming discussion on a contentious issue or putting a guard rail on things); or "edits" are locked (hard lock, only task owner can edit things).

Roughly, "comments" is a soft/advisory lock. "edits" is a hard/strict lock. (I think both types of locks have reasonable use cases, which is why I'm not just making locks stronger across the board.)

When "edits" are locked:

  - The edit policy looks like "no one" to normal callers.
  - In one special case, we sneak the real value through a back channel using PolicyCodex in the specific narrow case that you're editing the object. Otherwise, the policy selector control incorrectly switches to "No One".
  - We also have to do a little more validation around applying a mixture of status + owner transactions that could leave the task uneditable.

For now, I'm allowing you to reassign a hard-locked task to someone else. If you get this wrong, we can end up in a state where no one can edit the task. If this is an issue, we could respond in various ways: prevent these edits; prevent assigning to disabled users; provide a `bin/task reassign`; uh maybe have a quorum convene?

Test Plan:
  - Defined "Soft Locked" and "Hard Locked" statues.
  - "Hard Locked" a task, hit errors (trying to unassign myself, trying to hard lock an unassigned task).
  - Saw nice new policy guidance icon in header.

{F6210362}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20165
2019-02-15 19:18:40 -08:00
epriestley
0b2d25778d Add basic, rough support for changing field behavior based on object subtype
Summary:
Ref T13248. This will probably need quite a bit of refinement, but we can reasonably allow subtype definitions to adjust custom field behavior.

Some places where we use fields are global, and always need to show all the fields. For example, on `/maniphest/`, where you can search across all tasks, you need to be able to search across all fields that are present on any task.

Likewise, if you "export" a bunch of tasks into a spreadsheet, we need to have columns for every field.

However, when you're clearly in the scope of a particular task (like viewing or editing `T123`), there's no reason we can't hide fields based on the task subtype.

To start with, allow subtypes to override "disabled" and "name" for custom fields.

Test Plan:
  - Defined several custom fields and several subtypes.
  - Disabled/renamed some fields for some subtypes.
  - Viewed/edited tasks of different subtypes, got desired field behavior.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13248

Differential Revision: https://secure.phabricator.com/D20161
2019-02-15 19:17:57 -08:00
epriestley
c5e16f9bd9 Give HarbormasterBuildUnitMessage a real Query class
Summary: Ref T13088. Prepares for putting test names in a separate table to release the 255-character limit.

Test Plan: Viewed revisions, buildables, builds, test lists, specific tests.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D20179
2019-02-15 19:16:47 -08:00
epriestley
8810cd2f4d Add a standalone view for the Maniphest task graph
Summary:
See PHI1073. Improve the UX here:

  - When there are a small number of connected tasks, no changes.
  - When there are too many total connected tasks, but not too many directly connected tasks, show hint text with a "View Standalone Graph" button to view more of the graph.
  - When there are too many directly connected tasks, show better hint text with a "View Standalone Graph" button.
  - Always show a "View Standalone Graph" option in the dropdown menu.
  - Add a standalone view which works the same way but has a limit of 2,000.
    - This view doesn't have "View Standalone Graph" links, since they'd just link back to the same page, but is basically the same otherwise.
  - Increase the main page task limit from 100 to 200.

Test Plan:
Mobile View:

{F6210326}

Way too much stuff:

{F6210327}

New persistent link to the standalone page:

{F6210328}

Kind of too much stuff:

{F6210329}

Standalone view:

{F6210330}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: 20after4

Differential Revision: https://secure.phabricator.com/D20164
2019-02-15 14:43:38 -08:00
epriestley
8f8e863613 When users follow an email login link but an install does not use passwords, try to get them to link an account
Summary:
Ref T13249. See PHI774. When users follow an email login link ("Forgot password?", "Send Welcome Email", "Send a login link to your email address.", `bin/auth recover`), we send them to a password reset flow if an install uses passwords.

If an install does not use passwords, we previously dumped them unceremoniously into the {nav Settings > External Accounts} UI with no real guidance about what they were supposed to do. Since D20094 we do a slightly better job here in some cases. Continue improving this workflow.

This adds a page like "Reset Password" for "Hey, You Should Probably Link An Account, Here's Some Options".

Overall, this stuff is still pretty rough in a couple of areas that I imagine addressing in the future:

  - When you finish linking, we still dump you back in Settings. At least we got you to link things. But better would be to return you here and say "great job, you're a pro".
  - This UI can become a weird pile of buttons in certain configs and generally looks a little unintentional. This problem is shared among all the "linkable" providers, and the non-login link flow is also weird.

So: step forward, but more work to be done.

Test Plan: {F6211115}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20170
2019-02-15 14:41:31 -08:00
epriestley
2ca316d652 When users confirm Duo MFA in the mobile app, live-update the UI
Summary: Ref T13249. Poll for Duo updates in the background so we can automatically update the UI when the user clicks the mobile phone app button.

Test Plan: Hit a Duo gate, clicked "Approve" in the mobile app, saw the UI update immediately.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20169
2019-02-15 14:38:15 -08:00
epriestley
454a762562 Queue search indexing tasks at a new PRIORITY_INDEX, not PRIORITY_IMPORT
Summary:
Depends on D20175. Ref T12425. Ref T13253. Currently, importing commits can stall search index rebuilds, since index rebuilds use an older priority from before T11677 and weren't really updated for D16585.

In general, we'd like to complete all indexing tasks before continuing repository imports. A possible exception is if you rebuild an entire index with `bin/search index --rebuild-the-world`, but we could queue those at a separate lower priority if issues arise.

Test Plan: Ran some search indexing through the queue.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13253, T12425

Differential Revision: https://secure.phabricator.com/D20177
2019-02-15 14:16:28 -08:00
epriestley
66060b294b Fix a URI construction in remarkup macro/meme rules
Summary: Ref T13250. Some of these parameters may be NULL, and `alter()` is no longer happy about that.

Test Plan: Ran daemon tasks that happened to render some memes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20176
2019-02-15 14:08:46 -08:00
epriestley
b09cf166a8 Clean up a couple more URI alter() calls
Summary:
See <https://discourse.phabricator-community.org/t/create-new-phriction-document-fails-with-unhandled-exception-invalidargumentexception/2406>.

These weren't obviously nullable from a cursory `grep`, but are sometimes nullable in practice.

Test Plan: Created, then saved a new Phriction document.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20184
2019-02-15 14:07:17 -08:00
epriestley
5892c78986 Replace all "setQueryParam()" calls with "remove/replaceQueryParam()"
Summary: Ref T13250. See D20149. Mostly: clarify semantics. Partly: remove magic "null" behavior.

Test Plan: Poked around, but mostly just inspection since these are pretty much one-for-one.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20154
2019-02-14 11:56:39 -08:00
epriestley
241f06c9ff Clean up final setQueryParams() callsites
Summary: Ref T13250. See D20149.

Test Plan: All trivial?

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20153
2019-02-14 11:54:56 -08:00
epriestley
4c12420162 Replace "URI->setQueryParams()" after initialization with a constructor argument
Summary: Ref T13250. See D20149. In a number of cases, we use `setQueryParams()` immediately after URI construction. To simplify this slightly, let the constructor take parameters, similar to `HTTPSFuture`.

Test Plan: See inlines.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20151
2019-02-14 11:46:37 -08:00
epriestley
eb73cb68ff Raise a setup warning when locked configuration has a configuration value stored in the database
Summary:
Ref T13249. See <https://discourse.phabricator-community.org/t/configuring-the-number-of-taskmaster-daemons/2394/>.

Today, when a configuration value is "locked", we prevent //writes// to the database. However, we still perform reads. When you upgrade, we generally don't want a bunch of your configuration to change by surprise.

Some day, I'd like to stop reading locked configuration from the database. This would defuse an escalation where an attacker finds a way to write to locked configuration despite safeguards, e.g. through SQL injection or policy bypass. Today, they could write to `cluster.mailers` or similar and substantially escalate access. A better behavior would be to ignore database values for `cluster.mailers` and other locked config, so that these impermissible writes have no effect.

Doing this today would break a lot of installs, but we can warn them about it now and then make the change at a later date.

Test Plan:
  - Forced a `phd.taskmasters` config value into the database.
  - Saw setup warning.
  - Used `bin/config delete --database phd.taskmasters` to clear the warning.
  - Reviewed documentation changes.
  - Reviewed `phd.taskmasters` documentation adjustment.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20159
2019-02-13 12:27:48 -08:00
epriestley
88d5233b77 Fix specifications of some "Visual Only" elements
Summary: See PHI823. These got "visual-only" but should acutally get "aural => false" to pick up "aria-hidden".

Test Plan: Viewed page source, saw both "visual-only" and "aria-hidden".

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20157
2019-02-13 12:26:28 -08:00
epriestley
9a9fa8bed2 Rate limit attempts to add payment methods in Phortune
Summary: Ref T13249. See D20132. Although we're probably a poor way to validate a big list of stolen cards in practice in production today (it's very hard to quickly generate a large number of small charges), putting rate limiting on "Add Payment Method" is generally reasonable, can't really hurt anything (no legitimate user will ever hit this limit), and might frustrate attackers in the future if it becomes easier to generate ad-hoc charges (for example, if we run a deal on support pacts and reduce their cost from $1,000 to $1).

Test Plan: Reduced limit to 4 / hour, tried to add a card several times, got rate limited.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20158
2019-02-13 12:25:56 -08:00
epriestley
a7bf279fd5 Don't try to publish build results to bare diffs
Summary:
See <https://discourse.phabricator-community.org/t/conduit-call-is-generating-phd-log-error-message/2380/>. If you run builds against a diff which is not attached to a revision (this is unusual) we still try to publish to the associated revision. This won't work since there is no associated revision.

Since bare diffs don't really have a timeline, just publish nowhere for now.

Test Plan:
  - Created a diff.
  - Did not attach it to a revision.
  - Created a build plan with "make http request + wait for response".
  - Manually ran the build plan against the bare diff.
  - Used `bin/phd debug task` to run the build and hit a "revision not attached" exception during publishing.
  - Applied patch.
  - Ran `bin/phd debug task`, got clean (no-op) publish.
  - Sent build a failure message with "harbormaster.sendmessage", got a failed build.

This isn't a real workflow, but shouldn't fail.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20156
2019-02-13 12:19:29 -08:00
epriestley
7d6d2c128a Make "bin/audit delete" synchronize commit audit status, and improve "bin/audit synchronize" documentation
Summary:
Depends on D20126. See PHI1056. Ref T13244.

  - `bin/audit delete` destroys audit requests, but does not update the overall audit state for associated commits. For example, if you destroy all audit requests for a commit, it does not move to "No Audit Required".
  - `bin/audit synchronize` does this synchronize step, but is poorly documented.

Make `bin/audit delete` synchronize affected commits.

Document `bin/audit synchronize` better.

There's some reasonable argument that `bin/audit synchronize` perhaps shouldn't exist, but it does let you recover from an accidentally (or intentionally) mangled database state. For now, let it live.

Test Plan:
  - Ran `bin/audit delete`, saw audits destroyed and affected commits synchornized.
  - Ran `bin/audit synchronize`, saw behavior unchanged.
  - Ran `bin/audit help`, got better help.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20127
2019-02-13 05:50:14 -08:00
epriestley
5a89da12e2 When users have no password on their account, guide them through the "reset password" flow in the guise of "set password"
Summary:
Depends on D20119. Fixes T9512. When you don't have a password on your account, the "Password" panel in Settings is non-obviously useless: you can't provide an old password, so you can't change your password.

The correct remedy is to "Forgot password?" and go through the password reset flow. However, we don't guide you to this and it isn't really self-evident.

Instead:

  - Guide users to the password reset flow.
  - Make it work when you're already logged in.
  - Skin it as a "set password" flow.

We're still requiring you to prove you own the email associated with your account. This is a pretty weak requirement, but maybe stops attackers who use the computer at the library after you do in some bizarre emergency and forget to log out? It would probably be fine to just let users "set password", this mostly just keeps us from having two different pieces of code responsible for setting passwords.

Test Plan:
  - Set password as a logged-in user.
  - Reset password on the normal flow as a logged-out user.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: revi

Maniphest Tasks: T9512

Differential Revision: https://secure.phabricator.com/D20120
2019-02-12 15:19:46 -08:00
epriestley
3f35c0068a Allow users to register with non-registration providers if they are invited to an instance
Summary:
Depends on D20117. Fixes T10071. When you're sent an email invitation, it's intended to allow you to register an account even if you otherwise could not (see D11737).

Some time between D11737 and today, this stopped working (or perhaps it never worked and I got things wrong in D11737). I think this actually ended up not mattering for us, given the way Phacility auth was ultimately built.

This feature generally seems reasonable, though, and probably //should// work. Make it work in the "password" and "oauth" cases, at least. This may //still// not work for LDAP, but testing that is nontrivial.

Test Plan:
  - Enabled only passwords, turned off registration, sent an invite, registered with a password.
  - Enabled only Google OAuth, turned off registration, sent an invite, registered with Google OAuth.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T10071

Differential Revision: https://secure.phabricator.com/D20118
2019-02-12 15:19:03 -08:00
epriestley
d22495a820 Make external link/refresh use provider IDs, switch external account MFA to one-shot
Summary:
Depends on D20113. Ref T6703. Continue moving toward a future where multiple copies of a given type of provider may exist.

Switch MFA from session-MFA at the start to one-shot MFA at the actual link action.

Add one-shot MFA to the unlink action. This theoretically prevents an attacker from unlinking an account while you're getting coffee, registering `alIce` which they control, adding a copy of your profile picture, and then trying to trick you into writing a private note with your personal secrets or something.

Test Plan: Linked and unlinked accounts. Refreshed account. Unlinked, then registered a new account. Unlinked, then relinked to my old account.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20117
2019-02-12 15:18:08 -08:00
epriestley
e5ee656fff Make external account unlinking use account IDs, not "providerType + providerDomain" nonsense
Summary: Depends on D20112. Ref T6703. When you go to unlink an account, unlink it by ID. Crazy!

Test Plan: Unlinked and relinked Google accounts.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20113
2019-02-12 15:16:24 -08:00
epriestley
541d794c13 Give ExternalAccount a providerConfigPHID, tying it to a particular provider
Summary:
Depends on D20111. Ref T6703. Currently, each ExternalAccount row is tied to a provider by `providerType` + `providerDomain`. This effectively prevents multiple providers of the same type, since, e.g., two LDAP providers may be on different ports on the same domain. The `domain` also isn't really a useful idea anyway because you can move which hostname an LDAP server is on, and LDAP actually uses the value `self` in all cases. Yeah, yikes.

Instead, just bind each account to a particular provider. Then we can have an LDAP "alice" on seven different servers on different ports on the same machine and they can all move around and we'll still have a consistent, cohesive view of the world.

(On its own, this creates some issues with the link/unlink/refresh flows. Those will be updated in followups, and doing this change in a way with no intermediate breaks would require fixing them to use IDs to reference providerType/providerDomain, then fixing this, then undoing the first fix most of the way.)

Test Plan: Ran migrations, sanity-checked database. See followup changes for more comprehensive testing.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20112
2019-02-12 14:48:14 -08:00
epriestley
55c18bc900 During first-time setup, create an administrator account with no authentication instead of weird, detached authentication
Summary:
Ref T6703. Currently, when you create an account on a new install, we prompt you to select a password.

You can't actually use that password unless you set up a password provider, and that password can't be associated with a provider since a password provider won't exist yet.

Instead, just don't ask for a password: create an account with a username and an email address only. Setup guidance points you toward Auth.

If you lose the session, you can send yourself an email link (if email works yet) or `bin/auth recover` it. This isn't really much different than the pre-change behavior, since you can't use the password you set anyway until you configure password auth.

This also makes fixing T9512 more important, which I'll do in a followup. I also plan to add slightly better guideposts toward Auth.

Test Plan: Hit first-time setup, created an account.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: revi

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20111
2019-02-12 14:47:47 -08:00
epriestley
378a43d09c Remove the highly suspect "Import from LDAP" workflow
Summary: Depends on D20109. Ref T6703. This flow was contributed in 2012 and I'm not sure it ever worked, or at least ever worked nondestructively. For now, get rid of it. We'll do importing and external sync properly at some point (T3980, T13190).

Test Plan: Grepped for `ldap/`, grepped for controller.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20110
2019-02-12 14:45:58 -08:00
epriestley
fcd85b6d7b Replace "getRequestURI()->setQueryParams(array())" with "getPath()"
Summary:
Ref T13250. A handful of callsites are doing `getRequestURI()` + `setQueryParams(array())` to get a bare request path.

They can just use `getPath()` instead.

Test Plan: See inlines.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20150
2019-02-12 14:43:33 -08:00
epriestley
00ffb190cc In Webhooks, label HTTP response codes as "HTTP Status Code", not "HTTP Error"
Summary: See PHI1068. We currently show "HTTP Error - 200", which is misleading. Instead, label these results as "HTTP Status Code".

Test Plan: {F6206016}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20148
2019-02-12 14:41:10 -08:00
epriestley
1fd69f788c Replace "getQueryParams()" callsites in Phabricator
Summary: See D20136. This method is sort of inherently bad because it is destructive for some inputs (`x=1&x=2`) and had "PHP-flavored" behavior for other inputs (`x[]=1&x[]=2`). Move to explicit `...AsMap` and `...AsPairList` methods.

Test Plan: Bit of an adventure, see inlines in a minute.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20141
2019-02-12 06:37:03 -08:00
epriestley
51cca22d07 Support EU domains for Mailgun API
Summary:
See <https://discourse.phabricator-community.org/t/mailgun-eu-zone-not-working/2332/4>.

Mailgun has a couple of API domains depending on where your account is based.

Test Plan:
  - Sent normal mail successfully with my non-EU account.
  - Waiting on user confirmation that this works in the EU.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20055
2019-02-11 15:06:04 -08:00
epriestley
1275326ea6 Use "phutil_string_cast()" in TypeaheadDatasource
Summary:
Depends on D20138. Ref T13250. This improves exception behavior and gives us a standard page with a stack trace instead of a text fatal with no stack trace.

Truly a great day for PHP.

(Eventually we may want to replace all `(string)` with `phutil_string_cast()`, but let's let it have some time in the wild first?)

Test Plan: Triggered the error, got a more useful exception behavior.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20140
2019-02-11 14:39:12 -08:00
epriestley
77247084bd Fix inverted check in audit triggers for "uninvolved owner"
Summary: See D20126. I was trying to be a little too cute here with the names and ended up confusing myself, then just tested the method behavior. :/

Test Plan: Persudaded by arguments in D20126.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20135
2019-02-11 14:08:04 -08:00
epriestley
711871f6bc Allow typeaheads to pass nonscalar data to datasources
Summary:
Ref T13250. Currently, datasources have a `setParameters(...)` method. This method accepts a dictionary and adds the key/value pairs to the raw HTTP request to the datasource endpoint.

Since D20049, this no longer works. Since D20116, it fatals explicitly.

In general, the datasource endpoint accepts other values (like `query`, `offset`, and `limit`), and even before these changes, using secret reserved keys in `setParameters(...)` would silently cause program misbehavior.

To deal with this, pass parameters as a JSON string named "parameters". This fixes the HTTP query issue (the more pressing issue affecting users today) and prevents the "shadowing reserved keys" issue (a theoretical issue which might affect users some day).

(I may revisit the `phutil_build_http_querystring()` behavior and possibly let it make this work again, but I think avoiding the duplicate key issue makes this change desirable even if the querystring behavior changes.)

Test Plan:
  - Used "Land Revision", selected branches.
  - Configured a custom Maniphest "users" field, used the search typeahead, selected users.
  - Manually browsed to `/typeahead/class/PhabricatorPeopleDatasource/?query=hi&parameters=xyz` to see the JSON decode exception.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20134
2019-02-11 10:53:39 -08:00
epriestley
a20f108034 When an edit overrides an object lock, note it in the transaction record
Summary:
Ref T13244. See PHI1059. When you lock a task, users who can edit the task can currently override the lock by using "Edit Task" if they confirm that they want to do this.

Mark these edits with an emblem, similar to the "MFA" and "Silent" emblems, so it's clear that they may have bent the rules.

Also, make the "MFA" and "Silent" emblems more easily visible.

Test Plan:
Edited a locked task, overrode the lock, got marked for it.

{F6195005}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: aeiser

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20131
2019-02-09 06:10:07 -08:00
epriestley
2b718d78bb Improve UI/UX when users try to add an invalid card with Stripe
Summary: Ref T13244. See PHI1052. Our error handling for Stripe errors isn't great right now. We can give users a bit more information, and a less jarring UI.

Test Plan:
Before (this is in developer mode, production doesn't get a stack trace):

{F6197394}

After:

{F6197397}

- Tried all the invalid test codes listed here: https://stripe.com/docs/testing#cards

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20132
2019-02-09 05:54:42 -08:00
epriestley
ae54af32c1 When an Owners package accepts a revision, count that as an "involved owner" for the purposes of audit
Summary:
Depends on D20129. Ref T13244. See PHI1058. When a revision has an "Accept" from a package, count the owners as "involved" in the change whether or not any actual human owners are actually accepting reviewers.

If a user owns "/" and uses "force accept" to cause "/src/javascript" to accept, or a user who legitimately owns "/src/javascript" accepts on behalf of the package but not on behalf of themselves (for whatever reason), it generally makes practical sense that these changes have owners involved in them (i.e., that's what a normal user would expect in both cases) and don't need to trigger audits under "no involvement" rules.

Test Plan: Used `bin/repository reparse --force --owners <commit>` to trigger audit logic. Saw a commit owned by `O1` with a revision counted as "involved" when `O1` had accepted the revision, even though no actual human owner had accepted it.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20130
2019-02-07 15:41:56 -08:00
epriestley
31a0ed92d0 Support a wider range of "Audit" rules for Owners packages
Summary:
Depends on D20124. Ref T13244. See PHI1055. Add a few more builtin audit behaviors to make Owners more flexible.

(At the upper end of flexibility you can trigger audits in a very granular way with Herald, but you tend to need to write one rule per Owners package, and providing a middle ground here has worked reasonably well for "review" rules so far.)

Test Plan:
  - Edited a package to select the various different audit rules.
  - Used `bin/repository reparse --force --owners <commit>` to trigger package audits under varied conditions.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20126
2019-02-07 15:39:39 -08:00
epriestley
8fab8d8a18 Prepare owners package audit rules to become more flexible
Summary:
Ref T13244. See PHI1055. (Earlier, see D20091 and PHI1047.) Previously, we expanded the Owners package autoreview rules from "Yes/No" to several "Review (Blocking) If Non-Owner Author Not Subscribed via Package" kinds of rules. The sky didn't fall and this feature didn't turn into "Herald-in-Owners", so I'm comfortable doing something similar to the "Audit" rules.

PHI1055 is a request for a way to configure slightly different audit behavior, and expanding the options seems like a good approach to satisfy the use case.

Prepare to add more options by moving everything into a class that defines all the behavior of different states, and converting the "0/1" boolean column to a text column.

Test Plan:
  - Created several packages, some with and some without auditing.
  - Inspected database for: package state; and associated transactions.
  - Ran the migrations.
  - Inspected database to confirm that state and transactions migrated correctly.
  - Reviewed transaction logs.
  - Created and edited packages and audit state.
  - Viewed the "Package List" element in Diffusion.
  - Pulled package information with `owners.search`, got sensible results.
  - Edited package audit status with `owners.edit`.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20124
2019-02-07 15:38:12 -08:00
epriestley
509fbb6c20 When building audit queries, prefilter possible "authorPHID" values
Summary:
Ref T13244. See PHI1057. Currently, if you're a member of a lot of projects/packages, you can end up with a very large `commit.authorPHID IN (...)` clause in part of the "Active Audits" query, since your `alice` token in "Responsible Users: alice" expands into every package and project you can audit on behalf of.

It's impossible for a commit to be authored by anything but a user, and evidence in PHI1057 suggests this giant `IN (...)` list can prevent MySQL from making effective utilization of the `<authorPHID, auditStatus, ...>` key on the table.

Prefilter the list of PHIDs to only PHIDs which can possibly author a commit.

(We'll also eventually need to convert the `authorPHIDs` into `identityPHIDs` anyway, for T12164, and this moves us slightly toward that.)

Test Plan: Loaded "Active Audits" before and after change, saw a more streamlined and sensible `authorPHID IN (...)` clause afterwards.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20129
2019-02-07 15:36:56 -08:00
epriestley
a4bab60ad0 Don't show "registration might be too open" warnings unless an auth provider actually allows registration
Summary:
Depends on D20118. Fixes T5351. We possibly raise some warnings about registration (approval queue, email domains), but they aren't relevant if no one can register.

Hide these warnings if no providers actually support registration.

Test Plan: Viewed the Auth provider list with registration providers and with no registration providers, saw more tailored guidance.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5351

Differential Revision: https://secure.phabricator.com/D20119
2019-02-07 15:32:42 -08:00
epriestley
7469075a83 Allow users to be approved from the profile "Manage" page, alongside other similar actions
Summary:
Depends on D20122. Fixes T8029. Adds an "Approve User" action to the "Manage" page.

Users are normally approved from the "Approval Queue", but if you click into a user's profile to check them out in more detail it kind of dead ends you right now. I've occasionally hit this myself, and think this workflow is generally reasonable enough to support upstream.

Test Plan: {F6193742}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T8029

Differential Revision: https://secure.phabricator.com/D20123
2019-02-07 15:04:23 -08:00
epriestley
949afb02fd On login forms, autofocus the "username" field
Summary: Depends on D20120. Fixes T8907. I thought this needed some Javascript nonsense but Safari, Firefox and Chrome all support an `autofocus` attribute.

Test Plan: Loaded login page with password auth enabled in Safari, Firefox, and Chrome; saw username field automatically gain focus.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T8907

Differential Revision: https://secure.phabricator.com/D20122
2019-02-07 15:03:43 -08:00
epriestley
8054f7d191 Convert a manual query against external accounts into a modern Query
Summary: Depends on D20108. Ref T6703. Update this outdated callsite to a more modern appraoch.

Test Plan: Destroyed a user with `bin/remove destroy`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20109
2019-02-07 15:02:00 -08:00
epriestley
f0364eef8a Remove weird integration between Legalpad and the ExternalAccount table
Summary:
Depends on D20107. Ref T6703. Legalpad currently inserts "email" records into the external account table, but they're never used for anything and nothing else references them.

They also aren't necessary for anything important to work, and the only effect they have is making the UI say "External Account" instead of "None" under the "Account" column. In particular, the signatures still record the actual email address.

Stop doing this, remove all the references, and destroy all the rows.

(Long ago, Maniphest may also have done this, but no longer does. Nuance/Gatekeeper use a more modern and more suitable "ExternalObject" thing that I initially started adapting here before realizing that Legalpad doesn't actually care about this data.)

Test Plan: Signed documents with an email address, saw signature reflected properly in UI. Grepped for other callsites.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20108
2019-02-07 15:00:00 -08:00
epriestley
9f5e6bee90 Make the default behavior of getApplicationTransactionCommentObject() "return null" instead of "throw"
Summary:
Depends on D20115. See <https://discourse.phabricator-community.org/t/transaction-search-endpoint-does-not-work-on-differential-diffs/2369/>.

Currently, `getApplicationTransactionCommentObject()` throws by default. Subclasses must override it to `return null` to indicate that they don't support comments.

This is silly, and leads to a bunch of code that does a `try / catch` around it, and at least some code (here, `transaction.search`) which doesn't `try / catch` and gets the wrong behavior as a result.

Just make it `return null` by default, meaning "no support for comments". Then remove the `try / catch` stuff and all the `return null` implementations.

Test Plan:
  - Grepped for `getApplicationTransactionCommentObject()`, fixed each callsite / definition.
  - Called `transaction.search` on a diff with transactions (i.e., not a sourced-from-commit diff).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jbrownEP

Differential Revision: https://secure.phabricator.com/D20121
2019-02-07 14:56:38 -08:00
epriestley
4fa5a2421e Add formal setup guidance warning that "feed.http-hooks" will be removed in a future version of Phabricator
Summary: Depends on D20114. This is on the way out, so make that explicitly clear.

Test Plan: Read setup issue and configuration option.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20115
2019-02-07 14:54:09 -08:00
epriestley
e25dc2dfe2 Revert "feed.http-hooks" HTTP request construction to use "http_build_query()" so nested "storyData" is handled correctly
Summary:
See <https://discourse.phabricator-community.org/t/storydata-is-blank-in-outgoing-requests-to-the-configured-feed-http-hooks/2366/>.

This behavior was changed by D20049. I think it's generally good that we not accept/encode nested values in a PHP-specific way, but retain `feed.http-hooks` compatibility for now.

Test Plan: {F6190681}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20114
2019-02-07 14:53:03 -08:00
epriestley
26081594e2 Fix two very, very minor correctness issues in Slowvote
Summary:
See <https://hackerone.com/reports/492525> and <https://hackerone.com/reports/489531>. I previously awarded a bounty for <https://hackerone.com/reports/434116> so Slowvote is getting "researched" a lot.

  - Prevent users from undoing their vote by submitting the form with nothing selected.
  - Prevent users from racing between the `delete()` and `save()` to vote for multiple options in a plurality poll.

Test Plan:
  - Clicked the vote button with nothing selected in plurality and approval polls, got an error now.
  - Added a `sleep(5)` between `delete()` and `save()`. Submitted different plurality votes in different windows. Before: votes raced, invalid end state. After: votes waited on the lock, arrived in a valid end state.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20125
2019-02-07 12:45:11 -08:00
Austin McKinley
f2236eb061 Autofocus form control for adding TOTP codes
Summary: Ref D20122. This is something I wanted in a bunch of places. Looks like at some point the most-annoying one (autofocus for entering TOTOP codes) already got fixed at some point.

Test Plan: Loaded the form, got autofocus as expected.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20128
2019-02-07 11:56:49 -08:00
epriestley
fc3b90e1d1 Allow users to unlink their last external account with a warning, instead of preventing the action
Summary:
Depends on D20105. Fixes T7732. T7732 describes a case where a user had their Google credentials swapped and had trouble regaining access to their account.

Since we now allow email login even if password auth is disabled, it's okay to let users unlink their final account, and it's even reasonable for users to unlink their final account if it is mis-linked.

Just give them a warning that what they're doing is a little sketchy, rather than preventing the workflow.

Test Plan: Unlinked my only login account, got a stern warning instead of a dead end.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T7732

Differential Revision: https://secure.phabricator.com/D20106
2019-02-06 17:07:41 -08:00
epriestley
d6f691cf5d In "External Accounts", replace hard-to-find tiny "link" icon with a nice button with text on it
Summary:
Ref T6703. Replaces the small "link" icon with a more obvious "Link External Account" button.

Moves us toward operating against `$config` objects instead of against `$provider` objects, which is more modern and will some day allow us to resolve T6703.

Test Plan: Viewed page, saw a more obvious button. Linked an external account.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T6703

Differential Revision: https://secure.phabricator.com/D20105
2019-02-06 16:07:16 -08:00
epriestley
55286d49e8 Clarify "metamta.default-address" instructions and lock the option
Summary: This option no longer needs to be configured if you configure inbound mail (and that's the easiest setup approach in a lot of cases), so stop telling users they have to set it up.

Test Plan: Read documentation and configuration help.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20104
2019-02-06 16:03:49 -08:00
epriestley
113a2773dd Remove one-time login from username change email
Summary:
Depends on D20100. Ref T7732. Ref T13244. This is a bit of an adventure.

Long ago, passwords were digested with usernames as part of the salt. This was a mistake: it meant that your password becomes invalid if your username is changed.

(I think very very long ago, some other hashing may also have used usernames -- perhaps session hashing or CSRF hashing?)

To work around this, the "username change" email included a one-time login link and some language about resetting your password.

This flaw was fixed when passwords were moved to shared infrastructure (they're now salted more cleanly on a per-digest basis), and since D18908 (about a year ago) we've transparently upgraded password digests on use.

Although it's still technically possible that a username change could invalidate your password, it requires:

  - You set the password on a version of Phabricator earlier than ~2018 Week 5 (about a year ago).
  - You haven't logged into a version of Phabricator newer than that using your password since then.
  - Your username is changed.

This probably affects more than zero users, but I suspect not //many// more than zero. These users can always use "Forgot password?" to recover account access.

Since the value of this is almost certainly very near zero now and declining over time, just get rid of it. Also move the actual mail out of `PhabricatorUser`, ala the similar recent change to welcome mail in D19989.

Test Plan: Changed a user's username, reviewed resulting mail with `bin/mail show-outbound`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244, T7732

Differential Revision: https://secure.phabricator.com/D20102
2019-02-05 16:01:53 -08:00
epriestley
9632c704c6 Always allow users to login via email link, even if an install does not use passwords
Summary:
Depends on D20099. Ref T13244. See PHI774. When password auth is enabled, we support a standard email-based account recovery mechanism with "Forgot password?".

When password auth is not enabled, we disable the self-serve version of this mechanism. You can still get email account login links via "Send Welcome Mail" or "bin/auth recover".

There's no real technical, product, or security reason not to let everyone do email login all the time. On the technical front, these links already work and are used in other contexts. On the product front, we just need to tweak a couple of strings.

On the security front, there's some argument that this mechanism provides more overall surface area for an attacker, but if we find that argument compelling we should probably provide a way to disable the self-serve pathway in all cases, rather than coupling it to which providers are enabled.

Also, inch toward having things iterate over configurations (saved database objects) instead of providers (abstract implementations) so we can some day live in a world where we support multiple configurations of the same provider type (T6703).

Test Plan:
  - With password auth enabled, reset password.
  - Without password auth enabled, did an email login recovery.

{F6184910}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20100
2019-02-05 16:00:55 -08:00
epriestley
99e5ef84fc Remove obsolete "PhabricatorAuthLoginHandler"
Summary: Depends on D20096. Reverts D14057. This was added for Phacility use cases in D14057 but never used. It is obsoleted by {nav Auth > Customize Messages} for non-Phacility use cases.

Test Plan: Grepped for removed symbol.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20099
2019-02-05 14:20:14 -08:00
epriestley
4fcb38a2a9 Move the Auth Provider edit flow toward a more modern layout
Summary:
Depends on D20095. Ref T13244. Currently, auth providers have a list item view and a single gigantic edit screen complete with a timeline, piles of instructions, supplemental information, etc.

As a step toward making this stuff easier to use and more modern, give them a separate view UI with normal actions, similar to basically every other type of object. Move the timeline and "Disable/Enable" to the view page (from the edit page and the list page, respectively).

Test Plan: Created, edited, and viewed auth providers.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20096
2019-02-05 14:19:26 -08:00
epriestley
8c8d56dc56 Replace "Add Auth Provider" radio buttons with a more modern "click to select" UI
Summary:
Depends on D20094. Ref T13244. Ref T6703. See PHI774. Currently, we use an older-style radio-button UI to choose an auth provider type (Google, Password, LDAP, etc).

Instead, use a more modern click-to-select UI.

Test Plan: {F6184343}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244, T6703

Differential Revision: https://secure.phabricator.com/D20095
2019-02-05 14:18:16 -08:00
epriestley
6f3bd13cf5 Begin adding more guidance to the "One-Time Login" flow
Summary:
Ref T13244. See PHI774. If an install does not use password auth, the "one-time login" flow (via "Welcome" email or "bin/auth recover") is pretty rough. Current behavior:

  - If an install uses passwords, the user is prompted to set a password.
  - If an install does not use passwords, you're dumped to `/settings/external/` to link an external account. This is pretty sketchy and this UI does not make it clear what users are expected to do (link an account) or why (so they can log in).

Instead, improve this flow:

  - Password reset flow is fine.
  - (Future Change) If there are external linkable accounts (like Google) and the user doesn't have any linked, I want to give users a flow like a password reset flow that says "link to an external account".
  - (This Change) If you're an administrator and there are no providers at all, go to "/auth/" so you can set something up.
  - (This Change) If we don't hit on any other rules, just go home?

This may be tweaked a bit as we go, but basically I want to refine the "/settings/external/" case into a more useful flow which gives users more of a chance of surviving it.

Test Plan: Logged in with passwords enabled (got password reset), with nothing enabled as an admin (got sent to Auth), and with something other than passwords enabled (got sent home).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20094
2019-02-05 14:17:25 -08:00
epriestley
03eb989fd8 Give Duo MFA a stronger hint if users continue without answering the challenge
Summary: See PHI912. Also, clean up some leftover copy/pastey code here.

Test Plan: {F6182333}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20088
2019-02-05 14:14:41 -08:00
epriestley
ab467d52f4 Improve feed rendering of user rename story
Summary:
Ref T13244. This story publishes to the feed (and I think that's reasonable and desirable) but doesn't render as nicely as it could.

Improve the rendering.

(See T9233 for some context on why we render stories like this one in this way.)

Test Plan: {F6184490}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20097
2019-02-05 14:13:44 -08:00
epriestley
5dc6502293 Make the mobile menu available in "/mail/"
Summary: Ref T13244. See <https://discourse.phabricator-community.org/t/left-hand-menu-not-responsive-on-mobile/2358>.

Test Plan: {F6184160}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20093
2019-02-05 14:10:57 -08:00
epriestley
ee24eb60b7 In Owners Packages, make the API representation of the "Auditing" field more consistent
Summary:
Ref T13244. See PHI1047. A while ago, the "Review" field changed from "yes/no" to 20 flavors of "Non-Owner Blocking Under A Full Moon". The sky didn't fall, so we'll probably do this to "Audit" eventually too.

The "owners.search" API method anticipates this and returns "none" or "audit" to describe package audit statuses, so it can begin returning "audit-non-owner-reviewers" or whatever in the future.

However, the "owners.edit" API method doesn't work the same way, and takes strings, and the strings have to be numbers. This is goofy and confusing and generally bad.

Make "owners.edit" take the same strings that "owners.search" emits. For now, continue accepting the old values of "0" and "1".

Test Plan:
  - Edited audit status of packages via API using "none", "audit", "0", "1" (worked), and invalid values like "quack" (helpful error).
  - Edited audit status of packages via web UI.
  - Used `owners.search` to retrieve package information.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13244

Differential Revision: https://secure.phabricator.com/D20091
2019-02-05 14:07:23 -08:00
Leopold Schabel
b8fe991ba2 Improve description text in the "Create Diff" form
Summary: The textarea is, in fact, above the description!

Test Plan: Description text changed.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D20092
2019-02-05 13:47:53 +00:00
epriestley
db1e123706 Fix an issue where Duo validation could incorrectly apply to other factor types
See <https://discourse.phabricator-community.org/t/configuring-mfa-provider-totp-fails-for-missing-duo-only-options/2355>.

Test Plan: Created a TOTP provider; created a Duo provider (with missing and supplied values).
2019-02-03 06:36:49 -08:00
epriestley
7c795ab757 Let omnipotent actors skip MFA transactions
Summary: This seems generally reasonable, but is also a narrow fix to "Phacility scripts try to move instances into 'up', but the daemons can't MFA".

Test Plan: Launched a new instance locally, no more "daemons can't MFA" error.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20081
2019-02-01 13:32:09 -08:00
epriestley
942d6d60d5 Don't load unnecessary handle data on "transaction.search"
Summary: Ref T13242. Currently, the transaction query loads handles by default (this is unusual). We don't need them, so turn them off.

Test Plan: No apparent behavioral change, will compare production profiles.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242

Differential Revision: https://secure.phabricator.com/D20068
2019-02-01 08:14:26 -08:00
epriestley
7dd55a728f Make repository daemons periodically check for out-of-sync repositories
Summary:
See PHI1015. If you add new repository nodes to a cluster, we may not actually sync some repositories for up to 6 hours (if they've had no commits for 30 days).

Add an explicit check for out-of-sync repositories to trigger background sync.

Test Plan:
  - Ran `bin/phd debug pullocal`.
  - Fiddled with the `repository_workingcopy` version table to put the local node in and out of sync with the cluster.
  - Saw appropriate responses in the daemon (sync; wait if the last sync trigger was too recent).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20078
2019-01-31 22:12:39 -08:00
epriestley
f3e154eb02 Allow "inactive" repositories to be read over SSH for cluster sync
Summary:
Fixes T13192. See PHI1015. When you deactivate a repository, we currently stop serving it.

This creates a problem for intracluster sync, since new nodes can't sync it. If nothing else, this means that if you "ship of theseus" your cluster and turn nodes over one at a time, you will eventually lose the entire repository. Since that's clearly a bad outcome, support sync.

Test Plan:
Testing this requires a "real" cluster, so I mostly used `secure`.

I deactivated rGITTEST and ran this on `secure002`:

```
./bin/repository thaw --demote secure002.phacility.net --force GITTEST && ./bin/repository update GITTEST
```

Before the patch, this failed:

```
[2019-01-31 19:40:37] EXCEPTION: (CommandException) Command failed with error #128!
COMMAND
git fetch --prune -- 'ssh://172.30.0.64:22/diffusion/GITTEST/' '+refs/*:refs/*'

STDOUT
(empty)

STDERR
Warning: Permanently added '172.30.0.64' (RSA) to the list of known hosts.
phabricator-ssh-exec: This repository ("rGITTEST") is not available over SSH.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
```

After applying (a similar patch to) this patch to `secure001`, the sync worked.

I'll repeat this test with the actual patch once this deploys to `secure`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13192

Differential Revision: https://secure.phabricator.com/D20077
2019-01-31 22:12:13 -08:00
epriestley
bc61d09f55 Allow parent and child revisions to be modified via Conduit
Summary: See PHI1048. We have similar transactions elsewhere already (particularly, on tasks) and these edges don't have any special properties (like "Closes Txx As Wontfix" edges do) so this doesn't create any sort of peril.

Test Plan: {F6170556}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20075
2019-01-31 22:11:17 -08:00
epriestley
b33333ab8e Fix method name typo in new modular transaction text/html mail methods
See PHI1039. See D20057.
2019-01-31 08:52:27 -08:00
epriestley
138c07f32c Allow modular transactions to override transaction title and body text in mail
Summary:
Ref T12921. I'm moving Instances to modular transactions, and we have an "Alert" transaction type used to send notifications ("Your instance is going to be suspended for nonpayment.").

Currently, there's no way to specifically customize mail rendering under modular transactions. Add crude support for it.

Note that (per comment) this is fairly aspirational right now, since we actually always render everything as text (see T12921). But this API should (?) mostly survive intact when I fix this properly, and allows Instances to move to modular transactions so I can fix some more pressing issues in the meantime.

Test Plan: See next diff for Instances.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12921

Differential Revision: https://secure.phabricator.com/D20057
2019-01-30 19:48:30 -08:00
epriestley
87b0ef8839 Remove "iconv" PHP extension dependency
Summary: Depends on D20069. Ref T13232. This is a very, very weak dependency and we can reasonably polyfill it.

Test Plan: Grepped for `iconv` in libphutil, arcanist, and Phabricator.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13232

Differential Revision: https://secure.phabricator.com/D20070
2019-01-30 19:46:58 -08:00
epriestley
48a3760814 Correct a bug where milestone "spacePHID" columns could become desynchronized
Summary:
Depends on D20041. See PHI1046. If you do this:

- Create a parent project called "Crab" in Space 1.
- Create a milestone called "Left Claw".
- Shift "Crab" to Space 2.
- Create a milestone called "Right Claw".

...you currently end up with "Left Claw" in the wrong `spacePHID` in the database. At the application level it's in the correct space, but when we `WHERE ... AND spacePHID IN (...)` we can incorrectly filter it out.

Test Plan:
  - Did the above setup.
  - Saved "Crab", saw the space fix itself.
  - Put things back in the broken state.
  - Ran the migration script, saw things fix themselves again.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: aeiser, PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20063
2019-01-30 19:41:49 -08:00
epriestley
e9b2d667ee Improve handling of "Deny" responses from Duo
Summary:
Ref T13231. See <https://discourse.phabricator-community.org/t/duo-integration-crashes-if-user-is-not-enrolled-and-enrollment-is-disabled/2340/5>

(There's an actual bug here, although I'm not sure exactly what's going on on the Duo side in the report.)

Test Plan:
To reproduce this, I was only able to actually "Deny" my account explicitly in Duo.

  - With "Deny", tried to add a factor. Got a nice helpful error message.
  - Undenied, added a factor, re-denied, tried to pass an MFA gate. Got another nice helpful error message.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13231

Differential Revision: https://secure.phabricator.com/D20065
2019-01-30 19:33:15 -08:00
epriestley
93b512b63c Update a factor query in TransactionEditor for providers
Summary: This query didn't get updated and could let you through an explicit "Sign with MFA" action if you have only disabled factors on your account.

Test Plan:
  - Disabled all factors.
  - Used explicit "Sign With MFA".
    - Before: Went through.
    - After: Sensible error.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20072
2019-01-30 19:27:37 -08:00
epriestley
612e9c6e09 Hide "Signed with MFA" stories from feed
Summary:
See <https://discourse.phabricator-community.org/t/sign-with-mfa-action-on-differential-revisions-creates-strange-feed-entries/2346>.

If you "Sign with MFA" and only add a comment, we can produce a weird notification and feed story. Hide these stories from feed since they're broadly not interesting.

Test Plan:
  - Used "sign with MFA" and only posted a comment.
  - Observed feed.
    - Before: Janky "untitled story".
    - After: Sensible "added a comment" story.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20071
2019-01-30 19:26:33 -08:00
epriestley
6bcfc212eb Allow diff change detection to complete for Mercurial changes which remove a binary file
Summary:
See PHI1044. When you push a commit, we try to detect if the diff is the same as the last diff in Differential. This is a soft/heuristic check and only used to provide a hint in email ("CHANGED SINCE LAST DIFF").

For some changes, we can end up on this pathway with a binary changeset for a deleted file in the commit, and try to fetch the file data. This will fail since the file has been deleted. I'm not //entirely// sure why this hasn't cropped up before and didn't try to truly build an end-to-end test case, but it's a valid state that we shouldn't fatal in.

When we get here in this state, just detect a change. This is safe, even if it isn't always correct.

(This will be revisited in the future so I'm favoring fixing the broken behavior for now.)

Test Plan: This required some rigging, but I modified `bin/differential extract` to run the `isDiffChangedBeforeCommit()` code, then rigged a commit/diff to hit this case and reproduced the reported error. After the change, the comparison worked cleanly.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20056
2019-01-30 19:17:03 -08:00
epriestley
1767b80654 Replace manual query string construction with "phutil_build_http_querystring()"
Summary: Now that we have a nice function for this, use it to simplify some code.

Test Plan: Ran through the Duo enroll workflow to make sure signing still works.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20053
2019-01-30 19:14:57 -08:00
epriestley
a24e6deb57 Read "$_POST" before hooking the profiler, and remove "aphront.default-application-configuration-class"
Summary:
Ref T4369. Ref T12297. Ref T13242. Ref PHI1010. I want to take a quick look at `transaction.search` and see if there's anything quick and obvious we can do to improve performance.

On `secure`, the `__profile__` flag does not survive POST like it's supposed to: when you profile a page and then submit a form on the page, the result is supposed to be profiled. The intent is to make it easier to profile Conduit calls.

I believe this is because we're hooking the profiler, then rebuilding POST data a little later -- so `$_POST['__profile__']` isn't set yet when the profiler checks.

Move the POST rebuild a little earlier to fix this.

Also, remove the very ancient "aphront.default-application-configuration-class". I believe this was only used by Facebook to do CIDR checks against corpnet or something like that. It is poorly named and long-obsolete now, and `AphrontSite` does everything we might reasonably have wanted it to do.

Test Plan: Poked around locally without any issues. Will check if this fixes the issue on `secure`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242, T12297, T4369

Differential Revision: https://secure.phabricator.com/D20046
2019-01-30 06:22:41 -08:00
epriestley
70b474e550 Allow MFA enrollment guidance to be customized
Summary: Depends on D20039. Ref T13242. If installs want users to install a specific application, reference particular help, etc., let them customize the MFA enrollment message so they can make it say "if you have issues, see this walkthrough on the corporate wiki" or whatever.

Test Plan:
{F6164340}

{F6164341}

{F6164342}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242

Differential Revision: https://secure.phabricator.com/D20043
2019-01-30 06:21:58 -08:00
epriestley
2374c92544 Add a warning about MFA requirements to edit forms
Summary:
Depends on D20044. Ref T13242. Similar to D20044, add reminder text to edit forms.

It would be nice to "workflow" these so the MFA flow happens inline, but Maniphest's inline edit behavior currently conflicts with this. Set it aside for now since the next workboards iteration (triggers) is probably a good opportunity to revisit it.

Test Plan: {F6164496}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242

Differential Revision: https://secure.phabricator.com/D20045
2019-01-30 06:21:25 -08:00
epriestley
13c5b427d6 Warn users about MFA requirements when interacting with "MFA Required" objects via the comment form
Summary:
Ref T13242. Warn user that they'll need to MFA (so they can go dig their phone out of their bag first or whatever, or don't type a giant comment on mobile if their U2F key is back at the office) on the comment form.

Also, when they'll need MFA and won't be able to provide it (no MFA on account), stop them from typing up a big comment that they can't actually submit: point them at MFA setup first.

Test Plan:
{F6164448}

{F6164449}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242

Differential Revision: https://secure.phabricator.com/D20044
2019-01-30 06:20:52 -08:00
epriestley
f7e8fa0764 Provide an Editor extension point for transaction validation
Summary:
Depends on D20040. Ref T13242. See PHI1039. See PHI873. Two reasonable cases have arisen recently where extending validation rules would help solve problems.

We can do this in a pretty straightforward way with a standard extension pattern.

Test Plan:
Used this extension to keep ducks away from projects:

```lang=php
<?php

final class NoDucksEditorExtension
  extends PhabricatorEditorExtension {

  const EXTENSIONKEY = 'no.ducks';

  public function getExtensionName() {
    return pht('No Ducks!');
  }

  public function supportsObject(
    PhabricatorApplicationTransactionEditor $editor,
    PhabricatorApplicationTransactionInterface $object) {
    return ($object instanceof PhabricatorProject);
  }

  public function validateTransactions($object, array $xactions) {
    $errors = array();

    $name_type = PhabricatorProjectNameTransaction::TRANSACTIONTYPE;

    $old_value = $object->getName();
    foreach ($xactions as $xaction) {
      if ($xaction->getTransactionType() !== $name_type) {
        continue;
      }

      $new_value = $xaction->getNewValue();
      if ($old_value === $new_value) {
        continue;
      }

      if (preg_match('/duck/i', $new_value)) {
        $errors[] = $this->newInvalidTransactionError(
          $xaction,
          pht('Project names must not contain the substring "duck".'));
      }
    }

    return $errors;
  }

}
```

{F6162585}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242

Differential Revision: https://secure.phabricator.com/D20041
2019-01-30 06:18:41 -08:00
epriestley
c9760e8d64 Support subtypes in Projects
Summary:
Ref T13242. See PHI1039. Maniphest subtypes generally seem to be working well. I designed them as a general capability that might be extended to other `EditEngine` objects later, and PHI1039 describes a situation where extending subtypes to projects would give us some reasonable tools.

(Some installs also already use icons/colors as a sort of lightweight version of subtypes, so I believe this is generally useful capability.)

Some of this is a little bit copy-pasted and could probably be shared, but I'd like to wait a bit longer before merging it. For example, both configs have exactly the same structure right now, but Projects should possibly have some different flags (for example: to disable creating subprojects / milestones).

This implementation is pretty basic for now: notably, subprojects/milestones don't get the nice "choose from among subtype forms" treatment that tasks do. If this ends up being part of a solution to PHI1039, I'd plan to fill that in later on.

Test Plan: Defined multiple subtypes, created subtype forms, created projects with appropriate subtypes. Filtered them by subtype. Saw subtype information on list/detail views.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13242

Differential Revision: https://secure.phabricator.com/D20040
2019-01-30 06:17:55 -08:00
epriestley
d254c1f8b1 Use "null", not "-1", as a local "no version" marker when performing intracluster repository sync
Summary:
Ref T13242. See <https://discourse.phabricator-community.org/t/out-of-range-value-for-column-deviceversion/2218>.

The synchronization log column is `uint32?` and `-1` doesn't go into that column.

Since we're only using `-1` for convenience to cheat through `$max_version > $this_version` checks, use `null` instead and make the checks more explicit.

Test Plan: Reproducing this is a bit tricky and I cheated fairly heavily to force the code down this pathway without actually building a multi-device cluster, but I did reproduce the original exception, apply the patch, and observe that it fixed things.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242

Differential Revision: https://secure.phabricator.com/D20047
2019-01-28 18:50:01 -08:00
epriestley
9fd8343704 Bring Duo MFA upstream
Summary: Depends on D20038. Ref T13231. Although I planned to keep this out of the upstream (see T13229) it ended up having enough pieces that I imagine it may need more fixes/updates than we can reasonably manage by copy/pasting stuff around. Until T5055, we don't really have good tools for managing this. Make my life easier by just upstreaming this.

Test Plan: See T13231 for a bunch of workflow discussion.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13231

Differential Revision: https://secure.phabricator.com/D20039
2019-01-28 18:26:45 -08:00
epriestley
d8d4efe89e Require MFA to edit MFA providers
Summary: Depends on D20037. Ref T13222. Ref T7667. Although administrators can now disable MFA from the web UI, at least require that they survive MFA gates to do so. T7667 (`bin/auth lock`) should provide a sturdier approach here in the long term.

Test Plan: Created and edited MFA providers, was prompted for MFA.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222, T7667

Differential Revision: https://secure.phabricator.com/D20038
2019-01-28 09:44:39 -08:00
epriestley
44a0b3e83d Replace "Show Secret" in Passphrase with one-shot MFA
Summary: Depends on D20036. Ref T13222. Now that we support one-shot MFA, swap this from session MFA to one-shot MFA.

Test Plan: Revealed a credential, was no longer left in high-security mode.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20037
2019-01-28 09:44:08 -08:00
epriestley
d24e66724d Convert "Rename User" from session MFA to one-shot MFA
Summary:
Depends on D20035. Ref T13222.

  - Allow individual transactions to request one-shot MFA if available.
  - Make "change username" request MFA.

Test Plan:
  - Renamed a user, got prompted for MFA, provided it.
  - Saw that I no longer remain in high-security after performing the edit.
  - Grepped for other uses of `PhabricatorUserUsernameTransaction`, found none.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20036
2019-01-28 09:41:10 -08:00
epriestley
29b4fad941 Get rid of "throwResult()" for control flow in MFA factors
Summary: Depends on D20034. Ref T13222. This is just cleanup -- I thought we'd have like two of these, but we ended up having a whole lot in Duo and a decent number in SMS. Just let factors return a result explicitly if they can make a decision early. I think using `instanceof` for control flow is a lesser evil than using `catch`, on the balance.

Test Plan: `grep`, went through enroll/gate flows on SMS and Duo.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20035
2019-01-28 09:40:28 -08:00
epriestley
bce44385e1 Add more factor details to the Settings factor list
Summary:
Depends on D20033. Ref T13222. Flesh this UI out a bit, and provide bit-strength information for TOTP.

Also, stop users from adding multiple SMS factors since this is pointless (they all always text your primary contact number).

Test Plan:
{F6156245}

{F6156246}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20034
2019-01-28 09:40:00 -08:00
epriestley
8e5d9c6f0e Allow MFA providers to be deprecated or disabled
Summary: Ref T13222. Providers can now be deprecated (existing factors still work, but users can't add new factors for the provider) or disabled (factors stop working, also can't add new ones).

Test Plan:
  - Enabled, deprecated, and disabled some providers.
  - Viewed provider detail, provider list.
  - Viewed MFA settings list.
  - Verified that I'm prompted for enabled + deprecated only at gates.
  - Tried to disable final provider, got an error.
  - Hit the MFA setup gate by enabling "Require MFA" with no providers, got a more useful message.
  - Immediately forced a user to the "MFA Setup Gate" by disabling their only active provider with another provider enabled ("We no longer support TOTP, you HAVE to finish Duo enrollment to continue starting Monday.").

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20031
2019-01-28 09:29:27 -08:00
epriestley
c9ff6ce390 Add CSRF to SMS challenges, and pave the way for more MFA types (including Duo)
Summary:
Depends on D20026. Ref T13222. Ref T13231. The primary change here is that we'll no longer send you an SMS if you hit an MFA gate without CSRF tokens.

Then there's a lot of support for genralizing into Duo (and other push factors, potentially), I'll annotate things inline.

Test Plan: Implemented Duo, elsewhere.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13231, T13222

Differential Revision: https://secure.phabricator.com/D20028
2019-01-24 15:10:57 -08:00
epriestley
069160404f Add a Duo API future
Summary: Depends on D20025. Ref T13231. Although I'm not currently planning to actually upstream a Duo MFA provider, it's probably easiest to put most of the support pieces in the upstream until T5055.

Test Plan: Used a test script to make some (mostly trivial) API calls and got valid results back, so I think the parameter signing is correct.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13231

Differential Revision: https://secure.phabricator.com/D20026
2019-01-24 15:10:17 -08:00
epriestley
98c4cdc5be Make the "PHP 7" setup warning more explicit about what it means
Summary: See <https://discourse.phabricator-community.org/t/minimum-php-versions-supported-by-latest-stable-phabricator/2314/3>.

Test Plan: o~o

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20027
2019-01-24 15:09:00 -08:00
epriestley
5cfcef7f53 Fix bad "$this" references in "Must Encrypt" mail after MailEngine changes
Summary: See PHI1038. I missed these when pulling the code out.

Test Plan: Sent "Must encrypt" mail, verified it made it through the queue in one piece.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20029
2019-01-24 15:07:42 -08:00
Austin McKinley
e72684a4ba Add infrastructure for sending SMS via AWS SNS
Summary:
Ref T920. Ref T13235. This adds a `Future`, similar to `TwilioFuture`, for interacting with Amazon's SNS service.

Also updates the documentation.

Also makes the code consistent with the documentation by accepting a `media` argument.

Test Plan: Clicked the "send test message" button from the Settings UI.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T13235, T920

Differential Revision: https://secure.phabricator.com/D19982
2019-01-23 16:29:50 -08:00