1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-28 08:20:57 +01:00
Commit graph

15732 commits

Author SHA1 Message Date
epriestley
f3340c6335 Allow different MFA factor types (SMS, TOTP, Duo, ...) to share "sync" tokens when enrolling new factors
Summary:
Depends on D20019. Ref T13222. Currently, TOTP uses a temporary token to make sure you've set up the app on your phone properly and that you're providing an answer to a secret which we generated (not an attacker-generated secret).

However, most factor types need some kind of sync token. SMS needs to send you a code; Duo needs to store a transaction ID. Turn this "TOTP" token into an "MFA Sync" token and lift the implementation up to the base class.

Also, slightly simplify some of the HTTP form gymnastics.

Test Plan:
  - Hit the TOTP enroll screen.
  - Reloaded it, got new secrets.
  - Reloaded it more than 10 times, got told to stop generating new challenges.
  - Answered a challenge properly, got a new TOTP factor.
  - Grepped for removed class name.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20020
2019-01-23 14:13:50 -08:00
epriestley
7c1d1c13f4 Add a rate limit for enroll attempts when adding new MFA configurations
Summary:
Depends on D20018. Ref T13222. When you add a new MFA configuration, you can technically (?) guess your way through it with brute force. It's not clear why this would ever really be useful (if an attacker can get here and wants to add TOTP, they can just add TOTP!) but it's probably bad, so don't let users do it.

This limit is fairly generous because I don't think this actually part of any real attack, at least today with factors we're considering.

Test Plan:
  - Added TOTP, guessed wrong a ton of times, got rate limited.
  - Added TOTP, guessed right, got a TOTP factor configuration added to my account.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20019
2019-01-23 14:12:19 -08:00
epriestley
e91bc26da6 Don't rate limit users clicking "Wait Patiently" at an MFA gate even if they typed some text earlier
Summary:
Depends on D20017. Ref T13222. Currently, if you:

  - type some text at a TOTP gate;
  - wait ~60 seconds for the challenge to expire;
  - submit the form into a "Wait patiently" message; and
  - mash that wait button over and over again very patiently

...you still rack up rate limiting points, because the hidden text from your original request is preserved and triggers the "is the user responding to a challenge" test. Only perform this test if we haven't already decided that we're going to make them wait.

Test Plan:
  - Did the above; before patch: rate limited; after patch: not rate limited.
  - Intentionally typed a bunch of bad answers which were actually evaluated: rate limited properly.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20018
2019-01-23 14:11:24 -08:00
epriestley
bb20c13651 Allow MFA factors to provide more guidance text on create workflows
Summary:
Depends on D20016. Ref T920. This does nothing interesting on its own since the TOTP provider has no guidance/warnings, but landing it separately helps to simplify an upcoming SMS diff.

SMS will have these guidance messages:

  - "Administrator: you haven't configured any mailer which can send SMS, like Twilio."
  - "Administrator: SMS is weak."
  - "User: you haven't configured a contact number."

Test Plan: {F6151283} {F6151284}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D20017
2019-01-23 14:10:16 -08:00
epriestley
ee7b03bdf7 Correct an issue where the default "Settings" screen could show the wrong settings
Summary:
Depends on D20013. Recently, I renamed the "Account" panel to "Language".

When you land on "Settings" and the first panel is an "EditEngine" panel ("Account/Langauge", "Date and Time", and "Conpherence" are all "EditEngine" panels), the engine shows the controls for the first panel.

However, the "first panel" according to EditEngine and the "first panel" in the menu are currently different: the menu groups panels into topics.

When I renamed "Account" to "Language", it went from conicidentally being the first panel in both lists to being the second panel in the grouped menu list and the, uh, like 12th panel in the ungrouped raw list.

This made landing on "Settings" show you the right chrome, but show you a different panel's controls ("Conpherence", now alphabetically first).

Instead, use the same order in both places.

(This was also a pre-existing bug if you use a language which translates the panel names such that "Account" is not alphabetically first.)

Test Plan: Visited "Settings", saw "Date & Time" form controls instead of "Conpherence" form controls on the default screen with "Date & Time" selected in the menu.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20016
2019-01-23 14:09:03 -08:00
epriestley
f69fbf5ea6 Make the "Test" adapter support both SMS and email
Summary:
Depends on D20012. Ref T920. If you have a test adapter configured, it should swallow messages and prevent them from ever hitting a lower-priority adapter.

Make the test adapter support SMS so this actually happens.

Test Plan: Ran `bin/mail send-test --type sms ...` with a test adapter (first) and a Twilio adapter (second). Got SMS swallowed by test adapter instead of live SMS messages.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D20013
2019-01-23 14:07:07 -08:00
epriestley
af71c51f0a Give "MetaMTAMail" a "message type" and support SMS
Summary:
Depends on D20011. Ref T920. This change lets a "MetaMTAMail" storage object represent various different types of messages, and  makes "all" the `bin/mail` stuff "totally work" with messages of non-email types.

In practice, a lot of the related tooling needs some polish/refinement, but the basics work.

Test Plan: Used `echo beep boop | bin/mail send-test --to epriestley --type sms` to send myself SMS.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D20012
2019-01-23 14:05:46 -08:00
epriestley
596435b35e Support designating a contact number as "primary"
Summary:
Depends on D20010. Ref T920. Allow users to designate which contact number is "primary": the number we'll actually send stuff to.

Since this interacts in weird ways with "disable", just do a "when any number is touched, put all of the user's rows into the right state" sort of thing.

Test Plan:
  - Added numbers, made numbers primary, disabled a primary number, un-disabled a number with no primaries. Got sensible behavior in all cases.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D20011
2019-01-23 14:03:08 -08:00
epriestley
12203762b7 Allow contact numbers to be enabled and disabled
Summary: Depends on D20008. Ref T920. Continue fleshing out contact number behaviors.

Test Plan:
  - Enabled and disabled a contact number.
  - Saw list, detail views reflect change.
  - Added number X, disabled it, added it again (allowed), enabled the disabled one ("already in use" exception).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D20010
2019-01-23 13:59:55 -08:00
epriestley
c4244aa177 Allow users to access some settings at the "Add MFA" account setup roadblock
Summary:
Depends on D20006. Ref T13222. Currently, the "MFA Is Required" gate doesn't let you do anything else, but you'll need to be able to access "Contact Numbers" if an install provides SMS MFA.

Tweak this UI to give users limited access to settings, so they can set up contact numbers and change their language.

(This is a little bit fiddly, and I'm doing it early on partly so it can get more testing as these changes move forward.)

Test Plan: {F6146136}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D20008
2019-01-23 13:43:28 -08:00
epriestley
d6d93dd658 Add icons to Settings
Summary: Depends on D20005. I love icons.

Test Plan: {F6145996}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20006
2019-01-23 13:41:41 -08:00
epriestley
f713fa1fd7 Expand "Settings" UI to full-width
Summary:
Depends on D19988. See D19826 for the last UI expansion. I don't have an especially strong product rationale for un-fixed-width'ing Settings since it doesn't suffer from the "mystery meat actions" issues that other fixed-width UIs do, but I like the full-width UI better and the other other fixed-width UIs all (?) have some actual rationale (e.g., large tables, multiple actions on subpanels), so "consistency" is an argument here.

Also rename "account" to "language" since both settings are language-related.

This moves away from the direction in D18436.

Test Plan:
Clicked each Settings panel, saw sensible rendering at full-width.

{F6145944}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20005
2019-01-23 13:40:34 -08:00
epriestley
f0c6ee4823 Add "Contact Numbers" so we can send users SMS mesages
Summary:
Ref T920. To send you SMS messages, we need to know your phone number.

This adds bare-bone basics (transactions, storage, editor, etc).

From here:

**Disabling Numbers**: I'll let you disable numbers in an upcoming diff.

**Primary Number**: I think I'm just going to let you pick a number as "primary", similar to how email works. We could imagine a world where you have one "MFA" number and one "notifications" number, but this seems unlikely-ish?

**Publishing Numbers (Profile / API)**: At some point, we could let you say that a number is public / "show on my profile" and provide API access / directory features. Not planning to touch this for now.

**Non-Phone Numbers**: Eventually this could be a list of other similar contact mechanisms (APNS/GCM devices, Whatsapp numbers, ICQ number, twitter handle so MFA can slide into your DM's?). Not planning to touch this for now, but the path should be straightforward when we get there. This is why it's called "Contact Number", not "Phone Number".

**MFA-Required + SMS**: Right now, if the only MFA provider is SMS and MFA is required on the install, you can't actually get into Settings to add a contact number to configure SMS. I'll look at the best way to deal with this in an upcoming diff -- likely, giving you partial access to more of Setings before you get thorugh the MFA gate. Conceptually, it seems reasonable to let you adjust some other settings, like "Language" and "Accessibility", before you set up MFA, so if the "you need to add MFA" portal was more like a partial Settings screen, maybe that's pretty reasonable.

**Verifying Numbers**: We'll probably need to tackle this eventually, but I'm not planning to worry about it for now.

Test Plan: {F6137174}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: avivey, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D19988
2019-01-23 13:39:56 -08:00
epriestley
aa48373889 Update bin/auth MFA commands for the new "MFA Provider" indirection layer
Summary:
Ref T13222. This updates the CLI tools and documentation for the changes in D19975.

The flags `--type` and `--all-types` retain their current meaning. In most cases, `bin/auth strip --type totp` is sufficient and you don't need to bother looking up the relevant provider PHID. The existing `bin/auth list-factors` is also unchanged.

The new `--provider` flag allows you to select configs from a particular provider in a more granular way. The new `bin/auth list-mfa-providers` provides an easy way to get PHIDs.

(In the Phacility cluster, the "Strip MFA" action just reaches into the database and deletes rows manually, so this isn't terribly important. I verified that the code should still work properly.)

Test Plan:
  - Ran `bin/auth list-mfa-providers`.
  - Stripped by user / type / provider.
  - Grepped for `list-factors` and `auth strip`.
  - Hit all (?) of the various possible error cases.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D19976
2019-01-23 13:38:44 -08:00
epriestley
0fcff78253 Convert user MFA factors to point at configurable "MFA Providers", not raw "MFA Factors"
Summary:
Ref T13222. Users configure "Factor Configs", which say "I have an entry on my phone for TOTP secret key XYZ".

Currently, these point at raw implementations -- always "TOTP" in practice.

To support configuring available MFA types (like "no MFA") and adding MFA types that need some options set (like "Duo", which needs API keys), bind "Factor Configs" to a "Factor Provider" instead.

In the future, several "Factors" will be available (TOTP, SMS, Duo, Postal Mail, ...). Administrators configure zero or more "MFA Providers" they want to use (e.g., "Duo" + here's my API key). Then users can add configs for these providers (e.g., "here's my Duo account").

Upshot:

  - Factor: a PHP subclass, implements the technical details of a type of MFA factor (TOTP, SMS, Duo, etc).
  - FactorProvider: a storage object, owned by administrators, configuration of a Factor that says "this should be available on this install", plus provides API keys, a human-readable name, etc.
  - FactorConfig: a storage object, owned by a user, says "I have a factor for provider X on my phone/whatever with secret key Q / my duo account is X / my address is Y".

Couple of things not covered here:

  - Statuses for providers ("Disabled", "Deprecated") don't do anything yet, but you can't edit them anyway.
  - Some `bin/auth` tools need to be updated.
  - When no providers are configured, the MFA panel should probably vanish.
  - Documentation.

Test Plan:
  - Ran migration with providers, saw configs point at the first provider.
  - Ran migration without providers, saw a provider created and configs pointed at it.
  - Added/removed factors and providers. Passed MFA gates. Spot-checked database for general sanity.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D19975
2019-01-23 13:37:43 -08:00
Austin McKinley
c756bf3476 Fix bin/accountadmin when not making changes
Summary: If you go through the `accountadmin` flow and change nothing, you get an exception about the transaction not having any effect. Instead, let the `applyTransactions` call continue even on no effect.

Test Plan: Ran `accountadmin` without changing anything for an existing user. No longer got an exception about no-effect transactions.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20009
2019-01-21 12:40:59 -08:00
Austin McKinley
6138d5885d Update documentation to reflect bin/auth changes
Summary: See https://secure.phabricator.com/D18901#249481. Update the docs and a warning string to reflect the new reality that `bin/auth recover` is now able to recover any account, not just administrators.

Test Plan: Mk 1 eyeball

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20007
2019-01-21 12:19:54 -08:00
epriestley
881d79c1ea When dirtying repository cluster routing caches after an Almanac edit, discover linked bindings from devices
Summary:
See PHI1030. When you edit an Almanac object, we attempt to discover all the related objects so we can dirty the repository cluster routing cache: if you modify a Device or Service that's part of a clustered repository, we need to blow away our cached view of the layout.

Currently, we don't correctly find linked Bindings when editing a Device, so we may miss Services which have keys that need to be disabled. Instead, discover these linked objects.

See D17000 for the original implementation and more context.

Test Plan:
  - Used `var_dump()` to dump out the discovered objects and dirtied cache keys.
  - Before change: editing a Service dirties repository routing keys (this is correct), but editing a Device does not.
  - After change: editing a Device now correctly dirties repository routing keys.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20003
2019-01-21 10:32:48 -08:00
epriestley
afd2ace0dc Apply inverse edge edits after committing primary object edits
Summary:
Fixes T13082. When you create a revision (say, `D111`) with `Ref T222` in the body, we write a `D111 -> T222` edge ("revision 111 references task 222") and an inverse `T222 -> D111` edge ("task 222 is referenced by revision 111").

We also apply a transaction to `D111` ("alice added a task: Txxx.") and an inverse transaction to `T222` ("alice added a revision: Dxxx").

Currently, it appears that the inverse transaction can sometimes generate mail faster than `D111` actually commits its (database) transactions, so the mail says "alice added a revision: Unknown Object (Differential Revision)". See T13082 for evidence that this is true, and a reproduction case.

To fix this, apply the inverse transaction (to `T222`) after we commit the main object (here, `D111`).

This is tricky because when we apply transactions, the transaction editor automatically "fixes" them to be consistent with the database state. For example, if a task already has title "XYZ" and you set the title to "XYZ" (same title), we just no-op the transaction.

It also fixes edge edits. The old sequence was:

  - Open (database) transaction.
  - Apply our transaction ("alice added a task").
  - Apply the inverse transaction ("alice added a revision").
  - Write the edges to the database.
  - Commit (database) transaction.

Under this sequence, the inverse transaction was "correct" and didn't need to be fixed, so the fixing step didn't touch it.

The new sequence is:

  - Open (database) transaction.
  - Apply our transaction ("alice added a task").
  - Write the edges.
  - Commit (database) transaction.
  - Apply the inverse transaction ("alice added a revision").

Since the inverse transaction now happens after the database edge write, the fixing step detects that it's a no-op and throws it away if we do this naively.

Instead, add some special cases around inverse edits to skip the correction/fixing logic, and just pass the "right" values in the first place.

Test Plan:
Added and removed related tasks from revisions, saw appropriate transactions render on both objects.

(It's hard to be certain this completely fixes the issue since it only happened occasionally in the first place, but we can see if it happens any more on `secure`.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13082, T222

Differential Revision: https://secure.phabricator.com/D19969
2019-01-20 21:03:20 -08:00
epriestley
0db29e624c Provide a richer error when an intracluster request can not be satisfied by the target node
Summary: See PHI1030. When installs hit this error, provide more details about which node we ended up on and what's going on.

Test Plan:
```
$ git pull
phabricator-ssh-exec: This repository request (for repository "spellbook") has been incorrectly routed to a cluster host (with device name "local.phacility.net", and hostname "orbital-3.local") which can not serve the request.

The Almanac device address for the correct device may improperly point at this host, or the "device.id" configuration file on this host may be incorrect.

Requests routed within the cluster by Phabricator are always expected to be sent to a node which can serve the request. To prevent loops, this request will not be proxied again.

(This is a read request.)
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
```

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20002
2019-01-20 17:26:06 -08:00
epriestley
be8b7c9eba Fix "Welcome Mail" check for a message when no message exists
Summary: Fixes T13239. See that task for discussion.

Test Plan: Tried to send welcome mail with no "Welcome" message.

Maniphest Tasks: T13239

Differential Revision: https://secure.phabricator.com/D20001
2019-01-20 07:00:33 -08:00
epriestley
755c40221d Temporarily disable transaction story links in HTML mail for the deploy
Ref T12921. See that task for discussion. Behavioral revert of D19968.
2019-01-19 05:10:02 -08:00
epriestley
e6ca2b998f Allow Conduit method call logs to be exported with the standard export pipeline
Summary:
See PHI1026. Allow installs to export Conduit call logs to a flat format.

Also, add date range queries.

Test Plan:
  - Exported some call logs.
  - Filtered logs by date.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19996
2019-01-18 20:09:57 -08:00
epriestley
6bb31de305 Use the customizable "Welcome Mail" message in welcome mail
Summary:
Depends on D19994. See PHI1027. If an install has customized the "Welcome Mail" message, include it in welcome mail. A special custom message from the profile screen overrides it, if provided.

(I fiddled with putting the custom message as "placeholder" text in the remarkup area as a hint, but newlines in "placeholder" text appear to have issues in Safari and Firefox. I think this is probably reasonably clear as-is.)

Make both render remarkup-into-text so things like links work properly, as it's reasonably likely that installs will want to link to things.

Test Plan:
  - With custom "Welcome Mail" text, sent mail with no custom override (got custom text) and a custom override (got overridden text).
  - Linked to some stuff, got sensible links in the mail (`bin/mail show-outbound`).

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19995
2019-01-18 19:55:44 -08:00
epriestley
22ad1ff2c5 Show the customized "Login" message on the login screen
Summary: Depends on D19992. Ref T13222. If administrators provide a custom login message, show it on the login screen.

Test Plan:
{F6137930}

  - Viewed login screen with and without a custom message.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D19994
2019-01-18 19:54:02 -08:00
epriestley
2c713b2d25 Add "Auth Messages" to support customizing onboarding/welcome flows
Summary:
Ref T13222. Long ago, we had a Config option (`welcome.html`) to let you dump HTML onto the login screen, but this was relatively hard to use and not good from a security perspective.

In some cases this was obsoleted by Dashboards, but there's at least some remaining set of use cases for actual login instructions on the login screen. For example, WMF has some guidance on //which// SSO mechanism to use based on what types of account you have. On `secure`, users assume they can register by clicking "Log In With GitHub" or whatever, and it might reduce frustration to tell them upfront that registration is closed.

Some other types of auth messaging could also either use customization or defaults (e.g., the invite/welcome/approve mail).

We could do this with a bunch of Config options, but I'd generally like to move to a world where there's less stuff in Config and more configuration is contextual. I think it tends to be easier to use, and we get a lot of fringe benefits (granular permissions, API, normal transaction logs, more abililty to customize workflows and provide contextual help/hints, etc). Here, for example, we can provide a remarkup preview, which would be trickier with Config.

This does not actually do anything yet.

Test Plan: {F6137541}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D19992
2019-01-18 19:53:19 -08:00
epriestley
ab7aceaabf Allow administrators to provide custom welcome text when welcoming users on the profile workflow
Summary:
See PHI1027. Currently, we allow you to customize invite email, but not most other types of email (approve, welcome). As a step forward, also allow welcome email to be customized with a message.

I considered separating the custom text from the main text with something heavyhanded ("alice added this custom message:") or a beautiful ASCII art divider like one of these:

https://www.asciiart.eu/art-and-design/dividers

...but nothing truly sung to me.

This only works on the profile flow for now. I'm planning to let you set a default message. I may or may not let you customize from "Create New User", seems like the default message probably covers most of that. Probably won't touch `scripts/user/add_user.php` since that's not really exactly super supported.

Test Plan:
Sent mail with and without custom messages, reviewed it with `bin/mail show-outbound`.

{F6137410}

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19991
2019-01-18 19:52:40 -08:00
epriestley
7f950f520b When password auth is not enabled, don't tell users to set a password in welcome email
Summary:
See PHI1027. Currently, the "Welcome" mail always tells users to set a password. This definitely isn't helpful if an install doesn't have password auth enabled.

We can't necessarily guess what they're supposed to do, so just give them generic instructions ("set up your account"). Upcoming changes will give administrators more control over the mail content.

Test Plan: Sent both versions of the mail, used `bin/mail show-outbound` to inspect them for correctness.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19990
2019-01-18 19:51:16 -08:00
epriestley
5537e29ee8 Move "Welcome" mail generation out of PhabricatorUser
Summary:
Ref PHI1027. Currently, `PhabricatorUser` has a couple of mail-related methods which shouldn't really be there in the long term. Immediately, I want to make some adjusments to the welcome email.

Move "Welcome" mail generation to a separate class and consolidate all the error handling. (Eventually, "invite" and "verify address" email should move to similar subclasses, too.) Previously, a bunch of errors/conditions got checked in multiple places.

The only functional change is that we no longer allow you to send welcome mail to disabled users.

Test Plan:
  - Used "Send Welcome Mail" from profile pages to send mail.
  - Hit "not admin", "disabled user", "bot/mailing list" errors.
  - Used `scripts/user/add_user.php` to send welcome mail.
  - Used "Create New User" to send welcome mail.
  - Verified mail with `bin/mail show-outbound`. (Cleaned up a couple of minor display issues here.)

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19989
2019-01-18 19:50:35 -08:00
epriestley
98bf3a950d Add setup warnings for "local_infile" (MySQL Server) and "mysql[i].allow_local_infile" (PHP Client)
Summary: Ref T13238. Warn users about these horrible options and encourage them to defuse them.

Test Plan: Hit both warnings, fixed the issues, issues went away.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13238

Differential Revision: https://secure.phabricator.com/D19999
2019-01-18 19:49:48 -08:00
epriestley
310ad7f8f4 Put a hard limit on password login attempts from the same remote address
Summary:
Ref T13222. Currently, if a remote address fails a few login attempts (5) in a short period of time (15 minutes) we require a CAPTCHA for each additional attempt.

This relies on:

  - Administrators configuring ReCAPTCHA, which they may just not bother with.
  - Administrators being comfortable with Google running arbitrary trusted Javascript, which they may not be comfortable with.
  - ReCAPTCHA actually being effective, which seems likely true for unsophisticated attackers but perhaps less true for more sophisticated attackers (see <https://github.com/ecthros/uncaptcha2>, for example).

(For unsophisticated attackers and researchers, "Rumola" has been the standard CAPTCHA bypass tool for some time. This is an extension that pays humans to solve CAPTCHAs for you. This is not practical at "brute force a strong password" scale. Google appears to have removed it from the Chrome store. The "submit the captcha back to Google's APIs" trick probably isn't practical at brute-force-scale either, but it's easier to imagine weaponizing that than weaponizing human solvers.)

Add a hard gate behind the CAPTHCA wall so that we fail into a secure state if there's no CAPTCHA or the attacker can defeat CAPTCHAs at a very low cost.

The big downside to this is that an attacker who controls your remote address (e.g., is behind the same NAT device you're behind on corpnet) can lock you out of your account. However:

  - That //should// be a lot of access (although maybe this isn't that high of a barrier in many cases, since compromising a "smart fridge" or "smart water glass" or whatever might be good enough).
  - You can still do "Forgot password?" and login via email link, although this may not be obvious.

Test Plan:
  - Logged in normally.
  - Failed many many login attempts, got hard gated.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D19997
2019-01-18 19:48:42 -08:00
epriestley
c125ab7a42 Remove "metamta.*.subject-prefix" options
Summary:
In ~2012, the first of these options was added because someone who hates dogs and works at Asana also hated `[Differential]` in the subject line. The use case there was actually //removing// the text, not changing it, but I made the prefix editable since it seemed like slightly less of a one-off.

These options are among the dumbest and most useless config options we have and very rarely used, see T11760. A very small number of instances have configured one of these options.

Newer applications stopped providing these options and no one has complained.

You can get the same effect with `translation.override`. Although I'm not sure we'll keep that around forever, it's a reasonable replacement today. I'll call out an example in the changelog to help installs that want to preserve this option.

If we did want to provide this, it should just be in {nav Applications > Settings} for each application, but I think it's wildly-low-value and "hack via translations" or "local patch" are entirely reasonable if you really want to change these strings.

Test Plan: Grepped for `subject-prefix`.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19993
2019-01-17 19:18:50 -08:00
epriestley
ff220acae6 Don't bounce mail messages if any recipient was reserved
Summary:
Ref T13222. If we receive a message and nothing processes it, we normally try to send the user an error message like "hey, nothing handled this, maybe you got the address wrong".

Just skip the "send them an error message" part if any recipient was reserved, so if you "Reply All" to a message that is "From: noreply@phabricator" you don't get a relatively unhelpful error.

This also makes sure that the "void" address doesn't generate bounces if the "From" is a valid user email address (e.g., with `metamta.can-send-as-user`). That is:

  - Phabricator needs to send a mail with only "CC" users.
  - Phabricator puts the "void" address in "To" as a placeholder.
  - The "void" address happens to route back to Phabricator.

We don't want that mail to bounce to anywhere. Normally, it won't:

  - From is usually "noreply@phabricator", which isn't a user, so we won't send anything back: we only send mail to verified user email addresses.
  - The message will have "X-Phabricator-Sent-This-Message: true" so we won't process it at all.

...but this is another layer of certainty.

Test Plan: Used `bin/mail receive-test` to receive mail to an invalid, unreserved address (bounce/error email) and an invalid, reserved address (no bounce/error email).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D19987
2019-01-17 19:17:37 -08:00
epriestley
6b6c991ad4 Allow Phortune accounts to customize their billing address and name
Summary:
See PHI1023. Ref T7607. Occasionally, companies need their billing address (or some other custom text) to appear on invoices to satisfy process or compliance requirements.

Allow accounts to have a custom "Billing Name" and a custom "Billing Address" which appear on invoices.

Test Plan: {F6134707}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T7607

Differential Revision: https://secure.phabricator.com/D19979
2019-01-16 16:16:27 -08:00
epriestley
a1516fefb6 Fix an issue where "Import Columns" could fail on a board for a project with milestones
Summary:
See PHI1025. When you "Import Columns", we test if you're trying to import into a board that already has columns. However, this test is too broad (it incorrectly detects "proxy" columns for milestones as columns) and not user-friendly (it returns 400 instead of a readable error).

Correct these issues, and refine some of the logic around proxy columns.

Test Plan:
  - Created a project, A.
  - Created a milestone under that project.
  - Imported another project's columns to A's workboard.
    - Before change: Unhelpful 400.
    - After change: import worked fine.
  - Also, hit the new error dialogs and read through them.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19978
2019-01-16 16:15:45 -08:00
epriestley
0a0afa489a Wordsmith inbound mail documentation more thoroughly
Summary: See D19973. Fix a couple typos and try to make some sections more clear / less scary.

Test Plan: Read text.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19986
2019-01-16 14:44:57 -08:00
epriestley
bd077bfcb7 Update inbound and outbound email documentation
Summary: Fixes T8636. Mention Herald for inbound, update some outbound stuff, do some language / organization tweaks.

Test Plan: Read documentation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T8636

Differential Revision: https://secure.phabricator.com/D19973
2019-01-16 13:56:32 -08:00
epriestley
c5f446defb Prevent application email addresses from shadowing user email addresses
Summary:
Fixes T13234. Don't let application email addresses be configured with user addresses. This might prevent an unlikely bit of mischief where someone does this intentionally, detailed in T13234.

(Possibly, these tables should just be merged some day, similar to how the "Password" table is now a shared resource that's modular enough for multiple applications to use it.)

Test Plan: {F6132259}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13234

Differential Revision: https://secure.phabricator.com/D19974
2019-01-16 13:28:08 -08:00
epriestley
dc4d7f1f3e Reorder "Merge" transaction to make "Close as Duplicate" produce a "[Merged]" email subject
Summary:
Fixes T11782. When you "Close as Duplicate", generate a "[Merged]" email by making the merge the first transaction.

(There are other, more-deterministic ways to do this with action strength, but this is much simpler and I believe it suffices.)

Test Plan: Used "Close as Duplicate", got a "[Merged]" email out of it.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T11782

Differential Revision: https://secure.phabricator.com/D19972
2019-01-16 13:27:10 -08:00
epriestley
35f0e31ed3 Add a Twilio SMS message adapter
Summary: Ref T920. Adds a "phone number" object, an "SMS" message type, and Twilio glue.

Test Plan:
Used this test script to send myself some text messages after configuring Twilio in `cluster.mailers`.

```
<?php

require_once 'scripts/init/init-script.php';

if ($argc < 3) {
  throw new Exception('usage: test.php <number> <body>');
}
$to_number = $argv[1];
$text_body = $argv[2];

$mailers = PhabricatorMetaMTAMail::newMailers(
  array(
    'outbound' => true,
    'media' => array(
      PhabricatorMailSMSMessage::MESSAGETYPE,
    ),
  ));
if (!$mailers) {
  return new Aphront404Response();
}
$mailer = head($mailers);

$message = id(new PhabricatorMailSMSMessage())
  ->setToNumber(new PhabricatorPhoneNumber($to_number))
  ->setTextBody($text_body);

$mailer->sendMessage($message);
```

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D19971
2019-01-16 13:25:59 -08:00
epriestley
96d3e73eed Fix an issue where "CC"-only email improperly wiped CC addresses
Summary: Ref T920. See <https://discourse.phabricator-community.org/t/ccd-emails-not-working-with-sendgrid-since-2019-week-1-update/2294>.

Test Plan: Used `bin/mail send-test --cc ...` without `--to`, got email.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D19970
2019-01-16 13:22:43 -08:00
epriestley
0c0cbb1c09 Fix an issue where transactions in mail were always rendered as text
Summary:
Fixes T12921. Currently, we call `getTitleForHTMLMail()`, but that calls `getTitleForMail()` which forces us into text rendering mode.

Instead, have `getTitleForHTML/TextMail()` force the rendering mode, then call `getTitleForMail()` with the desired rendering mode.

This causes stories like "epriestely added dependent tasks: x, y." to appear as links in email instead of plain text.

Test Plan: Used `bin/mail show-outbound --id ... --dump-html > out.html` to verify HTML mail.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12921

Differential Revision: https://secure.phabricator.com/D19968
2019-01-16 13:21:05 -08:00
epriestley
c3cafffed7 Update the "SES" and "sendmail" mailers for the new API; remove "encoding"
Summary: Ref T13222. Ref T920. This is the last of the upstream adapter updates.

Test Plan:
  - Sent mail with SES.
  - Sent mail with "sendmail". I don't have sendmail actually configured to an upstream MTA so I'm not 100% sure this worked, but the `sendmail` binary didn't complain and almost all of the code is shared with SES, so I'm reasonably confident this actually works.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13222, T920

Differential Revision: https://secure.phabricator.com/D19965
2019-01-16 13:18:55 -08:00
epriestley
43a6f34e7f Update the SMTP (PHPMailer) adapter for the new mail API; remove "encoding" and "mailer"
Summary:
Ref T920. Ref T12404.

  - Update to the new "$message" API.
  - Remove "encoding". I believe "base64" is always the best value for this since we stopped seeing issues once we changed the default.
  - Remove "mailer". This is a legacy option that makes little sense given how configuration now works.
  - Rename to "SMTP". This doesn't affect users anymore since this mailer has been configured as `smtp` for about a year.
  - This does NOT add a timeout since the SMTP code is inside PHPMailer (see T12404).

Test Plan: Sent messages with many mail features via GMail SMTP and SendGrid SMTP.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12404, T920

Differential Revision: https://secure.phabricator.com/D19961
2019-01-16 13:10:57 -08:00
epriestley
966a93334c Don't require "CAN_EDIT" to watch/unwatch a project
Summary:
See T1024. When "CAN_EDIT" became default in T13186, this was missed as an exception.

Watching shouldn't require "CAN_EDIT", so exempt it.

Test Plan:
  - Before change: tried to watch a project I could not edit, got a policy error.
  - After change: watched/unwatched a project I could not edit.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19977
2019-01-16 13:09:59 -08:00
epriestley
64e3296fe6 Upgrade Sendgrid to the modern mailer API; removes "api-user" option
Summary:
Ref T920. Ref T5969.

  - Update to the new "$message" API.
  - Update to Sendgrid v3.
  - Add a timeout.
  - This removes the "api-user" option, which Sendgrid no longer seems to use.

Test Plan: Sent Sendgrid messages with `bin/mail send-test ...` using subject/headers/attachments/html/to/cc.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jbrownEP

Maniphest Tasks: T5969, T920

Differential Revision: https://secure.phabricator.com/D19960
2019-01-16 13:09:29 -08:00
epriestley
d7da3560ec Update Mailgun adapter for the new mail adapter API
Summary: Ref T920. Ref T5969. Update the Mailgun adapter for the API changes and add a timeout.

Test Plan: Configured Mailgun as a mailer, sent mail with subject/to/cc/headers/html/attachments using `bin/mail send-test`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5969, T920

Differential Revision: https://secure.phabricator.com/D19959
2019-01-16 13:02:04 -08:00
epriestley
bc97a7d755 Update Mail test adapter for the newer adapter API and make all tests pass
Summary: Depends on D19956. Ref T920. Move the TestAdapter to the new API and adjust a couple of tests for the changes.

Test Plan: All tests now pass.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D19957
2019-01-16 13:01:25 -08:00
epriestley
a8657e6ab6 Update Postmark adapter for multiple mail media
Summary:
Depends on D19955. Ref T920. Ref T5969. Update Postmark to accept new Message objects. Also:

  - Update the inbound whitelist.
  - Add a little support for `media` configuration.
  - Add a service call timeout (see T5969).
  - Drop the needless word "Implementation" from the Adapter class tree. I could call these "Mailers" instead of "Adapters", but then we get "PhabricatorMailMailer" which feels questionable.

Test Plan: Used `bin/mail send-test` to send mail via Postmark with various options (mulitple recipients, text vs html, attachments).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5969, T920

Differential Revision: https://secure.phabricator.com/D19956
2019-01-16 13:00:34 -08:00
epriestley
b5797ce60a Refactor mail to produce an intermediate "bag of strings" object in preparation for SMS
Summary:
Depends on D19954. Ref T920. This is a step toward a world where "Mailers" are generic and may send messages over a broader array of channels (email, SMS, postal mail).

There are a few major parts here:

  - Instead of calling `$mailer->setSubject()`, `$mailer->setFrom()`, etc., build in intermediate `$message` object first, then pass that to the mailer.
    - This breaks every mailer! This change on its own does not fix them. I plan to fix them in a series of "update mailer X", "update mailer Y" followups.
    - This generally makes the API easier to change in the far future, and in the near future supports mailers accepting different types of `$message` objects with the same API.
  - Pull the "build an email" stuff out into a `PhabricatorMailEmailEngine`. `MetaMTAMail` is already a huge object without also doing this translation step. This is just a separation/simplification change, but also tries to fight against `MetaMTAMail` getting 5K lines of email/sms/whatsapp/postal-mail code.
  - Try to rewrite the "build an email" stuff to be a bit more straightforward while making it generate objects. Prior to this change, it had this weird flow:

```lang=php
foreach ($properties as $key => $prop) {
  switch ($key) {
    case 'xyz':
      // ...
  }
}
```

This is just inherently somewhat hard to puzzle out, and it means that processing order depends on internal property order, which is quite surprising.

Test Plan: This breaks everything on its own; adapters must be updated to use the new API. See followups.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D19955
2019-01-16 12:58:29 -08:00