1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2025-02-28 14:39:36 +01:00
Commit graph

46 commits

Author SHA1 Message Date
epriestley
b3557796fe Remove product literal strings in "pht()", part 6
Summary: Ref T13658.

Test Plan:
This test plan is non-exhaustive.

  - Ran `bin/mail`.
  - Uninstalled and reinstalled an application.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13658

Differential Revision: https://secure.phabricator.com/D21770
2022-04-25 12:22:28 -07:00
epriestley
caaa1394ef Don't count "Cc: x@y.com" as a legitimate recipient if the user who has "x@y.com" attached to their account has not verified the address
Summary:
Fixes T13317. On `admin.phacility.com`, an enterprising user added `noreply@admin.phacility.com` to their account. This caused them to become CC'd on several support issues over the last year, because we send mail "From" this address and it can get CC'd via reply/reply all/whatever else.

The original driving goal here is that if I reply to a task email and CC you on my reply, that should count as a CC in Phabricator, since this aligns with user intent and keeps them in the loop.

This misfire on `noreply@` is ultimately harmless (being CC'd does not grant the user access permission, see T4411), but confusing and undesirable. Instead:

  - Don't allow reserved addresses ("noreply@", "ssladmin@", etc) to trigger this subscribe-via-CC behavior.
  - Only count verified addresses as legitimate user recipients.

Test Plan:
  - Added a `bin/mail receive-test --cc ...` flag to make this easier to test.
  - Sent mail as `bin/mail receive-test --to X --as alice --cc bailey@verified.com`. Bailey was CC'd both before and after the change.
  - Sent mail as `bin/mail receive-test --to X --as alice --cc unverified@imaginary.com`, an address which Bailey has added to her account but not verified.
    - Before change: Bailey was CC'd on the task anyway.
    - After change: Bailey is not CC'd on the task.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13317

Differential Revision: https://secure.phabricator.com/D20593
2019-06-19 12:51:00 -07:00
epriestley
af71c51f0a Give "MetaMTAMail" a "message type" and support SMS
Summary:
Depends on D20011. Ref T920. This change lets a "MetaMTAMail" storage object represent various different types of messages, and  makes "all" the `bin/mail` stuff "totally work" with messages of non-email types.

In practice, a lot of the related tooling needs some polish/refinement, but the basics work.

Test Plan: Used `echo beep boop | bin/mail send-test --to epriestley --type sms` to send myself SMS.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D20012
2019-01-23 14:05:46 -08:00
epriestley
5537e29ee8 Move "Welcome" mail generation out of PhabricatorUser
Summary:
Ref PHI1027. Currently, `PhabricatorUser` has a couple of mail-related methods which shouldn't really be there in the long term. Immediately, I want to make some adjusments to the welcome email.

Move "Welcome" mail generation to a separate class and consolidate all the error handling. (Eventually, "invite" and "verify address" email should move to similar subclasses, too.) Previously, a bunch of errors/conditions got checked in multiple places.

The only functional change is that we no longer allow you to send welcome mail to disabled users.

Test Plan:
  - Used "Send Welcome Mail" from profile pages to send mail.
  - Hit "not admin", "disabled user", "bot/mailing list" errors.
  - Used `scripts/user/add_user.php` to send welcome mail.
  - Used "Create New User" to send welcome mail.
  - Verified mail with `bin/mail show-outbound`. (Cleaned up a couple of minor display issues here.)

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19989
2019-01-18 19:50:35 -08:00
epriestley
b5797ce60a Refactor mail to produce an intermediate "bag of strings" object in preparation for SMS
Summary:
Depends on D19954. Ref T920. This is a step toward a world where "Mailers" are generic and may send messages over a broader array of channels (email, SMS, postal mail).

There are a few major parts here:

  - Instead of calling `$mailer->setSubject()`, `$mailer->setFrom()`, etc., build in intermediate `$message` object first, then pass that to the mailer.
    - This breaks every mailer! This change on its own does not fix them. I plan to fix them in a series of "update mailer X", "update mailer Y" followups.
    - This generally makes the API easier to change in the far future, and in the near future supports mailers accepting different types of `$message` objects with the same API.
  - Pull the "build an email" stuff out into a `PhabricatorMailEmailEngine`. `MetaMTAMail` is already a huge object without also doing this translation step. This is just a separation/simplification change, but also tries to fight against `MetaMTAMail` getting 5K lines of email/sms/whatsapp/postal-mail code.
  - Try to rewrite the "build an email" stuff to be a bit more straightforward while making it generate objects. Prior to this change, it had this weird flow:

```lang=php
foreach ($properties as $key => $prop) {
  switch ($key) {
    case 'xyz':
      // ...
  }
}
```

This is just inherently somewhat hard to puzzle out, and it means that processing order depends on internal property order, which is quite surprising.

Test Plan: This breaks everything on its own; adapters must be updated to use the new API. See followups.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D19955
2019-01-16 12:58:29 -08:00
epriestley
e3aa043a02 Allow multiple mail receivers to react to an individual email
Summary:
Fixes T7477. Fixes T13066. Currently, inbound mail is processed by the first receiver that matches any "To:" address. "Cc" addresses are ignored.

**To, CC, and Multiple Receivers**

Some users would like to be able to "Cc" addresses like `bugs@` instead of having to "To" the address, which makes perfect sense. That's the driving use case behind T7477.

Since users can To/Cc multiple "create object" or "update object" addresses, I also wanted to make the behavior more general. For example, if you email `bugs@` and also `paste@`, your mail might reasonably make both a Task and a Paste. Is this useful? I'm not sure. But it seems like it's pretty clearly the best match for user intent, and the least-surprising behavior we can have. There's also no good rule for picking which address "wins" when two or more match -- we ended up with "address order", which is pretty arbitrary since "To" and "Cc" are not really ordered fields.

One part of this change is removing `phabricator.allow-email-users`. In practice, this option only controlled whether users were allowed to send mail to "Application Email" addresses with a configured default author, and it's unlikely that we'll expand it since I think the future of external/grey users is Nuance, not richer interaction with Maniphest/Differential/etc. Since this option only made "Default Author" work and "Default Author" is optional, we can simplify behavior by making the rule work like this:

  - If an address specifies a default author, it allows public email.
  - If an address does not, it doesn't.

That's basically how it worked already, except that you could intentionally "break" the behavior by not configuring `phabricator.allow-email-users`. This is a backwards compatility change with possible security implications (it might allow email in that was previously blocked by configuration) that I'll call out in the changelog, but I suspect that no installs are really impacted and this new behavior is generally more intuitive.

A somewhat related change here is that each receiver is allowed to react to each individual email address, instead of firing once. This allows you to configure `bugs-a@` and `bugs-b@` and CC them both and get two tasks. Useful? Maybe not, but seems like the best execution of intent.

**Sender vs Author**

Adjacently, T13066 described an improvement to error handling behavior here: we did not distinguish between "sender" (the user matching the email "From" address) and "actor" (the user we're actually acting as in the application). These are different when you're some internet rando and send to `bugs@`, which has a default author. Then the "sender" is `null` and the "author" is `@bugs-robot` or whatever (some user account you've configured).

This refines "Sender" vs "Author". This is mostly a purity/correctness change, but it means that we won't send random email error messages to `@bugs-robot`.

Since receivers are now allowed to process mail with no "sender" if they have some default "actor" they would rather use instead, it's not an error to send from an invalid address unless nothing processes the mail.

**Other**

This removes the "abundant receivers" error since this is no longer an error.

This always sets "external user" mail recipients to be unverified. As far as I can tell, there's no pathway by which we send them email anyway (before or after this change), although it's possible I'm missing something somewhere.

Test Plan:
I did most of this with `bin/mail receive-test`. I rigged the workflow slightly for some of it since it doesn't support multiple addresses or explicit "CC" and adding either would be a bit tricky.

These could also be tested with `scripts/mail/mail_handler.php`, but I don't currently have the MIME parser extension installed locally after a recent upgrade to Mojave and suspect T13232 makes it tricky to install.

- Ran unit tests, which provide significant coverage of this flow.
- Sent mail to multiple Maniphest application emails, got multiple tasks.
- Sent mail to a Maniphest and a Paste application email, got a task and a paste.
- Sent mail to a task.
  - Saw original email recorded on tasks. This is a behavior particular to tasks.
- Sent mail to a paste.
- Sent mail to a mock.
- Sent mail to a Phame blog post.
- Sent mail to a Legalpad document.
- Sent mail to a Conpherence thread.
- Sent mail to a poll.
- This isn't every type of supported object but it's enough of them that I'm pretty confident I didn't break the whole flow.
- Sent mail to an object I could not view (got an error).
- As a non-user, sent mail to several "create an object..." addresses.
  - Addresses with a default user worked (e.g., created a task).
  - Addresses without a default user did not work.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13066, T7477

Differential Revision: https://secure.phabricator.com/D19952
2019-01-16 12:28:02 -08:00
epriestley
73e3057c52 Rename "MetaMTA" mail attachments and add more mail message objects
Summary:
Depends on D19953. Ref T9141. We have a "MetaMTAAttachment" object, rename it to "MailAttachment".

Also add a "Header" object and an "EmailMessage" object. Currently, mail adapters have a large number of methods like `setSubject()`, `addTo()`, etc, that I would like to remove.

I'd like the API to be more like `sendMessage(PhabricatorMailExternalMessage $message)`. This is likely a significant simplification anyway, since the implementations of all these methods are just copy/pasted boilerplate anyway (lots of `$this->subject = $subject;`) and this will let Adapters support other message media (SMS, APNS, Whatsapp, etc.)

That's a larger change, but move toward a world where we can build a concrete `$message` object for "email" or "sms".

The `PhabricatorMailEmailMessage` object is just a dumb, flat object representation of the information an adapter needs to actually send mail. The existing `PhabricatorMetaMTAMail` is a much more complex object and has a lot of rich data (delivery status, related object PHIDs, etc) and is a storage object.

The new flow will be something like:

  - `PhabricatorMetaMTAMail` (possibly renamed) is the storage object for any outbound message on this channel. It tracks message content, acceptable delivery media (SMS vs email), delivery status, related objects, has a PHID, and has a daemon worker associated with delivering it.
  - It builds a `PhabricatorMailExternalMessage`, which is a simple, flat description of the message it wants to send. The subclass of this object depends on the message medium. For email, this will be an `EmailMessage`. This is just a "bag of strings" sort of object, with appropriate flattened values for the adapter to work with (e.g., Email has email addresses, SMS has phone numbers).
  - It passes the `ExternalMessage` (which is a `MailMessage` or `SMSMessage` or whatever) to the adapter.
  - The adapter reads the nice flat properties off it and turns them into an API request, SMTP call, etc.

This is sort of how things work today anyway. The major change is that I want to hand off a "bag of strings" object instead of calling `setX()`, `setY()`, `setZ()` with each individual value.

Test Plan: Grepped for `MetaMTAAttachment`. This doesn't change any behavior yet.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T9141

Differential Revision: https://secure.phabricator.com/D19954
2019-01-04 15:23:44 -08:00
epriestley
690a460c8e Allow mailers to be explicitly marked as inbound or outbound
Summary:
See PHI785. Ref T13164. In this case, an install wants to receive mail via Mailgun, but not configure it (DKIM + SPF) for outbound mail.

Allow individual mailers to be marked as not supporting inbound or outbound mail.

Test Plan:
  - Added and ran unit tests.
  - Went through some mail pathways locally, but I don't have every inbound/outbound configured so this isn't totally conclusive.
  - Hit `bin/mail send-test` with a no-outbound mailer.
  - I'll hold this until after the release cut so it can soak on `secure` for a bit.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19546
2018-07-30 08:25:06 -07:00
epriestley
1b24b486f5 Manage object mailKeys automatically in Mail instead of storing them on objects
Summary:
Ref T13065. `mailKey`s are a private secret for each object. In some mail configurations, they help us ensure that inbound mail is authentic: when we send you mail, the "Reply-To" is "T123+456+abcdef".

  - The `T123` is the object you're actually replying to.
  - The `456` is your user ID.
  - The `abcdef` is a hash of your user account with the `mailKey`.

Knowing this hash effectively proves that Phabricator has sent you mail about the object before, i.e. that you legitimately control the account you're sending from. Without this, anyone could send mail to any object "From" someone else, and have comments post under their username.

To generate this hash, we need a stable secret per object. (We can't use properties like the PHID because the secret has to be legitimately secret.)

Today, we store these in `mailKey` properties on the actual objects, and manually generate them. This results in tons and tons and tons of copies of this same ~10 lines of code.

Instead, just store them in the Mail application and generate them on demand. This change also anticipates possibly adding flags like "must encrypt" and "original subject", which are other "durable metadata about mail transmission" properties we may have use cases for eventually.

Test Plan:
  - See next change for additional testing and context.
  - Sent mail about Herald rules (next change); saw mail keys generate cleanly.
  - Destroyed a Herald rule with a mail key, saw the mail properties get nuked.
  - Grepped for `getMailKey()` and converted all callsites I could which aren't the copy/pasted boilerplate present in 50 places.
  - Used `bin/mail receive-test --to T123` to test normal mail receipt of older-style objects and make sure that wasn't broken.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065

Differential Revision: https://secure.phabricator.com/D19399
2018-04-25 06:46:58 -07:00
epriestley
19b3fb8863 Add a Postmark mail adapter so it can be configured as an outbound mailer
Summary: Depends on D19007. Ref T12677.

Test Plan: Used `bin/mail send-test ... --mailer postmark` to deliver some mail via Postmark.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12677

Differential Revision: https://secure.phabricator.com/D19009
2018-02-08 06:18:23 -08:00
epriestley
7b2b5cd91e Add basic support for a "Must Encrypt" mail flag which prevents unsecured content transmission
Summary:
Ref T13053. See PHI291. For particularly sensitive objects (like security issues), installs may reasonably wish to prevent details from being sent in plaintext over email.

This adds a "Must Encrypt" mail behavior, which discards mail content and all identifying details, replacing it with a link to the `/mail/` application. Users can follow the link to view the message over HTTPS.

The flag discards body content, attachments, and headers which imply things about the content of the object. It retains threading headers and headers which may uniquely identify the object as long as they don't disclose anyting about the content.

The `bin/mail list-outbound` command now flags these messages with a `#` mark.

The `bin/mail show-outbound` command now shows sent/suppressed headers and the body content as delivered (if it differs from the original body content).

The `/mail/` web UI now shows a tag for messages marked with this flag.

For now, there is no way to actually set this flag on mail.

Test Plan:
  - Forced this flag on, made comments and took actions to send mail.
  - Reviewed mail with `bin/mail` and `/mail/` in the web UI, saw all content information omitted.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13053

Differential Revision: https://secure.phabricator.com/D18983
2018-02-02 14:34:34 -08:00
Dmitri Iouchtchenko
9bd6a37055 Fix spelling
Summary: Noticed a couple of typos in the docs, and then things got out of hand.

Test Plan:
  - Stared at the words until my eyes watered and the letters began to swim on the screen.
  - Consulted a dictionary.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D18693
2017-10-09 10:48:04 -07:00
epriestley
8377bb3637 Raise a tailored error message on "show-outbound --id cat"
Summary: Fixes T12579. Unclear why the user ran this command.

Test Plan: Ran with `--id cat`. Ran with `--id 123`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12579

Differential Revision: https://secure.phabricator.com/D17719
2017-04-18 09:51:26 -07:00
epriestley
2b5bf4b911 Allow "bin/mail send-test" to accept raw email addresses via "--to"
Summary: Ref T12372. This supports testing the `wordwrap()` patch discussed in that task.

Test Plan:
  - Ran `bin/mail send-test --to email@domain.com`
  - Ran `bin/mail send-test --to username`

Reviewers: chad, lvital

Reviewed By: lvital

Maniphest Tasks: T12372

Differential Revision: https://secure.phabricator.com/D17489
2017-03-10 14:52:33 -08:00
epriestley
3cf6f746f0 Raise an "Account Setup Issue" if your primary address is unverified
Summary:
Ref T12237. This adds a UI cue for users who have unverified primary addresses, since we no longer send them mail.

Also adds a new `bin/mail unverify` to unverify an address (for example, because mail is bouncing).

Test Plan:
  - Unverified my address, saw setup issue.
  - Verified my address, no more setup issue.

{F2861820}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12237

Differential Revision: https://secure.phabricator.com/D17344
2017-02-13 10:41:32 -08:00
cburroughs
786b135a66 variable days back for bin/mail volume
Summary:
Email is so exciting I can't wait 30 days for initial results.

ref T9161

Test Plan:
 * `./bin/mail volume --days 60` took longer and gave plausibly larger
   results.
 * `./bin/mail volume --days 0` quickly told me no mail had been sent.
 * `./bin/mail volume` Said it was still looking 30 days back.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Maniphest Tasks: T9161

Differential Revision: https://secure.phabricator.com/D13901
2015-08-27 04:40:45 -07:00
epriestley
3b987a93ce Consolidate outbound mail status in a new class
Summary: Ref T5791. This collects outbound mail status in one place and makes the list view a little spiffier.

Test Plan: Looked at list and detail views. Grepped for changed classes/constants.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13884
2015-08-14 04:31:42 -07:00
Joshua Spence
368f359114 Use PhutilClassMapQuery instead of PhutilSymbolLoader
Summary: Use `PhutilClassMaQuery` instead of `PhutilSymbolLoader`, mostly for consistency. Depends on D13588.

Test Plan: Poked around a bunch of pages.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13589
2015-08-14 07:49:01 +10:00
epriestley
8c06d89070 Flesh out web UI for mail a bit to prepare for Herald outbound rules
Summary:
Ref T9141. Ref T5791. Ref T7013. Major changes here is:

  - Currently, we don't store the headers we actually sent, or the reasons we actually did or did not deliver a mail.
    - Start storing these (as `headers.sent` and `actors.sent`).
    - Show them in the web UI.
    - Show them in `bin/mail show-outbound` (previously, we sort of re-computed them in a hacky way).
    - Take them into account in `bin/mail volume`.

Then some minor changes:

  - Show mail bodies.
  - Show more mail information.
  - Start renaming "MetaMTA" to "Mail", at least in the web UI.

Test Plan:
{F707501}
{F707502}
{F707503}
{F707504}
{F707505}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5791, T7013, T9141

Differential Revision: https://secure.phabricator.com/D13878
2015-08-12 12:27:31 -07:00
epriestley
50e084dcda Add a rough bin/mail volume command for showing mail volume
Summary: Ref T7013. This might help us understand the problem better.

Test Plan:
```
$ ./bin/mail volume
+==============+============+
| User         | Unfiltered |
+==============+============+
| admin        | 136        |
| dog          | 31         |
| epriestley   | 24         |
| ducksey      | 18         |
| saurus       | 8          |
| example-list | 7          |
| squeakybirdo | 3          |
| nnn          | 3          |
| facebooker   | 2          |
+==============+============+

Mail sent in the last 30 days.
"Unfiltered" is raw volume before preferences were applied.
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7013

Differential Revision: https://secure.phabricator.com/D13813
2015-08-06 11:32:17 -07:00
Joshua Spence
36e2d02d6e phtize all the things
Summary: `pht`ize a whole bunch of strings in rP.

Test Plan: Intense eyeballing.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12797
2015-05-22 21:16:39 +10:00
epriestley
c0e26c65e0 Make mail delivery reasons code-based; include positive and negative reasons
Summary:
Ref T7731. Looking forward to T5791, I eventually anticipate writing an interface which looks like a webmail UI where users can review mail they've been sent and understand why they recieved (or did not receive) the mail. Roughly like `bin/mail list-outbound` / `bin/mail show-outbound` work today, but policy-aware (so you can only see messages where delivery was attempted to you).

We currently record a list of "reasons" why a mail is undeliverable, but this list is string-based (so it can not be translated once we start persisting it) and has only negative reasons (so it can not be used to fully understand reasons for delivery or nondelivery).

Make it code-based (so it can be translated) and allow both positive and negative reasons to be listed (so positive reasons can be understood).

Test Plan: Used `bin/mail show-outbound` to review mail delivery reasons, including the positive reason we currently have (forced delivery of authentication mail).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7731

Differential Revision: https://secure.phabricator.com/D12297
2015-04-06 10:01:11 -07:00
Bob Trahan
799dada3ad MetaMTA - add support for "Default Author"
Summary: Ref T5952. This adds support for a "default author" and deploys it on Maniphest.

Test Plan: used augmented (by this diff) bin/mail receive-test to test creation via an application email with a default author configured and no author specified. a task was created with the author as the default author i configured.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5952

Differential Revision: https://secure.phabricator.com/D11446
2015-01-28 11:13:29 -08:00
Bob Trahan
9219645287 Daemons - add "objectPHID" to task tables.
Summary: Ref T5402. This more or less "fixes" it but there's probably some polish to do?

Test Plan:
stopped and started daemons. error logs look good.

ran bin/storage upgrade.  noted that `adjust` added the appropriate indices for active and archive task.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5402

Differential Revision: https://secure.phabricator.com/D11044
2014-12-23 16:30:05 -08:00
Joshua Spence
0151c38b10 Apply some autofix linter rules
Summary: Self-explanatory.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10454
2014-09-10 06:55:05 +10:00
epriestley
7d31ea7c55 Minor tweaks to bin/mail send-test
Summary: Clean up some arg handling stuff.

Test Plan: Used this while debugging.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D10314
2014-08-21 11:25:44 -07:00
epriestley
8403812e15 Make HTML email a little easier to debug
Summary:
Ref T992.

  - Format text/HTML bodies explicitly in `bin/mail show-outbound`.
  - Provide `bin/mail show-outbound --dump-html` so you can do something like `bin/mail show-outbound --dump-html > dump.html; open dump.html` to get a browser preview somewhat easily.

Test Plan: Ran `bin/mail show-outbound` with and without `--dump-html` flag.

Reviewers: talshiri, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T992

Differential Revision: https://secure.phabricator.com/D10272
2014-08-15 11:07:33 -07:00
Tal Shiri
4c57e6d34d HTML emails
Summary:
Added support for side-by-side HTML and plaintext email building.

We can control if the HTML stuff is sent by by a new config, metamta.html-emails

Test Plan:
Been running this in our deployment for a few months now.

====Well behaved clients====
 - Gmail
 - Mail.app

====Bad clients====

- [[ http://airmailapp.com/ | Airmail ]]. They confuse Gmail too, though.

====Need testing====
 - Outlook (Windows + Mac)

Reviewers: chad, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: webframp, taoqiping, chad, epriestley, Korvin

Maniphest Tasks: T992

Differential Revision: https://secure.phabricator.com/D9375
2014-08-15 08:12:21 -07:00
epriestley
500506bfef Persist excluded recipients when saving mail
Summary:
Fixes T5185. The fundamental issue is that this `excludePHIDs` property was not saved, so the logic went like this:

  - Generate `excludePHIDs` correctly.
  - Pass `excludePHIDs` through the stack.
  - Perform some other computations correctly.
  - Queue the mail for the daemons, throwing it away. {icon bomb}
  - Daemons process mail with empty `excludePHIDs` list.

Store it in the persistent properties array instead.

Also remove the "override self mail" thing, since it's only used by `bin/mail send-test` and suffers from the same issue. I think it's too useless to fix, since even if you get caught by it, `bin/mail` makes it clear why the message was dropped.

Test Plan:
Notable:

  - `exclude` present in properties
  - Exclusion reason under RECIPIENTS header

{P1229}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5185

Differential Revision: https://secure.phabricator.com/D10234
2014-08-12 12:28:07 -07:00
Joshua Spence
9a679bf374 Allow worker tasks to have priorities
Summary: Fixes T5336. Currently, `PhabricatorWorkerLeaseQuery` is basically FIFO. It makes more sense for the queue to be a priority-queue, and to assign higher priorities to alerts (email and SMS).

Test Plan: Created dummy tasks in the queue (with different priorities). Verified that the priority field was set correctly in the DB and that the priority was shown on the `/daemon/` page. Started a `PhabricatorTaskmasterDaemon` and verified that the higher priority tasks were executed before lower priority tasks.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5336

Differential Revision: https://secure.phabricator.com/D9871
2014-07-12 03:02:06 +10:00
Joshua Spence
47964077ef Convert more scripts to use PhutilConsoleTable.
Summary: Convert `./bin/mail` and a`./bin/sms` to use `PhutilConsoleTable` for formatting output.

Test Plan: I don't actually have mail and SMS setup on my dev box, but this is a pretty straightforward change.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9621
2014-06-19 02:36:58 +10:00
Joshua Spence
0a62f13464 Change double quotes to single quotes.
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.

Test Plan: Eyeballed it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9431
2014-06-09 11:36:50 -07:00
epriestley
fcf5149b36 Clean up numerous rough edges in Mail configuration
Summary:
  - Support file attachments in Mailgun, after D8831.
  - Fix `bin/mail send-test --attach ...` flag.
  - Make `bin/mail send-test` route mail through the daemons.
  - Remove the `workerTaskID` on MetaMTAMail, which is only used (needlessly) by `bin/mail resend` and creates a huge mess elsewhere.
  - Currently, when mail fails, the daemon exits with a very generic and useless message. Instead, make `sendNow()` throw when it fails, so the real reason is surfaced. This is OK now because mail is always sent via the daemons.
  - Now that Mailgun supports attachments, document it.
  - Update a bunch of mail docs.

Test Plan:
  - Sent mail.
  - Sent mail with attachments.
  - Read documentation.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8832
2014-04-21 15:45:29 -07:00
epriestley
4b0ef353e4 Remove retry/failure mechanisms from MetaMTA
Summary:
Fixes T4202. We have old code in MetaMTA which implements gradual backoff and maximum retries.

However, we have more general code in the task queue which does this, too. We can just use the more general stuff in the task queue; it obsoletes the specific stuff in MetaMTA, which is more complex and ran into some kind of issue in T4202.

Remove `retryCount`, `nextRetry` (obsoleted by task queue retry mechanisms) and "simulated failures" (no longer in use).

Generally, modern infrastructure has replaced these mechanisms with more general ones.

Test Plan:
  - Sent mail.
  - Observed unsendable mail failing in reasonable ways in the queue.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4202

Differential Revision: https://secure.phabricator.com/D8115
2014-02-01 14:35:42 -08:00
epriestley
ce78bf1de4 Make all bin/* scripts locate their workflows dynamically
Summary:
Ref T2015. Not directly related to Drydock, but I bumped into this. All these scripts currently enumerate their workflows explicitly.

Instead, use `PhutilSymbolLoader` to automatically discover workflows. This reduces code duplication and errors (see all the bad `extends` this diff fixes) and lets third parties add new workflows (not clearly valuable?).

Test Plan: Ran `bin/x help` for each modified script.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7840
2013-12-27 13:15:48 -08:00
epriestley
e397103bf2 Extend all "ManagementWorkflow" classes from a base class
Summary:
Ref T2015. Not directly related to Drydock, but I've wanted to do this for a bit.

Introduce a common base class for all the workflows in the scripts in `bin/*`. This slightly reduces code duplication by moving `isExecutable()` to the base, but also provides `getViewer()`. This is a little nicer than `PhabricatorUser::getOmnipotentUser()` and gives us a layer of indirection if we ever want to introduce more general viewer mechanisms in scripts.

Test Plan: Lint; ran some of the scripts.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7838
2013-12-27 13:15:40 -08:00
epriestley
a3c811f281 Accept case-insensitive mail replies
Summary:
Mailbox sometimes (?) changes the case of the email address (?). Be more liberal in what we accept.

Also fix a minor output bug.

Test Plan: Sent mail to `e1+...` instead of `E1+...`, verified it arrived.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7575
2013-11-12 21:23:23 -08:00
epriestley
1ee455c441 Add defualt view and default edit policies for tasks
Summary: Ref T603. Allow global default policies to be configured for tasks.

Test Plan:
  - Created task via web UI.
  - Created task via Conduit.
  - Created task via email.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7267
2013-10-09 13:53:17 -07:00
Bob Trahan
b902005bed Kill PhabricatorObjectDataHandle
Summary: Ref T603. Killing this class is cool because the classes that replace it are policy-aware. Tried to keep my wits about me as I did this and fixed a few random things along the way. (Ones I remember right now are pulling a query outside of a foreach loop in Releeph and fixing the text in UIExample to note that the ace of hearts if "a powerful" card and not the "most powerful" card (Q of spades gets that honor IMO))

Test Plan: tested the first few changes (execute, executeOne X handle, object) then got real mechanical / careful with the other changes.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran, FacebookPOC

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D6941
2013-09-11 12:27:28 -07:00
epriestley
34356c7154 Fix outdated link to MetaMTA web UI in MetaMTA tasks
Summary: Missed this when moving most MetaMTA responsibilities to the CLI. Show the correct command to get data rather than linking to a 404.

Test Plan: {F56733}

Reviewers: wez, btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6846
2013-08-29 17:37:17 -07:00
epriestley
544a84ebb9 Move outbound mail lists to CLI and enhance details
Summary: Finish off moving all this stuff to the CLI. Ref T3306.

Test Plan:
  PROPERTIES
  ID: 6483
  Status: void
  Retry Count: 0
  Next Retry: 1373494457
  Related PHID: PHID-DREV-5bnb33yeuhuaulyc3exg
  Message: Message has no valid recipients: all To/Cc are disabled, invalid, or configured not to receive this mail.

  PARAMETERS
  from: PHID-USER-lqiz3yd7wmk64ejugvov
  is-html:
  parent-message-id: null
  thread-id: differential-rev-PHID-DREV-5bnb33yeuhuaulyc3exg-req
  is-first-message: null
  is-bulk: 1
  mailtags: ["differential-comment"]
  cc: ["PHID-USER-cluwcdowc35gmperlkbi"]
  subject: D22: quack quack
  subject-prefix: [Differential]
  vary-subject-prefix: [Commented On]
  worker-task: 936546

  HEADERS
  Thread-Topic: D22: quack quack
  X-Herald-Rules: none
  X-Differential-Author: <PHID-USER-lqiz3yd7wmk64ejugvov>
  X-Differential-CC: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Differential-CC: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Differential-CC: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Differential-CC: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Differential-CC: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Differential-CC: <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-CCs: <PHID-USER-ly3pvrtdkw7lbgs72jvr>, <PHID-USER-cluwcdowc35gmperlkbi>, <PHID-MLST-wkxaantg3q6pgdkty5pt>, <PHID-USER-aeabc4ipqbifny3rw4ok>, <PHID-USER-zqxtb3oi4pouwxnxlv3f>, <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-Explicit-CC: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Differential-Explicit-CC: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Differential-Explicit-CC: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Differential-Explicit-CC: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Differential-Explicit-CC: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Differential-Explicit-CC: <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-Explicit-CCs: <PHID-USER-ly3pvrtdkw7lbgs72jvr>, <PHID-USER-cluwcdowc35gmperlkbi>, <PHID-MLST-wkxaantg3q6pgdkty5pt>, <PHID-USER-aeabc4ipqbifny3rw4ok>, <PHID-USER-zqxtb3oi4pouwxnxlv3f>, <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Phabricator-To: <PHID-USER-lqiz3yd7wmk64ejugvov>
  X-Phabricator-Cc: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Phabricator-Cc: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Phabricator-Cc: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Phabricator-Cc: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Phabricator-Cc: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Phabricator-Cc: <PHID-USER-cknqtm2dzw7twnwyiaye>

  RECIPIENTS
  ! dog (dog)
      - This user is disabled; disabled users do not receive mail.

  BODY
  epriestley has commented on the revision "quack quack".

    zxcbzxcb

  REVISION DETAIL
    http://local.aphront.com:8080/D22

  To: epriestley
  Cc: Unknown User, dog, list, duck, epriestley992, asana

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3306

Differential Revision: https://secure.phabricator.com/D6423
2013-07-10 18:52:22 -07:00
epriestley
e8f679fa88 Move the inbound mail list to the CLI
Summary: Same as D6419. See D6413.

Test Plan:
      13  epriestley        D6                    Re: [local/Differential] [Request, 83 lines] D6: aabd
      14  epriestley        D6                    Re: [local/Differential] [Request, 83 lines] D6: aabd
      15  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      16  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      17  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      18  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      19  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      20  -                 -                     (No subject.)
      21  epriestley        D22                   (No subject.)
      22  epriestley        D12                   (No subject.)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6420
2013-07-10 15:18:37 -07:00
epriestley
6bac7e390e Move "send test" to bin/mail
Summary: Like D6417. See D6413.

Test Plan:
  >>> orbital ~/devtools/phabricator $ echo "hello" | ./bin/mail send-test --to dog --cc duck --from epriestley --subject 'hey there' --attach README --tag blip --tag bloop
  Reading message body from stdin...
  Mail sent! You can view details by running this command:

      phabricator/ $ ./bin/mail show-outbound --id 6480

  >>> orbital ~/devtools/phabricator $ ./bin/mail show-outbound --id 6480
  PROPERTIES
  ID: 6480
  Status: sent
  Retry Count: 0
  Next Retry: 1373493443
  Related PHID:
  Message:

  PARAMETERS
  to: ["PHID-USER-cluwcdowc35gmperlkbi"]
  cc: ["PHID-USER-aeabc4ipqbifny3rw4ok"]
  subject: hey there
  is-html:
  is-bulk:
  mailtags: ["blip","bloop"]
  from: PHID-USER-lqiz3yd7wmk64ejugvov
  worker-task: 936543

  BODY
  hello

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6419
2013-07-10 15:18:24 -07:00
epriestley
7fa2343822 Move mail "Receive Test" from web UI to CLI
Summary:
Ref T3306. Moves this from the web to the CLI, which is a tiny bit clunkier but way better as far as policies go and more repeatable for development.

See discussion in D6413.

Test Plan: Ran `bin/mail receive-test`, verified mail was received. Used and abused various options.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3306

Differential Revision: https://secure.phabricator.com/D6417
2013-07-10 15:13:24 -07:00
epriestley
e4525f9de1 Provide some diagnostic tools for examining inbound and outbound mail
Summary: We can't show this stuff on the web UI because it has password reset links and private reply-to addresses, but we can provide easier CLI tools than "root around in the database". Land a rough version of `bin/mail show-inbound` and `bin/mail show-outbound`.

Test Plan: Used both commands to examine mail from the CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: tido, euresti, aran

Differential Revision: https://secure.phabricator.com/D5963
2013-05-20 10:13:42 -07:00
epriestley
e80c59cbc6 Introduce basic bin/mail with a resend workflow
Summary:
Fixes T2458. Ref T2843. @tido's email from T2843 has exhausted its retries and failed, but we want to try it again with the patch from D5464 to capture the actual error. This sort of thing has come up a few times in debugging, too.

Also fixed some stuff that came up while debugging this.

Test Plan:
  - Ran command with no args.
  - Ran resend with no args.
  - Ran resend with bad IDs.
  - Ran resend with already-queued messages, got "already queued" error.
  - Ran resend with already-sent message, got requeue.

Reviewers: btrahan, tido

Reviewed By: tido

CC: aran

Maniphest Tasks: T2458, T2843

Differential Revision: https://secure.phabricator.com/D5493
2013-03-30 15:53:49 -07:00