1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-01 03:02:43 +01:00
Commit graph

173 commits

Author SHA1 Message Date
epriestley
d011f8fdc6 Add a setting to disable all notification email
Summary: Ref T5861. Adds an option to opt out of all notification email. We'll still send you password resets, email verifications, etc.

Test Plan:
{F189484}

  - Added unit tests.
  - With preference set to different things, tried to send myself mail. Mail respected preferences.
  - Sent password reset email, which got through the preference.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: rush898, epriestley

Maniphest Tasks: T5861

Differential Revision: https://secure.phabricator.com/D10237
2014-08-12 12:28:29 -07:00
epriestley
500506bfef Persist excluded recipients when saving mail
Summary:
Fixes T5185. The fundamental issue is that this `excludePHIDs` property was not saved, so the logic went like this:

  - Generate `excludePHIDs` correctly.
  - Pass `excludePHIDs` through the stack.
  - Perform some other computations correctly.
  - Queue the mail for the daemons, throwing it away. {icon bomb}
  - Daemons process mail with empty `excludePHIDs` list.

Store it in the persistent properties array instead.

Also remove the "override self mail" thing, since it's only used by `bin/mail send-test` and suffers from the same issue. I think it's too useless to fix, since even if you get caught by it, `bin/mail` makes it clear why the message was dropped.

Test Plan:
Notable:

  - `exclude` present in properties
  - Exclusion reason under RECIPIENTS header

{P1229}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5185

Differential Revision: https://secure.phabricator.com/D10234
2014-08-12 12:28:07 -07:00
epriestley
e4049e8797 Fix verbose email addresses being passed to mail adapters
Summary:
Fixes T5233.

  - The mail adapter API currently expects plain addresses (like `a@b.com`) in `addTos()`, and some adapters can not accept fancy verbose addresses (like `"name" <a@b.com>`).
  - When we try to send error email, we pass the entire "From" header into the API. This is incorrect.
  - Since it would be nice to make this just work in the future, fix it inside the API.
  - Specifically, this is reached with: send email -> generates error -> we try to send you an email back -> we send it to your "From" -> some mailers choke on the fancy name if you have one.

Test Plan: Processed an errorneous email with a fancy "From", got a response error.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5233

Differential Revision: https://secure.phabricator.com/D10232
2014-08-12 12:27:13 -07:00
Tal Shiri
570a78d42a don't add email addresses to CC if they are already in TO
Summary:
Some mailers remove the duplicate entries themselves, but some (Mailgun) don't.
This affects installations with metamta.one-mail-per-recipient set to false, and will cause
- ugly looking "to" entries. Gmail, for example, collapses to+cc entries to one list, so you get something that looks like "to: me me john"
- It sometimes causes duplicate delivery of the same message when used in conjuction with Google Groups. I suspect that their message de-dup mechanism is confused by it (I fuzzed it directly with Mailgun, and saw the same message delivered twice - once directly through mailgun, and bounced again through Google Groups). This doesn't happen when the entries are not duplicated.

Test Plan: Created some tasks. Added subscribers. Things seem to work reasonably well.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9978
2014-07-17 18:32:44 -07:00
Joshua Spence
9a679bf374 Allow worker tasks to have priorities
Summary: Fixes T5336. Currently, `PhabricatorWorkerLeaseQuery` is basically FIFO. It makes more sense for the queue to be a priority-queue, and to assign higher priorities to alerts (email and SMS).

Test Plan: Created dummy tasks in the queue (with different priorities). Verified that the priority field was set correctly in the DB and that the priority was shown on the `/daemon/` page. Started a `PhabricatorTaskmasterDaemon` and verified that the higher priority tasks were executed before lower priority tasks.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5336

Differential Revision: https://secure.phabricator.com/D9871
2014-07-12 03:02:06 +10:00
Joshua Spence
0a62f13464 Change double quotes to single quotes.
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.

Test Plan: Eyeballed it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9431
2014-06-09 11:36:50 -07:00
epriestley
fcf5149b36 Clean up numerous rough edges in Mail configuration
Summary:
  - Support file attachments in Mailgun, after D8831.
  - Fix `bin/mail send-test --attach ...` flag.
  - Make `bin/mail send-test` route mail through the daemons.
  - Remove the `workerTaskID` on MetaMTAMail, which is only used (needlessly) by `bin/mail resend` and creates a huge mess elsewhere.
  - Currently, when mail fails, the daemon exits with a very generic and useless message. Instead, make `sendNow()` throw when it fails, so the real reason is surfaced. This is OK now because mail is always sent via the daemons.
  - Now that Mailgun supports attachments, document it.
  - Update a bunch of mail docs.

Test Plan:
  - Sent mail.
  - Sent mail with attachments.
  - Read documentation.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8832
2014-04-21 15:45:29 -07:00
epriestley
95a405da10 Record build success or failure on buildable objects
Summary:
Fixes T4810. When a buildable completes, make an effort to update the corresponding object with a success or failure message. Commits don't support this yet, but revisions do.

{F144614}

Test Plan:
  - Used `bin/harbormaster build` and `bin/harbormaster update` to run a pile of builds.
  - Tried good/bad builds.
  - Sent some normal mail to make sure the mail reentrancy change didn't break stuff.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4810

Differential Revision: https://secure.phabricator.com/D8803
2014-04-17 16:04:14 -07:00
epriestley
b03912213f Add original mail headers to the bottom of error mail
Summary: From IRC, this is sometimes helpful for debugging if there's a mailing list issue or something like that. For example, it can show "To" and "Cc".

Test Plan: Got some email, saw headers in it.

Reviewers: dctrwatson, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8708
2014-04-06 07:57:03 -07:00
epriestley
c8cf7bb506 Simplify some more older mail error handling code
Summary: Ref T4371. We can reuse more code for this "your stuff is empty" error, now, and benefit from global rate limiting and being able to reply to arbitrary addresses.

Test Plan: Sent valid, empty, and empty-ignored email via `mail_handler.php`, got appropriate actions/errors/states.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4371

Differential Revision: https://secure.phabricator.com/D8701
2014-04-04 11:14:33 -07:00
epriestley
d9cdbdb9fa When we fail to process mail, tell the user about it
Summary:
Ref T4371. Ref T4699. Fixes T3994.

Currently, we're very conservative about sending errors back to users. A concern I had about this was that mistakes could lead to email loops, massive amounts of email spam, etc. Because of this, I was pretty hesitant about replying to email with more email when I wrote this stuff.

However, this was a long time ago. We now have Message-ID deduplication, "X-Phabricator-Sent-This-Mail", generally better mail infrastructure, and rate limiting. Together, these mechanisms should reasonably prevent anything crazy (primarily, infinite email loops) from happening.

Thus:

  - When we hit any processing error after receiving a mail, try to send the author a reply with details about what went wrong. These are limited to 6 per hour per address.
  - Rewrite most of the errors to be more detailed and informative.
  - Rewrite most of the errors in a user-facing voice ("You sent this mail..." instead of "This mail was sent..").
  - Remove the redundant, less sophisticated code which does something similar in Differential.

Test Plan:
  - Using `scripts/mail/mail_receiver.php`, artificially received a pile of mail.
  - Hit a bunch of different errors.
  - Saw reasonable error mail get sent to me.
  - Saw other reasonable error mail get rate limited.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3994, T4371, T4699

Differential Revision: https://secure.phabricator.com/D8692
2014-04-03 18:43:18 -07:00
Ben Alpert
133772d7eb Set from name when $can_send_as_user is true
Test Plan: Crossed fingers.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D8694
2014-04-03 17:51:23 -07:00
Joshua Spence
e11adc4ad7 Added some additional assertion methods.
Summary:
There are quite a few tests in Arcanist, libphutil and Phabricator that do something similar to `$this->assertEqual(false, ...)` or `$this->assertEqual(true, ...)`.

This is unnecessarily verbose and it would be cleaner if we had `assertFalse` and `assertTrue` methods.

Test Plan: I contemplated adding a unit test for the `getCallerInfo` method but wasn't sure if it was required / where it should live.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8460
2014-03-08 19:16:21 -08:00
epriestley
eca7d3feda Expand aggregate email recipients prior to multiplexing
Summary:
Ref T4361. Before we figure out which To/CC are addressable, try to expand To/CC. Specifically, the supported expansion right now is project PHIDs expanding to all their members.

Because of the way multiplexing works, we have to do this in two places: explicitly in `multiplexMail()`, and when sending mail that wasn't multiplexed. This is messy; eventually we can get rid of it (after ApplicationTransactions are everywhere).

This has some rough edges, but should basically give us what we need to make stuff like projects mailable. Particularly, it deals with most issues in D7436:

  - I got around the resolution/multiplexing issue by resolving aggregate mailables separately from mailable actors.
  - We get to keep the Project PHID as a To/CC/Reviewer/Whatever until the last second.
  - Users won't get two emails for being a CC and also a member of a CC'd project.
  - We can degrade to the list stuff this way if we want, by having the project aggregate yield a single list PHID.

Test Plan: Added a comment to a revision with a project reviewer, got mail to all the project's members.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4361

Differential Revision: https://secure.phabricator.com/D8117
2014-02-01 14:35:55 -08:00
epriestley
4b0ef353e4 Remove retry/failure mechanisms from MetaMTA
Summary:
Fixes T4202. We have old code in MetaMTA which implements gradual backoff and maximum retries.

However, we have more general code in the task queue which does this, too. We can just use the more general stuff in the task queue; it obsoletes the specific stuff in MetaMTA, which is more complex and ran into some kind of issue in T4202.

Remove `retryCount`, `nextRetry` (obsoleted by task queue retry mechanisms) and "simulated failures" (no longer in use).

Generally, modern infrastructure has replaced these mechanisms with more general ones.

Test Plan:
  - Sent mail.
  - Observed unsendable mail failing in reasonable ways in the queue.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4202

Differential Revision: https://secure.phabricator.com/D8115
2014-02-01 14:35:42 -08:00
epriestley
e4deb7faad Remove metamta.send-immediately
Summary:
Ref T3857.

  - Always send mail via daemons. This lets us get rid of this config, and is generally much more performant.
  - After D7964, we warn if daemons aren't running.

Test Plan: Sent some mail.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3857

Differential Revision: https://secure.phabricator.com/D7965
2014-01-14 13:22:47 -08:00
Bob Trahan
d0127f95e5 Maniphest - add support for !assign command
Summary:
also try to centralize some of the command parsing logic. note that differential is still an exception here. it uses a whitelist-style regex. i think long-term we should have this for every app but changing it seemed too big for this diff.

Fixes T3937.

Test Plan:
echo '!assign btrahan' | ./bin/mail receive-test --as xerxes --to T22 ; echo '!claim' | ./bin/mail receive-test --as xerxes --to T22

unit tests passed, though my new one is silly

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3937

Differential Revision: https://secure.phabricator.com/D7307
2013-10-14 12:29:41 -07:00
Bob Trahan
db71bf6128 Fix issue reported from github
Summary:
we filter the $actors above such that its possible to have no $actor anymore (if $actor is not a deliverable email address). ergo, make sure we have actor before we start calling methods.

Fixes github issue 403

Test Plan: logic on this one - not 100% sure how to easily reproduce

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7284
2013-10-10 15:17:37 -07:00
epriestley
6d45a2e09b Restore some missing features from Maniphest mail
Summary:
Ref T2217. Fixes two issues:

  # The "task created" email didn't include the task description, but should.
  # We were treaging the "status" event as the "create", but that's kind of a mess. Treat the "title" event as the "create" instead. This makes initial emails say "[Created]".

Test Plan: Created some tasks, got better emails.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7115
2013-09-25 11:16:55 -07:00
epriestley
cf0bf34255 Allow MetaMTA adapters to indicate that a mail is permanently undeliverable
Summary: Currently, adapters can only fail mail temporarily. Allow them to indicate a permanent failure by throwing a special exception.

Test Plan: Added and ran unit tests.

Reviewers: wez, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6847
2013-08-30 08:21:50 -07:00
epriestley
293a475e39 Show why recipients were excluded from mail
Summary:
Ref T3306. This interface has a hard time balancing security/policy issues and I'm not sure what the best way forward is. Some possibilities:

  # We just let you see everything from the web UI.
    - This makes debugging easier.
    - Anyone who can see this stuff can trivially take over any user's account with five seconds of work and no technical expertise (reset their password from the web UI, then go read the email and click the link).
  # We let you see everything, but only for messages you were a recipient of or author of.
    - This makes it much more difficult to debug issues with mailing lists.
      - But maybe we could just say mailing list recipients are "public", or define some other ruleset.
    - Generally this gets privacy and ease of use right.
  # We could move the whole thing to the CLI.
    - Makes the UI/UX way worse.
  # We could strike an awkward balance between concerns, as we do now.
    - We expose //who// sent and received messages, but not the content of the messages. This doesn't feel great.

I'm inclined to probably go with (2) and figure something out for mailing lists?

Anyway, irrespective of that this should generally make things more clear, and improves the code a lot if nothing else.

Test Plan:
{F49546}

  - Looked at a bunch of mail.
  - Sent mail from different apps.
  - Checked that recipients seem correct.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3306

Differential Revision: https://secure.phabricator.com/D6413
2013-07-10 15:17:38 -07:00
Bryan Cuccioli
d78386584f Globally limit the size of generated emails.
Summary: At the global level, truncate emails at a user-configured size.

Test Plan: Untested, as I could not get PHP to send emails on my box, but if you can this should be very easy to test. Just set the max size to something like .001 kilobytes and make sure it does the right thing.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T1392

Differential Revision: https://secure.phabricator.com/D6118
2013-06-05 05:47:31 -07:00
epriestley
5cd13c3c65 Remove the last hardcoding from PhabricatorMetaMTAReceivedMail
Summary:
Moves all remaining mail handling into ReplyHandlers.

Farewell, `getPhabricatorToInformation()`! You were a bad method and no one liked you.

Ref T1205.

Test Plan:
  - Used test console to send mail to Revisions, Tasks, Conpherences and Commits (these all actually work).
  - Used test console to send mail to Requests, Macros, Questions and Mocks (these accept the mail but don't do anything with it, but didn't do anything before either).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5953
2013-05-17 10:00:49 -07:00
epriestley
c967141f92 Make "new task" and "new conpherence" not-so-awful
Summary: Ref T1205. Moves the handling logic for these email types to reply handlers.

Test Plan: Used test form to send conpherence and maniphest mail.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5945
2013-05-17 03:51:57 -07:00
epriestley
2676e91dd8 Move sender validation into MailReceiver classes
Summary:
Ref T1205. Finally able to delete a big chunk of this nastiness.

Make MailReceivers responsible for validating senders. For object creation receivers (bugs, conpherences) this just means that users must not be disabled. For other receivers the senders must be able to see the objects, have the right hashes, etc., according to policy.

Test Plan: Added a bunch of test cases (everything except policy). Verified behavior via the Receive test console.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5943
2013-05-17 03:49:29 -07:00
epriestley
5243b0d653 Move computeMailHash() to PhabricatorObjectMailReceiver
Summary: Kick this out of here. Ref T1205.

Test Plan: Grep.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5942
2013-05-17 03:49:00 -07:00
epriestley
ff4073c2f4 Copy mail sender selection logic into MailReceivers
Summary: Copies sender identification logic into MailReceivers and makes it basically sane. The mess we run into after this try/catch is terrifying so I'm avoiding actually getting rid of any of it quite yet. Ref T1205.

Test Plan: Added a bit of test coverage. Used Receiver test console to verify some additional behaviors.

Reviewers: btrahan

Reviewed By: btrahan

CC: Afaque_Hussain, aran

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5931
2013-05-15 08:44:54 -07:00
Afaque Hussain
cab28cc91d Adding Email Reply support for External Users.
Summary: Adding Email reply support for external users.

Test Plan: Please let me know if I have approached it correctly. Had few doubts. Will proceed after your comments :)

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan, jennis.mekwan3

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5912
2013-05-14 15:42:14 -07:00
epriestley
341079c3cf Move some received mail responsibility to applications
Summary:
Ref T1205. Continuation of D5915.

Currently, `PhabricatorMetaMTAReceivedMail` has //all// the logic for routing mail. In particular:

  - New mail receivers in applications must edit it.
  - Mail receivers don't drop out when applications are uninstalled.

Applications have some logic in subclasses of `PhabricatorMailReplyHandler`, but this class is a bit of a mess. It is also heavily based on the assumption that mail receivers are objects (like revisions), but this is not true in at least two cases today (creating new tasks with `bugs@`, creating a new Conpherence thread) and likely other cases in the future (e.g., revision-by-mail).

Move this logic into a new `PhabricatorMailReceiver` classtree. This is similar to `PhabricatorMailReplyHandler` but a bit cleaner and more general. I plan to heavily reduce the responsibilities of `PhabricatorMailReplyHandler` or possibly eliminate it entirely.

For now, the new classtree doesn't do much of interest. The only behavioral change this diff causes is that Phabricator will now reject mail to an application when that application is uninstalled.

I also moved all the `ReplyHandler` classes into `mail/` directories in their respective applications.

Test Plan: Unit tests, used receive test to route mail to various objects.

Reviewers: btrahan

Reviewed By: btrahan

CC: Afaque_Hussain, edward, aran

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5922
2013-05-14 10:57:41 -07:00
epriestley
eabe3a4d33 Begin improving the soundness of received mail
Summary:
We/I broke a couple of things here recently (see D5911) and are doing some work here in general (see D5912, etc.).

Generally, this code is pretty oldschool and not especially well architected for modern application-oriented Phabricator. It hardcodes a lot of stuff which should be applications' responsibilites.

Take the first steps toward making it more solid to reduce the risk here. In particular:

  - Factor out the "self mail" and "duplicate mail" checks and add unit tests.
  - Make Message-ID hash handling automatic.

Test Plan: Ran unit tests.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5915
2013-05-13 16:32:19 -07:00
epriestley
9e6da42206 Fix some issues with Maniphest inbound email
Summary:
Fixes T3181.

  - Inbound `bugs@` mail is broken right now if it doesn't use the new external user stuff, because it calls `$user->getPhabricatorUser()` on an object which is already a `PhabricatorUser`. Instead, build the right `$user` object from the external user earlier on.
  - Maniphest mail is nuking or otherwise awkwardly altering CCs. Make this work properly.
  - Make sure "!unsubscribe" works correctly.

Test Plan: Sent `bugs@` mail. Sent CC mail. Sent "!unsubscribe" mail.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran, tido

Maniphest Tasks: T3181

Differential Revision: https://secure.phabricator.com/D5911
2013-05-13 08:10:02 -07:00
Afaque Hussain
6cd4a564dc Enabling Maniphest to send email to External Users.
Summary: Maniphest sends email to External users.

Test Plan:
{F42649}
It seems that maniphest tries to send an email, my install is not configured to deliver email.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5856
2013-05-12 19:22:39 -07:00
Afaque Hussain
f76ecd9e76 Adding Handle Suport
Summary: Added PHID_TYPE_XUSR to loadObjectsofType() of PhabricatorObjectHandle data. Was trying to solve the setActor() issue. Created a getPhabricator method in PhabricatorExternalAccount. When I try to set authorPHID using setAuthorPHID() in PhabricatorExternalAccount it's saying bad setter call, if I don't, then it's saying authorPHID can't be null. Sending you diff for comments on this.

Test Plan:
{F42523}

{F42525}

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5839
2013-05-06 11:34:48 -07:00
Zedstar
f53cde8f92 Using PhabricatorExternalAccount
Summary: Using PhabricatorExternalAccount in place maniphest.default-public-author.

Test Plan:
Using receivemail to see if the a new entry is made in the 'phabircator_user.user_externalaccount' table. Few things, I noticed that phabricator creates table 'user_externalaccout'. And now it throws up error 'Unknown column 'dateCreated' in 'field list''. Awaiting your comments.
{F41370}

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T1205

Differential Revision: https://secure.phabricator.com/D5747
2013-04-28 13:22:33 -07:00
epriestley
e80c59cbc6 Introduce basic bin/mail with a resend workflow
Summary:
Fixes T2458. Ref T2843. @tido's email from T2843 has exhausted its retries and failed, but we want to try it again with the patch from D5464 to capture the actual error. This sort of thing has come up a few times in debugging, too.

Also fixed some stuff that came up while debugging this.

Test Plan:
  - Ran command with no args.
  - Ran resend with no args.
  - Ran resend with bad IDs.
  - Ran resend with already-queued messages, got "already queued" error.
  - Ran resend with already-sent message, got requeue.

Reviewers: btrahan, tido

Reviewed By: tido

CC: aran

Maniphest Tasks: T2458, T2843

Differential Revision: https://secure.phabricator.com/D5493
2013-03-30 15:53:49 -07:00
epriestley
eca49cb91f Exclusion of actor as a mail recipient per preference can be overridden
Summary:
Fixes T2797

Currently, emails are sent in no case to the actor if he has disabled the preference to send emails for his own actions. This won't let us send any test emails from the MetaMTA application to ourselves.

This revision corrects the latter by specifically overriding the option for test emails.

Test Plan:
tried to send emails to myself. verified test mail got blocked with change not applied.

applied the change. verified new change with both preference enabled and disabled did work fine.

Reviewers: epriestley

Reviewed By: epriestley

CC: kai, aran, Korvin

Maniphest Tasks: T2797

Differential Revision: https://secure.phabricator.com/D5398
2013-03-20 15:50:02 -07:00
epriestley
a22bea2a74 Apply lint rules to Phabricator
Summary: Mostly applies a new call spacing rule; also a few things that have slipped through via pull requests and such

Test Plan: `find src/ -type f -name '*.php' | xargs -n16 arc lint --output summary --apply-patches`

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D5002
2013-02-19 13:33:10 -08:00
Bob Trahan
275f708f14 Conpherence - make messages to btrahan@metamta.domain start conpherences
Summary: I'm not super happy with the prettiness of the code, but I wasn't able to come up with a good way to clean it up. Happy for suggestions.

Test Plan: sent message to btrahan@phabricator.dev from gmail. Copied raw email and piped it to mail_handler.php -- it created a conpherence! Repeated but sent to btrahan and xerxes and noted that the conpherence was created for both users

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2431

Differential Revision: https://secure.phabricator.com/D4854
2013-02-07 15:17:11 -08:00
Bob Trahan
3f91559b8a fix mail handler for conpherence try 2
Summary: the editor methods are protected so just build a reply handler directly

Test Plan: push it live and try again

Reviewers: epriestley, chad

Reviewed By: chad

CC: aran, Korvin

Maniphest Tasks: T2399

Differential Revision: https://secure.phabricator.com/D4670
2013-01-26 10:49:20 -08:00
Bob Trahan
3a4e0045be make conpherence reply handler work
Summary: it doesn't work right now. updating this regex should fix it methinks

Test Plan: gotta push it live

Reviewers: chad, epriestley

Reviewed By: chad

CC: aran, Korvin

Maniphest Tasks: T2399

Differential Revision: https://secure.phabricator.com/D4663
2013-01-26 09:14:19 -08:00
epriestley
f6622f43e6 Replace all array_combine(x, x) with array_fuse(x) in Phabricator
Summary: Fixes various array_combine() warnings for PHP < 5.4

Test Plan: lint/unit/grep

Reviewers: btrahan, vrana, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4660
2013-01-25 17:06:55 -08:00
Bob Trahan
4d22c9104f Conpherence - reply handler integration
Summary: Added a reply handler. A few problems -- first, I can't seem to get this to actually send me email so I haven't been able to reply (which I would have done by generating a reply, then copying the raw email into scripts/mail_handler.php). Second, the subject is often terrible on these emails -- unless the conpherence is named its something gross like "E4:" Third, on create I am noticing an error on array_combine() which I think is related to the need to write array_combine_not_broken or what have you I saw go by... (PhabricatorTransactionEditor does array_combine(xaction->getOldValue(), xaction->getOldValue()) and complains that the arrays are empty)

Test Plan: noted that /mail/ said mails were being sent

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T2399

Differential Revision: https://secure.phabricator.com/D4656
2013-01-25 16:03:54 -08:00
Bob Trahan
9d23a49c91 random cleanup from long ago
Summary: was doing some work in here and noticed this old crap lying around. T547 was last updated in December 2011 so I think its okay to delete these old mail hashes now.

Test Plan: careful code inspection, though I will be testing mail like whoa for the rest of the day probably

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T547

Differential Revision: https://secure.phabricator.com/D4650
2013-01-25 13:11:20 -08:00
Nick Pellegrino
3e6fa43658 getConfigEnv fails fast when key is not found and no default value is given.
Summary:
T2345
getConfig throws an Exception when the key does not exist.
Also removes dead code that throws an Exception.

Test Plan:
Reloaded the Phabricator home page.  In the process, found
2 Exceptions thrown due to nonexistent keys.  After addressing these problems,
the home page loads without Exceptions.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4541
2013-01-19 12:11:28 -08:00
epriestley
7752717125 Fix a MetaMTA array_combine() warning
Summary:
Fixes T2290. Older versions of PHP (prior to PHP 5.4) raised a warning if you tried to combine empty arrays. (Newer versions don't, which is why I missed this in testing, although I may also not have tried sending empty mail.)

If mail has no recipients, we reach this with an empty array. Just skip the function body and return immediately, the result is empty array.

(You can get mail with no recipients in various valid ways, currently by, e.g., commenting on a Macro with no subscribers.)

Test Plan: Sent mail with zero, nonzero recipients. Received the nonzero recipient mail. Verified on php.net that this is a version issue.

Reviewers: codeblock, btrahan

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2290

Differential Revision: https://secure.phabricator.com/D4360
2013-01-08 10:39:49 -08:00
Bob Trahan
3448781c40 de-duplicate emails received by phabricator multiple times
Summary: this can happen if you have Phabricator and email lists co-mingling such that Phabricator receives an email multiple times. we can prevent this from then spamming everyone or otherwise taking the action multiple times by storing a message id hash and dropping the message if we have more than one message that matches.

Test Plan: simulated sending the same email multiple times on the command line. noted only the first one made it through.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1726

Differential Revision: https://secure.phabricator.com/D4328
2013-01-03 17:04:30 -08:00
Bob Trahan
06384f5f87 fix mail handling error -- return empty array if we have no $addresses
Test Plan: pushing it live to test
2012-11-16 06:57:15 -08:00
Bob Trahan
bd1bc6d71a fix error where blank email addresses were sneaking in the stack
Summary: array_filter to the rescue

Test Plan: mostly lots of reasoning (as opposed to making a fake user with a blank email address to reproduce)

Reviewers: epriestley, chad

Reviewed By: chad

CC: aran, Korvin

Maniphest Tasks: T2052

Differential Revision: https://secure.phabricator.com/D3927
2012-11-08 12:21:50 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
Bob Trahan
afae26ad94 robustify Differential and Maniphest mailhandlers wrt attachments
Summary:
a few things

- make the parent mailhandler class not send "blank body" error if you have attachments
- make both differential and maniphest append a list of attachments to the body if any exist
- BONUS - made the cc stuff work in Maniphest

Test Plan: I haven't actually tested this yet. :(  i need to figure out how to send a mail with an attachment from the command-line and figured I'd serve this up first.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2012

Differential Revision: https://secure.phabricator.com/D3868
2012-11-01 15:18:06 -07:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
epriestley
1a8232f4c9 Fix an issue where excluded recipients are not respected
Summary: I broke this in D3778. We modify `$parameters` and then ignore it in favor of `$params` for the rest of the method. Unit tests work great since they're one level below this.

Test Plan: Verified "Send email about my own actions" behaved correctly.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3796
2012-10-23 10:48:03 -07:00
epriestley
90ccfe4da1 Clean up and expose filtered recipient lists for PhabricatorMetaMTAMail
Summary: Provide a public interface to get all the filtered recipients of an email. The intent is to pass this along to Notifications so it can mark notifications as read if the user is also receiving an email, possibly based on some preference (see T1403). This also simplifies the enormous sendNow() method a little bit.

Test Plan: Added unit tests, and sent a few mails that should cover most/all of these cases. They appeared to produce the correct recipients.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1403

Differential Revision: https://secure.phabricator.com/D3778
2012-10-22 16:25:43 -07:00
Bob Trahan
d9c6e07f2c If users are on the email to Phabricator, do not send them the Phabricator reply.
Summary: When we receive an email, figure out if any of the other tos and ccs are users. If they are, pass their phids through the stach as "exclude phids" and exclude them from getting the email.

Test Plan: used the various applications (audit, differential, maniphest) and noted emails were sent as expected.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: aran, Korvin, vrana

Maniphest Tasks: T1676

Differential Revision: https://secure.phabricator.com/D3645
2012-10-10 10:18:23 -07:00
Bob Trahan
a754fdfca3 Create metamta.user-address-format configuration option
Summary: this lets users specify what "name" to use in email addresses

Test Plan: changed the conf setting in my local instance and used phabricator. observed name format changes

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1862

Differential Revision: https://secure.phabricator.com/D3639
2012-10-05 11:27:58 -07:00
Bob Trahan
cdfc71ced5 Only send the "this is blank silly" error message email if the email is sent *just* to Phabricator.
Summary: said differently, if the user included another to address or one or more cc's, don't send the error message email.

Test Plan: played around in the metamta test console and verified that blank replies generated the error handler.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1643

Differential Revision: https://secure.phabricator.com/D3345
2012-08-28 14:09:37 -07:00
epriestley
21ebd1a609 Fix an issue with To/CC placeholders
Summary: Currently, if no placeholder is configured we always move "Cc" up to "To", even if we have a valid "To". Instead, move "Cc" to "To" only if there's no "To" and no placeholder.

Test Plan: Sent email with "to" and "cc", email with "cc" only with a placeholder, and email with "cc" only without a placeholder. Verified recipients ended up in the right location in all cases.

Reviewers: nh, btrahan

Reviewed By: btrahan

CC: klimek, aran

Differential Revision: https://secure.phabricator.com/D3342
2012-08-20 14:08:45 -07:00
epriestley
74b438db13 Move "Mailing Lists" to a separate application
Summary:
There's currently no way to get here from the UI since nav tabs don't exist anymore. It's also always been hard to find this feature even when we had the tabs, since it's surprising that it's inside "MetaMTA".

  - Move mailing lists to a separate application.
  - Add `buildApplicationPage()`, since we don't really need `buildStandardPageResponse()` any more -- we can infer all the information from `PhabricatorApplication`. This will let us get rid of a lot of the `PhabricatorXXXController` classes which just define application information.
  - Add `getApplicationURI()` to reduce code duplication, and in case we want to let you move applications around some day.

Test Plan: Looked/edited/saved mailing lists.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T631

Differential Revision: https://secure.phabricator.com/D3248
2012-08-12 19:19:46 -07:00
epriestley
59db282624 Fix an issue with email "Re:" prefix
Summary: See detailed discussion in T1543.

Test Plan:
  - Enabled multiplexing.
  - Set user A to "enable Re".
  - Created a task owned by user B with user A cc'd.
  - Verified A got no "Re:" before this patch.
  - Applied patch.
  - Verified A got "Re:" after this patch.

Reviewers: nh, btrahan, vrana

Reviewed By: nh

CC: aran

Maniphest Tasks: T1543

Differential Revision: https://secure.phabricator.com/D3031
2012-07-23 09:23:54 -07:00
Nick Harper
67c302ae4f Send messages with only a CC
Summary: This keeps people in the correct To or CC field on multiplexed messages.

Test Plan:
with multiplexing on, checked that I received an email with me in the CC
field instead of the To field for a diff I'm CC'd on.

Reviewers: epriestley, jungejason, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2999
2012-07-18 11:45:46 -07:00
epriestley
2b690372de Fix several issues with Differential exception email
Summary:
  - Assigning $cc_phids clobbers the correct value assigned on line 80.
  - Remove stack trace noise, the trace is always meaningless and well-known.
  - Actually show the original body.

Test Plan: Piped mail to the mail receiver and verified the errors didn't CC revision CCs, no longer had traces, and included the original raw text body.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2969
2012-07-12 13:33:26 -07:00
epriestley
16c20c92fe Minor, fix an issue where we may fail to select a translation for mail. 2012-07-02 07:17:56 -07:00
vrana
c5c0324e1b Send several X-Differential-CC headers
Summary: People want to create filters checking if they are in CC which is almost impossible with multiplexing in Outlook.

Test Plan: Sent e-mail with multiple CCs, verified headers.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2829
2012-06-22 11:10:23 -07:00
vrana
e84f9f9ec9 Send Differential e-mails in user's language
Summary:
Works this way:

- Select users' language with multiplexing.
- Select default language otherwise (it can be different from current user's language).
- Build body and subject for each user individually.
- Set the original language after sending the mails.

Test Plan:
- Comment on a diff of user with custom translation.
- Set default to a custom translation. Comment on a diff of user with default translation.
- Set default to a default translation. Comment on a diff of user with default translation.

Repeat with/without multiplexing.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D2774
2012-06-18 12:41:09 -07:00
vrana
2793828795 Refactor setting e-mail subjects
Summary:
It seems that Outlook and Mail.app mostly ignores the threading headers and thread primarily by subject.
They are also very picky about the Re: part in the header.
I guess that's because users of these clients often hit Reply when they want to create a new message to the sender of an e-mail.

We need both of these applications to work with the same setting because we don't use multiplexing to prevent sending multiple e-mails to people in lists.
I also believe that the default behavior should just work in most setups.

I've tried several different combinations of putting "Re:" and none of them seems to always work in both clients.

This diff at least adds more abstraction to the code which should prevent copy/paste errors (two fixed by this diff!).

Test Plan: Sent several e-mails with varying subject, verified that they look as before in Outlook and Mail.app.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2709
2012-06-11 19:07:21 -07:00
vrana
6cc196a2e5 Move files in Phabricator one level up
Summary:
- `kill_init.php` said "Moving 1000 files" - I hope that this is not some limit in `FileFinder`.
- [src/infrastructure/celerity] `git mv utils.php map.php; git mv api/utils.php api.php`
- Comment `phutil_libraries` in `.arcconfig` and run `arc liberate`.

NOTE: `arc diff` timed out so I'm pushing it without review.

Test Plan:
/D1234
Browsed around, especially in `applications/repository/worker/commitchangeparser` and `applications/` in general.

Auditors: epriestley

Maniphest Tasks: T1103
2012-06-01 12:32:44 -07:00
vrana
1ebf9186b4 Depend on class autoloading
Test Plan:
Run setup.
/differential/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2612
2012-05-30 16:57:21 -07:00
epriestley
09c8af4de0 Upgrade phabricator to libphutil v2
Summary: Mechanical changes from D2588. No "Class.php" moves yet.

Test Plan: See D2588.

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2591
2012-05-30 14:26:29 -07:00
epriestley
0461cd6e4f Prevent loops in received mail
Summary:
It's currently possible to configure Phabricator to send mail to some address it recognizes as relating to an object.

When we receive mail from Phabricator, drop it unconditionally.

Test Plan: Wrote two emails, one with the header and one without. Piped them to `mail_handler.php`, one was dropped immediately.

Reviewers: btrahan, nh, mikaaay, jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2529
2012-05-22 06:02:05 -07:00
Nick Harper
c2a9a8079f Remove email from handles
Summary:
Since user emails aren't in the user table, we had to do extra data fetching
for handles, and the emails are only used in MetaMTA, so we move the email
code into MetaMTA and remove it from handles.

Test Plan: send test emails

Reviewers: jungejason, vrana, epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2494
2012-05-21 17:37:26 -07:00
epriestley
87207b2f4e Allow users to have multiple email addresses, and verify emails
Summary:
  - Move email to a separate table.
  - Migrate existing email to new storage.
  - Allow users to add and remove email addresses.
  - Allow users to verify email addresses.
  - Allow users to change their primary email address.
  - Convert all the registration/reset/login code to understand these changes.
  - There are a few security considerations here but I think I've addressed them. Principally, it is important to never let a user acquire a verified email address they don't actually own. We ensure this by tightening the scoping of token generation rules to be (user, email) specific.
  - This should have essentially zero impact on Facebook, but may require some minor changes in the registration code -- I don't exactly remember how it is set up.

Not included here (next steps):

  - Allow configuration to restrict email to certain domains.
  - Allow configuration to require validated email.

Test Plan:
This is a fairly extensive, difficult-to-test change.

  - From "Email Addresses" interface:
    - Added new email (verified email verifications sent).
    - Changed primary email (verified old/new notificactions sent).
    - Resent verification emails (verified they sent).
    - Removed email.
    - Tried to add already-owned email.
  - Created new users with "accountadmin". Edited existing users with "accountadmin".
  - Created new users with "add_user.php".
  - Created new users with web interface.
  - Clicked welcome email link, verified it verified email.
  - Reset password.
  - Linked/unlinked oauth accounts.
  - Logged in with oauth account.
  - Logged in with email.
  - Registered with Oauth account.
  - Tried to register with OAuth account with duplicate email.
  - Verified errors for email verification with bad tokens, etc.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2393
2012-05-07 10:29:33 -07:00
epriestley
9531496d66 Add "X-Auto-Response-Suppress" header to all outgoing mail
Summary: This appears to sometimes be effective (for MS clients), and we've seen it in the wild on inbound mail.

Test Plan: Sent myself some mail, verified it had the right header.

Reviewers: btrahan, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T571

Differential Revision: https://secure.phabricator.com/D2241
2012-04-17 06:35:28 -07:00
epriestley
c458768415 Fix various threading issues, particularly in Gmail
Summary:
  - Add an explicit multiplexing option, and enable it by default. This is necessary for Mail.app to coexist with other clients ("Re:" breaks outlook at the very least, and generally sucks in the common case), and allows users with flexible clients to enable subject variance.
  - Add an option for subject line variance. Default to not varying the subject, so mail no longer says [Committed], [Closed], etc. This is so the defaults thread correctly in Gmail (not entirely sure this actually works).
  - Add a preference to enable subject line variance.
  - Unless all mail is multiplexed, don't enable or respect the "Re" or "vary subject" preferences. These are currently shown and respected in non-multiplex cases, which creates inconsistent results.

NOTE: @jungejason @nh @vrana This changes the default behavior (from non-multiplexing to multiplexing), and might break Facebook's integration. You should be able to keep the same behavior by setting the options appropriately, although if you can get the new defaults working they're probably better.

Test Plan:
Send mail from Maniphest, Differential and Audit. Updated preferences. Enabled/disabled multiplexing. Things seem OK?

NOTE: I haven't actually been able to repro the Gmail threading issue so I'm not totally sure what's going on there, maybe it started respecting "Re:" (or always has), but @cpiro and @20after4 both reported it independently. This fixes a bunch of bugs in any case and gives us more conservative set of defaults.

I'll see if I can buff out the Gmail story a bit but every client is basically a giant black box of mystery. :/

Reviewers: btrahan, vrana, jungejason, nh

Reviewed By: btrahan

CC: cpiro, 20after4, aran

Maniphest Tasks: T1097, T847

Differential Revision: https://secure.phabricator.com/D2206
2012-04-12 09:31:03 -07:00
epriestley
23fd936b47 Add some basic signature stripping
Summary: See discussion in T789. Covered the obvious cases, at least. We can refine this as we get a larger sample size.

Test Plan: Unit test coverage.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T789

Differential Revision: https://secure.phabricator.com/D2154
2012-04-08 15:04:12 -07:00
vrana
8813c7be0e Use assert_instances_of() everywhere but Differential and Diffusion
Summary: NOTE: This is not produced by a script so there might be errors. Please review carefully.

Test Plan: Browse around.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2091
2012-04-03 14:53:20 -07:00
vrana
4fba549a99 Use PhabricatorEnv::newObjectFromConfig() wherever possible
Test Plan:
/mail/send/
scripts/aphront/aphrontpath.php /

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1983
2012-03-21 14:57:52 -07:00
epriestley
d0af617818 Add "final" to (almost) everything else
Summary: Last of the big final patches. Left a few debatable classes (12 out of about 400) that I'll deal with individually eventually.

Test Plan: Ran testEverythingImplemented.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T795

Differential Revision: https://secure.phabricator.com/D1881
2012-03-13 16:21:04 -07:00
Nick Harper
d8c601f21b Move functionality of PhabricatorMetaMTADaemon to a worker task
Summary:
This will allow sending mail to be done by task workers. See T750.

Task ID: #

Blame Rev:

Test Plan:
- started taskmaster daemon in test env
- used "send new test message" feature in MetMTA (with send now unchecked)
- confirmed receipt of 1 email
- repeated 2 & 3 with send now checked

Revert Plan:

Tags:

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T388, T750

Differential Revision: https://secure.phabricator.com/D1723
2012-03-01 22:01:55 -08:00
epriestley
94daf63ca5 Add an explicit "this mail came from Phabricator" header
Summary:
See T926. If you want to write a mail rule that, e.g., captures Differential
mail but ignores people replying to it, it's kind of tricky right now. You can
use the 'X-Mail-Transport-Agent' header but that's not obvious and it's not
necessarily stable.

Add a nice, obvious "X-Phabricator-Sent-This-Message" header.

Test Plan: Sent myself some mail, verified the header appeared.

Reviewers: vrana, btrahan, fugalh, jungejason

Reviewed By: jungejason

CC: aran, epriestley

Maniphest Tasks: T926

Differential Revision: https://secure.phabricator.com/D1732
2012-02-29 15:30:56 -08:00
epriestley
d7a7bca85c Enable email for audits
Summary:
When users submit an audit, send email to relevant parties informing them.

Allow email to be replied to. Just basic support so far; no "!raise" stuff and
no threading with the Herald commit notification.

Test Plan: Made comments, got email. Replied to email, got comments.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1698
2012-02-27 12:57:57 -08:00
epriestley
bfea830d09 Add email preferences to receive fewer less-important notifications
Summary:
A few similar requests have come in across several tools and use cases that I
think this does a reasonable job of resolving.

We currently send one email for each update an object receives, but these aren't
always appreciated:

  - Asana does post-commit review via Differential, so the "committed" mails are
useless.
  - Quora wants to make project category edits to bugs without spamming people
attached to them.
  - Some users in general are very sensitive to email volumes, and this gives us
a good way to reduce the volumes without incurring the complexity of
delayed-send-batching.

The technical mechanism is basically:

  - Mail may optionally have "mail tags", which indicate content in the mail
(e.g., "maniphest-priority, maniphest-cc, maniphest-comment" for a mail which
contains a priority change, a CC change, and a comment).
  - If a mail has tags, remove any recipients who have opted out of all the
tags.
  - Some tags can't be opted out of via the UI, so this ensures that important
email is still delivered (e.g., cc + assign + comment is always delivered
because you can't opt out of "assign" or "comment").

Test Plan:
  - Disabled all mail tags in the web UI.
  - Used test console to send myself mail with an opt-outable tag, it was
immediately dropped.
  - Used test console to send myself mail with an opt-outable tag and a custom
tag, it was delivered.
  - Made Differential updates affecting CCs with and without comments, got
appropriate delivery.
  - Made Maniphest updates affecting project, priority and CCs with and without
comments, got appropriate delivery.
  - Verified mail headers in all cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley, moskov

Maniphest Tasks: T616, T855

Differential Revision: https://secure.phabricator.com/D1635
2012-02-17 22:57:07 -08:00
epriestley
ad36865e50 Add optional "Re:" prefix to all threaded mail and allow disabling mail about
your own actions

Summary:
  - Mail.app on Lion has cumbersome threading rules, see T782. Add an option to
stick "Re: " in front of all threaded mail so it behaves. This is horrible, but
apparently the least-horrible option.
  - While I was in there, I added an option for T228.

Test Plan:
  - Sent a bunch of threaded and unthreaded mail with varous "Re:" settings,
seemed to get "Re:" in the right places.
  - Disabled email about my stuff, created a task with just me, got voided mail,
added a CC, got mail to just the CC.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, mkjones

Maniphest Tasks: T228, T782

Differential Revision: https://secure.phabricator.com/D1448
2012-01-18 15:20:50 -08:00
vrana
6472dbe168 Change fileName to filename
Summary: There are lots of callsites to $changeset->getFilename() so it seemed
easier to rename getFileName() to getFilename() even if it includes database
change. Plus I think that getFilename() is better.

Test Plan:
Alter database.
Open revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D1437
2012-01-17 10:50:14 -08:00
epriestley
79205481e6 Minor, fix \n in MetaMTA headers. 2012-01-08 10:26:16 -08:00
epriestley
e45ffda55a Move most remaining sha1() calls to HMAC
Summary:
  - For context, see T547. This is the last (maybe?) in a series of diffs that
moves us off raw sha1() calls in order to make it easier to audit the codebase
for correct use of hash functions.
  - This breaks CSRF tokens. Any open forms will generate an error when
submitted, so maybe upgrade off-peak.
  - We now generate HMAC mail keys but accept MAC or HMAC. In a few months, we
can remove the MAC version.
  - The only remaining callsite is Conduit. We can't use HMAC since Arcanist
would need to know the key. {T550} provides a better solution to this, anyway.

Test Plan:
  - Verified CSRF tokens generate properly.
  - Manually changed CSRF to an incorrect value and got an error.
  - Verified mail generates with a new mail hash.
  - Verified Phabricator accepts both old and new mail hashes.
  - Verified Phabricator rejects bad mail hashes.
  - Checked user log, things look OK.

Reviewers: btrahan, jungejason, benmathews

Reviewed By: btrahan

CC: aran, epriestley, btrahan

Maniphest Tasks: T547

Differential Revision: 1237
2011-12-19 08:56:53 -08:00
epriestley
98c8e150b0 Prevent delivery of email to disabled objects
Summary: See T625. Facebook's REST-based MTA layer had a check for this so I
overlooked it in porting it out. We should not attempt to deliver email to
disabled users.

Test Plan:
Used MetaMTA console to send email to:

  - No users: received "no To" exception.
  - A disabled user: received "all To disabled" exception.
  - A valid user: received email.
  - A valid user and a disabled user: received email to valid user only.

(Note that you can't easily send to disabled users directly since they don't
appear in the typeahead, but you can prefill it and then disable the user by
hitting "Send".)

Reviewers: btrahan, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: skrul, aran, epriestley

Differential Revision: 1120
2011-11-16 11:07:50 -08:00
Marek Sapota
b71a55900a Allow tweaking of Differential mail by using events
Summary: Allow tweaking Differential mail before sending.

Test Plan:
Wrote a listener renaming Differential attachments and it worked without
problems.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, mareksapota, davidreuss

Differential Revision: 1091
2011-11-09 10:13:53 -08:00
David Reuss
b81231b3dc Corrected manipest reference in exception thrown
Summary: This should hopefully kill off the last of these :P

Test Plan: Should be self explanatory

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1043
2011-10-28 08:05:49 -07:00
David Reuss
c20608f066 use correct key looking for a public author of a task
Test Plan:
Used the scripts/mail_handler.php with and without patch and saw
the maniphest task being created with patch applied.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, davidreuss

Differential Revision: 1041
2011-10-28 08:04:25 -07:00
epriestley
4156cf6bd9 Add an optional configuration option to set 'Precedence: bulk' headers on
transactional mail

Summary: See T571. SES refuses to deliver mail with this header and there are
various reports of other issues on the internet so I'm defaulting it to off.

Test Plan: Set config to true, tried to send mail, SES rejected it because of
"Precedence: bulk" header.

Reviewers: bmaurer, ola, jungejason, nh, aran

Reviewed By: aran

CC: aran, epriestley, bmaurer

Differential Revision: 1032
2011-10-23 14:25:13 -07:00
epriestley
9a4bb3901e Allow bugs@ addresses to blanket-accept tasks
Summary: Allow configuration of a default author for bugs@ emails which don't
correspond to a known system user.

Test Plan: Configured a default author, sent some mails from nonsense addresses,
tasks were created.

Reviewers: davidreuss, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley, ide

Differential Revision: 1013
2011-10-20 14:26:19 -07:00
Marek Sapota
5148741ab2 Prevent duplicated emails with send-immedialtely = true and MTA daemon running
Test Plan:
Set 'metamta.send-immediately' to true.  Start up several MTA daemons, without
the patch you'll probably get multiple emails, with the patch you should get
only one.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, mareksapota, epriestley

Differential Revision: 1021
2011-10-19 14:51:28 -07:00
Marek Sapota
fee7184350 Phabricator mail
Test Plan: EMPTY

Reviewers: aran, epriestley

Reviewed By: epriestley

CC: aran, mareksapota, epriestley, jungejason, nh, drnikki

Differential Revision: 1002
2011-10-14 12:12:41 -07:00
epriestley
d1134810d6 Allow insecure mail auth with "Reply-To" header
Summary: Quora wants to handle some moderation tasks with Phabricator, but want
to lower the barrier to entry for the install and let moderators adopt it
gradually. One request is to allow auth rules to be relaxed so we can auth based
on Reply-To to make things easier. This is insecure if configured but not really
a big deal and the patch isn't big or complicated.

Test Plan: Sent a test email with bogus "From" but valid "Reply-To". It was
rejected with this setting off, and allowed with this setting on.

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 842
2011-08-23 14:13:04 -07:00
David Reuss
c236e4ad72 Enable support for a single reply-handler for outbound emails
Summary:
This allows you to configure a single mailbox for all mail sent by phabricator,
so you
can keep a mailaddress like bugs@example.com and don't need a catchall on your
domain/subdomain.

Test Plan:
Enabled and disabled suffix. Saw mails generated have to correct prefix. Also
piped raw mails
into the scripts/mail/mail_handler.php and ensured comments went into
phabricator for both maniphest
and differential.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 815
2011-08-22 10:20:49 +02:00
epriestley
6deddbfc70 Better enforce uniqueness for email delivery
Summary:
@skrul reports receiving multiple copies of notification emails since
@hunterbridges configured some bizarre dystopian email replication factory on
their outbound route. Two fixes:

  - Ensure "To" and "Cc" are unique. Email shouldn't be replicated for "To:
x@y.com, x@y.com" but it's silly that we do this.
  - Remove "To" addresses from "Cc". Email shouldn't be replicated here either,
but we don't really lose anything by accommodating this.

Test Plan:
Sent a mail to the same to/cc, verified I was to'd only and not cc'd when the
mail was delivered.

@hunterbridges, can you apply this patch locally and verify it fixes the issue?
You can test by going to MetaMTA -> Send New Message and sending a message to
yourself as both To and CC.

Reviewed By: skrul
Reviewers: skrul, hunterbridges, jungejason, tuomaspelkonen, aran
CC: aran, skrul, epriestley
Differential Revision: 751
2011-08-01 15:52:51 -07:00
epriestley
f9599f4499 Allow configuration of a task-creation email address
Summary: This lets you configure an email address which will create tasks when
emails are sent to it. It's pretty basic but should get us most of the way
there.
Test Plan: Configured an address and created a task via email. Replied to a task
via email to check that I didn't break that.
Reviewed By: tuomaspelkonen
Reviewers: davidreuss, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, tuomaspelkonen
Differential Revision: 590
2011-07-05 17:17:27 -07:00
epriestley
a15f07cc33 Allow Phabricator to be configured to use a public Reply-To address
Summary:
We already support this (and Facebook uses it) but it is difficult to configure
and you have to write a bunch of code. Instead, provide a simple flag.

See the documentation changes for details, but when this flag is enabled we send
one email with a reply-to like "D2+public+23hf91fh19fh@phabricator.example.com".
Anyone can reply to this, and we figure out who they are based on their "From"
address instead of a unique hash. This is less secure, but a reasonable tradeoff
in many cases.

This also has the advantage over a naive implementation of at least doing object
hash validation.

@jungejason: I don't think this affects Facebook's implementation but this is an
area where we've had problems in the past, so watch out for it when you deploy.
Also note that you must set "metamta.public-replies" to true since Maniphest now
looks for that key specifically before going into public reply mode; it no
longer just tests for a public reply address being generateable (since it can
always generate one now).

Test Plan:
Swapped my local install in and out of public reply mode and commented on
objects. Got expected email behavior. Replied to public and private email
addresses.

Attacked public addresses by using them when the install was configured to
disallow them and by altering the hash and the from address. All this stuff was
rejected.

Reviewed By: jungejason
Reviewers: moskov, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, moskov, jungejason
Differential Revision: 563
2011-07-03 12:31:00 -07:00
epriestley
c915a064a9 Include both the Gmail and "natural" Message-IDs in the "References" header
Summary:
See T251, where gregprice correctly argues that we need both:

  None of the other people on the thread will have seen that message, so it
seems
  like a lot of clients would put the server's message in a new thread. In
  general, I think you want the References: header to mention every ancestor
  message in the thread that you know about, because that's how MUAs keep a
thread
  together in the face of missing some of its messages.

Test Plan:
Sent a reply email locally, got a response with both Message-IDs in
"references".

Reviewed By: rm
Reviewers: gregprice, rm
Commenters: gregprice
CC: aran, gregprice, epriestley, rm
Differential Revision: 499
2011-06-23 10:58:52 -07:00
epriestley
d6bfdf6ce7 Carry "Message-ID" across email replies to prevent Gmail conversation splitting
Summary:
See T251. In Gmail, conversations split if you reply to them and the next email
does not "In-Reply-To" your message ID. When an action is triggered by an email,
carry its Message-ID through the stack and use it for "In-Reply-To" and
"References" on the subsequent message.

Test Plan:
Live-patched phabricator.com and replied to a Maniphest thread in Gmail without
disrupting the thread. Locally replied to Maniphest and Differential threads and
verified Message-ID was carried across the reply boundary.

Reviewed By: rm
Reviewers: tcook, jungejason, aran, tuomaspelkonen, rm
CC: aran, epriestley, rm
Differential Revision: 498
2011-06-22 14:59:40 -07:00