Summary:
Ref T4361. Before we figure out which To/CC are addressable, try to expand To/CC. Specifically, the supported expansion right now is project PHIDs expanding to all their members.
Because of the way multiplexing works, we have to do this in two places: explicitly in `multiplexMail()`, and when sending mail that wasn't multiplexed. This is messy; eventually we can get rid of it (after ApplicationTransactions are everywhere).
This has some rough edges, but should basically give us what we need to make stuff like projects mailable. Particularly, it deals with most issues in D7436:
- I got around the resolution/multiplexing issue by resolving aggregate mailables separately from mailable actors.
- We get to keep the Project PHID as a To/CC/Reviewer/Whatever until the last second.
- Users won't get two emails for being a CC and also a member of a CC'd project.
- We can degrade to the list stuff this way if we want, by having the project aggregate yield a single list PHID.
Test Plan: Added a comment to a revision with a project reviewer, got mail to all the project's members.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4361
Differential Revision: https://secure.phabricator.com/D8117
Summary:
Fixes T4202. We have old code in MetaMTA which implements gradual backoff and maximum retries.
However, we have more general code in the task queue which does this, too. We can just use the more general stuff in the task queue; it obsoletes the specific stuff in MetaMTA, which is more complex and ran into some kind of issue in T4202.
Remove `retryCount`, `nextRetry` (obsoleted by task queue retry mechanisms) and "simulated failures" (no longer in use).
Generally, modern infrastructure has replaced these mechanisms with more general ones.
Test Plan:
- Sent mail.
- Observed unsendable mail failing in reasonable ways in the queue.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4202
Differential Revision: https://secure.phabricator.com/D8115
Summary:
Ref T3857.
- Always send mail via daemons. This lets us get rid of this config, and is generally much more performant.
- After D7964, we warn if daemons aren't running.
Test Plan: Sent some mail.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3857
Differential Revision: https://secure.phabricator.com/D7965
Summary:
also try to centralize some of the command parsing logic. note that differential is still an exception here. it uses a whitelist-style regex. i think long-term we should have this for every app but changing it seemed too big for this diff.
Fixes T3937.
Test Plan:
echo '!assign btrahan' | ./bin/mail receive-test --as xerxes --to T22 ; echo '!claim' | ./bin/mail receive-test --as xerxes --to T22
unit tests passed, though my new one is silly
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T3937
Differential Revision: https://secure.phabricator.com/D7307
Summary:
we filter the $actors above such that its possible to have no $actor anymore (if $actor is not a deliverable email address). ergo, make sure we have actor before we start calling methods.
Fixes github issue 403
Test Plan: logic on this one - not 100% sure how to easily reproduce
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7284
Summary:
Ref T2217. Fixes two issues:
# The "task created" email didn't include the task description, but should.
# We were treaging the "status" event as the "create", but that's kind of a mess. Treat the "title" event as the "create" instead. This makes initial emails say "[Created]".
Test Plan: Created some tasks, got better emails.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T2217
Differential Revision: https://secure.phabricator.com/D7115
Summary: Currently, adapters can only fail mail temporarily. Allow them to indicate a permanent failure by throwing a special exception.
Test Plan: Added and ran unit tests.
Reviewers: wez, btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6847
Summary:
Ref T3306. This interface has a hard time balancing security/policy issues and I'm not sure what the best way forward is. Some possibilities:
# We just let you see everything from the web UI.
- This makes debugging easier.
- Anyone who can see this stuff can trivially take over any user's account with five seconds of work and no technical expertise (reset their password from the web UI, then go read the email and click the link).
# We let you see everything, but only for messages you were a recipient of or author of.
- This makes it much more difficult to debug issues with mailing lists.
- But maybe we could just say mailing list recipients are "public", or define some other ruleset.
- Generally this gets privacy and ease of use right.
# We could move the whole thing to the CLI.
- Makes the UI/UX way worse.
# We could strike an awkward balance between concerns, as we do now.
- We expose //who// sent and received messages, but not the content of the messages. This doesn't feel great.
I'm inclined to probably go with (2) and figure something out for mailing lists?
Anyway, irrespective of that this should generally make things more clear, and improves the code a lot if nothing else.
Test Plan:
{F49546}
- Looked at a bunch of mail.
- Sent mail from different apps.
- Checked that recipients seem correct.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3306
Differential Revision: https://secure.phabricator.com/D6413
Summary: At the global level, truncate emails at a user-configured size.
Test Plan: Untested, as I could not get PHP to send emails on my box, but if you can this should be very easy to test. Just set the max size to something like .001 kilobytes and make sure it does the right thing.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, AnhNhan
Maniphest Tasks: T1392
Differential Revision: https://secure.phabricator.com/D6118
Summary:
Moves all remaining mail handling into ReplyHandlers.
Farewell, `getPhabricatorToInformation()`! You were a bad method and no one liked you.
Ref T1205.
Test Plan:
- Used test console to send mail to Revisions, Tasks, Conpherences and Commits (these all actually work).
- Used test console to send mail to Requests, Macros, Questions and Mocks (these accept the mail but don't do anything with it, but didn't do anything before either).
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5953
Summary: Ref T1205. Moves the handling logic for these email types to reply handlers.
Test Plan: Used test form to send conpherence and maniphest mail.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5945
Summary:
Ref T1205. Finally able to delete a big chunk of this nastiness.
Make MailReceivers responsible for validating senders. For object creation receivers (bugs, conpherences) this just means that users must not be disabled. For other receivers the senders must be able to see the objects, have the right hashes, etc., according to policy.
Test Plan: Added a bunch of test cases (everything except policy). Verified behavior via the Receive test console.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5943
Summary: Copies sender identification logic into MailReceivers and makes it basically sane. The mess we run into after this try/catch is terrifying so I'm avoiding actually getting rid of any of it quite yet. Ref T1205.
Test Plan: Added a bit of test coverage. Used Receiver test console to verify some additional behaviors.
Reviewers: btrahan
Reviewed By: btrahan
CC: Afaque_Hussain, aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5931
Summary: Adding Email reply support for external users.
Test Plan: Please let me know if I have approached it correctly. Had few doubts. Will proceed after your comments :)
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, AnhNhan, jennis.mekwan3
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5912
Summary:
Ref T1205. Continuation of D5915.
Currently, `PhabricatorMetaMTAReceivedMail` has //all// the logic for routing mail. In particular:
- New mail receivers in applications must edit it.
- Mail receivers don't drop out when applications are uninstalled.
Applications have some logic in subclasses of `PhabricatorMailReplyHandler`, but this class is a bit of a mess. It is also heavily based on the assumption that mail receivers are objects (like revisions), but this is not true in at least two cases today (creating new tasks with `bugs@`, creating a new Conpherence thread) and likely other cases in the future (e.g., revision-by-mail).
Move this logic into a new `PhabricatorMailReceiver` classtree. This is similar to `PhabricatorMailReplyHandler` but a bit cleaner and more general. I plan to heavily reduce the responsibilities of `PhabricatorMailReplyHandler` or possibly eliminate it entirely.
For now, the new classtree doesn't do much of interest. The only behavioral change this diff causes is that Phabricator will now reject mail to an application when that application is uninstalled.
I also moved all the `ReplyHandler` classes into `mail/` directories in their respective applications.
Test Plan: Unit tests, used receive test to route mail to various objects.
Reviewers: btrahan
Reviewed By: btrahan
CC: Afaque_Hussain, edward, aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5922
Summary:
We/I broke a couple of things here recently (see D5911) and are doing some work here in general (see D5912, etc.).
Generally, this code is pretty oldschool and not especially well architected for modern application-oriented Phabricator. It hardcodes a lot of stuff which should be applications' responsibilites.
Take the first steps toward making it more solid to reduce the risk here. In particular:
- Factor out the "self mail" and "duplicate mail" checks and add unit tests.
- Make Message-ID hash handling automatic.
Test Plan: Ran unit tests.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D5915
Summary:
Fixes T3181.
- Inbound `bugs@` mail is broken right now if it doesn't use the new external user stuff, because it calls `$user->getPhabricatorUser()` on an object which is already a `PhabricatorUser`. Instead, build the right `$user` object from the external user earlier on.
- Maniphest mail is nuking or otherwise awkwardly altering CCs. Make this work properly.
- Make sure "!unsubscribe" works correctly.
Test Plan: Sent `bugs@` mail. Sent CC mail. Sent "!unsubscribe" mail.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran, tido
Maniphest Tasks: T3181
Differential Revision: https://secure.phabricator.com/D5911
Summary: Maniphest sends email to External users.
Test Plan:
{F42649}
It seems that maniphest tries to send an email, my install is not configured to deliver email.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, AnhNhan
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5856
Summary: Added PHID_TYPE_XUSR to loadObjectsofType() of PhabricatorObjectHandle data. Was trying to solve the setActor() issue. Created a getPhabricator method in PhabricatorExternalAccount. When I try to set authorPHID using setAuthorPHID() in PhabricatorExternalAccount it's saying bad setter call, if I don't, then it's saying authorPHID can't be null. Sending you diff for comments on this.
Test Plan:
{F42523}
{F42525}
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, AnhNhan
Differential Revision: https://secure.phabricator.com/D5839
Summary: Using PhabricatorExternalAccount in place maniphest.default-public-author.
Test Plan:
Using receivemail to see if the a new entry is made in the 'phabircator_user.user_externalaccount' table. Few things, I noticed that phabricator creates table 'user_externalaccout'. And now it throws up error 'Unknown column 'dateCreated' in 'field list''. Awaiting your comments.
{F41370}
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, AnhNhan
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5747
Summary:
Fixes T2458. Ref T2843. @tido's email from T2843 has exhausted its retries and failed, but we want to try it again with the patch from D5464 to capture the actual error. This sort of thing has come up a few times in debugging, too.
Also fixed some stuff that came up while debugging this.
Test Plan:
- Ran command with no args.
- Ran resend with no args.
- Ran resend with bad IDs.
- Ran resend with already-queued messages, got "already queued" error.
- Ran resend with already-sent message, got requeue.
Reviewers: btrahan, tido
Reviewed By: tido
CC: aran
Maniphest Tasks: T2458, T2843
Differential Revision: https://secure.phabricator.com/D5493
Summary:
Fixes T2797
Currently, emails are sent in no case to the actor if he has disabled the preference to send emails for his own actions. This won't let us send any test emails from the MetaMTA application to ourselves.
This revision corrects the latter by specifically overriding the option for test emails.
Test Plan:
tried to send emails to myself. verified test mail got blocked with change not applied.
applied the change. verified new change with both preference enabled and disabled did work fine.
Reviewers: epriestley
Reviewed By: epriestley
CC: kai, aran, Korvin
Maniphest Tasks: T2797
Differential Revision: https://secure.phabricator.com/D5398
Summary: Mostly applies a new call spacing rule; also a few things that have slipped through via pull requests and such
Test Plan: `find src/ -type f -name '*.php' | xargs -n16 arc lint --output summary --apply-patches`
Reviewers: chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D5002
Summary: I'm not super happy with the prettiness of the code, but I wasn't able to come up with a good way to clean it up. Happy for suggestions.
Test Plan: sent message to btrahan@phabricator.dev from gmail. Copied raw email and piped it to mail_handler.php -- it created a conpherence! Repeated but sent to btrahan and xerxes and noted that the conpherence was created for both users
Reviewers: epriestley, chad
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2431
Differential Revision: https://secure.phabricator.com/D4854
Summary: the editor methods are protected so just build a reply handler directly
Test Plan: push it live and try again
Reviewers: epriestley, chad
Reviewed By: chad
CC: aran, Korvin
Maniphest Tasks: T2399
Differential Revision: https://secure.phabricator.com/D4670
Summary: it doesn't work right now. updating this regex should fix it methinks
Test Plan: gotta push it live
Reviewers: chad, epriestley
Reviewed By: chad
CC: aran, Korvin
Maniphest Tasks: T2399
Differential Revision: https://secure.phabricator.com/D4663
Summary: Fixes various array_combine() warnings for PHP < 5.4
Test Plan: lint/unit/grep
Reviewers: btrahan, vrana, chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D4660
Summary: Added a reply handler. A few problems -- first, I can't seem to get this to actually send me email so I haven't been able to reply (which I would have done by generating a reply, then copying the raw email into scripts/mail_handler.php). Second, the subject is often terrible on these emails -- unless the conpherence is named its something gross like "E4:" Third, on create I am noticing an error on array_combine() which I think is related to the need to write array_combine_not_broken or what have you I saw go by... (PhabricatorTransactionEditor does array_combine(xaction->getOldValue(), xaction->getOldValue()) and complains that the arrays are empty)
Test Plan: noted that /mail/ said mails were being sent
Reviewers: epriestley
Reviewed By: epriestley
CC: chad, aran, Korvin
Maniphest Tasks: T2399
Differential Revision: https://secure.phabricator.com/D4656
Summary: was doing some work in here and noticed this old crap lying around. T547 was last updated in December 2011 so I think its okay to delete these old mail hashes now.
Test Plan: careful code inspection, though I will be testing mail like whoa for the rest of the day probably
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T547
Differential Revision: https://secure.phabricator.com/D4650
Summary:
T2345
getConfig throws an Exception when the key does not exist.
Also removes dead code that throws an Exception.
Test Plan:
Reloaded the Phabricator home page. In the process, found
2 Exceptions thrown due to nonexistent keys. After addressing these problems,
the home page loads without Exceptions.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4541
Summary:
Fixes T2290. Older versions of PHP (prior to PHP 5.4) raised a warning if you tried to combine empty arrays. (Newer versions don't, which is why I missed this in testing, although I may also not have tried sending empty mail.)
If mail has no recipients, we reach this with an empty array. Just skip the function body and return immediately, the result is empty array.
(You can get mail with no recipients in various valid ways, currently by, e.g., commenting on a Macro with no subscribers.)
Test Plan: Sent mail with zero, nonzero recipients. Received the nonzero recipient mail. Verified on php.net that this is a version issue.
Reviewers: codeblock, btrahan
Reviewed By: codeblock
CC: aran
Maniphest Tasks: T2290
Differential Revision: https://secure.phabricator.com/D4360
Summary: this can happen if you have Phabricator and email lists co-mingling such that Phabricator receives an email multiple times. we can prevent this from then spamming everyone or otherwise taking the action multiple times by storing a message id hash and dropping the message if we have more than one message that matches.
Test Plan: simulated sending the same email multiple times on the command line. noted only the first one made it through.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T1726
Differential Revision: https://secure.phabricator.com/D4328
Summary: array_filter to the rescue
Test Plan: mostly lots of reasoning (as opposed to making a fake user with a blank email address to reproduce)
Reviewers: epriestley, chad
Reviewed By: chad
CC: aran, Korvin
Maniphest Tasks: T2052
Differential Revision: https://secure.phabricator.com/D3927
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).
We are removing the headers for these reasons:
- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.
This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).
Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.
Reviewers: epriestley, davidrecordon
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2035
Differential Revision: https://secure.phabricator.com/D3886
Summary:
a few things
- make the parent mailhandler class not send "blank body" error if you have attachments
- make both differential and maniphest append a list of attachments to the body if any exist
- BONUS - made the cc stuff work in Maniphest
Test Plan: I haven't actually tested this yet. :( i need to figure out how to send a mail with an attachment from the command-line and figured I'd serve this up first.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2012
Differential Revision: https://secure.phabricator.com/D3868
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:
- Supporting the idea of permanent failure (e.g., after N failures just stop trying).
- Showing the user how fast taskmasters are completing tasks.
- Showing the user how long tasks took to complete.
Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.
Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D3852
Summary: I broke this in D3778. We modify `$parameters` and then ignore it in favor of `$params` for the rest of the method. Unit tests work great since they're one level below this.
Test Plan: Verified "Send email about my own actions" behaved correctly.
Reviewers: vrana, btrahan
Reviewed By: vrana
CC: aran
Differential Revision: https://secure.phabricator.com/D3796
Summary: Provide a public interface to get all the filtered recipients of an email. The intent is to pass this along to Notifications so it can mark notifications as read if the user is also receiving an email, possibly based on some preference (see T1403). This also simplifies the enormous sendNow() method a little bit.
Test Plan: Added unit tests, and sent a few mails that should cover most/all of these cases. They appeared to produce the correct recipients.
Reviewers: btrahan, vrana, nh
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1403
Differential Revision: https://secure.phabricator.com/D3778
Summary: When we receive an email, figure out if any of the other tos and ccs are users. If they are, pass their phids through the stach as "exclude phids" and exclude them from getting the email.
Test Plan: used the various applications (audit, differential, maniphest) and noted emails were sent as expected.
Reviewers: epriestley, vrana
Reviewed By: vrana
CC: aran, Korvin, vrana
Maniphest Tasks: T1676
Differential Revision: https://secure.phabricator.com/D3645
Summary: this lets users specify what "name" to use in email addresses
Test Plan: changed the conf setting in my local instance and used phabricator. observed name format changes
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T1862
Differential Revision: https://secure.phabricator.com/D3639
Summary: said differently, if the user included another to address or one or more cc's, don't send the error message email.
Test Plan: played around in the metamta test console and verified that blank replies generated the error handler.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T1643
Differential Revision: https://secure.phabricator.com/D3345
Summary: Currently, if no placeholder is configured we always move "Cc" up to "To", even if we have a valid "To". Instead, move "Cc" to "To" only if there's no "To" and no placeholder.
Test Plan: Sent email with "to" and "cc", email with "cc" only with a placeholder, and email with "cc" only without a placeholder. Verified recipients ended up in the right location in all cases.
Reviewers: nh, btrahan
Reviewed By: btrahan
CC: klimek, aran
Differential Revision: https://secure.phabricator.com/D3342
Summary:
There's currently no way to get here from the UI since nav tabs don't exist anymore. It's also always been hard to find this feature even when we had the tabs, since it's surprising that it's inside "MetaMTA".
- Move mailing lists to a separate application.
- Add `buildApplicationPage()`, since we don't really need `buildStandardPageResponse()` any more -- we can infer all the information from `PhabricatorApplication`. This will let us get rid of a lot of the `PhabricatorXXXController` classes which just define application information.
- Add `getApplicationURI()` to reduce code duplication, and in case we want to let you move applications around some day.
Test Plan: Looked/edited/saved mailing lists.
Reviewers: btrahan, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T631
Differential Revision: https://secure.phabricator.com/D3248
Summary: See detailed discussion in T1543.
Test Plan:
- Enabled multiplexing.
- Set user A to "enable Re".
- Created a task owned by user B with user A cc'd.
- Verified A got no "Re:" before this patch.
- Applied patch.
- Verified A got "Re:" after this patch.
Reviewers: nh, btrahan, vrana
Reviewed By: nh
CC: aran
Maniphest Tasks: T1543
Differential Revision: https://secure.phabricator.com/D3031
Summary: This keeps people in the correct To or CC field on multiplexed messages.
Test Plan:
with multiplexing on, checked that I received an email with me in the CC
field instead of the To field for a diff I'm CC'd on.
Reviewers: epriestley, jungejason, vrana
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D2999
Summary:
- Assigning $cc_phids clobbers the correct value assigned on line 80.
- Remove stack trace noise, the trace is always meaningless and well-known.
- Actually show the original body.
Test Plan: Piped mail to the mail receiver and verified the errors didn't CC revision CCs, no longer had traces, and included the original raw text body.
Reviewers: btrahan, jungejason
Reviewed By: jungejason
CC: aran
Differential Revision: https://secure.phabricator.com/D2969
Summary: People want to create filters checking if they are in CC which is almost impossible with multiplexing in Outlook.
Test Plan: Sent e-mail with multiple CCs, verified headers.
Reviewers: epriestley, nh
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D2829
