Allow multiple mail receivers to react to an individual email

Summary:
Fixes T7477. Fixes T13066. Currently, inbound mail is processed by the first receiver that matches any "To:" address. "Cc" addresses are ignored.

**To, CC, and Multiple Receivers**

Some users would like to be able to "Cc" addresses like `bugs@` instead of having to "To" the address, which makes perfect sense. That's the driving use case behind T7477.

Since users can To/Cc multiple "create object" or "update object" addresses, I also wanted to make the behavior more general. For example, if you email `bugs@` and also `paste@`, your mail might reasonably make both a Task and a Paste. Is this useful? I'm not sure. But it seems like it's pretty clearly the best match for user intent, and the least-surprising behavior we can have. There's also no good rule for picking which address "wins" when two or more match -- we ended up with "address order", which is pretty arbitrary since "To" and "Cc" are not really ordered fields.

One part of this change is removing `phabricator.allow-email-users`. In practice, this option only controlled whether users were allowed to send mail to "Application Email" addresses with a configured default author, and it's unlikely that we'll expand it since I think the future of external/grey users is Nuance, not richer interaction with Maniphest/Differential/etc. Since this option only made "Default Author" work and "Default Author" is optional, we can simplify behavior by making the rule work like this:

  - If an address specifies a default author, it allows public email.
  - If an address does not, it doesn't.

That's basically how it worked already, except that you could intentionally "break" the behavior by not configuring `phabricator.allow-email-users`. This is a backwards compatility change with possible security implications (it might allow email in that was previously blocked by configuration) that I'll call out in the changelog, but I suspect that no installs are really impacted and this new behavior is generally more intuitive.

A somewhat related change here is that each receiver is allowed to react to each individual email address, instead of firing once. This allows you to configure `bugs-a@` and `bugs-b@` and CC them both and get two tasks. Useful? Maybe not, but seems like the best execution of intent.

**Sender vs Author**

Adjacently, T13066 described an improvement to error handling behavior here: we did not distinguish between "sender" (the user matching the email "From" address) and "actor" (the user we're actually acting as in the application). These are different when you're some internet rando and send to `bugs@`, which has a default author. Then the "sender" is `null` and the "author" is `@bugs-robot` or whatever (some user account you've configured).

This refines "Sender" vs "Author". This is mostly a purity/correctness change, but it means that we won't send random email error messages to `@bugs-robot`.

Since receivers are now allowed to process mail with no "sender" if they have some default "actor" they would rather use instead, it's not an error to send from an invalid address unless nothing processes the mail.

**Other**

This removes the "abundant receivers" error since this is no longer an error.

This always sets "external user" mail recipients to be unverified. As far as I can tell, there's no pathway by which we send them email anyway (before or after this change), although it's possible I'm missing something somewhere.

Test Plan:
I did most of this with `bin/mail receive-test`. I rigged the workflow slightly for some of it since it doesn't support multiple addresses or explicit "CC" and adding either would be a bit tricky.

These could also be tested with `scripts/mail/mail_handler.php`, but I don't currently have the MIME parser extension installed locally after a recent upgrade to Mojave and suspect T13232 makes it tricky to install.

- Ran unit tests, which provide significant coverage of this flow.
- Sent mail to multiple Maniphest application emails, got multiple tasks.
- Sent mail to a Maniphest and a Paste application email, got a task and a paste.
- Sent mail to a task.
  - Saw original email recorded on tasks. This is a behavior particular to tasks.
- Sent mail to a paste.
- Sent mail to a mock.
- Sent mail to a Phame blog post.
- Sent mail to a Legalpad document.
- Sent mail to a Conpherence thread.
- Sent mail to a poll.
- This isn't every type of supported object but it's enough of them that I'm pretty confident I didn't break the whole flow.
- Sent mail to an object I could not view (got an error).
- As a non-user, sent mail to several "create an object..." addresses.
  - Addresses with a default user worked (e.g., created a task).
  - Addresses without a default user did not work.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13066, T7477

Differential Revision: https://secure.phabricator.com/D19952

src/applications/audit/mail/PhabricatorAuditMailReceiver.php

@@ -12,7 +12,7 @@ final class PhabricatorAuditMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)preg_replace('/^COMMIT/', '', $pattern);
+    $id = (int)preg_replace('/^COMMIT/i', '', $pattern);
 
     return id(new DiffusionCommitQuery())
       ->setViewer($viewer)

src/applications/calendar/mail/PhabricatorCalendarEventMailReceiver.php

@@ -13,7 +13,7 @@ final class PhabricatorCalendarEventMailReceiver
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'E');
+    $id = (int)substr($pattern, 1);
 
     return id(new PhabricatorCalendarEventQuery())
       ->setViewer($viewer)

src/applications/config/check/PhabricatorExtraConfigSetupCheck.php

@@ -394,6 +394,10 @@ final class PhabricatorExtraConfigSetupCheck extends PhabricatorSetupCheck {
 
       'metamta.insecure-auth-with-reply-to' => pht(
         'Authenticating users based on "Reply-To" is no longer supported.'),
+
+      'phabricator.allow-email-users' => pht(
+        'Public email is now accepted if the associated address has a '.
+        'default author, and rejected otherwise.'),
     );
 
     return $ancient_config;

src/applications/config/option/PhabricatorCoreConfigOptions.php

@@ -234,14 +234,6 @@ EOREMARKUP
       $this->newOption('phabricator.cache-namespace', 'string', 'phabricator')
         ->setLocked(true)
         ->setDescription(pht('Cache namespace.')),
-      $this->newOption('phabricator.allow-email-users', 'bool', false)
-        ->setBoolOptions(
-          array(
-            pht('Allow'),
-            pht('Disallow'),
-          ))
-        ->setDescription(
-           pht('Allow non-members to interact with tasks over email.')),
       $this->newOption('phabricator.silent', 'bool', false)
         ->setLocked(true)
         ->setBoolOptions(

src/applications/conpherence/mail/ConpherenceThreadMailReceiver.php

@@ -13,7 +13,7 @@ final class ConpherenceThreadMailReceiver
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'Z');
+    $id = (int)substr($pattern, 1);
 
     return id(new ConpherenceThreadQuery())
       ->setViewer($viewer)

src/applications/countdown/mail/PhabricatorCountdownMailReceiver.php

@@ -13,7 +13,7 @@ final class PhabricatorCountdownMailReceiver
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)substr($pattern, 4);
+    $id = (int)substr($pattern, 1);
 
     return id(new PhabricatorCountdownQuery())
       ->setViewer($viewer)

src/applications/differential/mail/DifferentialCreateMailReceiver.php

@@ -9,14 +9,16 @@ final class DifferentialCreateMailReceiver
 
   protected function processReceivedMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
+    PhutilEmailAddress $target) {
+
+    $author = $this->getAuthor();
 
     $attachments = $mail->getAttachments();
     $files = array();
     $errors = array();
     if ($attachments) {
       $files = id(new PhabricatorFileQuery())
-        ->setViewer($sender)
+        ->setViewer($author)
         ->withPHIDs($attachments)
         ->execute();
       foreach ($files as $index => $file) {
@@ -37,7 +39,7 @@ final class DifferentialCreateMailReceiver
         array(
           'diff' => $file->loadFileData(),
         ));
-      $call->setUser($sender);
+      $call->setUser($author);
       try {
         $result = $call->execute();
         $diffs[$file->getName()] = $result['uri'];
@@ -56,7 +58,7 @@ final class DifferentialCreateMailReceiver
         array(
           'diff' => $body,
         ));
-      $call->setUser($sender);
+      $call->setUser($author);
       try {
         $result = $call->execute();
         $diffs[pht('Mail Body')] = $result['uri'];
@@ -108,10 +110,10 @@ final class DifferentialCreateMailReceiver
     }
 
     id(new PhabricatorMetaMTAMail())
-      ->addTos(array($sender->getPHID()))
+      ->addTos(array($author->getPHID()))
       ->setSubject($subject)
       ->setSubjectPrefix($subject_prefix)
-      ->setFrom($sender->getPHID())
+      ->setFrom($author->getPHID())
       ->setBody($body->render())
       ->saveAndSend();
   }

src/applications/differential/mail/DifferentialRevisionMailReceiver.php

@@ -13,7 +13,7 @@ final class DifferentialRevisionMailReceiver
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'D');
+    $id = (int)substr($pattern, 1);
 
     return id(new DifferentialRevisionQuery())
       ->setViewer($viewer)

src/applications/files/mail/FileCreateMailReceiver.php

@@ -9,7 +9,8 @@ final class FileCreateMailReceiver
 
   protected function processReceivedMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
+    PhutilEmailAddress $target) {
+    $author = $this->getAuthor();
 
     $attachment_phids = $mail->getAttachments();
     if (empty($attachment_phids)) {
@@ -21,6 +22,11 @@ final class FileCreateMailReceiver
     $first_phid = head($attachment_phids);
     $mail->setRelatedPHID($first_phid);
 
+    $sender = $this->getSender();
+    if (!$sender) {
+      return;
+    }
+
     $attachment_count = count($attachment_phids);
     if ($attachment_count > 1) {
       $subject = pht('You successfully uploaded %d files.', $attachment_count);

src/applications/files/mail/FileMailReceiver.php

@@ -12,7 +12,7 @@ final class FileMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'F');
+    $id = (int)substr($pattern, 1);
 
     return id(new PhabricatorFileQuery())
       ->setViewer($viewer)

src/applications/legalpad/mail/LegalpadMailReceiver.php

@@ -12,7 +12,7 @@ final class LegalpadMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'L');
+    $id = (int)substr($pattern, 1);
 
     return id(new LegalpadDocumentQuery())
       ->setViewer($viewer)

src/applications/maniphest/mail/ManiphestCreateMailReceiver.php

@@ -9,15 +9,20 @@ final class ManiphestCreateMailReceiver
 
   protected function processReceivedMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
+    PhutilEmailAddress $target) {
 
-    $task = ManiphestTask::initializeNewTask($sender);
+    $author = $this->getAuthor();
-    $task->setOriginalEmailSource($mail->getHeader('From'));
+    $task = ManiphestTask::initializeNewTask($author);
+
+    $from_address = $mail->newFromAddress();
+    if ($from_address) {
+      $task->setOriginalEmailSource((string)$from_address);
+    }
 
     $handler = new ManiphestReplyHandler();
     $handler->setMailReceiver($task);
 
-    $handler->setActor($sender);
+    $handler->setActor($author);
     $handler->setExcludeMailRecipientPHIDs(
       $mail->loadAllRecipientPHIDs());
     if ($this->getApplicationEmail()) {

src/applications/maniphest/mail/ManiphestTaskMailReceiver.php

@@ -12,7 +12,7 @@ final class ManiphestTaskMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'T');
+    $id = (int)substr($pattern, 1);
 
     return id(new ManiphestTaskQuery())
       ->setViewer($viewer)

src/applications/metamta/applicationpanel/PhabricatorMetaMTAApplicationEmailPanel.php

@@ -261,8 +261,12 @@ final class PhabricatorMetaMTAApplicationEmailPanel
           ->setName($config_default)
           ->setLimit(1)
           ->setValue($v_default)
-          ->setCaption(pht(
+          ->setCaption(
-            'Used if the "From:" address does not map to a known account.')));
+            pht(
+              'Used if the "From:" address does not map to a user account. '.
+              'Setting a default author will allow anyone on the public '.
+              'internet to create objects in Phabricator by sending email to '.
+              'this address.')));
 
     if ($is_new) {
       $title = pht('New Address');
@@ -369,9 +373,17 @@ final class PhabricatorMetaMTAApplicationEmailPanel
         $email_space = null;
       }
 
+      $default_author_phid = $email->getDefaultAuthorPHID();
+      if (!$default_author_phid) {
+        $default_author = phutil_tag('em', array(), pht('None'));
+      } else {
+        $default_author = $viewer->renderHandle($default_author_phid);
+      }
+
       $rows[] = array(
         $email_space,
         $email->getAddress(),
+        $default_author,
         $button_edit,
         $button_remove,
       );
@@ -383,11 +395,13 @@ final class PhabricatorMetaMTAApplicationEmailPanel
       array(
         pht('Space'),
         pht('Email'),
+        pht('Default'),
         pht('Edit'),
         pht('Delete'),
       ));
     $table->setColumnClasses(
       array(
+        '',
         '',
         'wide',
         'action',
@@ -398,6 +412,7 @@ final class PhabricatorMetaMTAApplicationEmailPanel
       array(
         PhabricatorSpacesNamespaceQuery::getViewerSpacesExist($viewer),
         true,
+        true,
         $is_edit,
         $is_edit,
       ));

src/applications/metamta/constants/MetaMTAReceivedMailStatus.php

@@ -6,7 +6,6 @@ final class MetaMTAReceivedMailStatus
   const STATUS_DUPLICATE            = 'err:duplicate';
   const STATUS_FROM_PHABRICATOR     = 'err:self';
   const STATUS_NO_RECEIVERS         = 'err:no-receivers';
-  const STATUS_ABUNDANT_RECEIVERS   = 'err:multiple-receivers';
   const STATUS_UNKNOWN_SENDER       = 'err:unknown-sender';
   const STATUS_DISABLED_SENDER      = 'err:disabled-sender';
   const STATUS_NO_PUBLIC_MAIL       = 'err:no-public-mail';
@@ -23,7 +22,6 @@ final class MetaMTAReceivedMailStatus
       self::STATUS_DUPLICATE => pht('Duplicate Message'),
       self::STATUS_FROM_PHABRICATOR => pht('Phabricator Mail'),
       self::STATUS_NO_RECEIVERS => pht('No Receivers'),
-      self::STATUS_ABUNDANT_RECEIVERS => pht('Multiple Receivers'),
       self::STATUS_UNKNOWN_SENDER => pht('Unknown Sender'),
       self::STATUS_DISABLED_SENDER => pht('Disabled Sender'),
       self::STATUS_NO_PUBLIC_MAIL => pht('No Public Mail'),

src/applications/metamta/management/PhabricatorMailManagementReceiveTestWorkflow.php

@@ -33,6 +33,7 @@ final class PhabricatorMailManagementReceiveTestWorkflow
   }
 
   public function execute(PhutilArgumentParser $args) {
+    $viewer = $this->getViewer();
     $console = PhutilConsole::getConsole();
 
     $to = $args->getArg('to');
@@ -95,14 +96,23 @@ final class PhabricatorMailManagementReceiveTestWorkflow
     if (preg_match('/.+@.+/', $to)) {
       $header_content['to'] = $to;
     } else {
+
       // We allow the user to use an object name instead of a real address
       // as a convenience. To build the mail, we build a similar message and
       // look for a receiver which will accept it.
+
+      // In the general case, mail may be processed by multiple receivers,
+      // but mail to objects only ever has one receiver today.
+
       $pseudohash = PhabricatorObjectMailReceiver::computeMailHash('x', 'y');
+
+      $raw_target = $to.'+1+'.$pseudohash;
+      $target = new PhutilEmailAddress($raw_target.'@local.cli');
+
       $pseudomail = id(new PhabricatorMetaMTAReceivedMail())
         ->setHeaders(
           array(
-            'to' => $to.'+1+'.$pseudohash,
+            'to' => $raw_target,
           ));
 
       $receivers = id(new PhutilClassMapQuery())
@@ -112,7 +122,11 @@ final class PhabricatorMailManagementReceiveTestWorkflow
 
       $receiver = null;
       foreach ($receivers as $possible_receiver) {
-        if (!$possible_receiver->canAcceptMail($pseudomail)) {
+        $possible_receiver = id(clone $possible_receiver)
+          ->setViewer($viewer)
+          ->setSender($user);
+
+        if (!$possible_receiver->canAcceptMail($pseudomail, $target)) {
           continue;
         }
         $receiver = $possible_receiver;

src/applications/metamta/query/PhabricatorMetaMTAActorQuery.php

@@ -121,12 +121,12 @@ final class PhabricatorMetaMTAActorQuery extends PhabricatorQuery {
 
       $actor->setEmailAddress($xuser->getAccountID());
 
-      // NOTE: This effectively drops all outbound mail to unrecognized
+      // Circa T7477, it appears that we never intentionally send email to
-      // addresses unless "phabricator.allow-email-users" is set. See T12237
+      // external users (even when they email "bugs@" to create a task).
-      // for context.
+      // Mark these users as unverified so mail to them is always dropped.
-      $allow_key = 'phabricator.allow-email-users';
+      // See also T12237. In the future, we might change this behavior.
-      $allow_value = PhabricatorEnv::getEnvConfig($allow_key);
+
-      $actor->setIsVerified((bool)$allow_value);
+      $actor->setIsVerified(false);
     }
   }
 

src/applications/metamta/receiver/PhabricatorApplicationMailReceiver.php

@@ -3,32 +3,105 @@
 abstract class PhabricatorApplicationMailReceiver
   extends PhabricatorMailReceiver {
 
+  private $applicationEmail;
+  private $emailList;
+  private $author;
+
   abstract protected function newApplication();
 
+  final protected function setApplicationEmail(
+    PhabricatorMetaMTAApplicationEmail $email) {
+    $this->applicationEmail = $email;
+    return $this;
+  }
+
+  final protected function getApplicationEmail() {
+    return $this->applicationEmail;
+  }
+
+  final protected function setAuthor(PhabricatorUser $author) {
+    $this->author = $author;
+    return $this;
+  }
+
+  final protected function getAuthor() {
+    return $this->author;
+  }
+
   final public function isEnabled() {
     return $this->newApplication()->isInstalled();
   }
 
-  final public function canAcceptMail(PhabricatorMetaMTAReceivedMail $mail) {
+  final public function canAcceptMail(
-    $application = $this->newApplication();
+    PhabricatorMetaMTAReceivedMail $mail,
+    PhutilEmailAddress $target) {
+
     $viewer = $this->getViewer();
+    $sender = $this->getSender();
 
-    $application_emails = id(new PhabricatorMetaMTAApplicationEmailQuery())
+    foreach ($this->loadApplicationEmailList() as $application_email) {
-      ->setViewer($viewer)
+      $create_address = $application_email->newAddress();
-      ->withApplicationPHIDs(array($application->getPHID()))
-      ->execute();
 
-    foreach ($mail->newTargetAddresses() as $address) {
+      if (!PhabricatorMailUtil::matchAddresses($create_address, $target)) {
-      foreach ($application_emails as $application_email) {
+        continue;
-        $create_address = $application_email->newAddress();
+      }
-        if (PhabricatorMailUtil::matchAddresses($create_address, $address)) {
+
-          $this->setApplicationEmail($application_email);
+      if ($sender) {
-          return true;
+        $author = $sender;
+      } else {
+        $author_phid = $application_email->getDefaultAuthorPHID();
+
+        // If this mail isn't from a recognized sender and the target address
+        // does not have a default author, we can't accept it, and it's an
+        // error because you tried to send it here.
+
+        // You either need to be sending from a real address or be sending to
+        // an address which accepts mail from the public internet.
+
+        if (!$author_phid) {
+          throw new PhabricatorMetaMTAReceivedMailProcessingException(
+            MetaMTAReceivedMailStatus::STATUS_UNKNOWN_SENDER,
+            pht(
+              'You are sending from an unrecognized email address to '.
+              'an address which does not support public email ("%s").',
+              (string)$target));
+        }
+
+        $author = id(new PhabricatorPeopleQuery())
+          ->setViewer($viewer)
+          ->withPHIDs(array($author_phid))
+          ->executeOne();
+        if (!$author) {
+          throw new Exception(
+            pht(
+              'Application email ("%s") has an invalid default author ("%s").',
+              (string)$create_address,
+              $author_phid));
         }
       }
+
+      $this
+        ->setApplicationEmail($application_email)
+        ->setAuthor($author);
+
+      return true;
     }
 
     return false;
   }
 
+  private function loadApplicationEmailList() {
+    if ($this->emailList === null) {
+      $viewer = $this->getViewer();
+      $application = $this->newApplication();
+
+      $this->emailList = id(new PhabricatorMetaMTAApplicationEmailQuery())
+        ->setViewer($viewer)
+        ->withApplicationPHIDs(array($application->getPHID()))
+        ->execute();
+    }
+
+    return $this->emailList;
+  }
+
 }

src/applications/metamta/receiver/PhabricatorMailReceiver.php

@@ -2,167 +2,40 @@
 
 abstract class PhabricatorMailReceiver extends Phobject {
 
-  private $applicationEmail;
+  private $viewer;
+  private $sender;
 
-  public function setApplicationEmail(
+  final public function setViewer(PhabricatorUser $viewer) {
-    PhabricatorMetaMTAApplicationEmail $email) {
+    $this->viewer = $viewer;
-    $this->applicationEmail = $email;
     return $this;
   }
 
-  public function getApplicationEmail() {
+  final public function getViewer() {
-    return $this->applicationEmail;
+    return $this->viewer;
+  }
+
+  final public function setSender(PhabricatorUser $sender) {
+    $this->sender = $sender;
+    return $this;
+  }
+
+  final public function getSender() {
+    return $this->sender;
   }
 
   abstract public function isEnabled();
-  abstract public function canAcceptMail(PhabricatorMetaMTAReceivedMail $mail);
+  abstract public function canAcceptMail(
+    PhabricatorMetaMTAReceivedMail $mail,
+    PhutilEmailAddress $target);
 
   abstract protected function processReceivedMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender);
+    PhutilEmailAddress $target);
 
   final public function receiveMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
+    PhutilEmailAddress $target) {
-    $this->processReceivedMail($mail, $sender);
+    $this->processReceivedMail($mail, $target);
-  }
-
-  public function getViewer() {
-    return PhabricatorUser::getOmnipotentUser();
-  }
-
-  public function validateSender(
-    PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
-
-    $failure_reason = null;
-    if ($sender->getIsDisabled()) {
-      $failure_reason = pht(
-        'Your account (%s) is disabled, so you can not interact with '.
-        'Phabricator over email.',
-        $sender->getUsername());
-    } else if ($sender->getIsStandardUser()) {
-      if (!$sender->getIsApproved()) {
-        $failure_reason = pht(
-          'Your account (%s) has not been approved yet. You can not interact '.
-          'with Phabricator over email until your account is approved.',
-          $sender->getUsername());
-      } else if (PhabricatorUserEmail::isEmailVerificationRequired() &&
-               !$sender->getIsEmailVerified()) {
-        $failure_reason = pht(
-          'You have not verified the email address for your account (%s). '.
-          'You must verify your email address before you can interact '.
-          'with Phabricator over email.',
-          $sender->getUsername());
-      }
-    }
-
-    if ($failure_reason) {
-      throw new PhabricatorMetaMTAReceivedMailProcessingException(
-        MetaMTAReceivedMailStatus::STATUS_DISABLED_SENDER,
-        $failure_reason);
-    }
-  }
-
-  /**
-   * Identifies the sender's user account for a piece of received mail. Note
-   * that this method does not validate that the sender is who they say they
-   * are, just that they've presented some credential which corresponds to a
-   * recognizable user.
-   */
-  public function loadSender(PhabricatorMetaMTAReceivedMail $mail) {
-    $raw_from = $mail->getHeader('From');
-    $from = self::getRawAddress($raw_from);
-
-    $reasons = array();
-
-    // Try to find a user with this email address.
-    $user = PhabricatorUser::loadOneWithEmailAddress($from);
-    if ($user) {
-      return $user;
-    } else {
-      $reasons[] = pht(
-        'This email was sent from "%s", but that address is not recognized by '.
-        'Phabricator and does not correspond to any known user account.',
-        $raw_from);
-    }
-
-    // If we don't know who this user is, load or create an external user
-    // account for them if we're configured for it.
-    $email_key = 'phabricator.allow-email-users';
-    $allow_email_users = PhabricatorEnv::getEnvConfig($email_key);
-    if ($allow_email_users) {
-      $from_obj = new PhutilEmailAddress($from);
-      $xuser = id(new PhabricatorExternalAccountQuery())
-        ->setViewer($this->getViewer())
-        ->withAccountTypes(array('email'))
-        ->withAccountDomains(array($from_obj->getDomainName(), 'self'))
-        ->withAccountIDs(array($from_obj->getAddress()))
-        ->requireCapabilities(
-          array(
-            PhabricatorPolicyCapability::CAN_VIEW,
-            PhabricatorPolicyCapability::CAN_EDIT,
-          ))
-        ->loadOneOrCreate();
-      return $xuser->getPhabricatorUser();
-    } else {
-      // NOTE: Currently, we'll always drop this mail (since it's headed to
-      // an unverified recipient). See T12237. These details are still useful
-      // because they'll appear in the mail logs and Mail web UI.
-
-      $reasons[] = pht(
-        'Phabricator is also not configured to allow unknown external users '.
-        'to send mail to the system using just an email address.');
-      $reasons[] = pht(
-        'To interact with Phabricator, add this address ("%s") to your '.
-        'account.',
-        $raw_from);
-    }
-
-    if ($this->getApplicationEmail()) {
-      $application_email = $this->getApplicationEmail();
-      $default_user_phid = $application_email->getConfigValue(
-        PhabricatorMetaMTAApplicationEmail::CONFIG_DEFAULT_AUTHOR);
-
-      if ($default_user_phid) {
-        $user = id(new PhabricatorUser())->loadOneWhere(
-          'phid = %s',
-          $default_user_phid);
-        if ($user) {
-          return $user;
-        }
-
-        $reasons[] = pht(
-          'Phabricator is misconfigured: the application email '.
-          '"%s" is set to user "%s", but that user does not exist.',
-          $application_email->getAddress(),
-          $default_user_phid);
-      }
-    }
-
-    $reasons = implode("\n\n", $reasons);
-
-    throw new PhabricatorMetaMTAReceivedMailProcessingException(
-      MetaMTAReceivedMailStatus::STATUS_UNKNOWN_SENDER,
-      $reasons);
-  }
-
-  /**
-   * Reduce an email address to its canonical form. For example, an address
-   * like:
-   *
-   *  "Abraham Lincoln" < ALincoln@example.com >
-   *
-   * ...will be reduced to:
-   *
-   *  alincoln@example.com
-   *
-   * @param   string  Email address in noncanonical form.
-   * @return  string  Canonical email address.
-   */
-  public static function getRawAddress($address) {
-    $address = id(new PhutilEmailAddress($address))->getAddress();
-    return trim(phutil_utf8_strtolower($address));
   }
 
 }

src/applications/metamta/receiver/PhabricatorObjectMailReceiver.php

@@ -29,52 +29,23 @@ abstract class PhabricatorObjectMailReceiver extends PhabricatorMailReceiver {
 
   final protected function processReceivedMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
+    PhutilEmailAddress $target) {
 
-    $object = $this->loadObjectFromMail($mail, $sender);
+    $parts = $this->matchObjectAddress($target);
-    $mail->setRelatedPHID($object->getPHID());
+    if (!$parts) {
-
+      // We should only make it here if we matched already in "canAcceptMail()",
-    $this->processReceivedObjectMail($mail, $object, $sender);
+      // so this is a surprise.
-
+      throw new Exception(
-    return $this;
+        pht(
-  }
+          'Failed to parse object address ("%s") during processing.',
-
+          (string)$target));
-  protected function processReceivedObjectMail(
-    PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorLiskDAO $object,
-    PhabricatorUser $sender) {
-
-    $handler = $this->getTransactionReplyHandler();
-    if ($handler) {
-      return $handler
-        ->setMailReceiver($object)
-        ->setActor($sender)
-        ->setExcludeMailRecipientPHIDs($mail->loadAllRecipientPHIDs())
-        ->processEmail($mail);
     }
 
-    throw new PhutilMethodNotImplementedException();
-  }
-
-  protected function getTransactionReplyHandler() {
-    return null;
-  }
-
-  public function loadMailReceiverObject($pattern, PhabricatorUser $viewer) {
-    return $this->loadObject($pattern, $viewer);
-  }
-
-  public function validateSender(
-    PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
-
-    parent::validateSender($mail, $sender);
-
-    $parts = $this->matchObjectAddressInMail($mail);
     $pattern = $parts['pattern'];
+    $sender = $this->getSender();
 
     try {
-      $object = $this->loadObjectFromMail($mail, $sender);
+      $object = $this->loadObject($pattern, $sender);
     } catch (PhabricatorPolicyException $policy_exception) {
       throw new PhabricatorMetaMTAReceivedMailProcessingException(
         MetaMTAReceivedMailStatus::STATUS_POLICY_PROBLEM,
@@ -95,7 +66,6 @@ abstract class PhabricatorObjectMailReceiver extends PhabricatorMailReceiver {
     }
 
     $sender_identifier = $parts['sender'];
-
     if ($sender_identifier === 'public') {
       if (!PhabricatorEnv::getEnvConfig('metamta.public-replies')) {
         throw new PhabricatorMetaMTAReceivedMailProcessingException(
@@ -136,30 +106,52 @@ abstract class PhabricatorObjectMailReceiver extends PhabricatorMailReceiver {
           'is correct.',
           $pattern));
     }
+
+    $mail->setRelatedPHID($object->getPHID());
+    $this->processReceivedObjectMail($mail, $object, $sender);
+
+    return $this;
   }
 
+  protected function processReceivedObjectMail(
+    PhabricatorMetaMTAReceivedMail $mail,
+    PhabricatorLiskDAO $object,
+    PhabricatorUser $sender) {
 
-  final public function canAcceptMail(PhabricatorMetaMTAReceivedMail $mail) {
+    $handler = $this->getTransactionReplyHandler();
-    if ($this->matchObjectAddressInMail($mail)) {
+    if ($handler) {
-      return true;
+      return $handler
+        ->setMailReceiver($object)
+        ->setActor($sender)
+        ->setExcludeMailRecipientPHIDs($mail->loadAllRecipientPHIDs())
+        ->processEmail($mail);
     }
 
-    return false;
+    throw new PhutilMethodNotImplementedException();
   }
 
-  private function matchObjectAddressInMail(
+  protected function getTransactionReplyHandler() {
-    PhabricatorMetaMTAReceivedMail $mail) {
-
-    foreach ($mail->newTargetAddresses() as $address) {
-      $parts = $this->matchObjectAddress($address);
-      if ($parts) {
-        return $parts;
-      }
-    }
-
     return null;
   }
 
+  public function loadMailReceiverObject($pattern, PhabricatorUser $viewer) {
+    return $this->loadObject($pattern, $viewer);
+  }
+
+  final public function canAcceptMail(
+    PhabricatorMetaMTAReceivedMail $mail,
+    PhutilEmailAddress $target) {
+
+    // If we don't have a valid sender user account, we can never accept
+    // mail to any object.
+    $sender = $this->getSender();
+    if (!$sender) {
+      return false;
+    }
+
+    return (bool)$this->matchObjectAddress($target);
+  }
+
   private function matchObjectAddress(PhutilEmailAddress $address) {
     $address = PhabricatorMailUtil::normalizeAddress($address);
     $local = $address->getLocalPart();
@@ -188,16 +180,6 @@ abstract class PhabricatorObjectMailReceiver extends PhabricatorMailReceiver {
     return $regexp;
   }
 
-  private function loadObjectFromMail(
-    PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
-    $parts = $this->matchObjectAddressInMail($mail);
-
-    return $this->loadObject(
-      phutil_utf8_strtoupper($parts['pattern']),
-      $sender);
-  }
-
   public static function computeMailHash($mail_key, $phid) {
     $hash = PhabricatorHash::digestWithNamedKey(
       $mail_key.$phid,

src/applications/metamta/storage/PhabricatorMetaMTAApplicationEmail.php

@@ -67,6 +67,9 @@ final class PhabricatorMetaMTAApplicationEmail
     return idx($this->configData, $key, $default);
   }
 
+  public function getDefaultAuthorPHID() {
+    return $this->getConfigValue(self::CONFIG_DEFAULT_AUTHOR);
+  }
 
   public function getInUseMessage() {
     $applications = PhabricatorApplication::getAllApplications();

src/applications/metamta/storage/PhabricatorMetaMTAReceivedMail.php

@@ -125,6 +125,7 @@ final class PhabricatorMetaMTAReceivedMail extends PhabricatorMetaMTADAO {
   }
 
   public function processReceivedMail() {
+    $viewer = $this->getViewer();
 
     $sender = null;
     try {
@@ -132,26 +133,94 @@ final class PhabricatorMetaMTAReceivedMail extends PhabricatorMetaMTADAO {
       $this->dropMailAlreadyReceived();
       $this->dropEmptyMail();
 
-      $receiver = $this->loadReceiver();
+      $sender = $this->loadSender();
-      $sender = $receiver->loadSender($this);
+      if ($sender) {
-      $receiver->validateSender($this, $sender);
+        $this->setAuthorPHID($sender->getPHID());
 
-      $this->setAuthorPHID($sender->getPHID());
+        // If we've identified the sender, mark them as the author of any
+        // attached files. We do this before we validate them (below), since
+        // they still authored these files even if their account is not allowed
+        // to interact via email.
 
-      // Now that we've identified the sender, mark them as the author of
+        $attachments = $this->getAttachments();
-      // any attached files.
+        if ($attachments) {
-      $attachments = $this->getAttachments();
+          $files = id(new PhabricatorFileQuery())
-      if ($attachments) {
+            ->setViewer($viewer)
-        $files = id(new PhabricatorFileQuery())
+            ->withPHIDs($attachments)
-          ->setViewer(PhabricatorUser::getOmnipotentUser())
+            ->execute();
-          ->withPHIDs($attachments)
+          foreach ($files as $file) {
-          ->execute();
+            $file->setAuthorPHID($sender->getPHID())->save();
-        foreach ($files as $file) {
+          }
-          $file->setAuthorPHID($sender->getPHID())->save();
+        }
+
+        $this->validateSender($sender);
+      }
+
+      $receivers = id(new PhutilClassMapQuery())
+        ->setAncestorClass('PhabricatorMailReceiver')
+        ->setFilterMethod('isEnabled')
+        ->execute();
+
+      $any_accepted = false;
+      $receiver_exception = null;
+
+      $targets = $this->newTargetAddresses();
+      foreach ($receivers as $receiver) {
+        $receiver = id(clone $receiver)
+          ->setViewer($viewer);
+
+        if ($sender) {
+          $receiver->setSender($sender);
+        }
+
+        foreach ($targets as $target) {
+          try {
+            if (!$receiver->canAcceptMail($this, $target)) {
+              continue;
+            }
+
+            $any_accepted = true;
+
+            $receiver->receiveMail($this, $target);
+          } catch (Exception $ex) {
+            // If receivers raise exceptions, we'll keep the first one in hope
+            // that it points at a root cause.
+            if (!$receiver_exception) {
+              $receiver_exception = $ex;
+            }
+          }
         }
       }
 
-      $receiver->receiveMail($this, $sender);
+      if ($receiver_exception) {
+        throw $receiver_exception;
+      }
+
+      if (!$any_accepted) {
+        if (!$sender) {
+          // NOTE: Currently, we'll always drop this mail (since it's headed to
+          // an unverified recipient). See T12237. These details are still
+          // useful because they'll appear in the mail logs and Mail web UI.
+
+          throw new PhabricatorMetaMTAReceivedMailProcessingException(
+            MetaMTAReceivedMailStatus::STATUS_UNKNOWN_SENDER,
+            pht(
+              'This email was sent from an email address ("%s") that is not '.
+              'associated with a Phabricator account. To interact with '.
+              'Phabricator via email, add this address to your account.',
+              (string)$this->newFromAddress()));
+        } else {
+          throw new PhabricatorMetaMTAReceivedMailProcessingException(
+            MetaMTAReceivedMailStatus::STATUS_NO_RECEIVERS,
+            pht(
+              'Phabricator can not process this mail because no application '.
+              'knows how to handle it. Check that the address you sent it to '.
+              'is correct.'.
+              "\n\n".
+              '(No concrete, enabled subclass of PhabricatorMailReceiver can '.
+              'accept this mail.)'));
+        }
+      }
     } catch (PhabricatorMetaMTAReceivedMailProcessingException $ex) {
       switch ($ex->getStatusCode()) {
         case MetaMTAReceivedMailStatus::STATUS_DUPLICATE:
@@ -311,51 +380,6 @@ final class PhabricatorMetaMTAReceivedMail extends PhabricatorMetaMTADAO {
         'text, and signatures are discarded and ignored.'));
   }
 
-  /**
-   * Load a concrete instance of the @{class:PhabricatorMailReceiver} which
-   * accepts this mail, if one exists.
-   */
-  private function loadReceiver() {
-    $receivers = id(new PhutilClassMapQuery())
-      ->setAncestorClass('PhabricatorMailReceiver')
-      ->setFilterMethod('isEnabled')
-      ->execute();
-
-    $accept = array();
-    foreach ($receivers as $key => $receiver) {
-      if ($receiver->canAcceptMail($this)) {
-        $accept[$key] = $receiver;
-      }
-    }
-
-    if (!$accept) {
-      throw new PhabricatorMetaMTAReceivedMailProcessingException(
-        MetaMTAReceivedMailStatus::STATUS_NO_RECEIVERS,
-        pht(
-          'Phabricator can not process this mail because no application '.
-          'knows how to handle it. Check that the address you sent it to is '.
-          'correct.'.
-          "\n\n".
-          '(No concrete, enabled subclass of PhabricatorMailReceiver can '.
-          'accept this mail.)'));
-    }
-
-    if (count($accept) > 1) {
-      $names = implode(', ', array_keys($accept));
-      throw new PhabricatorMetaMTAReceivedMailProcessingException(
-        MetaMTAReceivedMailStatus::STATUS_ABUNDANT_RECEIVERS,
-        pht(
-          'Phabricator is not able to process this mail because more than '.
-          'one application is willing to accept it, creating ambiguity. '.
-          'Mail needs to be accepted by exactly one receiving application.'.
-          "\n\n".
-          'Accepting receivers: %s.',
-          $names));
-    }
-
-    return head($accept);
-  }
-
   private function sendExceptionMail(
     Exception $ex,
     PhabricatorUser $viewer = null) {
@@ -434,4 +458,74 @@ EOBODY
       ));
   }
 
+  public function newFromAddress() {
+    $raw_from = $this->getHeader('From');
+
+    if (strlen($raw_from)) {
+      return new PhutilEmailAddress($raw_from);
+    }
+
+    return null;
+  }
+
+  private function getViewer() {
+    return PhabricatorUser::getOmnipotentUser();
+  }
+
+  /**
+   * Identify the sender's user account for a piece of received mail.
+   *
+   * Note that this method does not validate that the sender is who they say
+   * they are, just that they've presented some credential which corresponds
+   * to a recognizable user.
+   */
+  private function loadSender() {
+    $viewer = $this->getViewer();
+
+    // Try to identify the user based on their "From" address.
+    $from_address = $this->newFromAddress();
+    if ($from_address) {
+      $user = id(new PhabricatorPeopleQuery())
+        ->setViewer($viewer)
+        ->withEmails(array($from_address->getAddress()))
+        ->executeOne();
+      if ($user) {
+        return $user;
+      }
+    }
+
+    return null;
+  }
+
+  private function validateSender(PhabricatorUser $sender) {
+    $failure_reason = null;
+    if ($sender->getIsDisabled()) {
+      $failure_reason = pht(
+        'Your account ("%s") is disabled, so you can not interact with '.
+        'Phabricator over email.',
+        $sender->getUsername());
+    } else if ($sender->getIsStandardUser()) {
+      if (!$sender->getIsApproved()) {
+        $failure_reason = pht(
+          'Your account ("%s") has not been approved yet. You can not '.
+          'interact with Phabricator over email until your account is '.
+          'approved.',
+          $sender->getUsername());
+      } else if (PhabricatorUserEmail::isEmailVerificationRequired() &&
+               !$sender->getIsEmailVerified()) {
+        $failure_reason = pht(
+          'You have not verified the email address for your account ("%s"). '.
+          'You must verify your email address before you can interact '.
+          'with Phabricator over email.',
+          $sender->getUsername());
+      }
+    }
+
+    if ($failure_reason) {
+      throw new PhabricatorMetaMTAReceivedMailProcessingException(
+        MetaMTAReceivedMailStatus::STATUS_DISABLED_SENDER,
+        $failure_reason);
+    }
+  }
+
 }

src/applications/metamta/storage/__tests__/PhabricatorMetaMTAReceivedMailTestCase.php

@@ -48,11 +48,15 @@ final class PhabricatorMetaMTAReceivedMailTestCase extends PhabricatorTestCase {
   }
 
   public function testDropUnreceivableMail() {
+    $user = $this->generateNewTestUser()
+      ->save();
+
     $mail = new PhabricatorMetaMTAReceivedMail();
     $mail->setHeaders(
       array(
         'Message-ID' => 'test@example.com',
         'To'         => 'does+not+exist@example.com',
+        'From'        => $user->loadPrimaryEmail()->getAddress(),
       ));
     $mail->setBodies(
       array(
@@ -70,10 +74,6 @@ final class PhabricatorMetaMTAReceivedMailTestCase extends PhabricatorTestCase {
   public function testDropUnknownSenderMail() {
     $this->setManiphestCreateEmail();
 
-    $env = PhabricatorEnv::beginScopedEnv();
-    $env->overrideEnvConfig('phabricator.allow-email-users', false);
-    $env->overrideEnvConfig('metamta.maniphest.default-public-author', null);
-
     $mail = new PhabricatorMetaMTAReceivedMail();
     $mail->setHeaders(
       array(

src/applications/paste/mail/PasteCreateMailReceiver.php

@@ -9,7 +9,8 @@ final class PasteCreateMailReceiver
 
   protected function processReceivedMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
+    PhutilEmailAddress $target) {
+    $author = $this->getAuthor();
 
     $title = $mail->getSubject();
     if (!$title) {
@@ -26,18 +27,23 @@ final class PasteCreateMailReceiver
       ->setTransactionType(PhabricatorPasteTitleTransaction::TRANSACTIONTYPE)
       ->setNewValue($title);
 
-    $paste = PhabricatorPaste::initializeNewPaste($sender);
+    $paste = PhabricatorPaste::initializeNewPaste($author);
 
     $content_source = $mail->newContentSource();
 
     $editor = id(new PhabricatorPasteEditor())
-      ->setActor($sender)
+      ->setActor($author)
       ->setContentSource($content_source)
       ->setContinueOnNoEffect(true);
     $xactions = $editor->applyTransactions($paste, $xactions);
 
     $mail->setRelatedPHID($paste->getPHID());
 
+    $sender = $this->getSender();
+    if (!$sender) {
+      return;
+    }
+
     $subject_prefix =
       PhabricatorEnv::getEnvConfig('metamta.paste.subject-prefix');
     $subject = pht('You successfully created a paste.');
@@ -56,5 +62,4 @@ final class PasteCreateMailReceiver
       ->saveAndSend();
   }
 
-
 }

src/applications/paste/mail/PasteMailReceiver.php

@@ -12,7 +12,7 @@ final class PasteMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'P');
+    $id = (int)substr($pattern, 1);
 
     return id(new PhabricatorPasteQuery())
       ->setViewer($viewer)

src/applications/phame/mail/PhamePostMailReceiver.php

@@ -13,7 +13,7 @@ final class PhamePostMailReceiver
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)substr($pattern, 4);
+    $id = (int)substr($pattern, 1);
 
     return id(new PhamePostQuery())
       ->setViewer($viewer)

src/applications/pholio/mail/PholioMockMailReceiver.php

@@ -12,7 +12,7 @@ final class PholioMockMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'M');
+    $id = (int)substr($pattern, 1);
 
     return id(new PholioMockQuery())
       ->setViewer($viewer)

src/applications/ponder/mail/PonderAnswerMailReceiver.php

@@ -12,7 +12,7 @@ final class PonderAnswerMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'ANSR');
+    $id = (int)substr($pattern, 4);
 
     return id(new PonderAnswerQuery())
       ->setViewer($viewer)

src/applications/ponder/mail/PonderQuestionCreateMailReceiver.php

@@ -9,7 +9,8 @@ final class PonderQuestionCreateMailReceiver
 
   protected function processReceivedMail(
     PhabricatorMetaMTAReceivedMail $mail,
-    PhabricatorUser $sender) {
+    PhutilEmailAddress $target) {
+    $author = $this->getAuthor();
 
     $title = $mail->getSubject();
     if (!strlen($title)) {
@@ -26,18 +27,17 @@ final class PonderQuestionCreateMailReceiver
       ->setTransactionType(PonderQuestionTransaction::TYPE_CONTENT)
       ->setNewValue($mail->getCleanTextBody());
 
-    $question = PonderQuestion::initializeNewQuestion($sender);
+    $question = PonderQuestion::initializeNewQuestion($author);
 
     $content_source = $mail->newContentSource();
 
     $editor = id(new PonderQuestionEditor())
-      ->setActor($sender)
+      ->setActor($author)
       ->setContentSource($content_source)
       ->setContinueOnNoEffect(true);
     $xactions = $editor->applyTransactions($question, $xactions);
 
     $mail->setRelatedPHID($question->getPHID());
-
   }
 
 

src/applications/ponder/mail/PonderQuestionMailReceiver.php

@@ -12,7 +12,7 @@ final class PonderQuestionMailReceiver extends PhabricatorObjectMailReceiver {
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)trim($pattern, 'Q');
+    $id = (int)substr($pattern, 1);
 
     return id(new PonderQuestionQuery())
       ->setViewer($viewer)

src/applications/slowvote/mail/PhabricatorSlowvoteMailReceiver.php

@@ -13,7 +13,7 @@ final class PhabricatorSlowvoteMailReceiver
   }
 
   protected function loadObject($pattern, PhabricatorUser $viewer) {
-    $id = (int)substr($pattern, 4);
+    $id = (int)substr($pattern, 1);
 
     return id(new PhabricatorSlowvoteQuery())
       ->setViewer($viewer)