Summary:
Fixes T4379. Several changes:
- Migrate all project members into subscribers.
- When members are added or removed, subscribe or unsubscribe them.
- Show sub/unsub in the UI.
- Determine mailable membership of projects by querying subscribers.
Test Plan:
- As `duck`, joined a project.
- Added the project as a reviewer to a revision.
- Commented on the revision.
- Observed `duck` receive mail.
- Unsubscribed as `duck`.
- Observed no mail.
- Resubscribed as `duck`.
- Mail again.
- Joined/left project, checked sub/unsub status.
- Ran migration, looked at database.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, asherkin
Maniphest Tasks: T4379
Differential Revision: https://secure.phabricator.com/D8189
Summary: Ref T4368. We don't currently GC these tables, and the sent mail table is one of the largest on `secure.phabricator.com`. There's no value in retaining this information indefinitely. Instead, retain it for 90 days, which should be plenty of time to debug/diagnose any issues.
Test Plan: Ran `phd debug garbage`, saw it clean up a reasonable amount of data from these tables.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4368
Differential Revision: https://secure.phabricator.com/D8127
Summary:
Ref T4361. Before we figure out which To/CC are addressable, try to expand To/CC. Specifically, the supported expansion right now is project PHIDs expanding to all their members.
Because of the way multiplexing works, we have to do this in two places: explicitly in `multiplexMail()`, and when sending mail that wasn't multiplexed. This is messy; eventually we can get rid of it (after ApplicationTransactions are everywhere).
This has some rough edges, but should basically give us what we need to make stuff like projects mailable. Particularly, it deals with most issues in D7436:
- I got around the resolution/multiplexing issue by resolving aggregate mailables separately from mailable actors.
- We get to keep the Project PHID as a To/CC/Reviewer/Whatever until the last second.
- Users won't get two emails for being a CC and also a member of a CC'd project.
- We can degrade to the list stuff this way if we want, by having the project aggregate yield a single list PHID.
Test Plan: Added a comment to a revision with a project reviewer, got mail to all the project's members.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4361
Differential Revision: https://secure.phabricator.com/D8117
Summary:
Fixes T4202. We have old code in MetaMTA which implements gradual backoff and maximum retries.
However, we have more general code in the task queue which does this, too. We can just use the more general stuff in the task queue; it obsoletes the specific stuff in MetaMTA, which is more complex and ran into some kind of issue in T4202.
Remove `retryCount`, `nextRetry` (obsoleted by task queue retry mechanisms) and "simulated failures" (no longer in use).
Generally, modern infrastructure has replaced these mechanisms with more general ones.
Test Plan:
- Sent mail.
- Observed unsendable mail failing in reasonable ways in the queue.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4202
Differential Revision: https://secure.phabricator.com/D8115
Summary:
As you've suggested, I took the SendGrid code and massaged it until it played nice with Mailgun.
btw - unless I'm missing something, it appears that the SendGrid receiver lets you spoof emails (it performs no validation on the data received).
Test Plan: Opened a task with Mailgun. Felt great.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T4326
Differential Revision: https://secure.phabricator.com/D7989
Summary: Adds "verified" and "secretKey" to Legalpad document signatures. For logged in users using an email address they own, things are verified right away. Otherwise, the email is sent a verification letter. When the user clicks the link the signature is marked verified.
Test Plan: signed the document with a bogus email address not logged in. verified the email that would be sent looked good from command line. followed link and successfully verified bogus email address
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran, asherkin
Maniphest Tasks: T4283
Differential Revision: https://secure.phabricator.com/D7930
Summary:
Ref T3857.
- Always send mail via daemons. This lets us get rid of this config, and is generally much more performant.
- After D7964, we warn if daemons aren't running.
Test Plan: Sent some mail.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3857
Differential Revision: https://secure.phabricator.com/D7965
Summary:
Ref T2015. Not directly related to Drydock, but I bumped into this. All these scripts currently enumerate their workflows explicitly.
Instead, use `PhutilSymbolLoader` to automatically discover workflows. This reduces code duplication and errors (see all the bad `extends` this diff fixes) and lets third parties add new workflows (not clearly valuable?).
Test Plan: Ran `bin/x help` for each modified script.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7840
Summary:
Ref T2015. Not directly related to Drydock, but I've wanted to do this for a bit.
Introduce a common base class for all the workflows in the scripts in `bin/*`. This slightly reduces code duplication by moving `isExecutable()` to the base, but also provides `getViewer()`. This is a little nicer than `PhabricatorUser::getOmnipotentUser()` and gives us a layer of indirection if we ever want to introduce more general viewer mechanisms in scripts.
Test Plan: Lint; ran some of the scripts.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2015
Differential Revision: https://secure.phabricator.com/D7838
Summary:
Fixes two issues:
- When rendering a task's details, we currently issue a policy-oblivious query. Instead, issue a policy-aware query.
- The formatting is a little bit weird, with the top half in a box and the bottom half with an older style. Make them consistent.
Test Plan: Looked at the detail pages for several tasks in queue.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D7812
Summary: Until we implement an "enum" type for config, make this a bit harder to get wrong. A user entered "TLS", but the correct value is "tls". The documentation is consistent about this, but the behavior is sitll surprsing.
Test Plan: eyeballed it
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7778
Summary:
A user sent a message to Phabricator which looked like:
On blah blah blah ?
On <date>, <user> wrote:
> blah blah blah
The current algorithm is too aggressive and thinks lines 1-3 are //all// the "On ... wrote:" string. Instead, patch only the most recent "On".
Test Plan: Added a failing test and made it pass.
Reviewers: btrahan, zeeg
Reviewed By: zeeg
CC: aran
Differential Revision: https://secure.phabricator.com/D7732
Summary:
Ref T4039. This is mostly to deal with that, to prevent the security issues associated with mutable local paths. The next diff will lock them in the web UI.
I also added a confirmation prompt to `bin/repository delete`, which was a little scary without one.
See one comment inline about the `--as` flag. I don't love this, but when I started adding all the stuff we'd need to let this transaction show up as "Administrator" it quickly got pretty big.
Test Plan: Ran `bin/repository edit ...`, saw an edit with a transaction show up on the web UI.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4039
Differential Revision: https://secure.phabricator.com/D7579
Summary:
Mailbox sometimes (?) changes the case of the email address (?). Be more liberal in what we accept.
Also fix a minor output bug.
Test Plan: Sent mail to `e1+...` instead of `E1+...`, verified it arrived.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7575
Summary:
Small step forward which improves existing stuff or lays groudwork for future stuff:
- Currently, to check for email verification, we have to single-query the email address on every page. Instead, denoramlize it into the user object.
- Migrate all the existing users.
- When the user verifies an email, mark them as `isEmailVerified` if the email is their primary email.
- Just make the checks look at the `isEmailVerified` field.
- Add a new check, `isUserActivated()`, to cover email-verified plus disabled. Currently, a non-verified-but-not-disabled user could theoretically use Conduit over SSH, if anyone deployed it. Tighten that up.
- Add an `isApproved` flag, which is always true for now. In a future diff, I want to add a default-on admin approval queue for new accounts, to prevent configuration mistakes. The way it will work is:
- When the queue is enabled, registering users are created with `isApproved = false`.
- Admins are sent an email, "[Phabricator] New User Approval (alincoln)", telling them that a new user is waiting for approval.
- They go to the web UI and approve the user.
- Manually-created accounts are auto-approved.
- The email will have instructions for disabling the queue.
I think this queue will be helpful for new installs and give them peace of mind, and when you go to disable it we have a better opportunity to warn you about exactly what that means.
Generally, I want to improve the default safety of registration, since if you just blindly coast through the path of least resistance right now your install ends up pretty open, and realistically few installs are on VPNs.
Test Plan:
- Ran migration, verified `isEmailVerified` populated correctly.
- Created a new user, checked DB for verified (not verified).
- Verified, checked DB (now verified).
- Used Conduit, People, Diffusion.
Reviewers: btrahan
Reviewed By: btrahan
CC: chad, aran
Differential Revision: https://secure.phabricator.com/D7572
Summary: See @scottmac's reply in T3982. It looks like his email client uses the standard quote string, but includes it in the quoted block.
Test Plan: Added a failing unit test, made it pass.
Reviewers: btrahan
Reviewed By: btrahan
CC: scottmac, aran
Differential Revision: https://secure.phabricator.com/D7440
Summary: Fixes T4001. I broke this some time ago and no one has complained. I don't think it gets much use, and we haven't added it for the newer apps. Just get rid of it rather than adapt the URIs for ApplicationSearch.
Test Plan: Unit tests, sent myself some email.
Reviewers: zeeg, btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4001
Differential Revision: https://secure.phabricator.com/D7355
Summary:
also try to centralize some of the command parsing logic. note that differential is still an exception here. it uses a whitelist-style regex. i think long-term we should have this for every app but changing it seemed too big for this diff.
Fixes T3937.
Test Plan:
echo '!assign btrahan' | ./bin/mail receive-test --as xerxes --to T22 ; echo '!claim' | ./bin/mail receive-test --as xerxes --to T22
unit tests passed, though my new one is silly
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Maniphest Tasks: T3937
Differential Revision: https://secure.phabricator.com/D7307
Summary:
we filter the $actors above such that its possible to have no $actor anymore (if $actor is not a deliverable email address). ergo, make sure we have actor before we start calling methods.
Fixes github issue 403
Test Plan: logic on this one - not 100% sure how to easily reproduce
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7284
Summary: Ref T603. Allow global default policies to be configured for tasks.
Test Plan:
- Created task via web UI.
- Created task via Conduit.
- Created task via email.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D7267
Summary: Ref T603. Swaps out most `PhabricatorFile` loads for `PhabricatorFileQuery`.
Test Plan:
- Viewed Differential changesets.
- Used `file.info`.
- Used `file.download`.
- Viewed a file.
- Deleted a file.
- Used `/Fnnnn` to access a file.
- Uploaded an image, verified a thumbnail generated.
- Created and edited a macro.
- Added a meme.
- Did old-school attach-a-file-to-a-task.
- Viewed a paste.
- Viewed a mock.
- Embedded a mock.
- Profiled a page.
- Parsed a commit with image files linked to a revision with image files.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D7178
Summary: ...and deploy on Maniphest. Ref T1638.
Test Plan: created a herald rule to be cc'd for tasks created via web. made a task via web and another via email and was cc'd appropriately. edited the herald to be cc'd for tasks created via not web. made 2 tasks again and got cc'd appropriately
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T1638
Differential Revision: https://secure.phabricator.com/D7145
Summary:
Ref T2217. Fixes two issues:
# The "task created" email didn't include the task description, but should.
# We were treaging the "status" event as the "create", but that's kind of a mess. Treat the "title" event as the "create" instead. This makes initial emails say "[Created]".
Test Plan: Created some tasks, got better emails.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T2217
Differential Revision: https://secure.phabricator.com/D7115
Summary: Ref T603. Killing this class is cool because the classes that replace it are policy-aware. Tried to keep my wits about me as I did this and fixed a few random things along the way. (Ones I remember right now are pulling a query outside of a foreach loop in Releeph and fixing the text in UIExample to note that the ace of hearts if "a powerful" card and not the "most powerful" card (Q of spades gets that honor IMO))
Test Plan: tested the first few changes (execute, executeOne X handle, object) then got real mechanical / careful with the other changes.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran, FacebookPOC
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D6941
Summary: Currently, adapters can only fail mail temporarily. Allow them to indicate a permanent failure by throwing a special exception.
Test Plan: Added and ran unit tests.
Reviewers: wez, btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6847
Summary: Missed this when moving most MetaMTA responsibilities to the CLI. Show the correct command to get data rather than linking to a 404.
Test Plan: {F56733}
Reviewers: wez, btrahan, chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D6846
Summary: Ref T2715. Had to start loading status information in the query class. Debated trying to clean up some of the attach / load stuff but decided to just add status under the new paradigm for now.
Test Plan: phid.query also made a status and checked that out. also played in conpherence.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2715
Differential Revision: https://secure.phabricator.com/D6585
Summary:
Move comments from the old table to ApplicationTransactions. Patch dances around which objects it uses since I intend to delete the comment table.
NOTE: This temporarily disables comment writes. I'll restore them shortly.
Test Plan: {F50166}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6454
Summary: Fixes T3528. We won't be able to load the user if they've been deleted, and will fatal a few lines later on `$user->getID()`.
Test Plan: I'm going with my gut on this one.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3528
Differential Revision: https://secure.phabricator.com/D6442
Summary:
Ref T3306. This interface has a hard time balancing security/policy issues and I'm not sure what the best way forward is. Some possibilities:
# We just let you see everything from the web UI.
- This makes debugging easier.
- Anyone who can see this stuff can trivially take over any user's account with five seconds of work and no technical expertise (reset their password from the web UI, then go read the email and click the link).
# We let you see everything, but only for messages you were a recipient of or author of.
- This makes it much more difficult to debug issues with mailing lists.
- But maybe we could just say mailing list recipients are "public", or define some other ruleset.
- Generally this gets privacy and ease of use right.
# We could move the whole thing to the CLI.
- Makes the UI/UX way worse.
# We could strike an awkward balance between concerns, as we do now.
- We expose //who// sent and received messages, but not the content of the messages. This doesn't feel great.
I'm inclined to probably go with (2) and figure something out for mailing lists?
Anyway, irrespective of that this should generally make things more clear, and improves the code a lot if nothing else.
Test Plan:
{F49546}
- Looked at a bunch of mail.
- Sent mail from different apps.
- Checked that recipients seem correct.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3306
Differential Revision: https://secure.phabricator.com/D6413
Summary: Ref T3306. I'm going to add more information about To/Cc here, but here's a little cleanup first.
Test Plan: {F49524}
Reviewers: chad, btrahan
Reviewed By: chad
CC: aran
Maniphest Tasks: T3306
Differential Revision: https://secure.phabricator.com/D6410
Summary:
Ref T3306. Moves this from the web to the CLI, which is a tiny bit clunkier but way better as far as policies go and more repeatable for development.
See discussion in D6413.
Test Plan: Ran `bin/mail receive-test`, verified mail was received. Used and abused various options.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3306
Differential Revision: https://secure.phabricator.com/D6417
Summary:
Ref T1536. This is the schema code for `PhabricatorExternalAccount` which was previously in D4647. I'm splitting it out so I can put it earlier in the sequence and because it's simple and standalone.
Expands `PhabricatorExternalAccount` to have everything we need for the rest of registration.
Test Plan: Implemented the remainder of new registration on top of this.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1536
Differential Revision: https://secure.phabricator.com/D6169
Summary: At the global level, truncate emails at a user-configured size.
Test Plan: Untested, as I could not get PHP to send emails on my box, but if you can this should be very easy to test. Just set the max size to something like .001 kilobytes and make sure it does the right thing.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, AnhNhan
Maniphest Tasks: T1392
Differential Revision: https://secure.phabricator.com/D6118
Summary:
Ref T2625. Fixes T2812. Implement ApplicationSearch in People.
{F44788}
Test Plan: Made People queries. Used Conduit. Used `@mentions`.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T2625, T2812
Differential Revision: https://secure.phabricator.com/D6092
Summary: Ref T3166. I moved the create logic into a static method in the editor class to keep things tidy.
Test Plan: created a conpherence from UI. purdy. tried errors and got UI to show "required". for conduit, created a thread with all the bells and whistles and it worked. verified i got proper exceptions with bum conduit calls
Reviewers: epriestley
Reviewed By: epriestley
CC: chad, aran, Korvin
Maniphest Tasks: T3166
Differential Revision: https://secure.phabricator.com/D6083
Summary: I introduced this helper at some point, clean up all the code duplication around content sources.
Test Plan: Grepped; hit edit interfaces for most/all of these.
Reviewers: btrahan, chad, edward
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D6030
Summary:
Ref T3217. @hlau has an email client which quotes text with:
________________________________________
From: ...
Add a regular expression to correctly detect this as quoted body text.
Test Plan: Add unit test; executed unit test.
Reviewers: btrahan, chad
Reviewed By: chad
CC: hlau, aran
Maniphest Tasks: T3217
Differential Revision: https://secure.phabricator.com/D5986
Summary: We can't show this stuff on the web UI because it has password reset links and private reply-to addresses, but we can provide easier CLI tools than "root around in the database". Land a rough version of `bin/mail show-inbound` and `bin/mail show-outbound`.
Test Plan: Used both commands to examine mail from the CLI.
Reviewers: btrahan
Reviewed By: btrahan
CC: tido, euresti, aran
Differential Revision: https://secure.phabricator.com/D5963
Summary:
Moves all remaining mail handling into ReplyHandlers.
Farewell, `getPhabricatorToInformation()`! You were a bad method and no one liked you.
Ref T1205.
Test Plan:
- Used test console to send mail to Revisions, Tasks, Conpherences and Commits (these all actually work).
- Used test console to send mail to Requests, Macros, Questions and Mocks (these accept the mail but don't do anything with it, but didn't do anything before either).
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5953
Summary: Ref T1205. Moves the handling logic for these email types to reply handlers.
Test Plan: Used test form to send conpherence and maniphest mail.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5945
Summary: Currently this is fairly hard-coded. Instead, make it use available receivers. Ref T1205.
Test Plan: Used mail form to send mail to various objects (Dnn, Tnn, Cnn, etc.). Only some of these work right now because the receiver thing still hard-codes a bunch of junk.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5944
Summary:
Ref T1205. Finally able to delete a big chunk of this nastiness.
Make MailReceivers responsible for validating senders. For object creation receivers (bugs, conpherences) this just means that users must not be disabled. For other receivers the senders must be able to see the objects, have the right hashes, etc., according to policy.
Test Plan: Added a bunch of test cases (everything except policy). Verified behavior via the Receive test console.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1205
Differential Revision: https://secure.phabricator.com/D5943
