1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-11 16:16:14 +01:00
Commit graph

522 commits

Author SHA1 Message Date
epriestley
2e45021250 Fix several issues with email-related global preferences
Summary:
Ref T11098. Mixture of issues here:

  - Similar problem to D16112, where users with no settings at all could fail to fall back to the global defaults.
    - I made `UserPreferencesQuery` responsible for building defaults instead to simplify this, since we have 4 or 5 callsites which need to do it and they aren't easily reducible.
  - Handle cases where `metamta.one-mail-per-recipient` is off (and thus users can not have any custom settings) more explicitly.
  - When `metamta.one-mail-per-recipient` is off, remove the "Email Format" panel for users only -- administrators can still access it in global preferences.

Test Plan:
  - Deleted a user's preferences, changed globals, purged cache, made sure defaults reflected global defaults.
  - Changed global mail tags, sent mail to the user, verified it was dropped in accordinace with global settings.
  - Changed user's settings to get the mail instead, verified mail was sent.
  - Toggled user's Re / Vary settings, verified mail subject lines reflected user settings.
  - Disabled `metamta.one-mail-per-recipient`, verified user "Email Format" panel vanished.
  - Edited "Email Format" in single-mail-mode in global prefs as an administrator.
  - Sent more mail, verified mail respected new global settings.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11098

Differential Revision: https://secure.phabricator.com/D16118
2016-06-14 12:35:31 -07:00
epriestley
cb7560d301 Remove "re prefix" and "vary subjects" config
Summary: Ref T11098. There is no reason to maintain these as separate values now that they can be configured in global settings.

Test Plan:
  - Hit and read setup issue.
  - Fiddled with settings.
  - I'll vet this more throughly in the next diff since I need to fix an issue with global defaults in mail and can explicitly test this at the same time.

Reviewers: chad

Reviewed By: chad

Subscribers: eadler

Maniphest Tasks: T11098

Differential Revision: https://secure.phabricator.com/D16117
2016-06-14 12:35:12 -07:00
epriestley
2b344b2bb5 Make caches misses throw by default intead of inline-generating
Summary:
Ref T4103. Ref T10078. Currently, when a user misses a cache we just build it for them.

This is the behavior we want for the the viewer (so we don't have to build every cache up front if we don't actually need them), but not the right behavior for other users (since it allows performance problems to go undetected).

Make inline cache generation strict by default, then make sure all the things that rely on cache data request the correct data (well, all of the things identified by unit tests, at least: there might be some more stuff I haven't hit yet).

This fixes test failures in D16040, and backports a piece of that change.

Test Plan: Identified and then fixed failures with `arc unit --everything`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103, T10078

Differential Revision: https://secure.phabricator.com/D16042
2016-06-05 08:51:54 -07:00
epriestley
7ef6c0a523 Modularize all the mail preferences
Summary:
Ref T4103. This isn't completely perfect but should let us move forward without also expanding scope into "too much mail".

I split the existing "Mail Preferences" into two panels: a "Mail Delivery" panel for the EditEngine settings, and a "2000000 dropdowns" panel for the two million dropdowns. This one retains the old code more or less unmodified.

Test Plan:
  - Ran unit tests, which cover most of this stuff.
  - Grepped for all removed constants.
  - Ran migrations, inspected database results.
  - Changed settings in both modified panels.
  - This covers a lot of ground, but anything I missed will hopefully be fairly obvious.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16038
2016-06-05 08:50:07 -07:00
epriestley
ebd8f3c987 Make translation, timezone and pronoun into real settings
Summary:
Ref T4103. These are currently stored on the user, for historic/performance reasons.

Since I want administrators to be able to set defaults for translations and timezones at a minimum and there's no longer a meaningful performance penalty for moving them off the user record, turn them into real preferences and then nuke the columns.

Test Plan:
  - Set settings to unusual values.
  - Ran migrations.
  - Verified my unusual settings survived.
  - Created a new user.
  - Edited all settings with old and new UIs.
  - Reconciled client/server timezone disagreement.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16005
2016-06-02 06:29:47 -07:00
epriestley
b256f2d7b2 Prepare UserPreferences for transactions
Summary:
Ref T4103. This give preferences a PHID, policy/transaction interfaces, a transaction table, and a Query class.

This doesn't actually change how they're edited, yet.

Test Plan:
- Ran migrations.
- Inspected database for date created, date modified, PHIDs.
- Changed some of my preferences.
- Deleted a user's preferences, verified they reset properly.
- Set some preferences as a new user, got a new row.
- Destroyed a user, verified their preferences were destroyed.
- Sent Conpherence messages.
- Send mail.
- Tried to edit another user's settings.
- Tried to edit a bot's settings as a non-admin.
- Edited a bot's settings as an admin (technically, none of the editable settings are actually stored in the settings table, currently).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D15991
2016-05-31 12:28:44 -07:00
epriestley
49eb6403a4 Send HTML email by default
Summary: Ref T10694. Switch default mode to HTML since it has a number of significant advantages and we haven't seen reports of significant problems.

Test Plan:
  - Switched preference to default (saw "HTML" in UI).
  - Sent myself some mail.
  - Got HTML mail.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10694

Differential Revision: https://secure.phabricator.com/D15885
2016-05-18 14:53:57 -07:00
epriestley
547abfe873 Make packages mailable and subscribable
Summary:
Ref T10939. Fixes T7834.

  - Make packages into mailable objects, like projects and users.
  - Packages resolve recipients by resolving project and user owners into recipients.

Test Plan:
  - Added a comment to a revision with a package subscriber.
  - Used `bin/mail show-outbound` to see that owners got mail.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7834, T10939

Differential Revision: https://secure.phabricator.com/D15912
2016-05-13 17:18:57 -07:00
epriestley
94c7bb605c Highlight inline diff context in HTML mail
Summary:
Ref T10694. Ref T9790. When generating inline diff context, highlight it and then mangle the highlighted output into `style="..."` so it works in HTML.

Also try to tighten up some spacing/formatting stuff.

Test Plan:
Got some output in this vein:

{F1259937}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9790, T10694

Differential Revision: https://secure.phabricator.com/D15852
2016-05-05 11:13:27 -07:00
Chad Little
4d9bbc539e Update MetaMTA to new UI
Summary: Swaps over to new hotness

Test Plan: Pull up mail view, see new UI

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15616
2016-04-05 07:26:46 -07:00
epriestley
a8c9a5597d Improve error and header behaviors for Mailgun received mail webhook
Summary:
Ref T10709. Two issues:

  - If a user sends an invalid `!command`, we can throw, which means we don't return HTTP 200. This makes Mailgun re-send the mail later.
  - We don't parse headers of the modern API correctly, so the "Message-ID" failsafe doesn't work. Parse them correctly. I //believe// Mailgun's API changed at some point.

Test Plan:
This is difficult to test exhaustively in isolation. I used Mailgun's web tools to verify the format of the hook request, and faked some requests locally.

I'll keep an eye on this as it goes to production and make sure the fix is correct there.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10709

Differential Revision: https://secure.phabricator.com/D15575
2016-04-02 10:51:29 -07:00
Chad Little
59efb7bdf3 Update Meta for new UI
Summary: Runs through Meta, cleaned up policies and editing email addresses to new UI

Test Plan: Set a new Email address for Maniphest, edit policies.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15569
2016-04-01 14:13:16 -07:00
epriestley
601aaa5a86 Modularize content sources
Summary:
Ref T10537. For Nuance, I want to introduce new sources (like "GitHub" or "GitHub via Nuance" or something) but this needs to modularize eventually.

Split ContentSource apart so applications can add new content sources.

Test Plan:
This change has huge surface area, so I'll hold it until post-release. I think it's fairly safe (and if it does break anything, the breaks should be fatals, not anything subtle or difficult to fix), there's just no reason not to hold it for a few hours.

- Viewed new module page.
- Grepped for all removed functions/constants.
- Viewed some transactions.
- Hovered over timestamps to get content source details.
- Added a comment via Conduit.
- Added a comment via web.
- Ran `bin/storage upgrade --namespace XXXXX --no-quickstart -f` to re-run all historic migrations.
- Generated some objects with `bin/lipsum`.
- Ran a bulk job on some tasks.
- Ran unit tests.

{F1190182}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10537

Differential Revision: https://secure.phabricator.com/D15521
2016-03-26 11:59:45 -07:00
epriestley
4a6589524b Add amazon-ses.endpoint configuration
Summary: Fixes T5116.

Test Plan: Will test in production.

Reviewers: chad

Maniphest Tasks: T5116

Differential Revision: https://secure.phabricator.com/D15515
2016-03-23 12:28:59 -07:00
epriestley
ba9cd64e51 Stop moving "Cc" addresses to "To" when building mail targets
Summary: Fixes T10539. When building mail targets, we fail to preserve whether a recipient was originally "To" or "Cc", and just move everyone to "To".

Test Plan:
Added a comment to a task with a "To" user and a "Cc" user, with `metamta.placeholder-to-recipient` set and `metamta.one-mail-per-recipient` set.

Got mail with me Cc'd as the Cc'd user:

{F1172020}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10539

Differential Revision: https://secure.phabricator.com/D15465
2016-03-12 07:30:00 -08:00
Chad Little
3d44a5c253 Polish up timeline for PHIUTwoColumnView
Summary: This inverts colors and icons a bit, so they're not as harsh. So instead of a dark green item with white icon, its now light green with a dark green icon. I've also changed all text and comment boxes to be "grey" visually to separate out the UI from converation/actions. Give it a spin and let me know how this feels. I still need to update the comment UI.

Test Plan:
UIExamples, lots of various tasks and diffs.

{F1163837}

{F1163839}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15442
2016-03-08 17:51:53 -08:00
Chad Little
fe5cd4ca2c Move FontIcon calls to Icon
Summary: Normalizes all `setFontIcon` calls to `setIcon`.

Test Plan: UIExamples, Almanac, Apps list, etc.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, hach-que, yelirekim

Differential Revision: https://secure.phabricator.com/D15129
2016-01-28 08:48:45 -08:00
Chad Little
36158dbdc0 Convert all calls to 'IconFont' to just 'Icon'
Summary: Mostly for consistency, we're not using other forms of icons and this makes all classes that use an icon call it in the same way.

Test Plan: tested uiexamples, lots of other random pages.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15125
2016-01-27 20:59:27 -08:00
epriestley
8463ad2659 Replace subscribe/unsubscribe for projects with explicit mail setting
Summary:
Ref T10054. Ref T6113. Users can currently subscribe to projects, which causes them to receive:

  # mail about project membership changes, description changes, etc; and
  # mail to the project, e.g. when the project is added as a subscriber on a task, or a reviewer on a revision.

Almost no one cares about (1), and after D15061 you can use Herald to get this stuff if you really want it. (It will get progressively more annoying in the future with external membership sources causing automated project membership updates.)

A lot of users are confused about (2) and how it relates to membership, watching, etc, and most users who want (2) don't want (1).

Instead, add an explicit option for this and explain what it does.

This is fairly verbose but I've hidden it on the member/watch screen, which is now the "explain how projects work" screen, I guess.

Test Plan:
{F1064929}

{F1064930}

{F1064931}

  - Disabled/enabled mail for a project.
  - Sent mail to a project with mail disabled, verified I didn't get a copy.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6113, T10054

Differential Revision: https://secure.phabricator.com/D15065
2016-01-19 19:39:02 -08:00
epriestley
516ba5e6c5 Fix an issue where generation of mail processing error email could fail in the presence of duplicate headers
Summary:
Ref T10121. If a user sends mail with duplicate headers, like:

```
X-Duplicate: A
X-Duplicate: B
```

...and we process it with `mail_handler.php`, we may end up with `array('A', 'B')` as the header value. When we try to write this back into an error response mail, it fails.

Test Plan:
  - Generated a message with duplicate headers.
  - Piped it into `mail_handler.php` with `--process-duplicates` and `--trace` to get a look at it.
  - Faked an exception.
  - Before patch: bad error email.
  - After patch: clean error email showing multiple header values.

```
$ ./bin/mail show-outbound --id 12386
PROPERTIES
ID: 12386
Status: queued
Related PHID:
Message:

PARAMETERS
sensitive: 1
is-error: 1
force: 1
subject: Error Processing Mail (Exception)
raw-to: ["epriestley@phacility.com"]

HEADERS
X-Phabricator-Sent-This-Message: Yes
X-Mail-Transport-Agent: MetaMTA
X-Auto-Response-Suppress: All

TEXT BODY
Your email to Phabricator was not processed, because an error occurred while
trying to handle it:

Exception: TEST

-- Original Message Body -----------------------------------------------------

testy testy

-- Original Message Headers --------------------------------------------------

from: Evan Priestley <epriestley@phacility.com>
content-type: text/plain; charset=us-ascii
content-transfer-encoding: 7bit
x-smtp-server: smtp.gmail.com:epriestley@phacility.com
subject: test outbound mail
message-id: 7isvptmllqvdvtdxthvdwzg3woj5au7csyuh3hopypjv6y6hqb32qm4bcrd4jtid
x-universally-unique-identifier: 4E489E20-F674-49B2-94BA-0DE44F504EAA
date: Mon, 11 Jan 2016 09:50:12 -0800
date: Mon, 11 Jan 2016 09:50:13 -0800
date: Mon, 11 Jan 2016 09:50:14 -0800
date: Mon, 11 Jan 2016 09:50:15 -0800
to: epriestley@yghe.net
mime-version: 1.0 (Mac OS X Mail 8.2 \(2104\))

HTML BODY
(This message has no HTML body.)
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10121

Differential Revision: https://secure.phabricator.com/D14996
2016-01-11 10:16:57 -08:00
epriestley
19b2eb57a9 Improve lipsum generation of projects
Summary:
Fixes T9156.

  - Fix hashtag generation.
  - Fix various badnesses.
  - Improve project name generator.

Test Plan:
```
$ ./bin/lipsum generate projects
 GENERATORS  Selected generators: Projects.
 WARNING  This command generates synthetic test data, including user accounts. It is intended for use in development environments so you can test features more easily. There is no easy way to delete this data or undo the effects of this command. If you run it in a production environment, it will pollute your data with large amounts of meaningless garbage that you can not get rid of.

    Are you sure you want to generate piles of garbage? [y/N] y

 LIPSUM  Generating synthetic test objects forever. Use ^C to stop when satisfied.
Generated "Project": Self-Flying Data Center Swag Performance
Generated "Project": Optimize Cars
Generated "Project": Triaging Culture Optimization
Generated "Project": Automating Experience
Generated "Project": Accelerating NUX Performance
Generated "Project": Optimizing Culture Optimization
Generated "Project": Optimize Hardware
```

{F1042949}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9156

Differential Revision: https://secure.phabricator.com/D14874
2015-12-24 09:06:56 -08:00
epriestley
b3d3130b71 Fix a potentially lax hash comparison
Summary: Via HackerOne. See D14025. I missed this comparison when making the original change.

Test Plan:
  - Used `cat mail.txt | scripts/mail/mail_handler.php --process-duplicates` to pipe mail in a whole lot of times.
  - Tried bad hashes, saw rejections.
  - Tried good hash, saw mail accepted.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14455
2015-11-10 10:22:25 -08:00
lkassianik
28b8c8e212 HTML emails for Calendar event description changes should respect remarkup rules
Summary: Ref T7964, HTML emails for Calendar event description changes should respect remarkup rules

Test Plan: Create event and edit description, check that email has a correctly formatted remarkup description section.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T7964

Differential Revision: https://secure.phabricator.com/D13554
2015-11-07 07:39:52 -08:00
Joshua Spence
c35b564f4d Various translation improvements
Summary: Depends on D14070.

Test Plan: Eyeball it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D14073
2015-11-03 07:02:46 +11:00
Joshua Spence
495cb7a2e0 Mark PhabricatorPHIDType::getPHIDTypeApplicationClass() as abstract
Summary: Fixes T9625. As explained in a `TODO` comment, seems reasonable enough.

Test Plan: Unit tests.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, hach-que

Maniphest Tasks: T9625

Differential Revision: https://secure.phabricator.com/D14068
2015-11-03 06:47:12 +11:00
epriestley
4afeebe834 Don't store IP addresses in content sources
Summary:
We don't use these for anything, we're inconsistent about recording them, and there's some mild interaction with privacy concerns and data retention. Every other log we store any kind of information in can be given a custom retention policy after recent GC changes.

If we did put this back eventually it would probably be better to store a session identifier anyway, since that's more granular and more detailed.

You can fetch this info out of access logs anyway, too.

Test Plan: Left a couple of comments.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14315
2015-10-21 12:37:37 -07:00
Chad Little
dac16264e4 Update metamta for handleRequest
Summary: Updates metamta for handleRequest

Test Plan: Unable to test this, but looks safe?

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14256
2015-10-12 12:02:11 -07:00
epriestley
9c798e5cca Provide bin/garbage for interacting with garbage collection
Summary:
Fixes T9494. This:

  - Removes all the random GC.x.y.z config.
  - Puts it all in one place that's locked and which you use `bin/garbage set-policy ...` to adjust.
  - Makes every TTL-based GC configurable.
  - Simplifies the code in the actual GCs.

Test Plan:
  - Ran `bin/garbage collect` to collect some garbage, until it stopped collecting.
  - Ran `bin/garbage set-policy ...` to shorten policy. Saw change in web UI. Ran `bin/garbage collect` again and saw it collect more garbage.
  - Set policy to indefinite and saw it not collect garabge.
  - Set policy to default and saw it reflected in web UI / `collect`.
  - Ran `bin/phd debug trigger` and saw all GCs fire with reasonable looking queries.
  - Read new docs.

{F857928}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9494

Differential Revision: https://secure.phabricator.com/D14219
2015-10-02 09:17:24 -07:00
epriestley
878a493301 Begin standardizing garbage collectors
Summary: Ref T9494. Improve support infrastructure for garbage collectors.

Test Plan:
  - Ran `bin/phd debug trigger`, saw collectors execute.

{F857852}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9494

Differential Revision: https://secure.phabricator.com/D14218
2015-10-01 16:58:43 -07:00
Chad Little
666f19e504 Make icon setting in Section Headers easier/consistent
Summary: You can already pass other icons, but this makes it a bit simpler.

Test Plan: Test Maniphest, Badges

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14131
2015-09-19 11:29:01 -07:00
epriestley
de01f3e2e0 Add Maniphest Task email creator to CCs
Summary: Fixes T9369.

Test Plan:
  - Sent a mail with Mail.app to `bugs@local.phacility.com`.
  - Used "View Raw Mail", copy-pasted it into `mail.txt` on disk.
  - Ran `cat mail.txt | ./scripts/mail/manage_mail.php --process-duplicates`.
  - Saw task get created and me get added as CC.
  - Changed "To" to include another user, ran command again, saw task get created and other user get added as CC.

Reviewers: chad

Reviewed By: chad

Subscribers: Korvin

Maniphest Tasks: T9369

Differential Revision: https://secure.phabricator.com/D14086
2015-09-09 14:07:07 -07:00
epriestley
29948eaa5b Use phutil_hashes_are_identical() when comparing hashes in Phabricator
Summary: See D14025. In all cases where we compare hashes, use strict, constant-time comparisons.

Test Plan: Logged in, logged out, added TOTP, ran Conduit, terminated sessions, submitted forms, changed password. Tweaked CSRF token, got rejected.

Reviewers: chad

Reviewed By: chad

Subscribers: chenxiruanhai

Differential Revision: https://secure.phabricator.com/D14026
2015-09-01 15:52:44 -07:00
cburroughs
786b135a66 variable days back for bin/mail volume
Summary:
Email is so exciting I can't wait 30 days for initial results.

ref T9161

Test Plan:
 * `./bin/mail volume --days 60` took longer and gave plausibly larger
   results.
 * `./bin/mail volume --days 0` quickly told me no mail had been sent.
 * `./bin/mail volume` Said it was still looking 30 days back.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Maniphest Tasks: T9161

Differential Revision: https://secure.phabricator.com/D13901
2015-08-27 04:40:45 -07:00
epriestley
779a612e41 Fix mail parameter error with old migrations
Summary:
Fixes T9251. Old mail could get saved with bad parameters for two reasons that I can come up with:

  - Nothing ever set a parameter on it -- not sure this could ever actually happen; or
  - some field contained non-UTF8 data prior to D13939 and we silently failed to encode it.

My guess is that the second case is probably the culprit here.

In any case, recover from this so `20150622.metamta.5.actor-phid-mig.php` can proceed.

Test Plan: Same effective patch as user patch in T9251; looked at some mail to make sure it was still pulling parameters properly.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9251

Differential Revision: https://secure.phabricator.com/D13990
2015-08-24 09:37:48 -07:00
epriestley
7a1bbe6634 Add basic support for Herald outbound rules
Summary: Ref T5791. This is still very basic (no global actions, no support for matching headers/bodies/recipients/etc) but gets the core in.

Test Plan:
{F715209}

{F715211}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13897
2015-08-15 10:54:33 -07:00
epriestley
3b987a93ce Consolidate outbound mail status in a new class
Summary: Ref T5791. This collects outbound mail status in one place and makes the list view a little spiffier.

Test Plan: Looked at list and detail views. Grepped for changed classes/constants.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13884
2015-08-14 04:31:42 -07:00
Joshua Spence
368f359114 Use PhutilClassMapQuery instead of PhutilSymbolLoader
Summary: Use `PhutilClassMaQuery` instead of `PhutilSymbolLoader`, mostly for consistency. Depends on D13588.

Test Plan: Poked around a bunch of pages.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13589
2015-08-14 07:49:01 +10:00
epriestley
8c06d89070 Flesh out web UI for mail a bit to prepare for Herald outbound rules
Summary:
Ref T9141. Ref T5791. Ref T7013. Major changes here is:

  - Currently, we don't store the headers we actually sent, or the reasons we actually did or did not deliver a mail.
    - Start storing these (as `headers.sent` and `actors.sent`).
    - Show them in the web UI.
    - Show them in `bin/mail show-outbound` (previously, we sort of re-computed them in a hacky way).
    - Take them into account in `bin/mail volume`.

Then some minor changes:

  - Show mail bodies.
  - Show more mail information.
  - Start renaming "MetaMTA" to "Mail", at least in the web UI.

Test Plan:
{F707501}
{F707502}
{F707503}
{F707504}
{F707505}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5791, T7013, T9141

Differential Revision: https://secure.phabricator.com/D13878
2015-08-12 12:27:31 -07:00
epriestley
50e084dcda Add a rough bin/mail volume command for showing mail volume
Summary: Ref T7013. This might help us understand the problem better.

Test Plan:
```
$ ./bin/mail volume
+==============+============+
| User         | Unfiltered |
+==============+============+
| admin        | 136        |
| dog          | 31         |
| epriestley   | 24         |
| ducksey      | 18         |
| saurus       | 8          |
| example-list | 7          |
| squeakybirdo | 3          |
| nnn          | 3          |
| facebooker   | 2          |
+==============+============+

Mail sent in the last 30 days.
"Unfiltered" is raw volume before preferences were applied.
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7013

Differential Revision: https://secure.phabricator.com/D13813
2015-08-06 11:32:17 -07:00
epriestley
5f76c71d78 Share target filtering code in HeraldAction
Ref T8726. This shares some target filtering code with the base class.
2015-08-03 14:34:37 -07:00
epriestley
6f6d88794b Modularize the Diffusion "Add Auditors" Herald action
Ref T8726.
2015-08-03 14:33:27 -07:00
epriestley
51fead17cf Modularize "Send an Email" Herald actions
Summary: Ref T8726. No surprises.

Test Plan:
Created rules using both action variants, applied upgrade, saw rules still work correctly.

{F658842}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13701
2015-08-03 14:33:23 -07:00
epriestley
0306eb70ed Group and order Herald fields
Summary:
Ref T8726. Some adapters now have a large number of fields, and we lost the sort-of-human-readable implicit ordering when fields were modularized.

Instead, group and sort fields.

Test Plan: {F603066}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13619
2015-07-16 14:13:13 -07:00
epriestley
715233fb61 Fully modularize Herald field values
Summary: Ref T8726. This gets rid of all the `VALUE_*` constants and lets Fields provide arbitrary typeaheads without upstream/JS changes.

Test Plan: Used all tokenizers. Used "Another Herald Rule". Grepped for all removed constants.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13616
2015-07-16 14:12:44 -07:00
epriestley
b2b739c709 Reduce code duplication in Herald fields
Summary:
Ref T8726. I want to modularize values and reduce how hard-coded / copypasta'd they are.

  - Rename `get...StandardCondition()` to `get...StandardType()`, since we can drive both conditions and values from it.
  - Rename `STANDARD_LIST` to `STANDARD_PHID_LIST` for consistency: all "lists" are lists of PHIDs.
  - For all standard types which don't require typehaeads, lift their logic into the base class.
  - I'll lift typeaheads soon, but need to generalize them first.

Test Plan: Edited various Herald rules, saw value UI generate correctly.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13612
2015-07-16 14:11:44 -07:00
epriestley
b36d4025cb Fix a typo in the new "ApplicationEmailHeraldField"
Summary: This rule isn't quite right.

Test Plan: Shipped email in with a rule in effect, saw correct value in transcript.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D13578
2015-07-07 06:22:19 -07:00
Joshua Spence
f695dcea9e Use PhutilClassMapQuery
Summary: Use `PhutilClassMapQuery` where appropriate.

Test Plan: Browsed around the UI to verify things seemed somewhat working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13429
2015-07-07 22:51:57 +10:00
epriestley
9f220995b2 Modularize Application Email Herald field
Summary:
Ref T8726. There's no interface we can check for this, so the adapter needs to opt in.

Also fix a spelling mistake.

Test Plan: Created rules with "Application Email" fields.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13499
2015-07-06 13:16:26 -07:00
Joshua Spence
f2435fd1d0 Return $this from setter methods
Summary: Return `$this` from setter methods for consistency. I started writing a [[https://secure.phabricator.com/differential/diff/32506/ | linter rule]] to detect this, but I don't think it is trivial to do this properly.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13422
2015-07-06 22:53:43 +10:00
Joshua Spence
acb1eb81cc Move some PhabricatorSearchField subclasses
Summary: Move some `PhabricatorSearchField` subclasses to be adjacent to the application to which they belong. This seems generally better to me than lumping them all together in the `src/applications/search/field/` directory. I was also wondering if it makes sense to rename these subclasses as `PhabricatorXSearchField` rather than `PhabricatorSearchXField` (as per T5655), but wasn't really sure if these objects are meant to be search-fields, or just fields belonging to the #search application.

Test Plan: N/A.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13374
2015-07-06 22:52:05 +10:00
epriestley
1e3c49086e Merge branch 'master' into redesign-2015 2015-06-28 07:41:46 -07:00
epriestley
ae00d08b88 Fix GC threshold for mail to be 90 days instead of 0 seconds
See D13408.
2015-06-28 07:39:36 -07:00
epriestley
14a395ed8e Merge branch 'master' into redesign-2015 2015-06-25 10:06:50 -07:00
Bob Trahan
189fb2660a MetaMTA - make sure mail garbage collection also cleans up recipient edges
Summary: Ref T5791. This edge table grows 2+X faster than the corresponding mail table depending on usage. Ergo, lets make sure to clean that up too in the delete code.

Test Plan: careful thought

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13408
2015-06-24 11:44:49 -07:00
Bob Trahan
c1dca8238f MetaMTA - make the new mail app UI a little less bad
Summary: Ref T5791. Makes the result page a little less ugly and adds the time created. Makes the detail page include more useful information about the message.

Test Plan: viewed results page and detail page and they looked better

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13410
2015-06-23 18:30:05 -07:00
epriestley
729606ba93 Update BulkJob and MetaMTA search engines for redesign-2015 2015-06-23 13:39:27 -07:00
epriestley
075f041bec Merge branch 'master' into redesign-2015 2015-06-23 13:36:47 -07:00
epriestley
3215899925 Execute Maniphest batch edits in the background with a web UI progress bar
Summary:
Ref T8637. This does nothing interesting, just has empty scaffolding for a bulk job queue.

Basic idea is that when you do something like a batch edit in Maniphest, we:

  - Create a BulkJob with all the details.
  - Queue a worker to start the job.
  - Send you to a progress bar page for the job.

In the background:

  - The "start job" worker creates a ton of Task objects, then queues worker tasks to do the work.

In the foreground:

  - Fancy ajax animates the progress bar and it goes wooosh.

In general:

  - Big jobs actually work.
  - Jobs get logged.
  - You can monitor jobs.
  - Terrible junk like T8637 should be much harder to write and much easier to catch and diagnose.

Test Plan:
No interesting code/beahavior yet. Clean `storage adjust`.

{F526411}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8637

Differential Revision: https://secure.phabricator.com/D13392
2015-06-23 13:36:16 -07:00
Bob Trahan
7e0249d68c MetaMTA - more progress to mail app
Summary:
Ref T5791. This diff adds a "sensitive" flag to `PhabricatorMetaMTAMail`, defaults it to true in the constructor, and then sets it to false in teh application transaction editor. Assumption here is that sensitive emails are basically all the emails that don't flow through the application transaction editor.

This diff also gets a basic "mail view" page up and going.

This diff also fixes a bug writing recipient edges; the actor was being included.

This bug also fixes a querying bug; we shouldn't do the automagic join of $viewer is recipient or $viewer is actor if folks are querying for recipients or actors already. The bug manifested itself as having the "inbox" be inbox + outbox.

Test Plan: viewd list of messages. viewed message detail.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13406
2015-06-23 12:55:44 -07:00
Bob Trahan
dfef8e2f07 MetaMTA - more progress towards a mail application
Summary:
Ref T5791. This diff does a few things...

 - Adds code to write recipients to edges on save
 - Makes Query performance for policy filtering okay-ish
 - Adds a Search Engine for PhabricatorMetaMTAMail
 - Adds "working" List Controller
   - Inbox and Outbox both work
 - Adds stub View Controller

Test Plan: ran `./bin/storage upgrade` and saw my inbox and outbox start getting data. played with application and saw new entries in inbox and outbox

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13397
2015-06-23 11:37:14 -07:00
epriestley
1ccdb941b9 Merge branch 'master' into redesign-2015 2015-06-22 15:52:40 -07:00
Bob Trahan
4be568d346 MetaMTA - save actorPHID as its own column
Summary: Ref T5791. This should make performance snappy wrt policy checks in some future diff where the Query is updated and in use somewhere in the application.

Test Plan: ran `./bin/storage upgrade`. commented on a task and saw actorPHID populated correctly in underlying MetaMTAMail object database entry

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13396
2015-06-22 14:14:21 -07:00
Bob Trahan
ea5462fb60 MetaMTA - lay some ground work for having an application
Summary:
Ref T5791. This does a few bits there. Namely:

 - Adds PHID column to PhabricatorMetaMTAMail
   - Implements a PhabricatorMetaMTAMailPHIDType
   - Script to backpopulate them.
 - Makes PhabricatorMetaMTAMail implement PolicyInterface.
   - View policy is NOONE and the author and recipients have automatic view capabilities
   - No edit capability.
 - Adds a PhabricatorMetaMTAMailQuery for PhabricatorMetaMTAMail.

Test Plan: ran `./bin/storage upgrade` successfully. commented on a maniphest task and verifed the metamta mail object in the database was created successfully with a shiny new phid

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5791

Differential Revision: https://secure.phabricator.com/D13394
2015-06-22 13:46:26 -07:00
epriestley
f1b7fd483e Merge branch 'master' into redesign-2015 2015-06-16 19:34:36 -07:00
epriestley
30c4783c42 Dramatically limit the range of failures which can cause duplicate mail
Summary:
Ref T8574. Currently, failures during mail body construction, feed publishing, or search indexing could cause us to retry the publishing task and potentially send duplicate mail.

Instead, build (but do not send) the mail first, then send all the mail at the very end.

This isn't completley perfect, but should make it enormously harder for duplicate mail to be generated.

Test Plan: Sent some mail, ran the daemons, saw it show up normally in the outbound queue.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8574

Differential Revision: https://secure.phabricator.com/D13320
2015-06-16 16:43:24 -07:00
epriestley
c5c5523c1f Add missing $mailer property to PhabricatorMailImplementationPHPMailerAdapter
Fixes T8541.

Auditors: joshuaspence
2015-06-16 13:55:17 -07:00
epriestley
53ef057b1b Merge branch 'master' into redesign-2015 2015-06-15 08:06:23 -07:00
Joshua Spence
62648237c2 Fix a few more undeclared properties
Summary: Ref T8538. Ref T8539. Ref T8541.

Test Plan: Eyeball it.

Reviewers: epriestley, de_jean_7777, benoittgt, #blessed_reviewers

Reviewed By: benoittgt

Subscribers: epriestley, Korvin

Maniphest Tasks: T8541, T8538, T8539

Differential Revision: https://secure.phabricator.com/D13293
2015-06-15 19:39:04 +10:00
Joshua Spence
1239cfdeaf Add a bunch of tests for subclass implementations
Summary: Add a bunch of tests to ensure that subclasses behave.

Test Plan: `arc unit`

Reviewers: eadler, #blessed_reviewers, epriestley

Reviewed By: eadler, #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13272
2015-06-15 18:13:27 +10:00
Joshua Spence
b6d745b666 Extend from Phobject
Summary: All classes should extend from some other class. See D13275 for some explanation.

Test Plan: `arc unit`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13283
2015-06-15 18:02:27 +10:00
epriestley
0bc8382dfd Support Spaces in ApplicationEmail
Summary:
Ref T8498. Allow ApplicationEmail addresses to be put into spaces:

  - You can only see and send to addresses in Spaces you have access to.
  - Objects are created into the same space their address is associated with.

Test Plan:
  - Used `bin/mail receive-test` to send mail to various `xyz-bugs@...` addresses.
  - Saw objects created in the proper space.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8498

Differential Revision: https://secure.phabricator.com/D13247
2015-06-11 10:23:56 -07:00
epriestley
6d6211d441 Use ApplicationTransactions in ApplicationEmail
Summary:
Ref T8498. I want to add Spaces to these, and the logic for getting Spaces right is a bit tricky, so swap these to ApplicationTransactions.

One new piece of tech: made it easier for Editors to raise DuplicateKeyException as a normal ValidationException, so callers don't have to handle this case specially.

One behavioral change: we no longer require these addresses to be at the `auth.email-domains` domains -- I think this wasn't quite right in the general case. It's OK to require users to have `@mycompany.com` addresses but add `@phabricator.mycompany-infrastructure.com` addresses here if you want.

Test Plan:
  - Tried to create a duplicate email.
  - Tried to create an empty email.
  - Tried to create an invalid email.
  - Created a new email.
  - Deleted an email.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8498

Differential Revision: https://secure.phabricator.com/D13246
2015-06-11 10:15:49 -07:00
epriestley
57b898af9a Merge branch 'master' into redesign-2015 2015-06-10 07:44:58 -07:00
Joshua Spence
f47e69c015 Mark some strings for translation
Summary: Add some more `pht`izations.

Test Plan: Eyeball it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13200
2015-06-09 23:06:52 +10:00
epriestley
c3b11439f2 Apply Herald subscription effects immediately
Summary:
Fixes T8464. We could incorrectly use a cached value when computing CC's.

Just load a fresh value. There are no other callers that would benefit from this cache, so it's more complicated to reload it correctly prior to publishing than to just skip it.

Also make the PHID headers unique.

Test Plan:
  - Verified that users received mail about the transactions which caused them to be added to an object.
  - Veirfied that headers no longer have redundant values.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8464

Differential Revision: https://secure.phabricator.com/D13206
2015-06-08 10:50:13 -07:00
Joshua Spence
5914bbd806 Remove *TransactionType classes
Summary: Remove the `*TransactionType` classes and define the constants in the corresponding `*Transaction` class instead.

Test Plan: `grep`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13188
2015-06-08 11:26:43 +10:00
epriestley
6db97bde12 Build separate mail for each recipient, honoring recipient access levels
Summary:
Ref T6367. Removes `multiplexMail()`!

We can't pass a single body into a function which splits it anymore: we need to split recipients first, then build bodies for each recipient list. This lets us build separate bodies for each recipient's individual translation/access levels.

The new logic does this:

  - First, split recipients into groups called "targets".
    - Each target corresponds to one actual mail we're going to build.
    - Each target has a viewer (whose translation / access levels will be used to generate the mail).
    - Each target has a to/cc list (the users who we'll ultimately send the mail to).
  - For each target, build a custom mail body based on the viewer's access levels and settings (language prefs not actually implemented).
  - Then, deliver the mail.

Test Plan:
  - Read new config help.

Then did a bunch of testing, primarily with `bin/mail list-outbound` and `bin/mail show-outbound` (to review generated mail), `bin/phd debug taskmaster` (to run daemons freely) and `bin/worker execute --id <id>` (to repeatedly test a specific piece of code after identifying an issue).

With `one-mail-per-recipient` on (default):

  - Sent mail to multiple users.
  - Verified mail showed up in `mail list-outbound`.
  - Examined mail with `mail show-outbound`.
  - Added a project that a subscriber could not see.
    - Verified it was not present in `X-Phabricator-Projects`.
    - Verified it was rendered as "Restricted Project" for the non-permissioned viewer.
  - Added a subscriber, then changed the object policy so they could not see it and sent mail.
    - Verified I received mail but the other user did not.
  - Enabled public replies and verified mail generated with public addresses.
  - Disabld public replies and verified mail generated with private addresses.

With `one-mail-per-recipient` off:

  - Verified that one mail is sent to all recipients.
  - Verified users who can not see the object are still filtered.
  - Verified that partially-visible projects are completely visible in the mail (this violates policies, as documented, as the best available compromise).
  - Enabled public replies and verified the mail generated with "Reply To".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: carlsverre, epriestley

Maniphest Tasks: T6367

Differential Revision: https://secure.phabricator.com/D13131
2015-06-03 18:59:31 -07:00
epriestley
ba6cb62b49 Remove mailing lists application
Summary: Ref T8387. This is now completely obsoleted by mailing list users.

Test Plan: Grepped for `mailinglist` and related symbols.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, epriestley

Maniphest Tasks: T8387

Differential Revision: https://secure.phabricator.com/D13129
2015-06-03 18:42:36 -07:00
Joshua Spence
c36b82c7b8 Remove unused PhabricatorMail class
Summary: This class is no longer used after D13032.

Test Plan: `grep`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13118
2015-06-03 06:58:24 +10:00
Bob Trahan
c7de17663a Conpherence - massage email notification to have proper link to how to update email settings
Summary: Fixes T8329. I was able to figure out a reasonable way to have the full conpherence default to the email settings panel. I think this is cleaner than making things a dialogue as I rambled about in the description for T8329.

Test Plan: using /bin/mail to verify correct email links were generated for conpherence notifications and maniphest (general) notifications.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8329

Differential Revision: https://secure.phabricator.com/D13058
2015-05-28 15:30:33 -07:00
epriestley
bfca11dbba Merge branch 'master' into redesign-2015 2015-05-22 12:57:32 -07:00
Joshua Spence
36e2d02d6e phtize all the things
Summary: `pht`ize a whole bunch of strings in rP.

Test Plan: Intense eyeballing.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12797
2015-05-22 21:16:39 +10:00
Chad Little
a4784e03ff [Redesign] Add Table, Collapse support to ObjectBox
Summary: Converts most all tables to be directly set via `setTable` to an ObjectBox. I think this path is more flexible design wise, as we can change the box based on children, and not just CSS. We also already do this with PropertyList, Forms, ObjectList, and Header. `setCollapsed` is added to ObjectBox to all children objects to bleed to the edges (like diffs).

Test Plan: I did a grep of `appendChild($table)` as well as searches for `PHUIObjectBoxView`, also with manual opening of hundreds of files. I'm sure I missed 5-8 places. If you just appendChild($table) nothing breaks, it just looks a little funny.

Reviewers: epriestley, btrahan

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12955
2015-05-20 12:48:43 -07:00
Joshua Spence
c896aeb62e Various linter fixes
Summary: Apply various linter fixes.

Test Plan: Unit tests + eyeballing.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12390
2015-05-20 07:27:41 +10:00
epriestley
b34d5bf44b Strip GMail French quote header
Summary: Ref T8199. Add pattern matching for GMail French.

Test Plan:
  - Switched Gmail to french.
  - Replied to some mail.
  - Extracted quoted text header.
  - Added and executed unit test.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: tycho.tatitscheff, epriestley

Maniphest Tasks: T8199

Differential Revision: https://secure.phabricator.com/D12845
2015-05-14 13:04:44 -07:00
Joshua Spence
acb45968d8 Use __CLASS__ instead of hard-coding class names
Summary: Use `__CLASS__` instead of hard-coding class names. Depends on D12605.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12806
2015-05-14 07:21:13 +10:00
epriestley
5f7ee505b6 Revert a third json_decode() which decodes possible scalars
See D12714, D12680.

Auditors: joshuaspence
2015-05-05 13:03:32 -07:00
Joshua Spence
70c8649142 Use phutil_json_decode instead of json_decode
Summary: Generally, `phutil_json_decode` should be preferred over `json_decode`.

Test Plan: Eyellballed.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12680
2015-05-05 20:48:55 +10:00
epriestley
3e9ebb8e20 Add a "mailable" function datasource
Summary: Ref T4100. Add a function datasource for filtering mailable objects (e.g., subscribers).

Test Plan: Searched for objects with "Current Viewer", "Members: Dog Project", etc., as a subscriber in global search.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100

Differential Revision: https://secure.phabricator.com/D12524
2015-04-23 11:49:33 -07:00
epriestley
e5e5974d9f Give typeahead browse dialogs sensible titles
Summary: Ref T4100. Let datasources specify a more meaningful title than the class name.

Test Plan: Browsed some sources.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: chad, epriestley

Maniphest Tasks: T4100

Differential Revision: https://secure.phabricator.com/D12469
2015-04-20 10:06:23 -07:00
epriestley
f5580c7a08 Make buildWhereClause() a method of AphrontCursorPagedPolicyAwareQuery
Summary:
Ref T4100. Ref T5595.

To support a unified "Projects:" query across all applications, a future diff is going to add a set of "Edge Logic" capabilities to `PolicyAwareQuery` which write the required SELECT, JOIN, WHERE, HAVING and GROUP clauses for you.

With the addition of "Edge Logic", we'll have three systems which may need to build components of query claues: ordering/paging, customfields/applicationsearch, and edge logic.

For most clauses, queries don't currently call into the parent explicitly to get default components. I want to move more query construction logic up the class tree so it can be shared.

For most methods, this isn't a problem, but many subclasses define a `buildWhereClause()`. Make all such definitions protected and consistent.

This causes no behavioral changes.

Test Plan: Ran `arc unit --everything`, which does a pretty through job of verifying this statically.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: yelirekim, hach-que, epriestley

Maniphest Tasks: T4100, T5595

Differential Revision: https://secure.phabricator.com/D12453
2015-04-20 10:06:09 -07:00
epriestley
7db362a4b6 Cheat my way through the rest of the typeahead datasources
Summary:
Ref T5750. These are a pain to modernize and most don't matter, so cheat:

  - Mark a bunch nonbrowsable, including some that probably should be browsable but which I don't want to deal with for now.
  - For static datasources, add an easy server-side filter (this isn't really cheating, and is appropriate for the status/priority/application datasources).
  - Make composite sources browsable if their components are browsable.

Test Plan:

  - Tried to browse an unbrowsable source, got a 404.
  - Browsed a composite source.
  - Browsed static sources (priority/status/applications).
  - Browsed normal sources (people/projects).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5750

Differential Revision: https://secure.phabricator.com/D12438
2015-04-17 11:06:58 -07:00
epriestley
4fba6e7730 Remove trivial implementations of getPagingColumn()
Summary:
Ref T7803. Some Query subclasses implement getPagingColumn() in a trivial way, usually to provide a table alias.

Formalize the concept of a primary table alias, and remove obsoleted getPagingColumn() implementations.

Test Plan: Issued affected queries.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12356
2015-04-13 11:58:19 -07:00
epriestley
3c6c6552d3 Strip more Mailbox signatures
Summary: Saw this variant in a thread.

Test Plan: Unit tests.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D12349
2015-04-11 08:52:33 -07:00
epriestley
b16db61a87 Allow "send me an email" in personal rules to punch through settings
Summary:
Fixes T7731. When a user writes a "Send me an email" rule, always try send them an email, even if their notification settings would normally downgrade it to a notification.

In particular, this is stronger than these downgrades:

  - Downgrades due to "self actions";
  - downgrades due to "mail tags".

Test Plan:
  - Wrote various Herald rules with "Send me an email" rules.
  - Used `bin/mail list-outbound` / `show-outbound` to vet generated mail.
  - Mail reacted properly to a variety of conditions (disabled accounts, settings, "send me an email" rule, forced delivery).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7731

Differential Revision: https://secure.phabricator.com/D12300
2015-04-06 10:01:32 -07:00
epriestley
c0e26c65e0 Make mail delivery reasons code-based; include positive and negative reasons
Summary:
Ref T7731. Looking forward to T5791, I eventually anticipate writing an interface which looks like a webmail UI where users can review mail they've been sent and understand why they recieved (or did not receive) the mail. Roughly like `bin/mail list-outbound` / `bin/mail show-outbound` work today, but policy-aware (so you can only see messages where delivery was attempted to you).

We currently record a list of "reasons" why a mail is undeliverable, but this list is string-based (so it can not be translated once we start persisting it) and has only negative reasons (so it can not be used to fully understand reasons for delivery or nondelivery).

Make it code-based (so it can be translated) and allow both positive and negative reasons to be listed (so positive reasons can be understood).

Test Plan: Used `bin/mail show-outbound` to review mail delivery reasons, including the positive reason we currently have (forced delivery of authentication mail).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7731

Differential Revision: https://secure.phabricator.com/D12297
2015-04-06 10:01:11 -07:00
epriestley
5b1d0f9ed7 Remove "metamta.precedence-bulk" option (always enable it)
Summary: Ref T7746. This might possibly improve deliverability. Or might make it worse. Who knows?

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7746

Differential Revision: https://secure.phabricator.com/D12266
2015-04-03 16:38:27 -07:00
epriestley
a804f0ab93 Make file policies for emailed files more consistent
Summary:
Fixes T7712. Currently, files sent via email get default policies, like they were dragged and dropped onto the home page.

User expectation is better aligned with giving files more restrictive policies, like they were draggged and dropped directly onto an object.

Make files sent via email have restricted default visibility. Once we identify the sender, set them as the file author. Later, the file will become visible to other users via attachment to a task, revision, etc.

Test Plan: Sent some files via email; verified they got restrictive policies, correct authorship, and appropriate object attachment.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7712

Differential Revision: https://secure.phabricator.com/D12255
2015-04-02 13:41:39 -07:00
epriestley
eb81fd1562 Expose all application mail receivers
Summary:
Fixes T7199. This still isn't a shining example of perfect code, but the raw amount of copy/paste is much lower than it used to be.

  - Reduce code duplication between existing receivers.
  - Expose receiving objects in help menus where appropriate.
  - Connect some "TODO" receivers.

Test Plan:
  - Sent mail to every supported object type.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12249
2015-04-01 11:52:02 -07:00
epriestley
6f95b325c6 Generate mail command documentation from active commands
Summary: Ref T7199. This needs some polish and isn't reachable from the UI, but technically has all of the information.

Test Plan:
{F355899}

{F355900}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12241
2015-04-01 11:51:47 -07:00
epriestley
c32fee0e48 Fully modularize mail commands
Summary: Ref T7199. Everyone can have a mail command! You can have a mail command! You can have a mail command! Mail commands for everyone!

Test Plan: Used `bin/mail receive-test` to issue commands against files and pastes.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12238
2015-04-01 08:40:00 -07:00
epriestley
afd86a0420 Improve rules for embedding files received via email
Summary:
Ref T7199. Ref T7712. This improves the file rules for email:

  - Embed visible images as thumbnails.
  - Put all other file types in a nice list.

This "fixes" an issue caused by the opposite of the problem described in T7712 -- files being dropped if the default ruleset is too restrictive. T7712 is the real solution here, but use a half-measure for now.

Test Plan:
  - Sent mail with two non-images and two images.
  - Got a nice list of non-images and embeds of images.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7712, T7199

Differential Revision: https://secure.phabricator.com/D12235
2015-04-01 08:39:03 -07:00
epriestley
bad645f1ec Remove all application-specific reply handler domains
Summary:
Ref T7199. These were a bad idea which got copy-pasted a bunch.

  - There is zero reason to ever set these to different things.
  - Unsurprisingly, I don't know of any install which has them set to different things.

Unless I've completely forgotten about it, this option was not motivated by some obscure business need, it was just a bad decision which didn't catch anyone's attention at the time.

We partially remedied the mistake at some point by introducing `metamta.reply-handler-domain`, which works as a default for all applications, but never cleaned this mess up.

Test Plan: Sent some mail from applications, verified it picked up appropraite reply handler domains.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12231
2015-03-31 16:48:40 -07:00
epriestley
0d99c84bd7 Modernize email command parsing
Summary:
Ref T7199. This prepares for an exciting new world of more powerful "!action" commands. In particular:

  - We parse multiple commands per mail.
  - We parse command arguments (these are currently not used).
  - We parse commands at the beginning or end of mail.

Additionally:

  - Do a quick modernization pass on all handlers.
  - Break legacy compatibility with really hacky Facebook stuff (see T1992). They've theoretically been on notice for a year and a half, and their setup relies on calling very old reply handler APIs directly.
  - Some of these handlers had some copy/paste fluff.
  - The Releeph handler is unreachable, but fix it //in theory//.

Test Plan:
- Sent mail to a file; used "!unsubscribe".
- Sent mail to a legalpad document; used "!unsubscribe".
- Sent mail to a task; used various "!close", "!claim", "!assign", etc.
- Sent mail to a paste.
- Sent mail to a revision; used various "!reject", "!claim", etc.
- Tried to send mail to a pull request but it's not actually reachable.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12230
2015-03-31 16:48:27 -07:00
epriestley
030e05aa4c Remove reply handler instructions from email
Summary:
Ref T7199. Although this is useful for discovery, it's un-useful enough that we already have an option to disable it, and most applications do not provide any meaningful instructions.

Throwing it away makes it easier to move forward and lets us get rid of a config option.

This is becoming a more advanced/power-user feature anyway, and the new syntax will be significantly more complex and hard to explain with a one-liner. I'm currently thinking that I'll maybe make the "help" menu a dropdown and give it some options like:

  +---+
  | O |
  +---+---------------------+
  | Maniphest Documentation |
  | Maniphest Email Actions |
  +-------------------------+

Then you click the "Email Actions" thing and get a runtime-derived list of available options. Not sure if I'll actually build that, but I think we can fairly throw the in-mail instructions away even if we don't go in that specific direction.

Test Plan: Grepped for `replyHandlerInstructions`, got no hits.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12229
2015-03-31 16:48:17 -07:00
epriestley
d403700e1f Convert all tokenizers to take token/scalar inputs
Summary: Ref T7689. Ref T4100. This advances the goals of removing `loadViewerHandles()` (only 67 callsites remain!) and letting tokenizers some day take token functions like `viewer()` and `members(differential)`.

Test Plan:
- Sent a new message; used "To".
  - I simplified the cancel URI construction slightly because it's moot in all normal cases.
- Edited a thread; used "Add Participants".
- Searched rooms; used "Participants".
- Searched countdowns; used "Authors".
- Created a diff; used "Repository".
- Edited a revision; edited "Projects"; edited "Reveiwers"; edited "Subscribers".
- Searched for revisions; edited "responsible users"; "authors"; "reviwers"; "subscribers"; "repositories".
- Added revision comments; edited "Add Reveiwers"; "Add Subscribers".
- Commented on a commit; edited "Add Auditors"; "Add subscribers".
- Edited a commit; edited "Projects".
- Edited a repository; edited "Projects".
- Searched feed, used "include Users"; "include Proejcts".
- Searched files, used "authors".
- Edited initiative; edited "Projects".
- Searched backers; used "Backers".
- Searched initiatives; used "Owners".
- Edited build plans; edited "Run Command".
- Searched Herald; used "Authors".
- Added signature exemption in Legalpad.
- Searhced legalpad; used "creators"; used "contributors".
- Searched signatures; used "documents"; used "signers".
- Created meme.
- Searched macros; used "Authors".
- Used "Projects" in Maniphest reports.
- Used Maniphest comment actions.
- Edited Maniphest tasks; edited "Assigned To"; edited "CC"; edited "projects".
- Used "parent" in Maniphest task creation workflow.
- Searched for projects; used "assigned to"; "in any projec"; "in all projects"; "not in projects"; "in users' projects"; "authors"; "subscribers".
- Edited Maniphest bug filing domains, used "Default Author".
- Searched for OAuth applications, used "Creators".
- Edited Owners pacakge; edited "Primary Owner"; edited "Owners".
- Searched for Owners packages; used "Owner".
  - OMG this UI is OLD
- Edited a paste; edited "Projects".
- Searched for paste; used "Authors".
- Searched user activity log; used "Actors"; used "Users".
- Edited a mock; edited "Projects"; edited "CC".
- Searched for mocks; used "Authors".
- Edited Phortune account; edited "Members".
- Edited Phortune merchant account; edited "Members".
- Searched Phrequent; used "Users".
- Edited Ponder question; sued "projects".
- Searched Ponder; used "Authors"; used "Answered By".
- Added project members.
- Searched for projects; used "Members".
- Edited a Releeph product; edited "Pushers".
- Searched pull requests; searched "Requestors".
- Edited an arcanist project; used "Uses Symbols From".
- Searhced push logs; used "Repositories"; used "Pushers".
- Searched repositories; used "In nay project".
- Used global search; used Authors/owners/Subscribers/In Any Project.
- Edited a slowvote; used "Projects".
- Searched slovotes; used "Authors".
- Created a custom "Users" field; edited and searched for it.
- Made a whole lot of typos in this list. ^^^^^^

Did not test:

- Lint is nontrivial to test locally, I'll test it in production.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100, T7689

Differential Revision: https://secure.phabricator.com/D12224
2015-03-31 14:10:55 -07:00
epriestley
86404a1a18 Fix handling of notifications with project members
Summary: Fixes T7377. We don't expand projects into members when sending notifications right now. Instead, expand them.

Test Plan:
  - Added a project as a reviewer to a revision, made a comment, saw project members receive a read notification + email (with appropriate preferences).
  - There's meaningful test coverage on the core mail stuff.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7377

Differential Revision: https://secure.phabricator.com/D12142
2015-03-24 12:47:38 -07:00
epriestley
bd2eaad04f Add "phabricator.silent" for stopping all outbound events from an install
Summary:
Ref T7522. This is mostly useful in the cluster, but could be useful for external installs too.

If you want to import an instance into a test/dry-run state (in the cluster, to test an import; in the general case, to do something like test new hardware or configuration), you currently risk spamming users with a lot of duplicate notifications. In particular, if Phabricator tracks remotes, both instances will continue importing commits and sending email about them. Both instances will try to publish to mirrors, too, which could be bad news, and both instances will try to update linked services.

Instead, provide a flag to let an instance run in "silent mode", which disables all outbound messaging and data.

We need to remember to support this flag on any new outbound channels, but we add about one of those per year so I think that's reasonable.

Test Plan:
  - Flipped config.
  - Saw it void email, feed and mirroring.
  - Didn't test SMS since it's not really in use yet and not convenient to test.
  - (Can you think of any publishing I missed?)

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7522

Differential Revision: https://secure.phabricator.com/D12109
2015-03-18 07:09:43 -07:00
epriestley
66075708d0 Allow MetaMTAMail to send with a raw "From" address
Summary:
Ref T7607. Ref T7522.

  - For the import tools, I want to send from "Phacility Support <support@phacility.com>".
  - In the general case, I want to send billing mail from merchants (T7607) later on.

Test Plan: Sent an email and saw the desired "From" address.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7607, T7522

Differential Revision: https://secure.phabricator.com/D12100
2015-03-17 14:43:21 -07:00
epriestley
973079a7da Modularize application configuration panels
Summary:
Ref T7149. This is a few steps away, but:

  - Generally, I'd like to reduce the amount of "Config" configuration we have.
  - One good way to do this is to move it into UIs in Application configuration. We did this with email recently.
  - I think this was a great change and I'd like to keep moving in this direction.
  - T7149 touches configuration related to file storage engines. Although I'm not planning to fully move configuration into applications yet, it would be easier to debug and test if I could drop a read-only panel there to show engines.
  - So, modularize the config stuff so I can add a new panel without hard-coding it.

Test Plan:
  - Added, edited, and deleted application emails.
  - Viewed non-email application detail pages.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12051
2015-03-12 13:28:37 -07:00
Bob Trahan
d598edc5f3 MetaMTA - update documentation and make config a tad easier
Summary: Fixes T7088. Mainly this updates the documentation but I also snuck in tweaking how the domain reply handler is built. This does two main things -- makes the behavior consistent as some applications who didn't override this behavior would send out emails with reply tos AND makes it easier for us to deprecate the custom domain thing on a per application basis, which is just silly. On that note, the main documentation doesn't get into how this can be overridden, though I left in that mini blurb on the config setting itself. We could deprecate this harder and LOCK things if you want as well.

Test Plan: read docs, looked good. reasoned through re-factor

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7088

Differential Revision: https://secure.phabricator.com/D11725
2015-02-12 11:05:39 -08:00
Bob Trahan
388d1ff7bd Policy - lock down file loading in mail reply handler path
Summary: Ref T7094. This one is really straight-forward since $this->actor is always populated and the right thing to do here.

Test Plan: used the ole thinking noodle since testing email w/ attachments is really hard

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7094

Differential Revision: https://secure.phabricator.com/D11589
2015-02-02 14:02:36 -08:00
Chad Little
8b06804394 Remove getIconName from all applications
Summary: Not used anymore

Test Plan: grep for 'getIconName'

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11582
2015-01-30 12:11:21 -08:00
Bob Trahan
c89dc19976 Application emails - move over paste and files
Summary: Fixes T3404 (post D11565), fixes T5952. This infrastructure has been getting deployed against Maniphest and its time to get these other two applications going on it.

Test Plan: created an email address for paste and used `./bin/mail receive-test` ; a paste was successfully created

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5952, T3404

Differential Revision: https://secure.phabricator.com/D11570
2015-01-29 14:47:32 -08:00
Bob Trahan
7d309a8e46 Application Emails - make various user email editing paths respect application emails
Summary: Ref T3404. The only mildly sketchy bit is these codepaths all load the application email directly, by-passing privacy. I think this is necessary because not getting to see an application doesn't mean you should be able to break the application by registering a colliding email address.

Test Plan:
Tried to add a registered application email to a user account via the web ui and got a pretty error.
Ran unit tests.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T3404

Differential Revision: https://secure.phabricator.com/D11565
2015-01-29 14:41:09 -08:00
Bob Trahan
ab8f7907de Herald - add support for application emails.
Summary:
Fixes T5039. The trick / possibly lame part here is we only match 1 application email and its undefined which one. e.g. if a user emails us at address x, y, and z only one of those will pick up the mail. Ergo, don't let users define non-sensical herald conditions like "matches all". Also document what I think was non-intuitive about the code with an inline comment; we have to return an array with just a phid from an object and out of context it feels very "what the...???"

Note this needs to be deployed to other applications still, but I think its okay to close T5039 aggressively here since its done from a user story perspective.

Test Plan: set up a herald rule to flag tasks created as blue via app email x. sent an email to x via `bin/mail receive-test` and verified the task had the blue flag

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5039

Differential Revision: https://secure.phabricator.com/D11564
2015-01-29 14:15:38 -08:00
Bob Trahan
fe0ca0abf2 Application Emails - add datasource so we can have a typeahead
Summary: Ref T5039. This will be necessary for Herald integration so users can make rules like "if app email is one of x, y, or z add projects foo, bar, and metallica." I think its best to do an actual typeahead here -- users select full email addresses -- rather than support prefix, suffix, etc stuff on the email address. I think the latter approach would yield lots of confusion, as well as prevent us from (more) easily providing diagnostic tools about what happened when and why.

Test Plan: hacked a maniphest tokenizer to use this new datasource and it worked

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5039

Differential Revision: https://secure.phabricator.com/D11546
2015-01-28 14:35:42 -08:00
Bob Trahan
799dada3ad MetaMTA - add support for "Default Author"
Summary: Ref T5952. This adds support for a "default author" and deploys it on Maniphest.

Test Plan: used augmented (by this diff) bin/mail receive-test to test creation via an application email with a default author configured and no author specified. a task was created with the author as the default author i configured.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5952

Differential Revision: https://secure.phabricator.com/D11446
2015-01-28 11:13:29 -08:00
Chad Little
5d8bb61dde Add FontIcon bridge to AppIcons
Summary: Select a similar or better FontAwesome icon to represent each application

Test Plan: Visual inspection

Reviewers: epriestley, btrahan

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11489
2015-01-24 23:43:01 -08:00
Bob Trahan
53b06408f4 MetaMTA - add (basic) application emails and deploy to Maniphest
Summary: Ref T5952, T3404. This lays the basic plumbing for how this will work, all the way to deploying on Maniphest. Aside from what is mentioned on T5952, I think page(s) on editing application emails could use a little more helpful text about what's going on, similar to how the config page that's getting deprecated works.

Test Plan: ran migration and noted my create email address migrated successfully. used bin/mail to make a task. added another email and used bin/mail to make a task. deleted an email. edited an email. invoked various error states and they all looked good.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T3404, T5952

Differential Revision: https://secure.phabricator.com/D11418
2015-01-19 16:07:26 -08:00
Joshua Spence
94b96ae533 Fix visibility of the PhabricatorWorker::doWork() methods
Summary: Ref T6822. This method is only called from within the `PhabricatorWorker::executeTask()` and `PhabricatorWorker::scheduleTask()` methods.

Test Plan: `grep`ped for `->doWork`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11406
2015-01-16 06:58:50 +11:00
Joshua Spence
d6b882a804 Fix visiblity of LiskDAO::getConfiguration()
Summary: Ref T6822.

Test Plan: `grep`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: hach-que, Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11370
2015-01-14 06:54:13 +11:00
Bob Trahan
9219645287 Daemons - add "objectPHID" to task tables.
Summary: Ref T5402. This more or less "fixes" it but there's probably some polish to do?

Test Plan:
stopped and started daemons. error logs look good.

ran bin/storage upgrade.  noted that `adjust` added the appropriate indices for active and archive task.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5402

Differential Revision: https://secure.phabricator.com/D11044
2014-12-23 16:30:05 -08:00
lkassianik
f7aa87311a Add email preference links to email footers
Summary: Ref T1217, Add link to email preferences to email template

Test Plan: Add comment to object like Maniphest task, check that email has a footer with a link to email preferences.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T1217

Differential Revision: https://secure.phabricator.com/D10883
2014-11-19 17:06:33 -08:00
lkassianik
1b438a8bd1 Process Remarkup in text and HTML email bodies appropriately
Summary: Ref T6343, adding HTMLMailMode to remarkup, and most objects should now be processed and appear pretty in emails.

Test Plan: Add a comment to a Maniphest task containing a mention of an object like '{T1}' or 'T1'. Emails should show a styled version of the object similar to how the object looks in the context of the Maniphest task in the UI.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: chad, Korvin, epriestley

Maniphest Tasks: T6343, T2617

Differential Revision: https://secure.phabricator.com/D10859
2014-11-17 18:27:21 -08:00
epriestley
ba778cffd7 Don't add "To/Cc" to HTML mail bodies if there's no HTML mail body
Summary: Ref T6576. This avoids generating almost-empty HTML mail bodies for mail which incorrectly has no HTML body.

Test Plan: Generated some mail locally; the specific hook case is a pain for me to hit right now. Will push and dig in if that doesn't fix it.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T6576

Differential Revision: https://secure.phabricator.com/D10863
2014-11-17 13:43:02 -08:00
Chad Little
1edcabe539 Have HTML mails return to and cc lines
Summary: Fixes T6525, adds cc and tos to html emails

Test Plan: send html and plain emails, see new stuff

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6525

Differential Revision: https://secure.phabricator.com/D10857
2014-11-15 09:12:17 -08:00
Chad Little
bcb360ea2b Fix HTML email section breaks
Summary: Fixes T6481. Provides better section handling.

Test Plan:
Test an email with multiple sections, multiple mail platforms.

{F231022}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6481

Differential Revision: https://secure.phabricator.com/D10811
2014-11-07 10:15:28 -08:00
Chad Little
1ac84c1b59 Add addLinkSection to MailBody to properly format URIs
Summary: Fixes T6343. Grepped for all callsites and added addLinkSection where needed.

Test Plan: Tested Differential, Maniphest, Conpherence, Ponder and Macro. Inspect HTML mail for anchor tags. Inspect text mails for non-disruption.

Reviewers: epriestley, btrahan

Reviewed By: btrahan

Subscribers: talshiri, Korvin, epriestley

Maniphest Tasks: T6343

Differential Revision: https://secure.phabricator.com/D10762
2014-10-30 15:24:10 -07:00
Bob Trahan
cd811232f3 MetaMTA - make test mail implementation support message id headers by default
Summary: Fixes T6372. Apparently ye olde error logs get some crazy spam action as is... Looking around at call sites, we do not specify $config (which could specify the supportage of message id header) so it seems correct to default this to something. I went with "true" as the spot we use this seems like pretty easy stuff that will always work.

Test Plan: lots of thinking

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6372

Differential Revision: https://secure.phabricator.com/D10749
2014-10-27 13:09:34 -07:00
Joshua Spence
3cf9a5820f Minor formatting changes
Summary: Apply some autofix linter rules.

Test Plan: `arc lint` and `arc unit`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D10585
2014-10-08 08:39:49 +11:00
epriestley
35dc510e18 Add more structure to Phortune product purchasing flow
Summary:
Ref T2787. When a user purchases a product in Phortune, transition the cart through a purchased state and invoke product callbacks so applications can respond to the workflow.

Also shore up some stuff like preventing negative amounts of funding.

Test Plan: Backed an initiative and saw it show up on the initiative after completing the purcahsing workflow.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D10635
2014-10-06 10:36:43 -07:00
epriestley
e333017c3e Strip email signatures from Mailbox
Summary: thanks mailbox

Test Plan: unit tests

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D10629
2014-10-02 12:33:43 -07:00
epriestley
8fa8415c07 Automatically build all Lisk schemata
Summary:
Ref T1191. Now that the whole database is covered, we don't need to do as much work to build expected schemata. Doing them database-by-database was helpful in converting, but is just reudndant work now.

Instead of requiring every application to build its Lisk objects, just build all Lisk objects.

I removed `harbormaster.lisk_counter` because it is unused.

It would be nice to autogenerate edge schemata, too, but that's a little trickier.

Test Plan: Database setup issues are all green.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, hach-que

Maniphest Tasks: T1191

Differential Revision: https://secure.phabricator.com/D10620
2014-10-02 09:51:20 -07:00
epriestley
4fcc634a99 Fix almost all remaining schemata issues
Summary:
Ref T1191. This fixes nearly every remaining blocker for utf8mb4 -- primarily, overlong keys.

Remaining issue is https://secure.phabricator.com/T1191#77467

Test Plan: I'll annotate inline.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, hach-que

Maniphest Tasks: T6099, T6129, T6133, T6134, T6150, T6148, T6147, T6146, T6105, T1191

Differential Revision: https://secure.phabricator.com/D10601
2014-10-01 08:18:36 -07:00
epriestley
03519c53bb Mark questionable column nullability for later
Summary:
Ref T1191. Ref T6203. While generating expected schemata, I ran into these columns which seem to have sketchy nullability.

  - Mark most of them for later resolution (T6203). They work fine today and don't need to block T1191. Changing them can break the application, so we can't autofix them.
  - Forgive a couple of them that are sort-of reasonable or going to get wiped out.

Test Plan: Saw 94 remaining warnings.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: hach-que, epriestley

Maniphest Tasks: T1191, T6203

Differential Revision: https://secure.phabricator.com/D10593
2014-10-01 07:59:44 -07:00
epriestley
d6639b68d5 Generate expected schemata for MetaMTA, Nuance, MetaData, OAuthServer
Summary: Ref T1191. Handful of minor things here (T6150, T6149, T6148, T6147, T6146) but nothing very noteworthy.

Test Plan: Viewed web UI, saw fewer errors.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1191

Differential Revision: https://secure.phabricator.com/D10527
2014-09-24 13:50:00 -07:00
Joshua Spence
0151c38b10 Apply some autofix linter rules
Summary: Self-explanatory.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10454
2014-09-10 06:55:05 +10:00
epriestley
957c1d6602 Add a setting for selecting SMTP mail encoding
Summary:
Fixes T5956. We changed the default mail encoding to `quoted-printable` to fix delivery via SendGrid via SMTP, but this broke multiple other mailers.

  - Change the default back to 8bit (which works everywhere except SendGrid).
  - Add a configuration setting for selecting `quoted-printable`.
  - Document this issue.
  - Discourage use of SendGrid in documentation.

(IMPORTANT) @klimek @nickz This reverts the `quoted-printable` fix for SendGrid. You will need to adjust your configurations (set `phpmailer.smtp-encoding` to `quoted-printable`) and restart your daemons or mail will get double newlines again.

Test Plan:
  - Sent mail via SendGrid with various `phpmailer.smtp-encoding` settings, saw mail arrive with specified encoding.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: klimek, nickz, epriestley

Maniphest Tasks: T5956

Differential Revision: https://secure.phabricator.com/D10397
2014-09-02 10:47:34 -07:00
Bob Trahan
b93bc7e479 phutil_utf8_shorten => PhutilUTF8StringTruncator
Summary: Ref T3307. Only one I thought was tricky was Excel; I went with bytes there like it was email.

Test Plan: played around on a few endpoints but mostly thought carefully

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T3307

Differential Revision: https://secure.phabricator.com/D10392
2014-08-29 15:15:13 -07:00
epriestley
7e655da977 Use a single newline between HTML mail sections
Summary: Ref T992. This makes HTML mail layout more consistent with text mail layout and fixes my greatest annoyance with it.

Test Plan: Used `bin/mail list-outbound --id <id> --dump-html` to view mail in Safari, saw it have a normal amount of whitespace between sections.

Reviewers: btrahan, talshiri, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T992

Differential Revision: https://secure.phabricator.com/D10344
2014-08-25 11:35:29 -07:00
epriestley
7d31ea7c55 Minor tweaks to bin/mail send-test
Summary: Clean up some arg handling stuff.

Test Plan: Used this while debugging.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D10314
2014-08-21 11:25:44 -07:00
epriestley
b7ad48aa50 Minor, also update PHPMailerLite to use quoted-printable
Summary: See T5927.

Auditors: btrahan
2014-08-20 17:18:49 -07:00
epriestley
5449200972 Send SMTP mail using quoted-printable encoding in all cases
Summary: See D10278. This works around a bug (?) in SendGrid, see http://stackoverflow.com/questions/6276181/extra-newlines-in-plain-text-emails-sent-via-sendgrid

Test Plan: @nickz confirmed this resolved his issue. See also D10278.

Reviewers: nickz, btrahan, chad

Reviewed By: chad

Subscribers: nickz, epriestley

Differential Revision: https://secure.phabricator.com/D10303
2014-08-19 17:36:38 -07:00
epriestley
8403812e15 Make HTML email a little easier to debug
Summary:
Ref T992.

  - Format text/HTML bodies explicitly in `bin/mail show-outbound`.
  - Provide `bin/mail show-outbound --dump-html` so you can do something like `bin/mail show-outbound --dump-html > dump.html; open dump.html` to get a browser preview somewhat easily.

Test Plan: Ran `bin/mail show-outbound` with and without `--dump-html` flag.

Reviewers: talshiri, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T992

Differential Revision: https://secure.phabricator.com/D10272
2014-08-15 11:07:33 -07:00
Tal Shiri
4c57e6d34d HTML emails
Summary:
Added support for side-by-side HTML and plaintext email building.

We can control if the HTML stuff is sent by by a new config, metamta.html-emails

Test Plan:
Been running this in our deployment for a few months now.

====Well behaved clients====
 - Gmail
 - Mail.app

====Bad clients====

- [[ http://airmailapp.com/ | Airmail ]]. They confuse Gmail too, though.

====Need testing====
 - Outlook (Windows + Mac)

Reviewers: chad, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: webframp, taoqiping, chad, epriestley, Korvin

Maniphest Tasks: T992

Differential Revision: https://secure.phabricator.com/D9375
2014-08-15 08:12:21 -07:00
epriestley
8ef1ea63dd Add "unblock" and "column" mail tags to Maniphest
Summary: Fixes T5769. Fixes T5861. Add mail tags for "unblock" and "column change".

Test Plan: Did unblocks and column changes, verified the mail got the right mailtags and recipient nondelivery flags.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5861, T5769

Differential Revision: https://secure.phabricator.com/D10241
2014-08-12 14:07:37 -07:00
epriestley
c443913c0b Allow users to set notifications to "Email", "Notification", or "Ignore"
Summary:
Ref T5861. Ref T5769. If users don't care at all about something, allow them to ignore it.

We have some higher-volume notifications either built now (column changes) or coming (mentions) which users might reasonably want to ignore completely.

Test Plan:
Ignored some notifications, then took appropraite actions. Saw my user culled from the notification subscriber list.

{F189531}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5769, T5861

Differential Revision: https://secure.phabricator.com/D10240
2014-08-12 12:29:03 -07:00
epriestley
d011f8fdc6 Add a setting to disable all notification email
Summary: Ref T5861. Adds an option to opt out of all notification email. We'll still send you password resets, email verifications, etc.

Test Plan:
{F189484}

  - Added unit tests.
  - With preference set to different things, tried to send myself mail. Mail respected preferences.
  - Sent password reset email, which got through the preference.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: rush898, epriestley

Maniphest Tasks: T5861

Differential Revision: https://secure.phabricator.com/D10237
2014-08-12 12:28:29 -07:00
epriestley
500506bfef Persist excluded recipients when saving mail
Summary:
Fixes T5185. The fundamental issue is that this `excludePHIDs` property was not saved, so the logic went like this:

  - Generate `excludePHIDs` correctly.
  - Pass `excludePHIDs` through the stack.
  - Perform some other computations correctly.
  - Queue the mail for the daemons, throwing it away. {icon bomb}
  - Daemons process mail with empty `excludePHIDs` list.

Store it in the persistent properties array instead.

Also remove the "override self mail" thing, since it's only used by `bin/mail send-test` and suffers from the same issue. I think it's too useless to fix, since even if you get caught by it, `bin/mail` makes it clear why the message was dropped.

Test Plan:
Notable:

  - `exclude` present in properties
  - Exclusion reason under RECIPIENTS header

{P1229}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5185

Differential Revision: https://secure.phabricator.com/D10234
2014-08-12 12:28:07 -07:00