1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-30 18:52:42 +01:00
Commit graph

7694 commits

Author SHA1 Message Date
epriestley
52a29be70d Introduce a request cache mechanism
Summary:
Ref T8424. This adds a standard KeyValueCache to serve as a request cache.

In particular, I need to cache Spaces (they are frequently accessed, sometimes by multiple viewers) but not have them survive longer than the scope of one request.

This request cache is explicitly destroyed by each web request and each daemon request.

In the very long term, building this kind of construct supports reusing PHP interpreters to run web requests (see some discussion in T2312).

Test Plan:
  - Added and executed unit tests.
  - Ran every daemon.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8424

Differential Revision: https://secure.phabricator.com/D13153
2015-06-04 17:27:31 -07:00
epriestley
e5b923743a Make CSRF salt per-user instead of per-request
Summary:
Fixes T8326. This removes calls to PhabricatorStartup from places that daemons may access.

This salt doesn't need to be global; it's embedded in the token we return. It's fine if we use a different salt every time. In practice, we always use the same viewer, so this change causes little or no behavioral change.

Ref T8424. For Spaces, I need a per-request cache for all spaces, because they have unusual access patterns and require repeated access, in some cases by multiple viewers.

We don't currently have a per-request in-process cache that we, e.g., clear in the daemons.

We do have a weak/theoretical/forward-looking attempt at this in `PhabricatorStartup::getGlobal()` but I'm going to throw that away (it's kind of junky, partly because of T8326) and replace it with a more formal mechanism.

Test Plan:
  - Submitted some forms.
  - Grepped for `csrf.salt`.
  - Viewed page source, saw nice CSRF tokens with salt.
  - All the salts are still the same on every page I checked, but it doesn't matter if this isn't true everywhere.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8326, T8424

Differential Revision: https://secure.phabricator.com/D13151
2015-06-04 17:26:23 -07:00
Joshua Spence
b9d004e9c4 Integrate Diviner with global search
Summary: Fixes T7458. Integrates #diviner into #applicationsearch by indexing `DivinerLiveBook` and `DivinerLiveSymbol` search documents. Depends on D13157.

Test Plan: Ran `./bin/search index --all --type BOOK` and `./bin/search index --all --type ATOM` and then searched for various symbols via global search.

Reviewers: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T7458

Differential Revision: https://secure.phabricator.com/D13090
2015-06-05 07:27:48 +10:00
Joshua Spence
0d50c40973 Fix a few Diviner method calls
Summary: I missed these in D13157.

Test Plan: Browsed around #diviner.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13158
2015-06-05 07:27:13 +10:00
Joshua Spence
32f669d566 Invert include/exclude logic on DivinerAtomQuery
Summary: Fixes T8401. Change `withIncludeGhosts()` to `withExcludeGhosts()` and `withIncludeUndocumentable()` to `withExcludeDocumentable()`. In particular, this allows querying for atoms by PHID to work as expected.

Test Plan: I got confused with double negatives so I might have gotten some of these wrong... I poked around Diviner and re-generated documentation to verify that this is working as expected.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T8401

Differential Revision: https://secure.phabricator.com/D13157
2015-06-05 07:23:19 +10:00
epriestley
0fc0af6443 Let Maniphest send mail again.
Ha ha ha! Fixes T8422.

Auditors: btrahan
2015-06-04 09:49:10 -07:00
epriestley
98aae51c3d Fix an issue with mentions in transactions
Ref T6367. Fixes T8415.

Maniphest filters transactions too early. This happens automatically later. Remove the code.

Transactions should be filtered per-user. If a transaction is hidden for some users, we shouldn't mail them. Move the filtering logic to be per-user.

Stack:

```
[Thu Jun 04 05:51:05.371061 2015] [:error] [pid 25111] [client 127.0.0.1:56497] [2015-06-04 05:51:05] EXCEPTION: (Exception) Transaction ("PHID-XACT-TASK-x4jlylat47s6ttr", of type "core:edge") requires a handle ("PHID-DREV-rs7jaoxbcb3av6biq4b5") that it did not load. at [<phabricator>/src/applications/transactions/storage/PhabricatorApplicationTransaction.php:277]
[Thu Jun 04 05:51:05.372546 2015] [:error] [pid 25111] [client 127.0.0.1:56497] arcanist(head=master, ref.master=4e83efb31d3e), instances(head=master, ref.master=db56d5d6ad91), ledger(head=master, ref.master=5694485699a4), libcore(), phabricator(head=xaction1, ref.master=04a22a8fa443, ref.xaction1=04a22a8fa443, custom=17), phutil(head=master, ref.master=c2cd90ee7aec), services(head=master, ref.master=2d76591c4f87)
[Thu Jun 04 05:51:05.372559 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #0 <#2> PhabricatorApplicationTransaction::getHandle(string) called at [<phabricator>/src/applications/transactions/storage/PhabricatorApplicationTransaction.php:463]
[Thu Jun 04 05:51:05.372564 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #1 <#2> PhabricatorApplicationTransaction::shouldHide() called at [<phabricator>/src/applications/maniphest/storage/ManiphestTransaction.php:163]
[Thu Jun 04 05:51:05.372568 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #2 <#2> ManiphestTransaction::shouldHide() called at [<phutil>/src/utils/utils.php:428]
[Thu Jun 04 05:51:05.372572 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #3 <#2> mfilter(array, string, boolean) called at [<phabricator>/src/applications/maniphest/editor/ManiphestTransactionEditor.php:380]
[Thu Jun 04 05:51:05.372576 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #4 <#2> ManiphestTransactionEditor::shouldSendMail(ManiphestTask, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:957]
[Thu Jun 04 05:51:05.372580 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #5 <#2> PhabricatorApplicationTransactionEditor::applyTransactions(ManiphestTask, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2858]
[Thu Jun 04 05:51:05.372585 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #6 <#2> PhabricatorApplicationTransactionEditor::applyInverseEdgeTransactions(DifferentialRevision, DifferentialTransaction, integer) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:569]
[Thu Jun 04 05:51:05.372589 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #7 <#2> PhabricatorApplicationTransactionEditor::applyBuiltinExternalTransaction(DifferentialRevision, DifferentialTransaction) called at [<phabricator>/src/applications/differential/editor/DifferentialTransactionEditor.php:615]
[Thu Jun 04 05:51:05.372594 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #8 <#2> DifferentialTransactionEditor::applyBuiltinExternalTransaction(DifferentialRevision, DifferentialTransaction) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:489]
[Thu Jun 04 05:51:05.372611 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #9 <#2> PhabricatorApplicationTransactionEditor::applyExternalEffects(DifferentialRevision, DifferentialTransaction) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:827]
[Thu Jun 04 05:51:05.372616 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #10 <#2> PhabricatorApplicationTransactionEditor::applyTransactions(DifferentialRevision, array) called at [<phabricator>/src/applications/differential/controller/DifferentialCommentSaveController.php:124]
[Thu Jun 04 05:51:05.372621 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #11 <#2> DifferentialCommentSaveController::processRequest() called at [<phabricator>/src/aphront/AphrontController.php:33]
[Thu Jun 04 05:51:05.372625 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #12 <#2> AphrontController::handleRequest(AphrontRequest) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:226]
[Thu Jun 04 05:51:05.372629 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #13 phlog(Exception) called at [<phabricator>/src/aphront/configuration/AphrontDefaultApplicationConfiguration.php:226]
[Thu Jun 04 05:51:05.372633 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #14 AphrontDefaultApplicationConfiguration::handleException(Exception) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:230]
[Thu Jun 04 05:51:05.372637 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #15 AphrontApplicationConfiguration::processRequest(AphrontRequest, PhutilDeferredLog, AphrontPHPHTTPSink, MultimeterControl) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:140]
[Thu Jun 04 05:51:05.372641 2015] [:error] [pid 25111] [client 127.0.0.1:56497]   #16 AphrontApplicationConfiguration::runHTTPRequest(AphrontPHPHTTPSink) called at [<phabricator>/webroot/index.php:19]
```

Auditors: btrahan
2015-06-04 06:05:24 -07:00
epriestley
04a22a8fa4 Fix slop with previous patch, perhaps
Auditors: btrahan
2015-06-03 20:11:33 -07:00
epriestley
ef43a98440 Reload commits before publishing mail about them
This fixes this, which only crops up some of the time:

```
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000] [2015-06-04 03:03:10] EXCEPTION: (PhutilProxyException) Error while executing Task ID 902271. {>} (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorRepositoryCommit (via getCommitData()), but the data is not actually attached. Before accessing attachable data on an object, you must load and attach it.
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000] Data is normally attached by calling the corresponding needX() method on the Query class when the object is loaded. You can also call the corresponding attachX() method explicitly. at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:166]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000] arcanist(head=master, ref.master=8c589f1f759f), phabricator(head=master, ref.master=4a45620b0458), phutil(head=master, ref.master=afc05a9a7f00)
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #0 <#2> PhabricatorLiskDAO::assertAttached(string) called at [<phabricator>/src/applications/repository/storage/PhabricatorRepositoryCommit.php:129]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #1 <#2> PhabricatorRepositoryCommit::getCommitData() called at [<phabricator>/src/applications/audit/editor/PhabricatorAuditEditor.php:669]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #2 <#2> PhabricatorAuditEditor::buildMailBody(PhabricatorRepositoryCommit, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2178]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #3 <#2> PhabricatorApplicationTransactionEditor::sendMailToTarget(PhabricatorRepositoryCommit, array, PhabricatorMailTarget) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2152]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #4 <#2> PhabricatorApplicationTransactionEditor::sendMail(PhabricatorRepositoryCommit, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:998]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #5 <#2> PhabricatorApplicationTransactionEditor::publishTransactions(PhabricatorRepositoryCommit, array) called at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:21]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #6 <#2> PhabricatorApplicationTransactionPublishWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:91]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #7 <#2> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:162]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #8 <#2> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:20]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #9 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:185]
Daemon 39827 STDE [Thu, 04 Jun 2015 03:03:11 +0000]   #10 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:125]
```

Auditors: btrahan
2015-06-03 20:08:04 -07:00
epriestley
4a45620b04 Reload revisions before publishing mail about them
Fixes this, which only triggers on some kinds of mail:

```
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000] [2015-06-04 02:56:38] EXCEPTION: (PhutilProxyException) Error while executing Task ID 902251. {>} (PhabricatorDataNotAttachedException) Attempting to access attached data on DifferentialRevision (via getActiveDiff()), but the data is not actually attached. Before accessing attachable data on an object, you must load and attach it.
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000] Data is normally attached by calling the corresponding needX() method on the Query class when the object is loaded. You can also call the corresponding attachX() method explicitly. at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:166]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000] arcanist(head=master, ref.master=8c589f1f759f), phabricator(head=master, ref.master=6dede2e2c513), phutil(head=master, ref.master=afc05a9a7f00)
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #0 <#2> PhabricatorLiskDAO::assertAttached(string) called at [<phabricator>/src/applications/differential/storage/DifferentialRevision.php:158]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #1 <#2> DifferentialRevision::getActiveDiff() called at [<phabricator>/src/applications/differential/customfield/DifferentialBranchField.php:72]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #2 <#2> DifferentialBranchField::updateTransactionMailBody(PhabricatorMetaMTAMailBody, DifferentialTransactionEditor, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2481]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #3 <#2> PhabricatorApplicationTransactionEditor::addCustomFieldsToMailBody(PhabricatorMetaMTAMailBody, DifferentialRevision, array) called at [<phabricator>/src/applications/differential/editor/DifferentialTransactionEditor.php:1208]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #4 <#2> DifferentialTransactionEditor::buildMailBody(DifferentialRevision, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2178]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #5 <#2> PhabricatorApplicationTransactionEditor::sendMailToTarget(DifferentialRevision, array, PhabricatorMailTarget) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2152]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #6 <#2> PhabricatorApplicationTransactionEditor::sendMail(DifferentialRevision, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:998]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #7 <#2> PhabricatorApplicationTransactionEditor::publishTransactions(DifferentialRevision, array) called at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:21]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #8 <#2> PhabricatorApplicationTransactionPublishWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:91]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #9 <#2> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:162]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #10 <#2> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:20]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #11 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:185]
Daemon 39819 STDE [Thu, 04 Jun 2015 02:56:38 +0000]   #12 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:125]
```

Auditors: btrahan
2015-06-03 20:01:23 -07:00
epriestley
6dede2e2c5 Make PhameBlog implement PhabricatorApplicationTransactionInterface
Summary: Ref T6367.

Test Plan: Static correctness.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6367

Differential Revision: https://secure.phabricator.com/D13143
2015-06-03 18:59:42 -07:00
epriestley
069e60d2ff Send mail to targets in the user's translation
Summary: Ref T6367.

Test Plan:
  - Added and executed unit tests.
  - Sent mail to A (en_US) and B (en_A*).
  - Got one mail in English and one mail in ENGLISH.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6367

Differential Revision: https://secure.phabricator.com/D13142
2015-06-03 18:59:33 -07:00
epriestley
6db97bde12 Build separate mail for each recipient, honoring recipient access levels
Summary:
Ref T6367. Removes `multiplexMail()`!

We can't pass a single body into a function which splits it anymore: we need to split recipients first, then build bodies for each recipient list. This lets us build separate bodies for each recipient's individual translation/access levels.

The new logic does this:

  - First, split recipients into groups called "targets".
    - Each target corresponds to one actual mail we're going to build.
    - Each target has a viewer (whose translation / access levels will be used to generate the mail).
    - Each target has a to/cc list (the users who we'll ultimately send the mail to).
  - For each target, build a custom mail body based on the viewer's access levels and settings (language prefs not actually implemented).
  - Then, deliver the mail.

Test Plan:
  - Read new config help.

Then did a bunch of testing, primarily with `bin/mail list-outbound` and `bin/mail show-outbound` (to review generated mail), `bin/phd debug taskmaster` (to run daemons freely) and `bin/worker execute --id <id>` (to repeatedly test a specific piece of code after identifying an issue).

With `one-mail-per-recipient` on (default):

  - Sent mail to multiple users.
  - Verified mail showed up in `mail list-outbound`.
  - Examined mail with `mail show-outbound`.
  - Added a project that a subscriber could not see.
    - Verified it was not present in `X-Phabricator-Projects`.
    - Verified it was rendered as "Restricted Project" for the non-permissioned viewer.
  - Added a subscriber, then changed the object policy so they could not see it and sent mail.
    - Verified I received mail but the other user did not.
  - Enabled public replies and verified mail generated with public addresses.
  - Disabld public replies and verified mail generated with private addresses.

With `one-mail-per-recipient` off:

  - Verified that one mail is sent to all recipients.
  - Verified users who can not see the object are still filtered.
  - Verified that partially-visible projects are completely visible in the mail (this violates policies, as documented, as the best available compromise).
  - Enabled public replies and verified the mail generated with "Reply To".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: carlsverre, epriestley

Maniphest Tasks: T6367

Differential Revision: https://secure.phabricator.com/D13131
2015-06-03 18:59:31 -07:00
epriestley
a9ceebbdb1 Move all ApplicationTransaction publishing to daemons
Summary:
Ref T6367. Do all mail, feed, notification and search stuff from the daemons, in all editors.

There are four relatively-stateful editors (Audit, Differential, Phriction, PhortuneCart) which needed special care to move state into the daemons properly.

Beyond that, I moved mailTo/mailCC/feedRelated/feedNotify to be computed before we enter the worker:

  - This is simpler, since a lot of editors rely on being able to call `$object->getReviewers()` or similar to compute them.
  - This is more correct, since we want to freeze the lists at this moment in time.

Finally, I renamed `loadEdges` to `willPublish` and made it a slightly more general hook.

---

This is a bit fragile and I'm not //thrilled// about it.

It would probably be cleaner to have separate Editor and Publisher classes (something like @fabe's D11329 did). However, I think that's quite a lot of work, and I'd like to see stronger motivation for it (either in this actually being more fragile than I think, or there being other things we get out of it). Overall, I'm comfortable with this change, just definitely not a big fan of the "save" + "load" pattern since I think it's really fragile, nonobvious, hard to debug/predict, etc.

Test Plan:
Directly updated editors:

- Created a new Phriction page, saw "Document Content".
- Edited a Phriction page, saw "Document Diff".
- Edited a revision, got normal looking mail.
- Faked in `changedPriorToCommitURI` and verified it survived the state boundary.
- Sent Audit mail.
- Sent invoice mail.

Indirect editors - for these, I just made a change and made sure the mail generated:

- Updated a paste.
- Updated an event.
- Updated a thread.
- Updated a task.
- Updated a mock.
- Updated a question.
- Updated a project.
- Updated a file.
- Updated an initiative.
- Updated a Legalpad document.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, fabe

Maniphest Tasks: T6367

Differential Revision: https://secure.phabricator.com/D13115
2015-06-03 18:59:29 -07:00
epriestley
1fc1114e29 Allow TransactionEditor to move publishing work to the daemons
Summary:
Ref T6367. This is similar to D11329, but not quite as ambitious.

Allow Editors to implement `supportsWorkers()` and move their publishing work into a daemon. So far, only Paste supports this.

Most of the complexity here is saving and restoring state across the barrier between the web process and the worker process, but I think this is ~90% of it and then we'll pick up a couple of random things in applications.

I'm primarily trying to keep this as gradual as possible.

Test Plan:
  - Published transactions with and without daemon support.
  - Looked at mail, feed.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: fabe, epriestley

Maniphest Tasks: T6367

Differential Revision: https://secure.phabricator.com/D13107
2015-06-03 18:59:28 -07:00
epriestley
d6247ffca5 Add support for "Extended Policies"
Summary:
Ref T7703. See that task and inline for a bunch of discussion.

Briefly, when we run implicit policy rules ("to see a revision, you must also be able to see its repository") at query time, they don't apply to other viewers we might check later.

We do this very rarely, but when we do we're often doing it for a bunch of different viewers (for example, in Herald) so I don't want to just reload the object a million times.

Test Plan:
  - Added and executed unit tests.
  - Wrote a "flag everything" Herald rule, as in the original report in T7703, and no longer got "Unknown Object" flags on revisions.
  - Rigged up a lot of cases in the web UI and couldn't find any inconsistencies, although this case is normally very hard to hit.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7703

Differential Revision: https://secure.phabricator.com/D13104
2015-06-03 18:59:27 -07:00
epriestley
908b426ee2 Put PhabricatorMetaMTAMailingList back to keep bin/storage adjust happy for now
Auditors: btrahan
2015-06-03 18:51:40 -07:00
epriestley
a4d874287f Allow administrators to configure "Email Settings" settings for lists
Summary:
Fixes T8387. This completes conversion of lists into users.

These settings allow administrators to reduce the amount of mail delivered to lists, ahead of T5791.

Test Plan: {F468206}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8387

Differential Revision: https://secure.phabricator.com/D13138
2015-06-03 18:42:41 -07:00
epriestley
d4a2034d2f Allow administrators to configure "Email Format" settings for lists
Summary: Ref T8387. Mostly completeness, but you might want to choose html vs text mail.

Test Plan: {F468203}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8387

Differential Revision: https://secure.phabricator.com/D13137
2015-06-03 18:42:40 -07:00
epriestley
14e318cb16 Allow administrators to edit the "Account" panel for mailing lists and bots
Summary:
Ref T8387. Ref T6367. This allows selection of a language, which will be respected in email delievered to the list users.

For example, you could have a German list that gets mail in German or something. I don't know that the feature is really useful, it's mostly just for completeness.

I also supported it for bots, mostly so their pronouns can be configured.

Test Plan: {F468186}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6367, T8387

Differential Revision: https://secure.phabricator.com/D13136
2015-06-03 18:42:39 -07:00
epriestley
02f0c099fa Fix some minor Spaces method visibility issues
Summary: Ref T8377. These were picked up by tests.

Test Plan: `arc unit --everything`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8377

Differential Revision: https://secure.phabricator.com/D13130
2015-06-03 18:42:38 -07:00
epriestley
ba6cb62b49 Remove mailing lists application
Summary: Ref T8387. This is now completely obsoleted by mailing list users.

Test Plan: Grepped for `mailinglist` and related symbols.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, epriestley

Maniphest Tasks: T8387

Differential Revision: https://secure.phabricator.com/D13129
2015-06-03 18:42:36 -07:00
epriestley
e0a3e6ba3b Allow administrators to manage addresses for mailing list users
Summary: Ref T8387. Mostly just modernizes this panel to work for viewers vs users. Auto-verify these edits since they aren't otherwise verifiable.

Test Plan:
  - Added, changed, removed addresses for a list.
  - Used panel normally for my own account.
  - Verified bots don't get a panel.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, epriestley

Maniphest Tasks: T8387

Differential Revision: https://secure.phabricator.com/D13125
2015-06-03 18:42:34 -07:00
epriestley
992c199577 Add "Mailing List" users
Summary:
Ref T8387. Adds new mailing list users.

This doesn't migrate anything yet. I also need to update the "Email Addresses" panel to let administrators change the list address.

Test Plan:
  - Created and edited a mailing list user.
  - Viewed profile.
  - Viewed People list.
  - Searched for lists / nonlists.
  - Grepped for all uses of `getIsDisabled()` / `getIsSystemAgent()` and added relevant corresponding behaviors.
  - Hit the web/api/ssh session blocks.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, tycho.tatitscheff, epriestley

Maniphest Tasks: T8387

Differential Revision: https://secure.phabricator.com/D13123
2015-06-03 18:42:33 -07:00
epriestley
13f0dac0ed Expand the power of user filtering
Summary:
Ref T8387. This mostly merges D10565 + D10480. I'm going to touch this to add mailing list stuff shortly so I wanted to clean those up.

This isn't super pretty but is fully flexible and consistent with other modern query UIs.

This should be more-or-less backward compatible.

Test Plan: Fiddled with the new options.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, cburroughs, epriestley

Maniphest Tasks: T8044, T8387

Differential Revision: https://secure.phabricator.com/D13122
2015-06-03 18:42:32 -07:00
epriestley
8440b3efc0 Update account roles documentation and remove actAsUser
Summary:
Ref T8387. This describes changes I haven't made yet, but plan to make.

Also removes the long-deprecated actAsUser capability so I can remove the caveat about it from the documentation.

Test Plan: `grep`, reading

Reviewers: btrahan, eadler

Reviewed By: btrahan, eadler

Subscribers: eadler, epriestley

Maniphest Tasks: T8387

Differential Revision: https://secure.phabricator.com/D13120
2015-06-03 18:42:09 -07:00
epriestley
8928aa2b67 Fix an issue with AlamancDeviceEditor
Summary: This got removed by accident in D12933.

Test Plan: Editing names works now.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D13144
2015-06-03 18:41:21 -07:00
Joshua Spence
0b87e462cd Remove flags and tokens upon object destruction
Summary: When destroying an object, also remove all associated flags and tokens.

Test Plan: Awarded a Maniphest task with a token and flagged it as well. Destroyed it with `./bin/remove destroy` and saw flag and token removed from the database.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13112
2015-06-04 07:23:48 +10:00
lkassianik
55c73b20ca Attempt to optimize ghost generating code.
Summary: Ref T8394, Attempt to optimize ghost generating code.

Test Plan: Open Upcoming, cancelled only, list view query. Should not hang.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T8394

Differential Revision: https://secure.phabricator.com/D13141
2015-06-03 13:01:26 -07:00
Mukunda Modell
dfac9d7f52 Add 'colors' and 'icons' fields to conduit api for Projects
Summary:
This simply adds colors, icons to ProjectQueryConduitAPIMethod and
ProjectCreateConduitAPIMethod.

Additionally, adds 'tags' to ProjectCreateConduitAPIMethod

Change-Id: Ic6332fe174a59ecfd60cea281ccb0ed938136141

Test Plan:
create a new project with project.create, specify icon, color and tags

call project.query with colors and icons args, check for results
containing matching projects.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: legoktm, aklapper, Korvin, epriestley, valhallasw, qgil

Differential Revision: https://secure.phabricator.com/D13098
2015-06-03 11:43:58 -07:00
lkassianik
d3a84687ba Query for recurring event parents should accept a "not a child of anything" clause
Summary: Closes T8371, Query for recurring event parents should accept a "not a child of anything" clause

Test Plan: querying for a list of events with recurring events should still work the same.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T8371

Differential Revision: https://secure.phabricator.com/D13140
2015-06-03 09:27:39 -07:00
Joshua Spence
c26ea062d1 Remove unused PhrictionActionConstants class
Summary: This class is no longer used since D10792.

Test Plan: `grep`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13116
2015-06-03 20:23:08 +10:00
lkassianik
446611e2e3 Cancelled recurring events should propogate to real child events
Summary: Ref T8371, Cancelled recurring events should propogate to real child events

Test Plan: Create recurring event, create and exception to a ghost event, cancel recurring event, real ghost event should be treated as cancelled while the recurring event remains cancelled.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T8371

Differential Revision: https://secure.phabricator.com/D13121
2015-06-02 19:44:31 -07:00
lkassianik
5e168b629d List view should only allow 100 ghost events per recurring event.
Summary: Ref T8394, List view should only allow 100 ghost events per recurring event

Test Plan: Open list view with many recurring events. Shouldn't hang.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8394

Differential Revision: https://secure.phabricator.com/D13132
2015-06-02 18:26:51 -07:00
Joshua Spence
1db6b9bafd Remove unused PhabricatorObjectHandleConstants class
Summary: This class is no longer used.

Test Plan: `grep`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13117
2015-06-03 06:58:32 +10:00
Joshua Spence
c36b82c7b8 Remove unused PhabricatorMail class
Summary: This class is no longer used after D13032.

Test Plan: `grep`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13118
2015-06-03 06:58:24 +10:00
Joshua Spence
dbd7e3c56d Remove unused CalendarColors class
Summary: This class is no longer used after D12850.

Test Plan: `grep`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13119
2015-06-03 06:58:06 +10:00
lkassianik
b576cef710 Edit link of an exception to a ghost event should not create new event
Summary: Fixes T8382, Edit link of an exception to a ghost event should not create new event

Test Plan: Create recurring event, open ghost, edit, save, edit again. Should edit event created after first edit, not new event.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T8382

Differential Revision: https://secure.phabricator.com/D13108
2015-06-02 07:29:48 -07:00
Joshua Spence
bf81fda036 Linter fixes
Summary: Apply various minor linter fixes.

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13109
2015-06-02 22:14:01 +10:00
Joshua Spence
2d15c3c062 Tidy up ManiphestTask destruction
Summary: Now that `ManiphestTask` implements `PhabricatorApplicationTransactionInterface`, the transaction removal happens automatically.

Test Plan: Used `./bin/remove destroy` to delete a `ManiphestTask` and saw related transactions removed as well.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13110
2015-06-02 22:12:45 +10:00
lkassianik
95551a1a5a DRAFT, recurring events need optional end dates
Summary: Ref T8357, DRAFT, recurring events need optional end dates

Test Plan: Edit recurring event, set end date, save, recurring ghosts should not generate after end date

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8357

Differential Revision: https://secure.phabricator.com/D13088
2015-06-01 18:56:11 -07:00
Bob Trahan
c83ddc3a0e Spaces - re-jigger application name for new Spaces application
Summary: got some errors in my sandbox when I updated. this fixes them and I think is the right naming convention.

Test Plan: /settings/panel/emailpreferences/ started working again!  /spaces/ also loaded  unit tests passed without errors

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8377

Differential Revision: https://secure.phabricator.com/D13103
2015-06-01 14:54:18 -07:00
epriestley
76523eec67 Implement DestructibleInterface on Spaces, add some basic tests
Summary: Ref T8377. Mostly just a framework for test coverage.

Test Plan: Tests pass.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8377

Differential Revision: https://secure.phabricator.com/D13102
2015-06-01 12:02:20 -07:00
epriestley
541b4c86b4 Add "Spaces", an application for managing policy namespaces
Summary: Ref T3820. This doesn't actually do anything yet, but dumps in all the plumbing.

Test Plan:
{F156989}

{F156990}

{F156991}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, wienczny, jdloft, devurandom, thz, hwinkel, 20after4, sascha-egerer, seporaitis, joshuaspence, chad, epriestley

Maniphest Tasks: T3820

Differential Revision: https://secure.phabricator.com/D9204
2015-06-01 11:28:38 -07:00
epriestley
7a9174fa21 Digest resource cachekeys to maximum length
Summary:
Fixes T8372. We currently store the entire resource path in the cachekey; shorten it to a maximum of 64 characters.

The key size is 128 and T8372 had an example of a 112-byte key which could run over 128 with a slightly-long cache namespace.

Test Plan:
  - Not 100% sure this is the issue, but it seems likely.
  - Dumped the cachekey before and after the change.
  - Saw values like `celerity:/res/1432859262T/phabricator/e2460e8f/core.pkg.css-6eGjUCJORU3j` now.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T8372

Differential Revision: https://secure.phabricator.com/D13095
2015-06-01 09:04:22 -07:00
Joshua Spence
c4bcfc85f1 Remove the repositories application
Summary: Fixes T7603. This application is no longer used.

Test Plan: Went to `/repository/` and got a 404.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley

Maniphest Tasks: T7603

Differential Revision: https://secure.phabricator.com/D12734
2015-06-01 15:37:13 +10:00
lkassianik
bb92819c09 Both ghost instances and exceptions to generated ghosts should be editable
Summary: Fixes T8369, Both ghost instances and exceptions to generated ghosts should be editable

Test Plan: Create recurring event, open /E{id}/1, edit, save. New exception should not be "recurring", and should be editable, as the regular ghost events.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8369

Differential Revision: https://secure.phabricator.com/D13089
2015-05-31 20:45:51 -07:00
Joshua Spence
9c5d9e3f47 Remove "arcanist project" fields
Summary: Ref T7604. Remove these "arcanist project" fields from the `LiskDAO` classes, but leave the data intact (there are no more read/writes to this data).

Test Plan:
- Grepped for calls to these methods.
- Ran `./bin/storage adjust` to check for schema issues.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7604

Differential Revision: https://secure.phabricator.com/D13012
2015-06-01 09:54:48 +10:00
Joshua Spence
ebde9357c7 Fix reverting commit language
Summary: I think the language here is inverted? On a commit which is reverted, the text states `$author added a reverted commit: $commit`. In reality, `$commit` is the //reverting// commit, not the //reverted// commit.

Test Plan: Think about English.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13067
2015-06-01 09:54:30 +10:00
lkassianik
65d9c0ea24 Edit recurring ghost events, not recurring event.
Summary: Closes T8358, Edit recurring ghost events, not recurring event.

Test Plan: Create recurring event, original event should not be editable, but ghost events should be editable.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8358

Differential Revision: https://secure.phabricator.com/D13075
2015-05-31 15:04:48 -07:00