1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-28 08:20:57 +01:00
Commit graph

1878 commits

Author SHA1 Message Date
epriestley
9827cc1622 Provide a missing timeout on the non-cluster connection pathway
Summary:
Ref T11232. The cluster connection pathway specifies a timeout when connecting, but this connection pathway does not. (I'm not sure if we just never did or if it got lost at some point.)

Soon, T11044 will obsolete this and unify the database connection pathways, but that's a more complicated change.

I'm not sure if this will fix T11232, but it can't hurt.

Test Plan: Put a `throw` on timeout specifications. Before the change: did not hit it in non-cluster configurations. After the change: hit it.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11232

Differential Revision: https://secure.phabricator.com/D16194
2016-06-29 11:20:01 -07:00
epriestley
a2cb5e1347 Log and continue when trying to destroy edges with no edge definition
Summary: Fixes T11201.

Test Plan:
Created bogus edges like this:

```
INSERT INTO edge (src, type, dst, dateCreated, seq) values ('PHID-TASK-vnddativbialb5p6ymis', 999999, 'quack', UNIX_TIMESTAMP(), 1);
```

Then ran `bin/remove destroy` on the relevant object.

Before the patch, destruction halted after hittin the bad edge.

After the patch, a warning is emitted but destruction continues.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11201

Differential Revision: https://secure.phabricator.com/D16171
2016-06-23 06:45:17 -07:00
epriestley
2cb779575d Split "Edit Blocking Tasks" into "Edit Parent Tasks" and "Edit Subtasks"
Summary:
Ref T11179. This splits "Edit Blocking Tasks" into two options now that we have more room ("Edit Parent Tasks", "Edit Subtasks").

This also renames "Blocking" tasks to "Subtasks", and "Blocked" tasks to "Parent" tasks. My goals here are:

  - Make the relationship direction more clear: it's more clear which way is up with "parent" and "subtask" at a glance than with "blocking" and "blocked" or "dependent" and "dependency".
  - Align language with "Create Subtask".
  - To some small degree, use more flexible/general-purpose language, although I haven't seen any real confusion here.

Fixes T6815. I think I narrowed this down to two issues:

  - Just throwing a bare exeception (we now return a dialog explicitly).
  - Not killing open transactions when the cyclec check fails (we now kill them).

Test Plan:
  - Edited parent tasks.
  - Edited subtasks.
  - Tried to introduce graph cycles, got a nice error dialog.

{F1697087}

{F1697088}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6815, T11179

Differential Revision: https://secure.phabricator.com/D16166
2016-06-22 11:20:38 -07:00
epriestley
72d554aa9b Fix parsing of anchors in Phriction document link syntax
Summary: Ref T4280. At some point (probably D15732) we started getting anchor parsing wrong. Just pop the anchor off before doing all the logic, then put it back on at the end.

Test Plan:
Tested various forms like:

```
[[ x ]]
[[ x | z ]]
[[ x#y | z ]]
[[ ./x#y | z ]]
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4280

Differential Revision: https://secure.phabricator.com/D16083
2016-06-08 13:14:33 -07:00
epriestley
2b344b2bb5 Make caches misses throw by default intead of inline-generating
Summary:
Ref T4103. Ref T10078. Currently, when a user misses a cache we just build it for them.

This is the behavior we want for the the viewer (so we don't have to build every cache up front if we don't actually need them), but not the right behavior for other users (since it allows performance problems to go undetected).

Make inline cache generation strict by default, then make sure all the things that rely on cache data request the correct data (well, all of the things identified by unit tests, at least: there might be some more stuff I haven't hit yet).

This fixes test failures in D16040, and backports a piece of that change.

Test Plan: Identified and then fixed failures with `arc unit --everything`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103, T10078

Differential Revision: https://secure.phabricator.com/D16042
2016-06-05 08:51:54 -07:00
epriestley
ebd8f3c987 Make translation, timezone and pronoun into real settings
Summary:
Ref T4103. These are currently stored on the user, for historic/performance reasons.

Since I want administrators to be able to set defaults for translations and timezones at a minimum and there's no longer a meaningful performance penalty for moving them off the user record, turn them into real preferences and then nuke the columns.

Test Plan:
  - Set settings to unusual values.
  - Ran migrations.
  - Verified my unusual settings survived.
  - Created a new user.
  - Edited all settings with old and new UIs.
  - Reconciled client/server timezone disagreement.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16005
2016-06-02 06:29:47 -07:00
epriestley
39cb5e7211 Improve some Phame custom domain remarkup and link behaviors
Summary:
Ref T6299. This makes more of the links point to the right places.

Not covered yet:

  - Projects and subscribers don't point to the right place (this is a little tricky to fix, I think).
  - `[[ #anchor ]]`s won't do the right thing in, uh, email, I guess, since `uri.here` is not set. This is also a little tricky.

Possibly we should just remove subscribers (although also kind of tricky).

Test Plan: On a custom-domain blog, observed that fewer things were broken.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6299

Differential Revision: https://secure.phabricator.com/D16007
2016-06-02 06:28:06 -07:00
epriestley
b352cae97d Don't stop on read-only mode for read-only storage workflows
Summary:
Fixes T11042. Currently, all `bin/storage` workflows stop if `cluster.read-only` is set:

```
$ ./bin/storage adjust
Usage Exception: Phabricator is currently in read-only mode. Use --force to override this mode.
```

However, some of them (`status`, `dump`, `databases`, etc) are read-only anyway and safe to run. Don't prompt in these cases.

Test Plan:
  - Set `cluster.read-only` to `true`.
  - Ran `bin/storage dump`, `bin/storage status`, etc. No longer received messages.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11042

Differential Revision: https://secure.phabricator.com/D15987
2016-05-30 09:30:43 -07:00
epriestley
10cc633b88 Warn and continue when failing to extract pht() strings
Summary:
Ref T5267. Fixes T9643. Currently, we can not parse/extract a small fraction of strings:

  - PHP files which contain obscure syntax that XHPAST can not parse.
  - HEREDOCs do not have a useable `evalStatic()` behavior right now.

Emit these as warnings, but continue and generate all usable/extractable translations.

Test Plan:
```
epriestley@orbital ~/dev/phabricator $ ./bin/i18n extract . >/dev/null
Found 4,548 files...
Done.
WARNING: Failed to extract strings from file "/Users/epriestley/dev/core/lib/phabricator/externals/stripe-php/lib/Stripe/ApiRequestor.php": XHPAST Parse Error: syntax error, unexpected T_STRING on line 357
WARNING: Failed to evaluate pht() call on line 141 in "/Users/epriestley/dev/core/lib/phabricator/scripts/repository/commit_hook.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 24 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorClusterConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 23 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 49 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 83 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 91 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 97 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 103 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 113 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 119 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 128 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 137 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 149 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 159 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 166 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 179 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 192 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorMetaMTAConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 24 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/config/option/PhabricatorNotificationConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 26 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/harbormaster/step/HarbormasterCircleCIBuildStepImplementation.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 154 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/maniphest/config/PhabricatorManiphestConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 282 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/maniphest/config/PhabricatorManiphestConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 84 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/maniphest/editor/ManiphestEditEngine.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 345 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/metamta/storage/PhabricatorMetaMTAReceivedMail.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 59 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/owners/editor/PhabricatorOwnersPackageEditEngine.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 26 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/project/config/PhabricatorProjectConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 53 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/project/config/PhabricatorProjectConfigOptions.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 98 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/search/engine/PhabricatorSearchEngineAPIMethod.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 165 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/search/engine/PhabricatorSearchEngineAPIMethod.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 252 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/search/engine/PhabricatorSearchEngineAPIMethod.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 299 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/search/engine/PhabricatorSearchEngineAPIMethod.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 360 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/search/engine/PhabricatorSearchEngineAPIMethod.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 439 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/search/engine/PhabricatorSearchEngineAPIMethod.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 529 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/search/engine/PhabricatorSearchEngineAPIMethod.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 54 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/transactions/controller/PhabricatorEditEngineConfigurationLockController.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 56 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/transactions/view/PhabricatorApplicationEditHTTPParameterHelpView.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 126 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/transactions/view/PhabricatorApplicationEditHTTPParameterHelpView.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 185 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/transactions/view/PhabricatorApplicationEditHTTPParameterHelpView.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 238 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/transactions/view/PhabricatorApplicationEditHTTPParameterHelpView.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 288 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/transactions/view/PhabricatorApplicationEditHTTPParameterHelpView.php": Unexpected node during static evaluation, of type: n_HEREDOC
WARNING: Failed to evaluate pht() call on line 17 in "/Users/epriestley/dev/core/lib/phabricator/src/applications/uiexample/examples/PhabricatorRemarkupUIExample.php": Unexpected node during static evaluation, of type: n_HEREDOC
epriestley@orbital ~/dev/phabricator $
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5267, T9643

Differential Revision: https://secure.phabricator.com/D15983
2016-05-26 20:11:27 -07:00
epriestley
5b77b86ffb Show translation option names natively, instead of in the current translation
Summary: Ref T5267. Put "Deutsch" in the list instead of "German", so you can find your language without knowing the English word for it.

Test Plan: {F1661598}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5267

Differential Revision: https://secure.phabricator.com/D15980
2016-05-26 08:07:57 -07:00
epriestley
3d3fff4991 Fix weird remarkup linewrapping on a few instructions forms, plus move toward fixing Phame/CORGI remarkup issues
Summary:
Fixes T10381. When we converted to `PHUIRemarkupView`, some instructional text got linebreaks added when it shouldn't have them (the source is written in PHP and wrapped at 80 characters, but the output should flow naturally).

Fix this so we don't preserve linebreaks.

This also makes `PHUIRemarkupView` a little more powerful and inches us toward fixing Phame/CORGI remarkup issues, getting rid of `PhabricatorMarkupInterface` / `PhabricatorMarkupOneOff`, and dropping all the application hard-coding in `PhabricatorMarkupEngine`.

Test Plan:
  - Grepped for all callsites, looking for callsites which accept remarkup written in `<<<HEREDOC` format.
  - Viewed form instructions, Conduit API methods, HTTP parameter edit instructions.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10381

Differential Revision: https://secure.phabricator.com/D15963
2016-05-22 12:23:05 -07:00
epriestley
de4312bcde Before executing svnserve, change the CWD to a readable directory
Summary: Fixes T10941. This avoids a confusing dead end when configuring Subversion hosting, where `svnserve` will fail to execute hooks if the CWD isn't readable by the vcs-user.

Test Plan:
  - Updated and committed in a hosted SVN repository.
  - Ran some git operations, too.
  - @dpotter confirmed this locally in T10941.

Reviewers: chad

Reviewed By: chad

Subscribers: dpotter

Maniphest Tasks: T10941

Differential Revision: https://secure.phabricator.com/D15879
2016-05-11 06:48:18 -07:00
epriestley
19aac8e8d3 Pass the new default syntax highlighting map to the remarkup engine
Summary: Ref T9790. This passes the map down so we can generate highlighted mail.

Test Plan:
Generated this relatively respectable-looking HTML mail:

{F1258558}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9790

Differential Revision: https://secure.phabricator.com/D15848
2016-05-05 02:51:19 -07:00
epriestley
01289f3f48 Generate syntax highlighting CSS from a reusable map
Summary:
Ref T9790. This prepares the syntax color rules to be reused in mail.

This goes about halfway toward T5701 by sort-of supporting different styles but not really.

Test Plan:
  - Ran `bin/celerity syntax` to regenerate syntax map.
  - Viewed some highlighted code, didn't see any differences.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9790

Differential Revision: https://secure.phabricator.com/D15846
2016-05-05 02:50:48 -07:00
epriestley
99718b61d8 Fill in new URI credential edit web UI interfaces
Summary:
Ref T10748. Ref T10366. Allows users to set credential for new URIs.

  - Ref T7221. Our handling of the "git://" protocol is currently incorrect. This protocol is not authenticated, but is considered an SSH protocol. In the new UI, it is considered an anonymous/unauthenticated protocol instead.
  - Ref T10241. This fixes the `PassphraseCredentialControl` so it doesn't silently edit the value if the current value is not visible to you and/or not valid.

Test Plan:
Performed a whole lot of credential edits, removals, and adjustments. I'll give this additional vetting before cutting over to it.

{F1253207}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7221, T10241, T10366, T10748

Differential Revision: https://secure.phabricator.com/D15829
2016-05-02 04:26:13 -07:00
epriestley
0630fef9fc Prevent web queries from running for more than 30 seconds
Summary:
Ref T10849. This enforces a global 30-second per-query time limit for anything not coming from the CLI.

If we run into another issue with MySQL hanging in the future, this should prevent it from being nearly as bad as it was.

Test Plan:
  - Set value to 0, verified the UI threw an exception immediately.
  - Set value back to 30, browsed around a bunch of pages.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10849

Differential Revision: https://secure.phabricator.com/D15799
2016-04-26 07:59:09 -07:00
epriestley
892a9a1f07 Make cluster repositories more resistant to freezing
Summary:
Ref T10860. This allows us to recover if the connection to the database is lost during a push.

If we lose the connection to the master database during a push, we would previously freeze the repository. This is very safe, but not very operator-friendly since you have to go manually unfreeze it.

We don't need to be quite this aggressive about freezing things. The repository state is still consistent after we've "upgraded" the lock by setting `isWriting = 1`, so we're actually fine even if we lost the global lock.

Instead of just freezing the repository immediately, sit there in a loop waiting for the master to come back up for a few minutes. If it recovers, we can release the lock and everything will be OK again.

Basically, the changes are:

  - If we can't release the lock at first, sit in a loop trying really hard to release it for a while.
  - Add a unique lock identifier so we can be certain we're only releasing //our// lock no matter what else is going on.
  - Do the version reads on the same connection holding the lock, so we can be sure we haven't lost the lock before we do that read.

Test Plan:
  - Added a `sleep(10)` after accepting the write but before releasing the lock so I could run `mysqld stop` and force this issue to occur.
  - Pushed like this:

```
$ echo D >> record && git commit -am D && git push
[master 707ecc3] D
 1 file changed, 1 insertion(+)
# Push received by "local001.phacility.net", forwarding to cluster host.
# Waiting up to 120 second(s) for a cluster write lock...
# Acquired write lock immediately.
# Waiting up to 120 second(s) for a cluster read lock on "local001.phacility.net"...
# Acquired read lock immediately.
# Device "local001.phacility.net" is already a cluster leader and does not need to be synchronized.
# Ready to receive on cluster host "local001.phacility.net".
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 254 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
BEGIN SLEEP
```

  - Here, I stopped `mysqld` from the CLI in another terminal window.

```
END SLEEP
# CRITICAL. Failed to release cluster write lock!
# The connection to the master database was lost while receiving the write.
# This process will spend 300 more second(s) attempting to recover, then give up.
```

  - Here, I started `mysqld` again.

```
# RECOVERED. Link to master database was restored.
# Released cluster write lock.
To ssh://local@localvault.phacility.com/diffusion/26/locktopia.git
   2cbf87c..707ecc3  master -> master
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10860

Differential Revision: https://secure.phabricator.com/D15792
2016-04-25 11:37:31 -07:00
Aviv Eyal
a3bb35e9d2 make Trigger Daemon sleep correctly when one-time triggers exist
Summary:
Trigger daemon is trying to find the next event to invoke before sleeping, but the query includes already-elapsed triggers.
It then tries to sleep for 0 seconds.

Test Plan:
On a new instance, schedule a single trigger of type `PhabricatorOneTimeTriggerClock` to a very near time.

Use top to see trigger daemon not going to 100% CPU once the event has elapsed.

Reviewers: #blessed_reviewers, epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15750
2016-04-18 14:17:10 -07:00
epriestley
595f203816 Correct RepositoryURI schema and propagate adjust exit code correctly
Summary:
Fixes T10830.

  - The return code from `storage adjust` did not propagate correct.
  - There was one column issue which I missed the first time around because I had a bunch of unrelated stuff locally.

Test Plan:
  - Ran `bin/storage upgrade -f` with failures, used `echo $?` to make sure it exited nonzero.
  - Got fully clean `bin/storage adjust` by dropping all my extra local tables.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10830

Differential Revision: https://secure.phabricator.com/D15746
2016-04-18 08:11:22 -07:00
epriestley
9352ed8abb Add missing RepositoryURI table + run storage adjustments in tests
Summary:
Fixes T10830. Ref T10366. I wasn't writing to this table yet so I didn't build it, but the fact that `bin/storage adjust` would complain slipped my mind.

  - Add the table.
  - Make the tests run `adjust`. This is a little slow (a few extra seconds) but we could eventually move some steps like this to run server-side only.

Test Plan: Ran `bin/storage upgrade -f`, got a clean `adjust`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10366, T10830

Differential Revision: https://secure.phabricator.com/D15744
2016-04-18 07:54:02 -07:00
epriestley
20bad9a4ba Reset umask to 022 for all Phabricator processes
Summary:
Fixes T7475. If you do something like:

  $ umask 123
  $ ./bin/phd start

...the daemons might inherit the weird umask, do a `git fetch` with the weird umask, and end up creating files with weird permissions in repositories.

Instead, just normalize the umask to 022 in all cases. This is overwhelmingly the most common setting, and the one we assume things are configured with.

(When we want to force permissions to a certain setting, we do so explicitly.)

Test Plan:
  - Added `var_dump(umask())` to observe umask.
  - Ran `bin/phd`, saw proper umask (`18`, which is decimal of `022` octal).
  - Set `umask 123`, then ran `bin/phd`, saw it correct properly again.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7475

Differential Revision: https://secure.phabricator.com/D15721
2016-04-15 10:03:01 -07:00
epriestley
7852ec1619 Use --master-data, not --dump-slave, in bin/storage dump
Summary: These flags do slightly different things, I actually want --master-data here. My test databases are setup half-weird and work with either statement, which is why I missed this.

Test Plan: Ran a dump against master, got the right CHANGE MASTER statement with no warnings.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15716
2016-04-14 14:56:21 -07:00
epriestley
bbb321395a Support Aphlict clustering
Summary:
Ref T6915. This allows multiple notification servers to talk to each other:

  - Every server has a list of every other server, including itself.
  - Every server generates a unique fingerprint at startup, like "XjeHuPKPBKHUmXkB".
  - Every time a server gets a message, it marks it with its personal fingerprint, then sends it to every other server.
  - Servers do not retransmit messages that they've already seen (already marked with their fingerprint).
  - Servers learn other servers' fingerprints after they send them a message, and stop sending them messages they've already seen.

This is pretty crude, and the first message to a cluster will transmit N^2 times, but N is going to be like 3 or 4 in even the most extreme cases for a very long time.

The fingerprinting stops cycles, and stops servers from sending themselves copies of messages.

We don't need to do anything more sophisticated than this because it's fine if some notifications get lost when a server dies. Clients will reconnect after a short period of time and life will continue.

Test Plan:
  - Wrote two server configs.
  - Started two servers.
  - Told Phabricator about all four services.
  - Loaded Chrome and Safari.
  - Saw them connect to different servers.
  - Sent messages in one, got notifications in the other (magic!).
  - Saw the fingerprinting stuff work on the console, no infinite retransmission of messages, etc.

(This pretty much just worked when I ran it the first time so I probably missed something?)

{F1218835}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6915

Differential Revision: https://secure.phabricator.com/D15711
2016-04-14 13:26:30 -07:00
epriestley
5a0b7398ca Give bin/storage some replica-aware options
Summary:
Fixes T10758.

  - Adds a "--host" flag. If you specify this, we read your cluster config. This lets you dump from a replica.
  - Adds a "--for-replica" flag to `storage dump`. This makes `mysqldump` include a `CHANGE MASTER ...` statement in the output, which is useful when setting up a replica for the first time.

Test Plan:
  - Dumped master and replica cluster databases.
  - Dumped non-cluster databases.
  - Ran various other commands (help, status, etc).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10758

Differential Revision: https://secure.phabricator.com/D15714
2016-04-14 13:23:35 -07:00
epriestley
2930733ac9 Complete modernization of Aphlict configuration
Summary:
Fixes T10697. This finishes bringing the rest of the config up to cluster power levels.

Phabricator is now given an arbitrarily long list of notification servers.

Each Aphlict server is given an arbitrarily long list of ports to run services on.

Users are free to make them meet in the middle by proxying whatever they want to whatever else they want.

This should also accommodate clustering fairly easily in the future.

Also rewrote the status UI and changed a million other things. 🐗

Test Plan:
{F1217864}

{F1217865}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15703
2016-04-14 04:57:00 -07:00
epriestley
0379cc10ac Fixes T10805. When clustering is not configured, this check should just
return.

Auditors: chad
2016-04-14 04:32:20 -07:00
epriestley
ac35246d0d Never sever non-cluster database; write more read-only documentation
Summary:
Ref T4571. Write more of the missing documentation sections and clarify a few things.

Since the "replicating master" check needs a special permission, imposes a performance penalty, is probably very difficult to misconfigure, and likely not a big deal anyway, just drop the idea of trying to automatically detect + prevent it. We still show if it's an issue on the status page, provided we have permission to check.

When you don't have any cluster databases configured, never stop trying to connect to the default master database. We might want to do this eventually as load reduction, but just don't muddy the waters too much for now while things stabilize.

Test Plan:
  - Tested functionality in cluster, non-cluster, and degraded-cluster modes.
  - Used status console to monitor a health check cycle.
  - Read docs.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15679
2016-04-11 08:44:11 -07:00
epriestley
ebff07d019 Automatically sever databases after prolonged unreachability
Summary:
Ref T4571. When a database goes down briefly, we fall back to replicas.

However, this fallback is slow (not good for users) and keeps sending a lot of traffic to the master (might be bad if the root cause is load-related).

Keep track of recent connections and fully degrade into "severed" mode if we see a sequence of failures over a reasonable period of time. In this mode, we send much less traffic to the master (faster for users; less load for the database).

We do send a little bit of traffic still, and if the master recovers we'll recover back into normal mode seeing several connections in a row succeed.

This is similar to what most load balancers do when pulling web servers in and out of pools.

For now, the specific numbers are:

  - We do at most one health check every 3 seconds.
  - If 5 checks in a row fail or succeed, we sever or un-sever the database (so it takes about 15 seconds to switch modes).
  - If the database is currently marked unhealthy, we reduce timeouts and retries when connecting to it.

Test Plan:
  - Configured a bad `master`.
  - Browsed around for a bit, initially saw "unrechable master" errors.
  - After about 15 seconds, saw "major interruption" errors instead.
  - Fixed the config for `master`.
  - Browsed around for a while longer.
  - After about 15 seconds, things recovered.
  - Used "Cluster Databases" console to keep an eye on health checks: it now shows how many recent health checks were good:

{F1213397}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15677
2016-04-11 08:43:52 -07:00
epriestley
146fb646f9 Automatically degrade to read-only mode when unable to connect to the master
Summary:
Ref T4571. If we fail to connect to the master, automatically try to degrade into a temporary read-only mode ("UNREACHABLE") for the remainder of the request, if possible.

If the request was something like "load the homepage", that'll work fine. If it was something like "submit a comment", there's nothing we can do and we just have to fail.

Detecting this condition imposes a performance penalty: every request checks the connection and gives the database a long time to respond, since we don't want to drop writes unless we have to. So the degraded mode works, but it's really slow, and may perpetuate the problem if the root issue is load-related.

This lays the groundwork for improving this case by degrading futher into a "SEVERED" mode which will persist across requests. In the future, if several requests in a short period of time fail, we'll sever the database host and refuse to try to connect to it for a little while, connecting directly to replicas instead (basically, we're "health checking" the master, like a load balancer would health check a web application server). This will give us a better (much faster) degraded mode in a major service disruption, and reduce load on the master if the root cause is load-related, giving it a better chance of recovering on its own.

Test Plan:
  - Disabled master in config by changing the host/username, got degraded automatically to UNREACAHBLE mode immediately.
  - Faked full SEVERED mode, requests hit replicas and put me in the mode properly.
  - Made stuff work, hit some good pages.
  - Hit some non-cluster pages.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15674
2016-04-10 12:20:13 -07:00
epriestley
e0a8cac703 When no master database is configured, automatically degrade to read-only mode
Summary: Ref T4571. If `cluster.databases` is configured but only has replicas, implicitly drop to read-only mode and send writes to a replica.

Test Plan:
  - Disabled the `master`, saw Phabricator automatically degrade into read-only mode against replicas.
  - (Also tested: explicit read-only mode, non-cluster mode, properly configured cluster mode).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15672
2016-04-10 12:19:55 -07:00
epriestley
071741c61d When Phabricator is in read-only mode, explain why
Summary:
Ref T4571. Allows users to click the "read-only mode" notification to get more information about why an install is in read-only mode.

Installs can be in this mode for several reasons (explicit administrative action, no masters defined, no masters reachable), and it's useful to be able to tell the difference.

Test Plan: {F1212930}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15671
2016-04-10 12:19:18 -07:00
epriestley
c178f29cdb Use new first-class MySQL timeout support in Phabricator
Summary: Fixes T6710. After D15669, we support a proper timeout parameter, so we don't need this hack anymore.

Test Plan: See D15669: forced a MySQL connector, set a low timeout, set a bad database, saw fast failures.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6710

Differential Revision: https://secure.phabricator.com/D15670
2016-04-10 12:19:00 -07:00
epriestley
6a4a9bb2d2 When cluster.databases is configured, read the master connection from it
Summary:
Ref T4571. Ref T10759. Ref T10758. This isn't complete, but gets most of the job done:

  - When `cluster.databases` is set up, most things ignore `mysql.host` now.
  - You can `bin/storage upgrade` and stuff works.
  - You can browse around in the web UI and stuff works.

There's still a lot of weird tricky stuff to navigate, and this has real no advantages over configuring a single server yet (no automatic failover, etc).

Test Plan:
  - Configured `cluster.databases` to point at my `t1.micro` hosts in EC2 (master + replica).
  - Ran `bin/storage upgrade`, got a new install setup on them properly.
  - Survived setup warnings, browsed around.
  - Switched back to local config, ran `bin/storage upgrade`, browsed around, went through setup checks.
  - Intentionally broke config (bad hosts, no masters) and things seemed to react reasonably well.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571, T10758, T10759

Differential Revision: https://secure.phabricator.com/D15668
2016-04-10 12:18:42 -07:00
epriestley
0439645d5b Add a "Database Cluster Status" console in Config
Summary: Ref T4571. The configuration option still doesn't do anything, but add a status panel for basic setup monitoring.

Test Plan:
Here's what a good version looks like:

{F1212291}

Also faked most of the errors it can detect and got helpful diagnostic messages like this:

{F1212292}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15667
2016-04-09 20:34:13 -07:00
epriestley
3f51b78539 Lay cluster.databases configuration groundwork for database clustering
Summary:
Ref T4571. This adds a new option which allows you to upgrade your one-host configuration to a multi-host configuration by configuring it.

Doing this currently does nothing. I wrote a lot of words about what it is //supposed// to do in the future, though.

Test Plan:
  - Tried to configure the option in all the possible bad ways, got errors.
  - Read documentation.

Reviewers: chad

Reviewed By: chad

Subscribers: eadler

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15663
2016-04-09 13:41:16 -07:00
epriestley
49d93dcf98 Add a cluster.read-only option
Summary:
Ref T4571. There will be a very long path beyond this, but add a basic read-only mode. You can explicitly enable this to put Phabricator in a sort of "maintenance" mode today if you're swapping databases or something.

In the long term, we'll automatically degrade into this mode if the master database is down.

Test Plan:
  - Enabled read-only mode.
  - Browsed around.
  - Didn't immediately see anything that was totally 100% broken.

Most stuff is 80-90% broken right now. For example:

  - Stuff like submitting comments doesn't work, and gives you a confusing, unhelpful error.
  - None of the UI really knows that it's read-only. EditEngine stuff should all hide itself and say "you can't add new comments while an install is in read-only mode", for example, but currently does not.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15662
2016-04-09 13:40:47 -07:00
epriestley
57f016b166 Convert OAuthServer to Transactions + EditEngine
Summary: Ref T7303. This application is currently stone-age tech (no transactions, hard "delete" action). Bring it up to modern specs.

Test Plan:
  - Created and edited an OAuth application.
  - Viewed transaction record.
  - Tried to create something with no name, invalid redirect URI, etc. Was gently rebuffed with detailed explanatory errors.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7303

Differential Revision: https://secure.phabricator.com/D15609
2016-04-05 01:55:49 -07:00
lkassianik
8d67629e9e Fix translation of badge feed stories.
Summary: Fixes T10688

Test Plan: Award badge, view main Feed

Reviewers: chad, #blessed_reviewers, epriestley

Reviewed By: chad, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T10688

Differential Revision: https://secure.phabricator.com/D15547
2016-03-29 10:56:36 -07:00
epriestley
7b0b820be1 Bridge GitHub users into Phabricator and attribute actions to them
Summary:
Ref T10538. Ref T10537. This creates PHIDs which represent GitHub users, and uses them as the actors for synchronized comments.

I've just made them Doorkeeper objects. There are three major kinds of objects they //could// possibly be:

  - Nuance requestor objects.
  - External account objects.
  - Doorkeeper objects.

I don't think we actually need distinct nuance requestor objects. These don't really do anything right now, and were originally created before Doorkeeper. I think Doorkeeper is a superset of nuance requestor functionality, and better developed and more flexible.

Likewise, doorkeeper objects are much more flexible than external account objects, and it's nice to imagine that we can import from Twootfeed or whatever without needing to build full OAuth for it. I also like less stuff touching auth code, when possible.

Making these separate from external accounts does make it a bit harder to reconcile external users with internal users, but I think that's OK, and that it's generally desirable to show the real source of a piece of content. That is, if I wrote a comment on GitHub but also have a Phabricator account, I think it's good to show "epriestley (GitHub)" (the GitHub user) as the author, not "epriestley" (the Phabricator user). I think this is generally less confusing overall, and we can add more linkage later to make it clearer.

Test Plan:
{F1194104}

{F1194105}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10537, T10538

Differential Revision: https://secure.phabricator.com/D15541
2016-03-28 13:10:32 -07:00
epriestley
bf3879b1c7 Fully fix a bad rule object aliasing issue custom remarkup rules
Summary:
Fixes T10234. This is a more thorough fix.

Root issue is that some time around D13589, we started hitting an object cache for `loadCustomInlineRules()`, but didn't adjust the code to account for that.

So if a page created multiple similar engines, we'd return the same `$rule` object for multiple engines, call `setEngine()` on it with different engines, and then possibly try to render using an already-expired engine the second time through.

Instead, create a separate `$rule` object for each separate `$engine`.

Test Plan:
Repro is something like this:

  - Create a custominlinerule which uses an engine.
  - Purge the remarkup cache.
  - Load a page which uses the rule in two engines (e.g., in a revision description, and also in an inline comment).
  - Before change: second one could fatal. After change: clean load.

Reviewers: thoughtpolice, chad

Reviewed By: thoughtpolice, chad

Subscribers: thoughtpolice, eadler

Maniphest Tasks: T10234

Differential Revision: https://secure.phabricator.com/D15535
2016-03-28 11:27:13 -07:00
lkassianik
3955ff719a Create feed transaction stories for awarding/revoking badges
Summary: Ref T10677, Awarding/revoking badge should create a feed story on homepage with badge handle recipient handles

Test Plan: Award/revoke badge, open Feed, should see story with badge link and recipient links.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T10677

Differential Revision: https://secure.phabricator.com/D15534
2016-03-28 10:25:24 -07:00
lkassianik
e6d2e66ea2 Adding basic transaction titles to awarding/revoking badges
Summary: Ref T10677, awarding/revoking a badge should create timeline entries with titles that are more clear (excludes homepage feed stories)

Test Plan: Award/revoke a badge to single or multiple users. See timeline entries that reflect those actions.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T10677

Differential Revision: https://secure.phabricator.com/D15533
2016-03-28 09:38:04 -07:00
epriestley
601aaa5a86 Modularize content sources
Summary:
Ref T10537. For Nuance, I want to introduce new sources (like "GitHub" or "GitHub via Nuance" or something) but this needs to modularize eventually.

Split ContentSource apart so applications can add new content sources.

Test Plan:
This change has huge surface area, so I'll hold it until post-release. I think it's fairly safe (and if it does break anything, the breaks should be fatals, not anything subtle or difficult to fix), there's just no reason not to hold it for a few hours.

- Viewed new module page.
- Grepped for all removed functions/constants.
- Viewed some transactions.
- Hovered over timestamps to get content source details.
- Added a comment via Conduit.
- Added a comment via web.
- Ran `bin/storage upgrade --namespace XXXXX --no-quickstart -f` to re-run all historic migrations.
- Generated some objects with `bin/lipsum`.
- Ran a bulk job on some tasks.
- Ran unit tests.

{F1190182}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10537

Differential Revision: https://secure.phabricator.com/D15521
2016-03-26 11:59:45 -07:00
epriestley
47dedfb152 Introduce "bridged" objects
Summary:
Ref T10537. These are objects which are bound to some external object, like a Maniphest task which is a representation of a GitHub issue.

This doesn't do much yet and may change, but my thinking is:

  - I'm putting these on-object instead of on edges because I think we want to actively change the UI for them (e.g., clearly call out that the object is bridged) but don't want every page to need to do extra queries in the common case where zero bridged objects exist anywhere in the system.
  - I'm making these one-to-one, more or less: an issue can't be bridged to a bunch of tasks, nor can a bunch of tasks be bridged to a single issue. Pretty sure this makes sense? I can't come up with any reasonable, realistic cases where you want a single GitHub issue to publish to multiple different tasks in Maniphest.
  - Technically, one type of each bridgable object could be bridged, but I expect this to never actually occur. Hopefully.

Test Plan: Ran storage upgrade, loaded some pages.

Reviewers: chad

Reviewed By: chad

Subscribers: Luke081515.2

Maniphest Tasks: T10537

Differential Revision: https://secure.phabricator.com/D15502
2016-03-22 15:06:57 -07:00
epriestley
66946c0996 Fix unusual use of Remarkup in Maniphest
Summary: Fixes T10234. This usage is unusual, out of date, and has some bad interactions with engines and custom rules.

Test Plan:
  - Added `CustomInlineCodeRule` from P1129 as an extension rule.
  - Put a custom `<code> ... </code>` block in a Maniphest task description.
  - Saw fatal as described in task; applied change; saw rule work properly.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10234

Differential Revision: https://secure.phabricator.com/D15501
2016-03-21 11:24:17 -07:00
Aviv Eyal
2b9d4f70ba Remarkup rule for rendering PHIDs as handles
Summary:
adds the `{{PHID....}}` rule. Should mostly be useful in UI code that refers to Objects.

It doesn't add any mention links/transactions.

Test Plan: Comment with this, see email (plain + html) and comment box.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15488
2016-03-17 20:24:03 +00:00
Chad Little
148a50e48b Convert Differential to new layout
Summary:
First pass at converting Differential, I likely have some buggy-poos but thought I'd toss this up now in case very bad bugs present.

To do:
- Need to put status back on Hovercards
- "Diff Detail" probably needs a better design

Test Plan: Looking at lots of diffs, admittedly I dont have harbormaster, etc, running locally. Checked Diffusion for Table of Content changes on small and large commits.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15463
2016-03-12 13:04:21 -08:00
epriestley
de23ba0002 Fix a minor issue in Nuance which could cause the trigger daemon to poll too often
Summary: Ref T10537. Currently, when you have at least two cursors, the daemon can poll too frequently when processing the last source because it never hits the end-of-list condition.

Test Plan:
  - Ran `bin/phd debug trigger`.
  - Observed huge volumes of output before change as triggers fired as fast as possible.
  - Observed reasonable poll frequency after change.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10537

Differential Revision: https://secure.phabricator.com/D15464
2016-03-12 05:04:42 -08:00
epriestley
68b468a846 Partially improve threading UI for adjacent inline comments
Summary:
Ref T10563. This isn't a complete fix, but should make viewing complex inline threads a little more manageable.

This just tries to put stuff in thread order instead of in pure chronological order. We can likely improve the display treatment -- this is a pretty minimal approach, but should improve clarity.

Test Plan:
T10563 has a "before" shot. Here's the "after":

{F1169018}

This makes it a bit easier to follow the conversations.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10563

Differential Revision: https://secure.phabricator.com/D15459
2016-03-10 17:40:13 -08:00
epriestley
8858b6cf8d When replying to a ghost comment, attach the reply to the same place
Summary:
Fixes T10562. I left this behavior sort of ambiguous in the original implementation because I didn't anticipate or stumble across this situation.

It's easy to fix: when you reply to a ghost, just put the reply in the exact same place as the ghost (even if it's a different diff), so they always move/ghost/port/thread together.

Test Plan:
See T10562 for reproduction steps and a "before" picture. Here's the after picture:

{F1168983}

The two comments at the bottom are pre-fix, and exhibit the bug. The comment at the top is post-fix, and appears adjacent to the original correctly.

Reviewers: chad

Reviewed By: chad

Subscribers: eadler

Maniphest Tasks: T10562

Differential Revision: https://secure.phabricator.com/D15458
2016-03-10 16:41:49 -08:00
epriestley
2a3c3b2b98 Provide bin/nuance import and ngram indexes for sources
Summary:
Ref T10537. More infrastructure:

  - Put a `bin/nuance` in place with `bin/nuance import`. This has no useful behavior yet.
  - Allow sources to be searched by substring. This supports `bin/nuance import --source whatever` so you don't have to dig up PHIDs.

Test Plan:
  - Applied migrations.
  - Ran `bin/nuance import --source ...` (no meaningful effect, but works fine).
  - Searched for sources by substring in the UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10537

Differential Revision: https://secure.phabricator.com/D15436
2016-03-08 10:30:24 -08:00
epriestley
3f4cc3ad6e Allow Nuances sources to provide import cursors
Summary:
Ref T10537. Some sources (like the future "GitHub Repository" source) need to poll remotes.

  - Provide a mechanism for sources to emit import cursors.
  - Hook them into the trigger daemon so they'll fire periodically.
  - Provide some storage.

This diff does nothing useful or interesting, and is pure infrastructure.

Test Plan:
  - Ran `bin/storage upgrade -f`, no adjustment issues.
  - Poked around Nuance.
  - Ran the trigger daemon, verified it didn't crash and checked for Nuance stuff to do.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10537

Differential Revision: https://secure.phabricator.com/D15435
2016-03-08 10:30:04 -08:00
epriestley
aaab1011e5 Give AphrontTagView a getViewer(), deprecate getUser()
Summary:
Two minor changes here:

  - Replace `get/setUser()` with `get/setViewer()` for consistency with everything else.
  - `getViewer()` now throws if no viewer is set. We had a lot of code that either "should" check this but didn't, or did check it in an identical way, duplicating work. In contrast, very little code checks for a viewer but works if one is not present.

Test Plan:
  - Grepped for `->user`.
  - Attempted to fix all callsites inside `*View` classes.
  - Browsed around a bunch of applications, particularly Calendar, Differential and Diffusion, which seemed most heavily affected.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15412
2016-03-06 09:27:38 -08:00
epriestley
abb4c03b47 Remove shouldShowSubscribersProperty() from SubscribableInterface
Summary:
Every caller returns `true`. This was added a long time ago for Projects, but projects are no longer subscribable.

I don't anticipate needing this in the future.

Test Plan: Grepped for this method.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15409
2016-03-06 06:01:36 -08:00
epriestley
1bdf988556 Convert DrydockBlueprints to EditEngine
Summary:
Ref T10457. Fixes T10024. This primarily just modernizes blueprints to use EditEngine.

This also fixes T10024, which was an issue with stored properties not being flagged correctly.

Also slightly improves typeaheads for blueprints (more information, disabled state).

Test Plan:
  - Created and edited various types of blueprints.
  - Set and removed limits.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10024, T10457

Differential Revision: https://secure.phabricator.com/D15390
2016-03-03 15:21:25 -08:00
Sébastien Santoro
a4db6f387d Fix typo: discsussions → discussions
Test Plan: Read again the sentence.

Reviewers: joshuaspence, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D15316
2016-02-21 01:51:03 -08:00
epriestley
50f910ce67 Always install the "icon" and "emoji" remarkup rules
Summary: Ref T10394. Currently, these rules are only active if the Macro application is installed. Instead, install them unconditionally.

Test Plan:
  - Used `{icon camera}` with Macro installed and uninstalled.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10394

Differential Revision: https://secure.phabricator.com/D15311
2016-02-19 11:51:53 -08:00
Chad Little
f35509e30e Update to use PHUIRemarkupView everywhere possible
Summary: Moves all the one off object calls to PHUIRemarkupView, adds a "Document" call as well (future plans).

Test Plan: Visited most pages I could get access to, but may want extra careful eyes on this diff.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15281
2016-02-16 14:05:53 -08:00
epriestley
a5bbe256c8 Fix a couple of missing translation strings
Summary: Clean the UI up a little.

Test Plan: {F1106533}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15259
2016-02-12 08:10:10 -08:00
epriestley
8934dee543 Add "does not match regexp" to Herald
Summary:
Fixes T10330.

  - Anywhere we support "matches regexp", also allow "does not match regexp". Although you can sometimes write a clever negative regexp, these rules are better expressed with "does not match <simple regexp>" anyway, and sometimes no regexp will work.
  - Always allow "does not contain" when we support "contains".
  - Fix some JS issues with certain rules affecting custom fields.

Test Plan:
  - Wrote an "Affected files do not match regexp" rule that required every diff to touch "MANUALCHANGELOG.md".
  - Tried to diff without the file; rejected.
  - Tried to diff with the file; accepted.
  - Wrote a bunch of "contains" and "does not contain" rules against text fields and custom fields, then edited tasks to trigger/observe them.
  - Swapped the editor into custom text, user, remarkup, etc fields, no more JS errors.

{F1105172}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10330

Differential Revision: https://secure.phabricator.com/D15254
2016-02-11 15:29:38 -08:00
Chad Little
41262150df Remove unused call to phui-text
Summary: I can't find any reference to these used. Fixes T10244

Test Plan: Grep for "phui-text" and "PHUI::TEXT"

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T10244

Differential Revision: https://secure.phabricator.com/D15142
2016-01-29 12:41:04 -08:00
Chad Little
fe5cd4ca2c Move FontIcon calls to Icon
Summary: Normalizes all `setFontIcon` calls to `setIcon`.

Test Plan: UIExamples, Almanac, Apps list, etc.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, hach-que, yelirekim

Differential Revision: https://secure.phabricator.com/D15129
2016-01-28 08:48:45 -08:00
Chad Little
36158dbdc0 Convert all calls to 'IconFont' to just 'Icon'
Summary: Mostly for consistency, we're not using other forms of icons and this makes all classes that use an icon call it in the same way.

Test Plan: tested uiexamples, lots of other random pages.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15125
2016-01-27 20:59:27 -08:00
epriestley
5f170847ca Throw a more tailored error when a storage upgrade patch can't access a database
Summary: Fixes T8762.

Test Plan: Ran `bin/storage upgrade --namespace ... --user limited`, saw a more specific error.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8762

Differential Revision: https://secure.phabricator.com/D15080
2016-01-21 14:54:56 -08:00
epriestley
d37153f003 Make bin/storage upgrade and bin/storage adjust emit detailed messages if the user has no access to databases
Summary:
Ref T10195. Distinguish between "database does not exist" and "database exists, you just don't have permission to access it".

We can't easily get this information out of INFORMATION_SCHEMA but can just `SHOW TABLES IN ...` every database that looks like it's missing and then look at the error code.

Test Plan:
  - Created a user `limited` with limited access.
  - Ran `bin/storage adjust`.
  - Got hopefully more helpful messages about access problems, instead of "Missing" errors.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10195

Differential Revision: https://secure.phabricator.com/D15079
2016-01-21 13:06:00 -08:00
epriestley
358240b804 Fix an issue with paginating queries which reverse vector ordering components
Summary:
Ref T10188. If you issue certain queries which use reverse ordering (like "All tasks, oldest update to newest update") and then try to page forward, we build the paging clause without reversing the column order correctly.

For example, the ordering of "oldest update to newest update" is "dateModified ASC, id ASC", so the second page should include an "id > X" query. Currently, this builds as "id < X" incorrectly instead.

The cause of this is just a failure to re-reverse a reversing flag when constructing the paging clause.

Test Plan:
  - Queried tasks by update, oldest to newest, with no grouping, etc.
  - Paged to second page.
  - After change, got a valid second page with a good query in the Services tab.
  - Made some other normal queries.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10188

Differential Revision: https://secure.phabricator.com/D15076
2016-01-21 11:11:24 -08:00
Richard van Velzen
958333c46d Preserve newlines in remarkup previews
Summary: Fixes T10177

Test Plan: Saw that the preview of the rendered task description showed newlines proper.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T10177

Differential Revision: https://secure.phabricator.com/D15068
2016-01-20 14:00:50 +01:00
epriestley
5c2e49a812 Allow any user to watch any project they can see
Summary:
Ref T6183. Ref T10054. Historically, only members could watch projects because there were some weird special cases with policies. These policy issues have been resolved and Herald is generally powerful enough to do equivalent watches on most objects anyway.

Also puts a "Watch Project" button on the feed panel to make the behavior and meaning more obvious.

Test Plan:
  - Watched a project I was not a member of.
  - Clicked the feed watch/unwatch button.

{F1064909}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6183, T10054

Differential Revision: https://secure.phabricator.com/D15063
2016-01-19 19:38:30 -08:00
epriestley
b812a2171a Make custom "date" field feed stories human-readable
Summary: Fixes T10130.

Test Plan: {F1060142}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10130

Differential Revision: https://secure.phabricator.com/D15004
2016-01-12 07:11:26 -08:00
epriestley
dd21761df9 Don't return any results for viewerprojects() if the viewer is in no projects
Summary:
Fixes T10135. When the viewer is a member of no projects, specify the constraint type as a new "EMPTY" type.

When a query has an "EMPTY" constraint, fail fast with no results.

Test Plan:
  - Viewed a viewerprojects() query result set as a user in no projects.
    - Before patch: got a lot of hits. After patch: no hits.
  - Viewed a normal result set, no changes.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10135

Differential Revision: https://secure.phabricator.com/D15003
2016-01-12 07:11:14 -08:00
epriestley
96b1665eaa Link "continue" action to confirm dialog in bulk jobs that are unconfirmed
Summary: See Q266.

Test Plan: Created a bulk job, clicked "Details" instead of "Confirm", clicked "Continue" to get back to confirmation dialog.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14985
2016-01-10 10:55:58 -08:00
epriestley
14ae3c099c Make queries for Project "X" mean "X, or any subproject of X"
Summary:
Ref T10010. I think this is the desired/expected default behavior (e.g., searching for "Maniphest" should find tasks in any subproject or sprint of that project).

I'll probably add an "exact(...)" function later to mean "only the Maniphest superproject, exactly, not any of its children".

Test Plan:
  - Added and executed unit tests.
  - Ran various queries from the web UI.
  - Got sensible-seeming results.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D14910
2015-12-29 10:41:13 -08:00
epriestley
6fe882e50a Convert projects to EditEngine
Summary: Ref T10010. This is pretty straightforward with a couple of very minor new behaviors, like the icon selector edit field.

Test Plan:
  - Created projects.
  - Edited projects.
  - Saw "Create Project" in quick create menu.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D14896
2015-12-27 15:42:50 -08:00
epriestley
e0ad791247 Fix hovercard behavior for multiple copies of the same object
Summary: Ref T8980. Previously, if you had like `T123 T123 T123` and waved your mouse over them, we wouldn't move the card. Now, move the card.

Test Plan: Waved mouse. Saw the card move.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8980

Differential Revision: https://secure.phabricator.com/D14881
2015-12-24 13:24:00 -08:00
epriestley
bdc517485c Modernize Hovercard implementation
Summary:
Ref T8980. Move away from events to EngineExtensions.

This also simplifies hovercards a bit:

  - Removes tasks from revision cards.
  - Removes blockers/blocked from task cards.
  - Removes "Send Message" from user cards.

These mostly felt cluttery to me. Open to arguments to retain them. I think we can make better use of the space, though (e.g., flags, projects + board columns).

Test Plan:
  - Viewed people, task, revision, commit and project hovercards.

{F1043256}

{F1043257}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8980

Differential Revision: https://secure.phabricator.com/D14878
2015-12-24 12:18:28 -08:00
epriestley
3ec07c4987 Show hovercards for most links in object property views
Summary:
Ref T8980. This isn't 100% coverage but should be pretty much all of the common ones.

These feel a touch iffy to me at first glance so I didn't go crazy trying to hunt all of them down. I have some other plans for them so maybe they'll feel better by the end of it.

Test Plan: Hovered over author, reviewers, blocked tasks, projects, etc.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8980

Differential Revision: https://secure.phabricator.com/D14877
2015-12-24 12:10:56 -08:00
epriestley
37f1f55557 Fix a possible deadlock in unit tests after an error
Summary:
After certain types of errors, we may deadlock when trying to destroy test databases.

Specifically, we still have connections open to, say, `phabricator_unittest_abasonaknlbaklnasb_herald` (or whatever) and MySQL sometimes (not sure exactly when?) waits for them before destorying the database.

Test Plan:
  - Added `$m = null; $m->method()` to a fixture test to force a fatal.
  - Saw consistent deadlock, with `storage destroy` never exiting.
  - Added `--trace` to the `storage destroy` command and made it use `phutil_passthru()` so I could see what was happening.
  - Saw it hang on some arbitrary database.
  - Conneced to MySQL, used `show full processlist;` to see what was wrong.
  - Saw the `DROP DATABASE ...` command waiting for locks to release on the database, and other connections still open.
  - Applied patch.
  - Saw consistent success.
  - Used `storage destroy --unittest-fixtures` to clean up extra databases.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14875
2015-12-24 09:11:47 -08:00
epriestley
e2edb1577c Improve error messages for bad hashtags and project names
Summary: Ref T8509. We currently give you a fairly obtuse error when trying to name a project something like "!!". The error is correct, but not as helpful as it could be. Give users a more specific, more helpful error.

Test Plan: {F1042883}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8509

Differential Revision: https://secure.phabricator.com/D14872
2015-12-24 08:11:02 -08:00
epriestley
92912a6072 Fix project hashtag bugs: duplicate tags, uppercase tags
Summary:
Ref T8509. This fixes three issues:

  - Adding a slug like `UPPERCASE` would not give you a normalized slug. (Now: normalizes as `uppercase`.)
  - Adding a slug like `UPPERCASE` would allow you to give two different projects the different tags `UPPERCASE` and `uppercase` (and `UpPeRcAsE`, etc). (Now: second tag is rejected as a duplicate.)
  - Adding multiple identical or similar slugs would produce a duplicate key exception. (Now: ignores the duplicates.)

Test Plan:
  - Added test coverage.
  - Made tests pass.
  - Hit these cases in the UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8509

Differential Revision: https://secure.phabricator.com/D14870
2015-12-24 08:10:03 -08:00
epriestley
96fe8c0b83 Implement basic ngram search for Owners Package names
Summary:
Ref T9979. This uses ngrams (specifically, trigrams) to build a reasonably efficient index for substring matching. Specifically, for a package like "Example", with ID 123, we store rows like this:

```
< ex, 123>
<exa, 123>
<xam, 123>
<amp, 123>
<mpl, 123>
<ple, 123>
<le , 123>
```

When the user searches for `exam`, we join this table for packages with tokens `exa` and `xam`. MySQL can do this a lot more efficiently than it can process a `LIKE "%exam%"` query against a huge table.

When the user searches for a one-letter or two-letter string, we only search the beginnings of words. This is probably what they want, the only thing we can do quickly, and a reasonable/expected behavior for typeaheads.

Test Plan:
  - Ran storage upgrades and search indexer.
  - Searched for stuff with "name contains".
  - Used typehaead and got sensible results.
  - Searched for `aabbccddeeffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz` and saw only 16 joins.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14846
2015-12-22 08:00:33 -08:00
epriestley
aab1574e33 Remove TYPE_SEARCH_DIDUPDATEINDEX event
Summary:
Ref T9979. This event had one weird callsite and no known third-party callers. It can be done more cleanly as an extension, now.

This index is used to allow us to "Group By: Project" in Maniphest without joining into the Projects database.

Test Plan:
  - Ran a query with "Group By: Project" in Maniphest.
  - Renamed project "Apples" to "Zebras".
  - Reloaded page.
  - UI properly moved "Zebras" tasks to the bottom of the list.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14836
2015-12-21 17:23:59 -08:00
epriestley
02f82c2af5 Modularize fulltext indexing of Projects, Subscriptions and Custom Fields
Summary: Ref T9979. This is going to become `FulltextEngine`, but pave the way for that by pulling extensions out of it.

Test Plan:
{F1036624}

  - Used `bin/search index Txxx`, saw projects, subscribers and custom fields rebuild in the index.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14835
2015-12-21 17:04:25 -08:00
epriestley
4bba3fd4c1 Fully modularize DestructionEngine
Summary: Ref T9979. Convert all DestructionEngine behaviors to extensions.

Test Plan:
{F1033244}

Destroyed an object, verifying:

  - Herald transcripts were destroyed;
  - edges were destroyed;
  - flags were destroyed;
  - tokens were destroyed;
  - transactions were destroyed;
  - worker tasks were cancelled.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14832
2015-12-21 17:03:44 -08:00
Chad Little
75ba2c0926 Correct some Pirate translations
Summary: Swapped these by mistake.

Test Plan: load pirate english, read

Reviewers: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14838
2015-12-21 08:04:23 -08:00
Chad Little
006321cce7 Add a pirate translation
Summary: Basic Pirate, mostly Maniphest

Test Plan: Play lots with Maniphest

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14830
2015-12-20 07:03:18 -08:00
epriestley
46e690b2fd Fix value reading in custom Text and Remarkup fields in EditEngine
Summary: Ref T10004. I missed these previously, so they didn't work quite right. Restore them to glory.

Test Plan: Edited remarkup and text custom fields on an Owners package.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10004

Differential Revision: https://secure.phabricator.com/D14821
2015-12-18 11:57:44 -08:00
epriestley
e9af4f8970 Fix an issue where Drydock followup tasks would not queue if the main task failed
Summary:
Ref T9994. This fixes the first issue discussed on that task, which is that when a merge fails after "arc land", we would not clean up all the leases properly.

Specifically, when a merge fails, we use `queueTask()` to schedule a followup task. This followup destroys the lease and frees the underlying resource.

However, the default behavior of `queueTask()` is to //not queue tasks// if the parent task fails. This is a reasonable, safe behavior that was originally introduced in D8774, where it kept us from sending too much mail if a task did "send some mail" and then failed a little later on and got retried.

Since I think the default behavior is correct, I just special cased the behavior for Drydock to make it queue even on failure. These are the only types of followup tasks we currently want to queue on main task failure.

(It's possible that future Blueprints might want some kind of more specialized behavior, where some tasks queue only on success, but we can cross that bridge when we come to it.)

Test Plan:
  - See T9994#149878 for test case setup.
  - I ran that test case again with this patch, and saw the followup task queue properly in the `--trace` log, a correspoinding update task show up in `/daemon/`, and the lease get destroyed when I ran it a moment later.

{F1029915}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9994

Differential Revision: https://secure.phabricator.com/D14818
2015-12-18 08:17:04 -08:00
epriestley
8a0dfa94d4 Make configured and EditEngine defaults work correctly for custom fields
Summary:
Ref T10004. Fixes T5158. There was a long-standing issue with defaults not working properly, but EditEngine has made it more obvious because it's a lot easier to set defaults now.

The issue is basically that the defaults are getting set as the field's real value early on, so when we go to generate the transaction "old value" later, we build a transaction that uses the //new// value as both the "new value" and "old value". Then the engine says "you didn't change anything, so I'm going to ignore this" and drops it.

To fix this, return `null` as the "old value" by default, and add a call to overwrite that after we load a legitimate old value.

This fix is a touch iffy, but I have some grand plans to clean up the CustomField stuff more broadly later on.

Test Plan:
  - Set config defaults on select/typeahead fields, created and edited tasks.
  - Set form defaults on select/typehaead fields, created and edited tasks.
  - In all cases, transactions and state accurately reflected edits.
  - Set defaults on //hidden// fields, verified forms respected them correctly.
  - This does generate some fluffy transactions, but I'll deal with those in T7661.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5158, T10004

Differential Revision: https://secure.phabricator.com/D14809
2015-12-17 10:44:41 -08:00
epriestley
a5d23c9f3e Allow custom fields to be ordered ascending or descending
Summary:
Fixes T6864. This creates a sort of busy menu but I think that's proably fine -- users are opting into activating these fields for search anyway.

In the future, we could refine this as, e.g.:

  - don't show these options in the dropdown;
  - do show them on some new "http prefilling" sort of page;
  - then you access them as an advanced user with `?order=secret-magic`.

But I'm not going to bother for now.

Test Plan: Ordered by an int field, then reversed the order.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6864

Differential Revision: https://secure.phabricator.com/D14800
2015-12-16 12:59:22 -08:00
epriestley
161ebad56d Improve Conduit type handling for *.edit endpoints
Summary:
Ref T9964. Three goals here:

  - Make it easier to supply Conduit documentation.
  - Make automatic documentation for `*.edit` endpoints more complete, particularly for custom fields.
  - Allow type resolution via Conduit types, so you can pass `["alincoln"]` to "subscribers" instead of needing to use PHIDs.

Test Plan:
  - Viewed and used all search and edit endpoints, including custom fields.
  - Used parameter type resolution to set subscribers to user "dog" instead of "PHID-USER-whatever".
  - Viewed HTTP parameter documentation.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14796
2015-12-16 08:45:46 -08:00
epriestley
0a50219f1b Formalize custom Conduit fields on objects
Summary: Ref T9964. This just adds more structure to application fields, to make it harder to make typos and easier to validate them later.

Test Plan: Viewed APIs, called some APIs, saw good documentation and correct results.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14776
2015-12-14 11:54:13 -08:00
epriestley
663dce5029 Flesh out Conduit parameter types for Owners + CustomFields
Summary:
Ref T9964. Fill in more parameter types and descriptions.

(No date support yet since it's a bit more involved.)

Test Plan: {F1024022}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14766
2015-12-14 04:23:44 -08:00
epriestley
1b325a0a89 Modularize SearchEngine extensions
Summary:
Ref T9964. ApplicationSearch currently has a bunch of hard-coded `if ($object instanceof thing)` stuff.

Pull that out so it can live in extensions.

Test Plan:
 - Searched by spaces, subscribers, projects.

{F1023921}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14764
2015-12-14 04:23:02 -08:00
epriestley
9499987cfe Add "owners.search" Conduit API endpoint, with CustomField support
Summary:
Ref T9964. Adds a new-style "owners.search" endpoint, and an extension for customfields.

Puts enough indirection in place to give us nice, consistent "custom.key" user-facing keys instead of "std:custom:owners:na0shf9a8dfdsafl" junk.

Test Plan:
  - Searched Owners via API.
  - Searched by ID.
  - Ordered by custom fields.
  - Reviewed API docs.
  - Used normal search with ordering.
  - Viewed custom field values in search results.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14758
2015-12-13 02:11:59 -08:00
epriestley
7c98cd85fe Implement DestructibleInterface for Owners Packages
Summary:
Fixes T9945. This is straightforward.

The two sub-object types are very lightweight so I just deleted them directly instead of loading + delete()'ing (or implementing DestructibleInterface on them, which would require they have PHIDs).

Also improve a US English localization.

Test Plan:
  - Used `bin/remove destroy PHID-... --trace` to destroy a package.
  - Verified it was gone.
  - Inspected the SQL in the log for general reasonableness.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9945

Differential Revision: https://secure.phabricator.com/D14729
2015-12-10 07:04:06 -08:00
epriestley
d57cc740ca Clean up some custom field strings in Feed
Summary: Fixes T9919. We were missing feed strings and US English localizations for some of this stuff.

Test Plan:
Before:

{F1018877}

After:

{F1018879}
{F1018880}
{F1018881}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9919

Differential Revision: https://secure.phabricator.com/D14721
2015-12-09 09:04:12 -08:00
epriestley
fb3c18349e Remove WILLEDITTASK and DIDEDITTASK events
Summary: Fixes T9851. I'll hold this for a while to give users some time to update per T9860.

Test Plan:
Edited a task via:

  - Conduit
  - Comments field
  - Edit form
  - New task form

Reviewers: chad

Reviewed By: chad

Subscribers: Krenair

Maniphest Tasks: T9851

Differential Revision: https://secure.phabricator.com/D14576
2015-12-09 07:03:15 -08:00
epriestley
5d5fd9e241 Explain why older changes are hidden more clearly
Summary:
Fixes T9920. When hiding changes, tell users why so they can learn the comment rule (usually, "Changes from before your most recent comment are hidden."; sometimes they're hidden for pagination reasons).

Also use "Show Older Comments" instead of "Show older comments." for the action since I think it's a little more consistent to use title case for links/actions?

Test Plan:
  - Viewed a task with a lot of comments, saw a "most recent comment" element.
  - Artificially set page size to 3, saw a "lots of changes" hide.
  - Grepped for removed string.
  - Clicked both "show older stuff" links.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9920

Differential Revision: https://secure.phabricator.com/D14719
2015-12-09 06:59:41 -08:00
epriestley
468f785845 Support "template objects" generically in EditEngine
Summary:
Ref T9132. Ref T9908. Fixes T5622. This allows you to copy some fields (projects, subscribers, custom fields, some per-application) from another object when creating a new object by passing the `?template=xyz` parameter.

Extend "copy" support to work with all custom fields.

Test Plan:
  - Created new pastes, packages, tasks using `?template=...`
  - Viewed new template docs page.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5622, T9132, T9908

Differential Revision: https://secure.phabricator.com/D14699
2015-12-07 13:44:07 -08:00
epriestley
618cec23d8 Make notification counts properly translatable
Summary:
Ref T9132. When I've touched `PhabricatorApplication` I keep hitting this bad `pht()` junk.

The warning is correct, these strings are not extactable and can not be translated.

Fix it so they can be extracted and translated.

Broadly, in all cases we want to render one of these:

> 95 Things (for fewer than some limit)
> 99+ Things (when we hit the limit)

Test Plan: Looked at homepage status counts, moused over them, saw reasonable strings. Grepped for removed method.

Reviewers: chad

Reviewed By: chad

Subscribers: joshuaspence

Maniphest Tasks: T9132

Differential Revision: https://secure.phabricator.com/D14638
2015-12-03 07:06:39 -08:00
Joshua Spence
9104867c71 Linter fixes
Summary: Minor linter fixes.

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14631
2015-12-03 07:44:23 +11:00
epriestley
773ecb9a44 Support Conduit application of most CustomField transactions in EditEngine
Summary:
Ref T9132. Give most standard custom fields reasonable Conduit support so you can use the new `application.x` endpoints to set them.

Major missing field type is dates, again.

Test Plan: Used Conduit to set various custom fields on a package.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9132

Differential Revision: https://secure.phabricator.com/D14635
2015-12-02 09:32:49 -08:00
epriestley
c1ae5321d7 Support HTTP parameter prefilling in EditEngine forms for CustomFields
Summary:
Ref T9132. This allows you to prefill custom fields with `?custom.x.y=value`, for most types of custom fields.

Dates (which are substantially more complicated) aren't supported. I'll just do those once the dust settles. Other types should work, I think.

Test Plan:
  - Verified custom fields appear on "HTTP Parameters" help UI.
  - Used `?x=y` to prefill custom fields on edit form.
  - Performed various normal edits.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9132

Differential Revision: https://secure.phabricator.com/D14634
2015-12-02 09:32:26 -08:00
epriestley
029b1b6733 Partially support CustomFields in EditEngine
Summary:
Ref T9132. This isn't perfect, but doesn't break any existing functionality. This stuff works:

  - Editing values.
  - Reordering fields.
  - All builtin field tyepes.

This stuff may not work yet:

  - Assigning custom field defaults.
  - Some conduit stuff.
  - Fully custom fields?
  - Locking/hiding fields? Didn't actually test this one.

I'll keep chipping away at that stuff. In some cases, it may be easier to convert all the CustomField apps first, although Differential might be a fair bit of work.

Test Plan:
Created a bunch of custom fields of every avialable type and edited them.

{F1008789}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9132

Differential Revision: https://secure.phabricator.com/D14617
2015-12-02 05:21:31 -08:00
Joshua Spence
7164606285 Add a lock to storage upgrade and adjustment
Summary: Fixes T9715. Adds a MySQL-based lock to ensure that schema migrations are not applied on multiple hosts simultaneously.

Test Plan: Ran `./bin/storage upgrade` concurrently. One invocation was successful whilst the other hit a `PhutilLockException`.

Reviewers: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T9715

Differential Revision: https://secure.phabricator.com/D14463
2015-12-02 06:18:28 +11:00
epriestley
b964f8873b Fix daemon restart behavior to check once every 10 seconds
Summary: This logic is flipped.

Test Plan:
  - Before change: ran `bin/phd debug task`, saw queries to the config table every second.
  - After change: ran `bin/phd debug task`, saw queries to the config table every 10 seconds.

Reviewers: chad, joshuaspence

Reviewed By: chad, joshuaspence

Differential Revision: https://secure.phabricator.com/D14542
2015-11-23 05:59:04 -08:00
cburroughs
d5cb3cd277 typo in storage message
Test Plan:
I didn't put any skill points in spelling since I need
combat skills to survive in a nuclear wasteland, but spell check says
this is better.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D14522
2015-11-19 12:40:50 -05:00
epriestley
2e09a93dc1 Improve efficiency of worker task GC for huge loads
Summary:
Fixes T9808.

An instance imported a very large repository, generating approximately 4 million tasks over the course of a few days. A week later, these tasks started expiring and became candidates for garbage collection.

The GC works by deleting 100 rows at at time over and over again. It finds the rows it's going to delete by querying for old rows.

Currently, this query generates a `WHERE dateCreated < X ORDER BY id DESC` query. This query can not efficiently execute using a single key, because it relies on `dateCreated` order to find the rows, then on `id` order to sort them. With a table with 4M rows, this is slow.

This would still be OK, except that the query has to execute a lot of times since it only deletes 100 rows each time. Particularly, it needs to execute a total of ~40K times.

Instead, generate `WHERE dateCreated < X ORDER BY dateCreated DESC, id DESC`. This should have the same effect in general and the GC definitely doesn't care about the difference, but it should be more efficient at large scales.

Test Plan:
I had to `TRUNCATE` the problem table so I don't have a perfect repro to completely convincingly test this anymore. Both queries behave fine at small scales, which is why we haven't seen this before.

I was able to run the newer query in production before I nuked the table and have it complete in a reasonable amount of time, while the old query hung longer than I wanted to wait (several minutes?). The query plan for the new query was also a good one, while the query plan for the old query was terrible.

I loaded the daemon console and ran `bin/garbage collect --collector worker.tasks --trace`. I verified the queries looked reasonable and produced reasonable results in production.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9808

Differential Revision: https://secure.phabricator.com/D14505
2015-11-17 17:05:10 -08:00
Joshua Spence
1f1c3f4075 Allow lint messages to be rendered as Remarkup
Summary: Some linter messages, such as those produced by `ArcanistPHPCompatibilityXHPASTLinterRule`, contain backticks but are currently rendered as Remarkup literals. I think that it is generally desirable to allow lint messages to be rendered as Remarkup, although we should ideally have a way to render Remarkup for use on the command line (I actually think that this already exists, but I don't think that `arc lint` does this when rendering linter messages).

Test Plan: Resubmitted D14481 to my dev install and saw Remarkuped lint messages.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14485
2015-11-15 19:50:10 +00:00
Joshua Spence
ca0b36c174 Rename XHPAST database
Summary: Rename the XHPAST database from `{$NAMESPACE}_xpastview` to `{$NAMESPACE}_xhpast`.

Test Plan: Ran `./bin/storage --namespace test upgrade --no-quickstart`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14442
2015-11-14 21:41:28 +11:00
Joshua Spence
a1737ef9c7 Fix a translation
Summary: Fixes T9763.

Test Plan: Merged tasks, saw translations.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T9763

Differential Revision: https://secure.phabricator.com/D14473
2015-11-13 07:04:48 +11:00
Joshua Spence
321c61a853 Remove daemon envHash and envInfo
Summary: Ref T7053. Remove the `envHash` and `envInfo` fields, which are no longer used now that the daemons restart automagically. Depends on D14458.

Test Plan: Saw no more setup issues.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: tycho.tatitscheff, epriestley

Maniphest Tasks: T7053

Differential Revision: https://secure.phabricator.com/D14446
2015-11-11 08:54:45 +11:00
Joshua Spence
a07a8aca24 Add a daemon overseer module to restart daemons when config changes
Summary: Fixes T7053. Depends on D14452.

Test Plan:
Created a custom daemon which dumps out the config hash (by querying `PhabricatorEnv::calculateEnvironmentHash()`). Ran this daemon with `./bin/phd debug PhabricatorDebugDaemon` and saw the config hash update within 30 seconds.

{P1886}

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T7053

Differential Revision: https://secure.phabricator.com/D14458
2015-11-11 08:44:18 +11:00
lkassianik
64ad44cffb Always override auth.email-domains when running unit tests
Summary: Fixes T9689, Always override `auth.email-domains` when running unit tests

Test Plan:
- Set `auth.email-domains`
- Run `arc unit --everything`.
Observe no errors.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T9689

Differential Revision: https://secure.phabricator.com/D14456
2015-11-10 10:15:10 -08:00
Joshua Spence
af7b16248e Fix a translation
Summary: Fixes T9655.

Test Plan: I haven't tested this... it seems simple enough.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T9655

Differential Revision: https://secure.phabricator.com/D14375
2015-11-06 13:39:35 +11:00
Chad Little
8bbcd896b8 Add styling for new Remarkup highlighter
Summary: Adds some basic style to new !!Remarkup Highlighter!! Ref T5560

Test Plan: Wait for next diff.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T5560

Differential Revision: https://secure.phabricator.com/D14383
2015-11-02 13:20:07 -08:00
Joshua Spence
c35b564f4d Various translation improvements
Summary: Depends on D14070.

Test Plan: Eyeball it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D14073
2015-11-03 07:02:46 +11:00
Joshua Spence
495cb7a2e0 Mark PhabricatorPHIDType::getPHIDTypeApplicationClass() as abstract
Summary: Fixes T9625. As explained in a `TODO` comment, seems reasonable enough.

Test Plan: Unit tests.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, hach-que

Maniphest Tasks: T9625

Differential Revision: https://secure.phabricator.com/D14068
2015-11-03 06:47:12 +11:00
epriestley
3f193cb9e0 Give Harbormaster build steps a "View" page
Summary:
Fixes T9519. Right now, build steps go straight from the build to the edit screen.

This means that there's no way to see their edit history or review details without edit permission. In particular, this makes it a bit harder to catch the Drydock Blueprint authorization warnings from T9519.

  - Add a standard view screen.
  - Add a little warning callout to blueprint authorizations.

This also does a bit of a touchup on the weird dropshadow element from T9586. Maybe not totally design-approved now but it's less ugly, at least.

Test Plan:
{F906695}

{F906696}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9519

Differential Revision: https://secure.phabricator.com/D14330
2015-10-26 12:38:32 -07:00
epriestley
ad53e7b878 Record how long storage patches took to apply
Summary:
It's hard for us to predict how long patches and migrations will take in the general case since it varies a lot from install to install, but we can give installs some kind of rough heads up about longer patches. I'm planning to just put a sort of hint for things in the changelog, something like this:

{F905579}

To make this easier, start storing how long stuff took. I'll write a little script to dump this into a table for the changelog.

Test Plan:
Ran `bin/storage status`:

{F905580}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14320
2015-10-24 05:58:44 -07:00
epriestley
4b43667086 Introduce PHUIRemarkupView, a sane way to work with Remarkup
Summary:
Fixes T9273. Remarkup has reasonably good fundamentals but the API is a giant pain to work with.

Provide a `PHUIRemarkupView` to make it easier. This object is way simpler to use by default.

It's not currently as powerful, but we can expand the power level later by adding more setters.

Eventually I'd expect to replace `PhabricatorRemarkupInterface` and `PhabricatorMarkupOneOff` with this, but no rush on those.

I converted a few callsites as a sanity check that it works OK.

Test Plan:
- Viewed remarkup in Passphrase.
- Viewed remarkup in Badges.
- Viewed a Conduit method.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9273

Differential Revision: https://secure.phabricator.com/D14289
2015-10-15 10:20:19 -07:00
epriestley
034ff3c870 Remove "_-_" -> "-" slug behavior
Summary: Fixes T9573. This incorrectly affected Phriction. I could restore it for only projects, but you didn't like the rule very much anyway and I don't feel strongly about it.

Test Plan: Unit tests.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9573

Differential Revision: https://secure.phabricator.com/D14287
2015-10-15 07:04:14 -07:00
epriestley
4169d7bfd5 Fix an issue where Harbormaster might cycle while saving
The way custom field interact with storage is a little odd, and can send us
down a bad path when applying external effect while saving changes.
2015-10-14 02:56:39 -07:00
epriestley
df5a031b54 Allow "Repository Automation" to be configured for repositories
Summary:
Ref T182. This allows you to assign blueprints that a repository can use to perform working copy operations. Eventually, this will support "merge this" in Differential, etc.

This is just UI for now, with no material effects.

Most of this diff is just taking logic that was in the existing "Blueprints" CustomField and putting it in more general places so Diffusion (which does not use CustomFields) can also access it.

Test Plan:
  - Configured repository automation for a repository.
  - Removed repository automation for a repository.

Reviewers: chad

Reviewed By: chad

Subscribers: avivey

Maniphest Tasks: T182

Differential Revision: https://secure.phabricator.com/D14259
2015-10-13 15:45:59 -07:00
epriestley
cd8be8106b Improve ruleset for generating project hashtags
Summary:
Ref T9551. We currently use the same logic for generating project hashtags and Phriction slugs, but should be a little more conservative with project hashtags.

Stop them from generating with stuff that won't parse in a "Reviewers:" field or generally in commments (commas, colons, etc).

Test Plan:
Created a bunch of projects with nonsense in them and saw them generate pretty reasonable hashtags.

{F873456}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9551

Differential Revision: https://secure.phabricator.com/D14261
2015-10-12 17:02:58 -07:00
epriestley
3ff5ca789a Fix /tag/aa%20bb project URIs
Summary:
Ref T9551. To set things up:

  - Name a project `aa bb`. This will have the tag `aa_bb`.
  - Try to visit `/tag/aa%20bb`.

Here's what happens now:

  - You get an Aphront redirect error as it tries to add the trailing `/`. Add `phutil_escape_uri()` so that works again.
  - Then, you 404, even though this tag is reasonably equivalent to the real project tag and could be redirected. Add a fallback to lookup, resolve, and redirect if we can find a hit for the tag.

This also fixes stuff like `/tag/AA_BB/`.

Test Plan: Visited URIs like `/tag/aa%20bb`, `/tag/aa%20bb/`, `/tag/Aa_bB/`, etc. None of them worked before and now they all do.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9551

Differential Revision: https://secure.phabricator.com/D14260
2015-10-12 17:02:42 -07:00
epriestley
2f6d3119f5 Rough cut of "Blueprint Authorizations"
Summary:
Ref T9519. This is like 80% of the way there and doesn't fully work yet, but roughly shows the shape of things to come. Here's how it works:

First, there's a new custom field type for blueprints which works like a normal typeahead but has some extra logic. It's implemented this way to make it easy to add to Blueprints in Drydock and Build Plans in Harbormaster. Here, I've added a "Use Blueprints" field to the "WorkingCopy" blueprint, so you can control which hosts the working copies are permitted to allocate on:

{F869865}

This control has a bit of custom rendering logic. Instead of rendering a normal list of PHIDs, it renders an annotated list with icons:

{F869866}

These icons show whether the blueprint on the other size of the authorization has approved this object. Once you have a green checkmark, you're good to go.

On the blueprint side, things look like this:

{F869867}

This table shows all the objects which have asked for access to this blueprint. In this case it's showing that one object is approved to use the blueprint since I already approved it, but by default new requests come in here as "Authorization Requested" and someone has to go approve them.

You approve them from within the authorization detail screen:

{F869868}

You can use the "Approve" or "Decline" buttons to allow or prevent use of the blueprint.

This doesn't actually do anything yet -- objects don't need to be authorized in order to use blueprints quite yet. That will come in the next diff, I just wanted to get the UI in reasonable shape first.

The authorization also has a second piece of state, which is whether the request from the object is active or inactive. We use this to keep track of the authorization if the blueprint is (maybe temporarily) deleted.

For example, you might have a Build Plan that uses Blueprints A and B. For a couple days, you only want to use A, so you remove B from the "Use Blueprints: ..." field. Later, you can add B back and it will connect to its old authorization again, so you don't need to go re-approve things (and if you're declined, you stay declined instead of being able to request authorization over and over again). This should make working with authorizations a little easier and less labor intensive.

Stuff not in this diff:

  - Actually preventing any allocations (next diff).
  - Probably should have transactions for approve/decline, at least, at some point, so there's a log of who did approvals and when.
  - Maybe should have a more clear/loud error state when no blueprints are approved?
  - Should probably restrict the typeahead to specific blueprint types.

Test Plan:
  - Added the field.
  - Typed some stuff into it.
  - Saw the UI update properly.
  - Approved an authorization.
  - Declined an authorization.
  - Saw active authorizations on a blueprint page.
  - Didn't see any inactive authroizations there.
  - Clicked "View All Authorizations", saw all authorizations.

Reviewers: chad, hach-que

Reviewed By: chad

Maniphest Tasks: T9519

Differential Revision: https://secure.phabricator.com/D14251
2015-10-10 07:15:25 -07:00
epriestley
5a874ba0a8 Put cows and figlet bannners in <pre> in HTML mail bodies
Summary: Fixes T9538. Ref T9408. `cowsay` and `figlet` Remarkup rules are being mangled in HTML mail right now. Put them in <pre> to unmangle them.

Test Plan:
Sent myself a cow + figlet in mail.

Used `bin/mail show-outbound --id ... --dump-html > dump.html` + open that HTML file in Safari to preview HTML mail.

Saw linebreaks and monospaced formatting.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9538, T9408

Differential Revision: https://secure.phabricator.com/D14248
2015-10-08 20:03:15 -07:00
epriestley
03fea70497 Fix some header formatting in bin/storage probe
Summary: Ref T9514. I missed these when I swapped out the console stuff recently.

Test Plan: Ran `bin/storage probe`, saw bold instead of escape sequences.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9514

Differential Revision: https://secure.phabricator.com/D14240
2015-10-06 07:38:09 -07:00
epriestley
de2bbfef7d Allow PhabricatorWorker->queueTask() to take full $options
Summary:
Ref T9252. Currently, `queueTask()` accepts `$priority` as its third argument. Allow it to take a full range of `$options` instead. This API just never got updated after we expanded avialable options.

Arguably this whole API should be some kind of "TaskQueueRequest" object but I'll leave that for another day.

Test Plan:
  - Grepped for `queueTask()` and verified no other callsites are affected by this API change.
  - Ran some daemons.
  - See also next diff.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14235
2015-10-05 09:46:29 -07:00
epriestley
4cf1270ecd In Harbormaster, make sure artifacts are destroyed even if a build is aborted
Summary:
Ref T9252. Currently, Harbormaster and Drydock work like this in some cases:

  # Queue a lease for activation.
  # Then, a little later, save the lease PHID somewhere.
  # When the target/resource is destroyed, destroy the lease.

However, something can happen between (1) and (2). In Drydock this window is very short and the "something" would have to be a lighting strike or something similar, but in Harbormaster we wait until the resource activates to do (2) so the window can be many minutes long. In particular, a user can use "Abort Build" during those many minutes.

If they do, the target is destroyed but it doesn't yet have a record of the artifact, so the artifact isn't cleaned up.

Make these things work like this instead:

  # Create a new lease and pre-generate a PHID for it.
  # Save that PHID as something that needs to be cleaned up.
  # Queue the lease for activation.
  # When the target/resource is destroyed, destroy the lease if it exists.

This makes sure there's no step in the process where we might lose track of a lease/resource.

Also, clean up and standardize some other stuff I hit.

Test Plan:
  - Stopped daemons.
  - Restarted a build in Harbormaster.
  - Stepped through the build one stage at a time using `bin/worker execute ...`.
  - After the lease was queued, but before it activated, aborted the build.
  - Processed the Harbormaster side of things only.
  - Saw the lease get destroyed properly.

Reviewers: chad, hach-que

Reviewed By: hach-que

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14234
2015-10-05 05:58:53 -07:00
epriestley
9c798e5cca Provide bin/garbage for interacting with garbage collection
Summary:
Fixes T9494. This:

  - Removes all the random GC.x.y.z config.
  - Puts it all in one place that's locked and which you use `bin/garbage set-policy ...` to adjust.
  - Makes every TTL-based GC configurable.
  - Simplifies the code in the actual GCs.

Test Plan:
  - Ran `bin/garbage collect` to collect some garbage, until it stopped collecting.
  - Ran `bin/garbage set-policy ...` to shorten policy. Saw change in web UI. Ran `bin/garbage collect` again and saw it collect more garbage.
  - Set policy to indefinite and saw it not collect garabge.
  - Set policy to default and saw it reflected in web UI / `collect`.
  - Ran `bin/phd debug trigger` and saw all GCs fire with reasonable looking queries.
  - Read new docs.

{F857928}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9494

Differential Revision: https://secure.phabricator.com/D14219
2015-10-02 09:17:24 -07:00
epriestley
878a493301 Begin standardizing garbage collectors
Summary: Ref T9494. Improve support infrastructure for garbage collectors.

Test Plan:
  - Ran `bin/phd debug trigger`, saw collectors execute.

{F857852}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9494

Differential Revision: https://secure.phabricator.com/D14218
2015-10-01 16:58:43 -07:00
epriestley
e431ab2189 Use getPhobjectClassConstant() to access class constants
Summary: Ref T9494. Depends on D14216. Remove 10 copies of this code.

Test Plan: Ran `arc unit --everything`, browsed Config > Modules, clicked around Herald / etc.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9494

Differential Revision: https://secure.phabricator.com/D14217
2015-10-01 16:56:21 -07:00
epriestley
4496176924 Add staging area support to Harbormaster/Drydock + various fixes
Summary:
Ref T9252. This primarily allows Harbormaster to request (and Drydock to fulfill) working copies with a patch from a staging area. Doing this means we can do builds on in-review changes from `arc diff`.

This is a little cobbled-together but should basically work.

Also fix some other issues:

  - Yielded, awakend workers are fine to update but could complain.
  - We can't log slot lock failures to resources if we don't end up saving them.
  - Killing the transaction would wipe out the log.
  - Fix some TODOs, etc.

Test Plan: Ran Harbormaster builds on a local revision.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14214
2015-10-01 16:55:01 -07:00
epriestley
4ac82be5ed Merge the DrydockLease workers into a single worker
Summary:
Ref T9252. This is the same as D14201, but for lease stuff instead of resource stuff.

This one is a little heavier but still feels pretty reasonable to me at the end of the day (worker is <1K lines and has a ton of comment stuff).

Also fixes a few random bugs I hit in the task queue.

Test Plan:
  - Restarted some Harbormaster builds, saw them go through cleanly.
  - Released pre-activation resources/leases.
  - Probably still kinda buggy but I'll iron the details out over time.

Logs are starting to look somewhat plausible:

{F855747}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14202
2015-10-01 08:11:02 -07:00
epriestley
fa943f744b Stop all object mentions from matching after "@"
Summary:
Fixes T9479. Currently, `@aaaaaaaa` may try to match as a commit hash, and `@C123456` may try to match as a Countdown reference. These should only match as user mentions.

Prevent object mention rules from matching after `@`. We already prevent them after `-` and `#`, and already prevented the username rule after `@` (i.e., preventing `@@user`).

Test Plan:
Created some "interesting" users locally and `@mentioned` them:

{F850779}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9479

Differential Revision: https://secure.phabricator.com/D14186
2015-09-29 06:43:49 -07:00
epriestley
55767aac0f Fix an issue where followup tasks could fail to queue with string priorities
Auditors: chad
2015-09-28 19:46:41 -07:00
epriestley
efaa8170c3 Simplify value decoding for PHID custom fields
Summary:
Ref T9123. The handling in D14183 didn't deal with new field values properly.

Make all this handling more consistent.

Test Plan: Created a new WorkignCopy build plan with some repos.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9123

Differential Revision: https://secure.phabricator.com/D14184
2015-09-28 18:44:40 -07:00
epriestley
bfaa93aa9b Allow Harbormaster build plans to request additional working copies
Summary:
Ref T9123. To run upstream builds in Harbormaster/Drydock, we need to be able to check out `libphutil`, `arcanist` and `phabricator` next to one another.

This adds an "Also Clone: ..." field to Harbormaster working copy build steps so I can type all three repos into it and get a proper clone with everything we need.

This is somewhat upstream-centric and a bit narrow, but I don't think it's totally unreasonable, and most of the underlying stuff is relatively general.

This adds some more typechecking and improves data/type handling for custom fields, too. In particular, it prevents users from entering an invalid/restricted value in a field (for example, you can't "Also Clone" a repository you don't have permission to see).

Test Plan: Restarted build, got a Drydock resource with multiple repositories in it.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9123

Differential Revision: https://secure.phabricator.com/D14183
2015-09-28 17:57:41 -07:00
epriestley
9b29d46e60 Make Drydock lease infrastructure more nimble
Summary:
Ref T9252. Currently, Harbormaster does this when trying to acquire a working copy:

  - Ask for a working copy.
  - Yield for 15 seconds.
  - Check if we have a working copy yet.

That's OK, but Drydock takes ~1s to acquire a working copy lease if a resource is already available, so we end up doing this:

  - T+0: Ask for a working copy.
  - T+0: Yield for 15 seconds.
  - T+1: Working copy lease activates.
  - T+15: Working copy lease is used.
  - T+16: Build finishes.

So we end up spending about 2 seconds doing work and 14 seconds sleeping.

One way to fix this would be to fiddle with the yield duration, so we yield for 1, 2, 4, ... seconds or something. This probably isn't a bad idea for longer leases (i.e., wait for 15, 30, 45 ... seconds or similar) but it implies a lot of churn for short leases.

Instead, let tasks "awaken" other tasks when they complete. The "awaken" operation means: if a task is in a yielded state (no failures, no owner, explicitly yielded, future expires time), pretend it only yielded until right now instead of whenever it really yielded to.

Basically, this rewrites history so that even though Harbormaster did a `yield(15)`, we pretend it did a `yield(4)` after we activate the lease if lease activation took 4 seconds.

If this misses, it's fine: we fall back to the normal yield behavior and things move forward normally a few seconds later.

If it hits, we get a more nimble process pretty cleanly.

Test Plan:
  - Restarted a build plan (lease working copy + run `ls`) with this patch no-op'd, took about 16 seconds.
  - Restarted a build plan with this patch active, took about 1 second.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14178
2015-09-28 09:35:40 -07:00
epriestley
ec6d69e74d Give Drydock resources a proper expiry mechanism
Summary:
Fixes T6569. This implements an expiry mechanism for Drydock resources which parallels the mechanism for leases.

A few things are missing that we'll probably need in the future:

  - An "EXPIRES" command to update the expiration time. This would let resources be permanent while leased, then expire after, say, 24 hours without any leases.
  - A callback like `shouldActuallyExpireRightNow()` for resources and leases that lets them decide not to expire at the last second.
  - A callback like `didAcquireLease()` for resource blueprints, to parallel `didReleaseLease()`, letting them clear or extend their timer.

However, this stuff would mostly just let us tune behaviors, not really open up new capabilities.

Test Plan: Changed host resources to expire after 60 seconds, leased one, saw it vanish 60 seconds later.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T6569

Differential Revision: https://secure.phabricator.com/D14176
2015-09-28 09:35:14 -07:00
epriestley
99d972fc81 Fix Herald rule actions on empty custom PHID fields
Summary: Fixes T9260. That task has a good description of the issue.

Test Plan: Followed steps in T9260 to reproduce the issue. Applied patch; issue went away.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9260

Differential Revision: https://secure.phabricator.com/D14169
2015-09-25 15:00:55 -07:00
epriestley
3379904237 Allow Drydock leases to expire after a time limit
Summary: Ref T6569. If a lease is activated with an expiration date, schedule a task to try to clean it up after that time.

Test Plan:
  - Used `bin/drydock lease ... --until ...` to activate a lease in the near future.
  - Waited for a bit.
  - Saw it expire and get destroyed at the scheduled time.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T6569

Differential Revision: https://secure.phabricator.com/D14148
2015-09-23 13:54:27 -07:00
epriestley
fcb6d1e2fa Strip some obsolete code out of Drydock
Summary:
Ref T9252. This simplifies some Drydock code.

Most of this code relates to the old notion of Drydock being able to enumerate all the tasks it needs to complete in order to acquire a lease. The code has stepped back from this, since it's unnecessary, the queue is more powerful than it used to be, and it would be a lot of work to keep track of.

The ~only thing that should ever wait for leases in modern code is `bin/drydock lease`, and it's fine for it to just sit there sleeping, so this just does that.

This reduces the granularity of logging, but I'll address that separately in future logging-focused changes.

Test Plan: Used `bin/drydock lease` to acquire a lease, saw it acquire cleanly.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14147
2015-09-23 13:21:41 -07:00
epriestley
f1119ffcf5 Support working copies and separate allocate + activate steps for resources/leases in Drydock
Summary:
Ref T9253. For resources and leases that need to do something which takes a lot of time or requires waiting, allow them to allocate/acquire first and then activate later.

When we allocate a resource or acquire a lease, the blueprint can either activate it immediately (if all the work can happen quickly/inline) or activate it later. If the blueprint activates it later, we queue a worker to handle activating it.

Rebuild the "working copy" blueprint to work with this model: it allocates/acquires and activates in a separate step, once it is able to acquire a host.

Test Plan: With some power of imagination, brought up a bunch of working copies with `bin/drydock lease --type working-copy ...`

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14127
2015-09-21 04:46:24 -07:00
epriestley
635e9c6075 Provide a generic "Datasource" StandardCustomField
Summary:
Ref T9253. See discussion in D13843.

I want to let Drydock blueprints for Almanac services choose those services from a typeahead, but only list appropriate services in the typeahead. To do this:

  - Provide a StandardCustomField for an arbitrary datasource.
  - Adjust the AlmanacServiceDatasource to allow filtering by service class.

This implementation is substantially the same as the one in D13843, with some adjustments:

  - I lifted most of the code in the `Users` standard custom field into a new `Tokenizer` standard custom field.
  - The `Users` and `Datasource` custom fields now extend the `Tokenizer` custom field and can share most of the code it uses.
  - I exposed this field fully as a configurable field. I don't think anyone will ever use it, but this generality costs us nearly nothing and improves consistency.
  - The code in D13843 didn't actually pass the parameters over the wire, since the object that responds to the request is not the same object that renders the field. Use the "parameters" mechanism in datasources to get things passed over the wire.

Test Plan:
  - Created a custom "users" field in Maniphest and made sure it still wokred.
  - Created a custom "almanc services" field in Maniphest and selected some services for a task.
  - With additional changes from D13843, selected an appropriate Almanac service in a new Drydock blueprint.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14111
2015-09-21 04:41:52 -07:00
epriestley
6bd8ee861c Use PEAR Text_Figlet to render figlet fonts
Summary:
Ref T7785. Makes Figlet available without installing the `figlet` package.

The PEAR Text_Figlet code is really sketchy and includes this API, which is quite marvelous:

```
    function loadFont($filename, $loadgerman = true)
```

At some point, this should probably be rewritten into a modern style, but it's not trivial since the figlet file format and rendering engine are somewhat complicated. I made some adjustments:

  - Broke the dependency on the PEAR core.
  - Prevented it from doing any wrong HTML escaping.
  - Looked through it for any glaring security or correctness problems.

This code isn't very pretty or modern, but as far as I can tell it's safe and does render Figlet fonts in a reasonable way.

Test Plan: {F803268}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9408, T7785

Differential Revision: https://secure.phabricator.com/D14102
2015-09-13 12:31:07 -07:00
epriestley
c705c8011e Use PHP implementation of Cowsay for cowsay rule
Summary:
Ref T7785. Convert the Cowsay Remarkup rule to use a PHP implementation so we don't have to execute an external `cowsay` binary.

I removed some of the default ".cow" files that come with Cowsay because they:

  - include Perl code which we can not interpret; or
  - are primarily in-jokes or standalone visual puns or artwork rather than usable actors on the grand stage of cowsay; or
  - offended my delicate sensibilities.

Users can add new cows to `resources/cows/custom/` if they want to make new cows available.

I have included a majestic original artwork depicting the "Companion Cube" character from //Portal//.

Test Plan: {F802535}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9408, T7785

Differential Revision: https://secure.phabricator.com/D14100
2015-09-13 12:27:30 -07:00
epriestley
c02f750267 Remove dot/Graphviz Remarkup rule
Summary: Ref T9408. This rule is unsafe in principle, and a practical vulnerability has been found by a security researcher.

Test Plan: `grep`

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9408

Differential Revision: https://secure.phabricator.com/D14103
2015-09-13 12:27:23 -07:00
epriestley
4e181a5611 Clean up browse/history links in Diffusion
Summary:
Fixes T9126. In particular:

  - Add "Browse" links to all history views.
  - Use icons to show "Browse" and "History" links, instead of text.
  - Use FontAwesome.
  - Generally standardize handling of these elements.

This might need a little design attention, but I think it's an improvement overall.

Test Plan:
  - Viewed repository history.
  - Viewed branch history.
  - Viewed file history.
  - Viewed table of contents on a commit.
  - Viewed merged changes on a merge commit.
  - Viewed a directory containing an external.
  - Viewed a deleted file.

{F788419}

{F788420}

{F788421}

{F788422}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9126

Differential Revision: https://secure.phabricator.com/D14096
2015-09-10 19:28:49 -07:00
epriestley
093a625698 Show users what's wrong when they try to edit an inline with an editor already open
Summary: Fixes T8572. Ideally we would probably just permit this, but clean up the behavior until the day arrives when inline code is actually rewritten.

Test Plan:
  - Tried to launch editors in Differential and Diffusion while comments were already open.
  - Verified that "Jump to inline" works in both cases.

{F788008}

{F788009}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8572

Differential Revision: https://secure.phabricator.com/D14094
2015-09-10 11:36:38 -07:00
epriestley
29948eaa5b Use phutil_hashes_are_identical() when comparing hashes in Phabricator
Summary: See D14025. In all cases where we compare hashes, use strict, constant-time comparisons.

Test Plan: Logged in, logged out, added TOTP, ran Conduit, terminated sessions, submitted forms, changed password. Tweaked CSRF token, got rejected.

Reviewers: chad

Reviewed By: chad

Subscribers: chenxiruanhai

Differential Revision: https://secure.phabricator.com/D14026
2015-09-01 15:52:44 -07:00
epriestley
51c52f50fd Prevent users from hiding unpublished inlines
Summary: Fixes T9135. This is (probably) never intended and can be confusing.

Test Plan: Saw no hide button on unpublished inlines. Saw hide button on published inlines. Clicked hide button.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9135

Differential Revision: https://secure.phabricator.com/D14014
2015-08-31 10:17:30 -07:00
Joshua Spence
7904d4c29c Add some missing translations
Summary: Fixes T8862.

Test Plan: Eyeballed it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T8862

Differential Revision: https://secure.phabricator.com/D14005
2015-08-29 23:29:16 +10:00
cburroughs
786b135a66 variable days back for bin/mail volume
Summary:
Email is so exciting I can't wait 30 days for initial results.

ref T9161

Test Plan:
 * `./bin/mail volume --days 60` took longer and gave plausibly larger
   results.
 * `./bin/mail volume --days 0` quickly told me no mail had been sent.
 * `./bin/mail volume` Said it was still looking 30 days back.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Maniphest Tasks: T9161

Differential Revision: https://secure.phabricator.com/D13901
2015-08-27 04:40:45 -07:00
epriestley
55b44f53f8 Fix possible recursive embeds in Dashboard text panels
Summary:
We currently detect tab panels embedding themselves, but do not detect text panels embedding themselves with `{Wxx}`.

Detect these self-embedding panels.

I had to add a bit of a hack to pass the parent panel PHIDs to the rule. Generally, I got the Markup API kind of wrong and want to update it, I'll file a followup with details about how I'd like to move forward.

Test Plan:
Created a text panel embedding itself, a tab panel embedding a text panel embedding itself, a tab panel embedding a text panel embedding the tab panel, etc.

Rendered all panels standalone and as `{Wxx}` from a different context.

{F761158}

{F761159}

{F761160}

{F761161}

{F761162}

Reviewers: chad, jbeta

Reviewed By: chad, jbeta

Differential Revision: https://secure.phabricator.com/D13999
2015-08-26 17:59:47 -07:00
epriestley
10966519e2 Prevent "commit message magic words" parser from exploding on "reverts aaaa.....aaz"
Summary:
Fixes T9268. Currently, we try to match any string like "a2f313f1" as a commit/revision, so short hashes will get picked up.

However, we don't require a word boundary or terminal after the match, so for input like "aaa...aaaaz" the engine can get stuck trying to split the string into sub-matches.

That is, in the original case, the input "aaaz" had valid matches against `[rA-Z0-9a-f]+` up to "z" of:

  aaa
  aa a
  a aa
  a a a

All of these will fail once it hits "z", but it has to try them all. This complexity is explosive with longer strings.

Instead, require a word boundary or EOL after the match, so this is the only valid match:

  aaa

Then the engine sees the "z", says "nope, no match" and doesn't have to backtrack across all possible combinations.

Test Plan: Added a failing unit test, applied patch, clean test.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9268

Differential Revision: https://secure.phabricator.com/D13997
2015-08-26 09:35:23 -07:00
epriestley
3ef270b292 Allow transaction publishers to pass binary data to workers
Summary:
Ref T8672. Ref T9187. Root issue in at least one case is:

  - User makes a commit including a file with some non-UTF8 text (say, a Japanese file full of Shift-JIS).
  - We pass the file to the TransactionEditor so it can inline or attach the patch if the server is configured for these things.
    - When inlining patches, we convert them to UTF8 before inlining. We must do this since the rest of the mail is UTF8.
    - When attaching patches, we send them in the original encoding (as file attachments). This is correct, and means we need to give the worker the raw patch in whatever encoding it was originally in: we can't just convert it to utf8 earlier, or we'd attach the wrong patch in some cases.
  - TransactionEditor does its thing (e.g., creates the commit), then gets ready to send mail about whatever it did.
  - The publishing work now happens in the daemon queue, so we prepare to queue a PublishWorker and pass it the patch (with some other data).
  - When we queue workers, we serialize the state data with JSON.

So far, so good. But this is where things go wrong:

  - JSON can't encode binary data, and can't encode Shift-JIS. The encoding silently fails and we ignore it.

Then we get to the worker, and things go wrong-er:

  - Since the data is bad, we fatal. This isn't a permanent failure, so we continue retrying the task indefinitely.

This applies several fixes:

  # When queueing tasks, fail loudly when JSON encoding fails.
  # In the worker, fail permanently when data can't be decoded.
  # Allow Editors to specify that some of their data is binary and needs special handling.

This is fairly messy, but some simpler alternatives don't seem like good ways forward:

  - We can't convert to UTF8 earlier, because we need the original raw patch when adding it as an attachment.
  - We could encode //only// this field, but I suspect some other fields will also need attention, so that adding a mechanism will be worthwhile. In particular, I suspect filenames //may// be causing a similar problem in some cases.
  - We could convert task data to always use a serialize()-based binary safe encoding, but this is a larger change and I think it's correct that things are UTF8 by default, even if it makes a bit of a mess. I'd rather have an explicit mess like this than a lot of binary data floating around.

The change to make `LiskDAO` will almost certainly catch some other problems too, so I'm going to hold this until after `stable` is cut. These problems were existing problems (i.e., the code was previously breaking or destroying data) so it's definitely correct to catch them, but this will make the problems much more obvious/urgent than they previously were.

Test Plan:
  - Created a commit with a bunch of Shift-JIS stuff in a file.
  - Tried to import it.

Prior to patch:

  - Broken PublishWorker with distant, irrelevant error message.

With patch partially applied (only new error checking):

  - Explicit, local error message about bad key in serialized data.

With patch fully applied:

  - Import went fine and mail generated.

Reviewers: chad

Reviewed By: chad

Subscribers: devurandom, nevogd

Maniphest Tasks: T8672, T9187

Differential Revision: https://secure.phabricator.com/D13939
2015-08-22 15:14:05 -07:00
epriestley
9790f93a83 Show all packages with a claim on a file in the table of contents view
Summary:
Ref T9218. See discussion there for rationale; I think this is the right behavior to pursue.

The screenshot below is pretty ugly. I think it's a lot worse than most real-world cases will be, since you have to sort of opt-in to having crazy levels of overlapping packages, and it's perfectly normal/reasonable for files owned by one package. Owners is powerful enough to let you specify sub-packages with exclusive ownership.

That said, this may be more typical than I hope. I don't think we can reduce the complexity here much for free, but it would might be reasonable to add some view options (e.g.: group by package?, show only packages I own?, show packages as icons with a tooltip?) if it's an issue.

Test Plan: {F734956}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9218

Differential Revision: https://secure.phabricator.com/D13940
2015-08-20 04:46:17 -07:00
epriestley
6010e80e5c Show packages in table of contents views in Diffusion and Differential
Summary:
Fixes T8004.

  - For paths which are part of a package, show the package.
  - Highlight paths which are part of a package you (the viewer) have authority over.

Test Plan:
{F725418}

  - Viewed owned and unowned chagnes in Diffusion and Differential.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8004

Differential Revision: https://secure.phabricator.com/D13923
2015-08-17 10:14:22 -07:00
epriestley
cfd1348975 Fix an issue with Table of Contents construction for copied files
Summary: Ref T9201.

Test Plan: Inspection.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9201

Differential Revision: https://secure.phabricator.com/D13918
2015-08-16 17:59:02 -07:00
epriestley
8149399312 Convert commits to use unified table of contents
Summary:
Fixes T2183. We now use the same rendering element in both places.

Intentional changes:

  - Package highlighting is out, coming back to both apps in next diff.
  - removed redundant-feeling "Change" link. The information is now shown with a character ("M", "V", etc.) and the page is a click away under "History". Clicking the path also jumps you to substantially similar content. (We could restore it fairly easily, I just think it's probably the least useful thing in the table right now.)

Test Plan: Viewed a bunch of commits in Diffusion.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2183

Differential Revision: https://secure.phabricator.com/D13910
2015-08-15 12:57:20 -07:00
epriestley
cb912d1735 Convert DifferentialRevision view to new PHUIDiffTableOfContentsListView
Summary:
Ref T2183. Introduces a new View which can (in theory) unify the Revision, Diff and Commit table of contents views.

This has the same behavior as before, but accepts slightly more general primitives and parameters and has somewhat cleaner code.

I've made one intentinoal behavior change: removing the "Open All in Editor" button. I suspect this is essentially unused, and is a pain to keep around. We can look at restoring it if anyone notices.

Test Plan: Looked at a bunch of revisions, no changes from before.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T2183

Differential Revision: https://secure.phabricator.com/D13908
2015-08-15 11:43:12 -07:00
Chad Little
fca716d699 More Ponder Answer polish
Summary: Fixes T9099, I think this is as much as I can come up with for unbeta. Cleans up the answer header (profile image, smaller font, smaller header). Cleans up voting (new, with color), and makes it a bit more readable.

Test Plan:
Review a number of answers in ponder with and without votes, comments.

{F720189}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9099

Differential Revision: https://secure.phabricator.com/D13907
2015-08-15 10:55:38 -07:00
Joshua Spence
368f359114 Use PhutilClassMapQuery instead of PhutilSymbolLoader
Summary: Use `PhutilClassMaQuery` instead of `PhutilSymbolLoader`, mostly for consistency. Depends on D13588.

Test Plan: Poked around a bunch of pages.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13589
2015-08-14 07:49:01 +10:00
Joshua Spence
7938d9a529 Add some missing translation strings
Summary: Self explanatory.

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D13880
2015-08-14 07:40:09 +10:00
Joshua Spence
2cf9ded878 Various linter fixes
Summary: Self explanatory.

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D13863
2015-08-11 22:36:55 +10:00
epriestley
a1431e53cc Link to Harbormaster build targets from the Daemon worker page
Summary:
Fixes T7370. Two changes:

  - Make the default to show nothing, instead of showing all the data. This is a better default because the data is sometimes sensitive. Workers should have to opt in to revealing it.
  - For TargetWorkers, link to the target (technically the build, for now, since there's no dedicated target detail page).

Test Plan: {F698325}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7370

Differential Revision: https://secure.phabricator.com/D13845
2015-08-10 14:15:19 -07:00
Joshua Spence
79f2e81f38 Various linter fixes
Summary: Self-explanatory.

Test Plan: Eyeball it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D13808
2015-08-08 10:19:45 +10:00
Chad Little
fdc1662bfd Add Projects to Ponder Search View
Summary: Ref T3578 Allows display of projects if available on individual ponder search results.

Test Plan: Added projects to my questions, see them display on search results.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T3578

Differential Revision: https://secure.phabricator.com/D13793
2015-08-05 09:31:37 -07:00
epriestley
5f76c71d78 Share target filtering code in HeraldAction
Ref T8726. This shares some target filtering code with the base class.
2015-08-03 14:34:37 -07:00
epriestley
6f6d88794b Modularize the Diffusion "Add Auditors" Herald action
Ref T8726.
2015-08-03 14:33:27 -07:00
epriestley
fdd379a026 Modularize the Legalpad "Require Signature" Herald Action
Ref T8726. Modularizes "Require Signatures" for Legalpad.
2015-08-03 14:33:26 -07:00
epriestley
a335004a91 Modularize Differential Reviewer actions in Herald
Ref T8726. Modularizes the "Add Reviewers" and "Add Blocking Reviewers" Herald actions.
2015-08-03 14:33:25 -07:00
epriestley
3782992670 Modularize "add projects" and "remove projects" Herald actions
Summary: Ref T8726. Convert these to be modular.

Test Plan:
  - Created rules using these actions.
  - Upgraded them.
  - Verified they still work.

{F659266}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13705
2015-08-03 14:33:24 -07:00
epriestley
51fead17cf Modularize "Send an Email" Herald actions
Summary: Ref T8726. No surprises.

Test Plan:
Created rules using both action variants, applied upgrade, saw rules still work correctly.

{F658842}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13701
2015-08-03 14:33:23 -07:00
epriestley
8ae08a3de7 Make "Add Subscribers" and "Remove Subscribers" Herald actions modular
Summary: Ref T8726. Converts these actions to be modular. No real surprises in this change.

Test Plan:
{F658709}

  - Wrote some rules.
  - Migrated them forward.
  - Used a bunch of these rules.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13699
2015-08-03 14:33:22 -07:00
lkassianik
930b6fec25 DRAFT - throw together Phurl skeleton.
Summary: DRAFT - throw together Phurl skeleton.

Test Plan: The idea is that `some/long/url` will become `install/Udet4d` and can be viewed and edited at `install/Udet4d/view` and `install/Udet4d/edit`, respectively?

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: joshuaspence, chad, epriestley, Korvin

Maniphest Tasks: T6049

Differential Revision: https://secure.phabricator.com/D13681
2015-07-24 11:06:27 -07:00
Chad Little
ffadf64751 Badges v0.1
Summary:
Basic plumbing for Badges application.

 - You can make Badges.
 - You can look at a list of them.
 - They can be edited.
 - They can be assigned to people.
 - You can revoke them from people.
 - You can subscribe to them.

Test Plan: Make Badges with various options. Give them to people. Take them away from people.

Reviewers: lpriestley, epriestley

Reviewed By: epriestley

Subscribers: tycho.tatitscheff, johnny-bit, epriestley, Korvin

Maniphest Tasks: T6526

Differential Revision: https://secure.phabricator.com/D13626
2015-07-22 13:37:20 -07:00
epriestley
f1222f956a Correctly clear draft markers when deleting an inline comment
Summary:
Fixes T8917. Prior to T2618, deleting inlines prompted users, then really deleted the rows.

After T2618, we delete immediately and offer "Undo". However, some interactions with drafts were missed, and we were only clearing the "this revision has a draft" flag on one of the delete pathways (when you delete all the comment text, then save the comment).

Make both the "Delete" action and the "Delete All Comment Text + Save" workflows do the same thing: mark the row as deleted, and clear any relevant drafts.

Test Plan:
  - Made an inline comment on a clean revision with no "draft comments" marker in the list view.
  - Used "delete" to delete it.
  - After applying the patch, verified that no "draft commetns" marker appears in the list view.
  - Used Delete and Edit + Remove Text + Save to delete comments in Differential and Diffusion.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T8917

Differential Revision: https://secure.phabricator.com/D13665
2015-07-21 11:36:46 -07:00
Chad Little
039be71b87 Fix a phutil_json_decode call in CustomField PHID
Summary: I assume this is correct, it fixes my test case and was inconsistent with the rest of the file.

Test Plan: Add and Remove a CustomFieldPHID in Maniphest, no longer get errors.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8866

Differential Revision: https://secure.phabricator.com/D13642
2015-07-16 21:02:01 -07:00
epriestley
0306eb70ed Group and order Herald fields
Summary:
Ref T8726. Some adapters now have a large number of fields, and we lost the sort-of-human-readable implicit ordering when fields were modularized.

Instead, group and sort fields.

Test Plan: {F603066}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13619
2015-07-16 14:13:13 -07:00
epriestley
97ccd93670 Support "Select" custom fields in Herald rules
Summary: Fixes T5016. Ref T655. Ref T8434. Ref T8726. For "select" custom fields, permit construction of Herald rules.

Test Plan: {F602997}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T655, T5016, T8434, T8726

Differential Revision: https://secure.phabricator.com/D13618
2015-07-16 14:12:54 -07:00
epriestley
715233fb61 Fully modularize Herald field values
Summary: Ref T8726. This gets rid of all the `VALUE_*` constants and lets Fields provide arbitrary typeaheads without upstream/JS changes.

Test Plan: Used all tokenizers. Used "Another Herald Rule". Grepped for all removed constants.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13616
2015-07-16 14:12:44 -07:00
epriestley
9979952e71 Refine "invalid cursor" exception to have a little more information
Summary: Ref T8780.

Test Plan: See @chad.

Reviewers: chad

Reviewed By: chad

Subscribers: chad, epriestley

Maniphest Tasks: T8780

Differential Revision: https://secure.phabricator.com/D13583
2015-07-07 12:52:12 -07:00
Joshua Spence
f695dcea9e Use PhutilClassMapQuery
Summary: Use `PhutilClassMapQuery` where appropriate.

Test Plan: Browsed around the UI to verify things seemed somewhat working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13429
2015-07-07 22:51:57 +10:00
epriestley
4c23d5cfae Make Herald custom field integration modular
Summary: Ref T8726. The existing implementation is largely made redundant by the newer modular implementation.

Test Plan: Created a custom field rule, set custom field to various values until it matched, saw effect and proper transcripts.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13497
2015-07-06 13:15:58 -07:00
Joshua Spence
f2435fd1d0 Return $this from setter methods
Summary: Return `$this` from setter methods for consistency. I started writing a [[https://secure.phabricator.com/differential/diff/32506/ | linter rule]] to detect this, but I don't think it is trivial to do this properly.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13422
2015-07-06 22:53:43 +10:00
Joshua Spence
acb1eb81cc Move some PhabricatorSearchField subclasses
Summary: Move some `PhabricatorSearchField` subclasses to be adjacent to the application to which they belong. This seems generally better to me than lumping them all together in the `src/applications/search/field/` directory. I was also wondering if it makes sense to rename these subclasses as `PhabricatorXSearchField` rather than `PhabricatorSearchXField` (as per T5655), but wasn't really sure if these objects are meant to be search-fields, or just fields belonging to the #search application.

Test Plan: N/A.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13374
2015-07-06 22:52:05 +10:00
Chad Little
54194acd39 [Redesign] Set a max-width on Graphviz
Summary: Ref T8750, Adds a maxwidth class for Graphviz images.

Test Plan: Generate a Graphviz image, really big, see it scale to the viewport.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8750

Differential Revision: https://secure.phabricator.com/D13548
2015-07-04 13:17:12 -07:00
epriestley
fe4bcde59e Merge branch 'master' into redesign-2015 2015-07-03 13:05:16 -07:00
epriestley
bcfbc5cfbf Remove CHECKREQUEST event
Summary: Fixes T8749.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8749

Differential Revision: https://secure.phabricator.com/D13546
2015-07-03 13:03:49 -07:00
epriestley
92b73fed6b Don't apply space constraints to omnipotent-viewer queries
Summary:
Fixes T8743. Fixes T8746. When running queries with the omnipotent viewer and no explicit space constraints, don't add implicit space constraints.

This prevents us from fataling when running older pre-space migrations and trying to load space-aware objects.

Test Plan: Manually ran migrations with `--trace`, verified no `WHERE spacePHID = ...`.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: eadler, epriestley

Maniphest Tasks: T8743, T8746

Differential Revision: https://secure.phabricator.com/D13542
2015-07-03 10:59:48 -07:00
epriestley
f7edabbc6a Merge branch 'master' into redesign-2015 2015-07-01 14:23:01 -07:00
epriestley
d5e4d96086 Don't put objects into the query workspace by default
Summary:
Ref T8731. I think the issue is that some `ProjectQuery` (without needImages()) populates the query workspace, then the "real" one hits the workspace.

Instead, only populate the workspace from ObjectQuery, so we know that objects in the workspace always have whatever ObjectQuery attaches to them.

Test Plan: Verified this didn't destroy the cache hitrate, but I can't repro the original issue locally per se.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: chad, epriestley

Maniphest Tasks: T8731

Differential Revision: https://secure.phabricator.com/D13516
2015-07-01 14:22:48 -07:00
Chad Little
3b755115c2 [Redesgin] Misc Inline Comment tweaks
Summary: Ref T8099. Better alignment with TODO and Ghost Icons.

Test Plan: Test more states of inline comments, ghosts, todos, unsubmitted, etc.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8099

Differential Revision: https://secure.phabricator.com/D13515
2015-07-01 11:32:32 -07:00
epriestley
4b298c1c44 Merge branch 'master' into redesign-2015 2015-06-30 11:20:16 -07:00
epriestley
4adaf53bf0 Dramatically increase cache hit rate for feed
Summary:
Ref T8631. The query plan for feed stories is really bad right now, because we miss caches we should be hitting:

  - The workspace cache is stored at each query, so adjacent queries can't benefit from the cache (only subqueries). Feed has primarily sibling queries.
    - There is no technical reason to do this. Store the workspace cache on the root query, so sibling queries can hit it.
  - In `ObjectQuery`, we check the workspace once, then load all the PHIDs. When the PHIDs are a mixture of transactions and objects, we always miss the workspace and load the objects twice.
    - Instead, check the workspace after loading each type of object.
  - `HandleQuery` does not set itself as the parent query for `ObjectQuery`, so handles never hit the workspace cache.
    - Pass it, so they can hit the workspace cache.
  - Feed's weird `PhabricatorFeedStory::loadAllFromRows()` method does not specify a parent query on its object/handle queries.
    - Just declare the object query to be the "root" query until this eventually gets cleaned up.

Test Plan: Saw queries for each object drop from 4-6x to 1x in `/feed/`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8631

Differential Revision: https://secure.phabricator.com/D13479
2015-06-30 11:19:41 -07:00
epriestley
12c6e63d5b Don't put the entire corpus into project edit feed stories
Summary: Fixes T8723. We override timeline titles for these fields to shorten them, but the parent class shows full values for feed edits, which can lead to screen-sized notifications.

Test Plan: Edited a project, saw a reasonably-sized notification for it.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T8723

Differential Revision: https://secure.phabricator.com/D13485
2015-06-30 06:05:30 -07:00
Chad Little
6fef37ddb5 [Redesign] Update Inline Comment UI
Summary: Ref T8099, Simplifies the button bar with a `borderless` option and implements in Differential Inline Commenting.

Test Plan:
Review new and old comments, submitted, unsubmitted, ghosts, done.

{F562765}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8099

Differential Revision: https://secure.phabricator.com/D13475
2015-06-29 14:42:29 -07:00
epriestley
729606ba93 Update BulkJob and MetaMTA search engines for redesign-2015 2015-06-23 13:39:27 -07:00
epriestley
075f041bec Merge branch 'master' into redesign-2015 2015-06-23 13:36:47 -07:00
epriestley
3215899925 Execute Maniphest batch edits in the background with a web UI progress bar
Summary:
Ref T8637. This does nothing interesting, just has empty scaffolding for a bulk job queue.

Basic idea is that when you do something like a batch edit in Maniphest, we:

  - Create a BulkJob with all the details.
  - Queue a worker to start the job.
  - Send you to a progress bar page for the job.

In the background:

  - The "start job" worker creates a ton of Task objects, then queues worker tasks to do the work.

In the foreground:

  - Fancy ajax animates the progress bar and it goes wooosh.

In general:

  - Big jobs actually work.
  - Jobs get logged.
  - You can monitor jobs.
  - Terrible junk like T8637 should be much harder to write and much easier to catch and diagnose.

Test Plan:
No interesting code/beahavior yet. Clean `storage adjust`.

{F526411}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8637

Differential Revision: https://secure.phabricator.com/D13392
2015-06-23 13:36:16 -07:00
epriestley
716bd4e4b4 Improve lint/unit limit, sort, view all, collapse behaviors
Summary:
Ref T8096. Various tweaks here:

  - Sort result lists by importance (even lint -- "errors first" seems better than "alphabetical by file", I think?).
  - Do sane stuff with display limits.
  - Add a "view all" view.
  - Don't show a huge table of passing tests in Differential.
  - Link to full results.

Test Plan: See screenshots.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8096

Differential Revision: https://secure.phabricator.com/D13407
2015-06-23 13:34:25 -07:00
epriestley
7d7e13d79b Merge branch 'master' into redesign-2015 2015-06-19 08:33:30 -07:00
Joshua Spence
68bb60e52f Use PhutilConsoleTable in ./bin/storage probe
Summary: Fixes T8477. Use `PhutilConsoleTable` to render the output from `./bin/storage probe`.

Test Plan: Ran `./bin/storage probe` and saw tabulated output.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T8477

Differential Revision: https://secure.phabricator.com/D13339
2015-06-19 13:09:22 +10:00
Joshua Spence
7f6508af5a Add missing execution on archived task query
Summary: Fixes T8599. I'm not sure how to reproduce the original issue, but I'm fairly confident that the issue is that the issue is that `execute()` is not called on the query object.

Test Plan: Created a Harbormaster build plan with a single "Lease Host" step. Ran `./bin/harbormaster build --plan 1 D1` from the command line and hit the exception as described in T8599. Applied patch and hit a different exception (which I think is just because I don't know how to use #drydock and #harbormaster).

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, epriestley, Korvin

Maniphest Tasks: T8599

Differential Revision: https://secure.phabricator.com/D13335
2015-06-18 22:40:05 +10:00
Joshua Spence
4761cb8d73 Use PhutilInvalidStateException
Summary: Use `PhutilInvalidStateException` where appropriate.

Test Plan: Eyeball it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13327
2015-06-18 07:09:02 +10:00
epriestley
f1b7fd483e Merge branch 'master' into redesign-2015 2015-06-16 19:34:36 -07:00
epriestley
6933ee5fb8 Don't spend ridiculous amounts of time rebuilding orderable columns
Summary:
Ref T8575. Because orderable columns need to build custom fields, they are relatively expensive to build. Use the request cache.

(The request cache is technically more correct than `static`, because configuration may change between requests and we may eventually reuse interpreters.)

Test Plan: Saw home page time drop 39% (from 462ms to 283ms).

Reviewers: btrahan, avivey

Reviewed By: avivey

Subscribers: avivey, epriestley

Maniphest Tasks: T8575

Differential Revision: https://secure.phabricator.com/D13322
2015-06-16 19:32:58 -07:00
epriestley
8484adcffd Cache application visibility in the request cache
Summary: Ref T8575. We check if users can see applications frequently, and caching on the Query isn't especially effective. Use the new Request cache instead.

Test Plan:
  - Saw `/feed/` drop 7% (from ~830ms to ~770ms) on profiles.

Reviewers: btrahan, avivey

Reviewed By: avivey

Subscribers: avivey, epriestley

Maniphest Tasks: T8575

Differential Revision: https://secure.phabricator.com/D13321
2015-06-16 19:32:37 -07:00
Joshua Spence
6b7d7401ca Modernize Diviner
Summary:
Ref T4558. This diff modernizes the #diviner application. Basically:

  - Add an edit controller, accessible at `/book/$BOOK/edit/`.
  - Add edit/view policies.
  - Added an action menu to the `DivinerBookController` to expose the edit interface.
  - Allows projects to be associated with books.
  - Implement edges and transactions.
  - Implemented `PhabricatorApplicationTransactionInterface` in `DivinerLiveBook`.

Test Plan:
  - Generated a Diviner book with `./bin/diviner generate`.
  - Added projects to a book and ensured that they persisted.
  - Changed the view policy on a book and made sure it was effective.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T4558

Differential Revision: https://secure.phabricator.com/D13091
2015-06-17 07:17:14 +10:00
epriestley
6af1c02f06 Merge branch 'master' into redesign-2015 2015-06-15 14:06:59 -07:00
epriestley
c2a2933848 Add missing property on PhabricatorBot
Auditors: joshuaspence
2015-06-15 13:53:18 -07:00
epriestley
53ef057b1b Merge branch 'master' into redesign-2015 2015-06-15 08:06:23 -07:00
epriestley
5878500e1f Fix missing property on PhabricatorYoutubeRemarkupRule
Fixes T8545.

Auditors: joshuaspence
2015-06-15 06:30:10 -07:00
Joshua Spence
b95fc8ba5d Fix another undeclared property
This was reported in IRC, see https://secure.phabricator.com/chatlog/channel/6/?at=204524.

Auditors: epriestley
2015-06-15 18:41:09 +10:00
Joshua Spence
1239cfdeaf Add a bunch of tests for subclass implementations
Summary: Add a bunch of tests to ensure that subclasses behave.

Test Plan: `arc unit`

Reviewers: eadler, #blessed_reviewers, epriestley

Reviewed By: eadler, #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13272
2015-06-15 18:13:27 +10:00
Joshua Spence
b6d745b666 Extend from Phobject
Summary: All classes should extend from some other class. See D13275 for some explanation.

Test Plan: `arc unit`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13283
2015-06-15 18:02:27 +10:00
epriestley
2fbc65e396 Call didRejectResult() in DiffusionCommitQuery properly
Summary:
Ref T4345. This error is per object-type in the query implementations, not a mail/permissions issue.

Without `didRejectResult()`, we can't distinguish between "restricted" and "unknown" for objects filtered by `willFilterPage()`.

  - Call `didRejectResult()` on commits.
  - Make `didRejectResult()` handle both existing policy exceptions and filtering.
  - Recover from partial objects (like commits) which are missing attached data required to figure out policies.

Test Plan: Saw "Restricted Diffusion Commit" instead of "Unknown Object (Diffusion Commit)" when viewing nonvisible commit handle in Maniphest.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4345

Differential Revision: https://secure.phabricator.com/D13289
2015-06-14 15:35:32 -07:00
Joshua Spence
32f4c962fc Remove "@stable" annotations
Summary: See D13285.

Test Plan: N/A

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13287
2015-06-15 07:55:08 +10:00
Joshua Spence
57e7c7c706 Tidy up some translation strings
Summary: Parameterize command names in translation strings.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley, eadler

Reviewed By: #blessed_reviewers, epriestley, eadler

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13270
2015-06-14 14:13:16 +10:00
Joshua Spence
ea7397f7e4 Rename PassphraseCredentialType subclasses for consistency
Summary: Ref T5655.

Test Plan: `arc unit` + `grep`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D13266
2015-06-14 14:11:55 +10:00
epriestley
88e7cd158f Allow Spaces to be archived
Summary:
Ref T8377. This adds a standard disable/enable feature to Spaces, with a couple of twists:

  - You can't create new stuff in an archived space, and you can't move stuff into an archived space.
  - We don't show results from an archived space by default in ApplicationSearch queries. You can still find these objects if you explicitly search for "Spaces: <the archived space>".

So this is a "put it in a box in the attic" sort of operation, but that seems fairly nice/reasonable.

Test Plan:
  - Archived and activated spaces.
  - Used ApplicationSearch, which omitted archived objects by default but allowed searches for them, specifically, to succeed.
  - Tried to create objects into an archived space (this is not allowed).
  - Edited objects in an archived space (this is OK).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8377

Differential Revision: https://secure.phabricator.com/D13238
2015-06-11 10:13:47 -07:00
epriestley
57b898af9a Merge branch 'master' into redesign-2015 2015-06-10 07:44:58 -07:00
Joshua Spence
f47e69c015 Mark some strings for translation
Summary: Add some more `pht`izations.

Test Plan: Eyeball it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13200
2015-06-09 23:06:52 +10:00
epriestley
1a091e5260 Drive Maniphest grouping and ordering through standard infrastructure
Summary: Ref T8441. Ref T7715. Ref T7909. Clean up all the ordering and grouping hacks in Maniphest so we can drive it through normal infrastructure, move it to SearchField, introduce Spaces, and eventually modernize the Conduit API.

Test Plan:
  - Executed all grouping/ordering queries, including custom queries.
  - Forced execution with old aliases; got modern results.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7909, T7715, T8441

Differential Revision: https://secure.phabricator.com/D13197
2015-06-08 12:23:13 -07:00
epriestley
bf87976d25 Support ordering in SearchField
Summary:
Ref T8441. Ref T7715. Automatically generate a modern "Order" control in ApplicationSearch for engines which fully support SearchField.

Notably, this allows the standard "Order" control to automatically support custom field orders. We do this in Maniphest today, but in an ad-hoc way.

Test Plan: Performed order-by queries in Almanac (Services), Pholio, Files, People, Projects, and Paste.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13193
2015-06-08 12:21:48 -07:00
epriestley
3cdaf52ce9 Make Subscribers automatically provide working SearchFields
Summary:
Ref T8441. Ref T7715. For modern Query classes, automatically make subscriber queries and SearchField integrations work.

In particular, we can just drive this query with EdgeLogic and don't need to do anything specific on these Query classes beyond making sure they're implemented in a way that picks up all of the EdgeLogic clauses.

Test Plan:
  - Searched for subscribers in Pholio, Files, Paste, and Projects.
  - Searched for all other fields in Projects to check that Query changes are OK.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13191
2015-06-08 12:20:53 -07:00
epriestley
ecc4c531c9 Stop threading handles through the CustomField + ApplicationSearch pipeline
Summary:
Ref T8441. Ref T7715.

  - These are obsolete after the Viewer/HandlePool changes.
  - These are unused after the typeahead parameterization changes.

Test Plan: `grep`, poked around.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13176
2015-06-07 07:32:46 -07:00
epriestley
2492fef029 Move Pholio to SearchFields
Summary:
Ref T8441. Ref T7715.

  - Update PholioSearchEngine.
  - Automatically add project fields.
  - Update Paste to support project search.
  - Simplify common Query class construction.

Test Plan:
  - Searched for pastes.
  - Searched for mocks.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13174
2015-06-07 07:31:28 -07:00
Joshua Spence
64b690f23f Add some missing translations
Summary: I saw these appear untranslated in the feed on https://secure.phabricator.com.

Test Plan: Created a dependency and checked the story text in `/feed/`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13149
2015-06-07 16:59:59 +10:00
epriestley
c1c897b961 Provide core policy support for Spaces
Summary:
Ref T8424. No UI or interesting behavior yet, but integrates Spaces checks:

  - `PolicyFilter` now checks Spaces.
  - `PolicyAwareQuery` now automatically adds Spaces constraints.

There's one interesting design decision here: **spaces are stronger than automatic capabilities**. That means that you can't see a task in a space you don't have permission to access, //even if you are the owner//.

I //think// this is desirable. Particularly, we need to do this in order to exclude objects at the query level, which potentially makes policy filtering for spaces hugely more efficient. I also like Spaces being very strong, conceptually.

It's possible that we might want to change this; this would reduce our access to optimizations but might be a little friendlier or make more sense to users later on.

For now, at least, I'm pursuing the more aggressive line. If we stick with this, we probably need to make some additional UI affordances (e.g., show when an owner can't see a task).

This also means that you get a hard 404 instead of a policy exception when you try to access something in a space you can't see. I'd slightly prefer to show you a policy exception instead, but think this is generally a reasonable tradeoff to get the high-performance filtering at the Query layer.

Test Plan:
  - Added and executed unit tests.
  - Put objects in spaces and viewed them with multiple users.
  - Made the default space visible/invisible, viewed objects.
  - Checked the services panel and saw `spacePHID` constraints.
  - Verified that this adds only one query to each page.

Reviewers: btrahan, chad

Reviewed By: btrahan

Subscribers: chad, epriestley

Maniphest Tasks: T8424

Differential Revision: https://secure.phabricator.com/D13156
2015-06-04 17:46:32 -07:00
epriestley
763b63a0fb Add spacePHID infrastructure and implement in Paste
Summary:
Ref T8424. I'm using Paste as a testbed application because Spaces make some degree of sense for it but it's also flat/simple.

This doesn't do anything interesting or useful and mostly just making the next (more interesting) diff smaller.

Test Plan:
  - Ran `bin/storage upgrade -f`.
  - Browsed pastes.
  - Created a paste.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8424

Differential Revision: https://secure.phabricator.com/D13154
2015-06-04 17:45:24 -07:00
epriestley
52a29be70d Introduce a request cache mechanism
Summary:
Ref T8424. This adds a standard KeyValueCache to serve as a request cache.

In particular, I need to cache Spaces (they are frequently accessed, sometimes by multiple viewers) but not have them survive longer than the scope of one request.

This request cache is explicitly destroyed by each web request and each daemon request.

In the very long term, building this kind of construct supports reusing PHP interpreters to run web requests (see some discussion in T2312).

Test Plan:
  - Added and executed unit tests.
  - Ran every daemon.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8424

Differential Revision: https://secure.phabricator.com/D13153
2015-06-04 17:27:31 -07:00
epriestley
069e60d2ff Send mail to targets in the user's translation
Summary: Ref T6367.

Test Plan:
  - Added and executed unit tests.
  - Sent mail to A (en_US) and B (en_A*).
  - Got one mail in English and one mail in ENGLISH.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6367

Differential Revision: https://secure.phabricator.com/D13142
2015-06-03 18:59:33 -07:00
Chad Little
e4788df9a5 [Redesign] PHUIPagerView
Summary: Ref T8099, Moves AphrontPagerView to PHUIPagerView, converts to standard PHUIButtons and adds some additional features for icon placement on buttons.

Test Plan: Tested Advanced Search and Searching files in Diffusion. Works as expected.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8342, T8099

Differential Revision: https://secure.phabricator.com/D13092
2015-06-02 14:34:04 -07:00
Joshua Spence
bf81fda036 Linter fixes
Summary: Apply various minor linter fixes.

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13109
2015-06-02 22:14:01 +10:00
epriestley
541b4c86b4 Add "Spaces", an application for managing policy namespaces
Summary: Ref T3820. This doesn't actually do anything yet, but dumps in all the plumbing.

Test Plan:
{F156989}

{F156990}

{F156991}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: eadler, wienczny, jdloft, devurandom, thz, hwinkel, 20after4, sascha-egerer, seporaitis, joshuaspence, chad, epriestley

Maniphest Tasks: T3820

Differential Revision: https://secure.phabricator.com/D9204
2015-06-01 11:28:38 -07:00
Chad Little
9708f68e58 Increase CSS specificity on Inline Comment Summary
Summary: Fixes T8354. Adds a class to each `th` and `td` for specificity.

Test Plan: Write a large table as an inline comment, see it wraps property on smaller screens.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8354

Differential Revision: https://secure.phabricator.com/D13076
2015-05-31 08:37:18 -07:00
Chad Little
13f9f209df [Redesign] Remove remaining barColor callsites
Summary: Ref T8099, Fixes T8341. Switches all remaining callsites to setBarColor to use setStatusIcon (sans workboards).

Test Plan: Test each of the applications I changed as I could (not Releeph).

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T8341, T8099

Differential Revision: https://secure.phabricator.com/D13059
2015-05-28 15:17:34 -07:00
epriestley
da9a61fb70 Use ApplicationTransactions for all non-path edits to Owners packages
Summary: Ref T8320.

Test Plan: {F437431}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8320

Differential Revision: https://secure.phabricator.com/D13028
2015-05-27 10:30:08 -07:00
epriestley
e9f4a84a89 Allow inline comments to be individually hidden
Summary:
Ref T7447. Implements per-viewer comment hiding. Once a comment is obsolete or uninteresting, you can hide it completely.

This is sticky per-user.

My hope is that this will strike a better balance between concerns than some of the other approaches (conservative porting, summarization, hide-all).

Specifically, this adds a new action here:

{F435621}

Clicking it completely collapses the comment into a small icon on the previous line, and saves the comment state as hidden for you:

{F435626}

You can click the icon to reveal all hidden comments below the line.

Test Plan:
  - Hid comments.
  - Showed comments.
  - Created, edited, deleted and submitted comments.
  - Used Diffusion comments (hiding is not implemented there yet, but I'd plan to bring it there eventually if it works out in Differential).

Reviewers: btrahan, chad

Reviewed By: btrahan

Subscribers: jparise, yelirekim, epriestley

Maniphest Tasks: T7447

Differential Revision: https://secure.phabricator.com/D13009
2015-05-27 10:28:38 -07:00
lkassianik
e4c9914697 Multi-day events should be correctly calculated for new js layout
Summary: Fixes T8304, Multi-day events should be correctly calculated for new js layout

Test Plan: A day with events starting before midnight of the displayed day should be correctly placed and fitted for day view.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T8304

Differential Revision: https://secure.phabricator.com/D13008
2015-05-25 11:19:57 -07:00
epriestley
ea131bb2ef Improve UX for Diffusion mail fields
Summary:
  - Give the fields names and descriptions.
  - When new, default-disabled fields are added, disable them by default even if there's already a config.
  - Be a bit less hacky about `$faux_spec`.

Test Plan: {F432383}

Reviewers: joshuaspence, fabe

Reviewed By: fabe

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D13006
2015-05-25 07:18:48 -07:00
epriestley
3d2c6f1ed2 Make "Tags" and "Branches" custom fields work properly in Diffusion
Summary: Ref T8295 (vaguely related). This old method has no callsites.

Test Plan: Generated mail, saw "Tags" and "Branches".

Reviewers: fabe, joshuaspence

Reviewed By: fabe, joshuaspence

Subscribers: epriestley

Maniphest Tasks: T8295

Differential Revision: https://secure.phabricator.com/D13000
2015-05-25 05:34:41 -07:00
epriestley
969a6a2b38 Fix custom field selector control in cases where the user doesn't edit anything
Summary: The JS and PHP representations of state can differ; just have the JS write the state out immediately on page load.

Test Plan: Saved `diffusion.fields` without making changes, reloaded, saw no effective change.

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D12998
2015-05-25 05:34:23 -07:00
Joshua Spence
36e2d02d6e phtize all the things
Summary: `pht`ize a whole bunch of strings in rP.

Test Plan: Intense eyeballing.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12797
2015-05-22 21:16:39 +10:00
epriestley
985eb26c7e Increase severity of bin/remove destroy warning
Summary:
Make sure we're 100% clear that this is really, truly not recommended.

Also improve the text itself, and show the objects which are being destroyed more clearly.

Test Plan: Removed objects with `bin/remove destroy`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D12952
2015-05-20 14:20:53 -07:00