Summary:
Ref T2625. Ref T3794. Ref T418. Ref T1703.
This is a more general version of D5278. It expands CustomField support to include real integration with ApplicationSearch.
Broadly, custom fields may elect to:
- build indicies when objects are updated;
- populate ApplicationSearch forms with new controls;
- read inputs entered into those controls out of the request; and
- apply constraints to search queries.
Some utility/helper stuff is provided to make this easier. This part could be cleaner, but seems reasonable for a first cut. In particular, the Query and SearchEngine must manually call all the hooks right now instead of everything happening magically. I think that's fine for the moment; they're pretty easy to get right.
Test Plan:
I added a new searchable "Company" field to People:
{F58229}
This also cleaned up the disable/reorder view a little bit:
{F58230}
As it did before, this field appears on the edit screen:
{F58231}
However, because it has `search`, it also appears on the search screen:
{F58232}
When queried, it returns the expected results:
{F58233}
And the actually good bit of all this is that the query can take advantage of indexes:
mysql> explain SELECT * FROM `user` user JOIN `user_customfieldstringindex` `appsearch_0` ON `appsearch_0`.objectPHID = user.phid AND `appsearch_0`.indexKey = 'mk3Ndy476ge6' AND `appsearch_0`.indexValue IN ('phacility') ORDER BY user.id DESC LIMIT 101;
+----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
| 1 | SIMPLE | appsearch_0 | ref | key_join,key_find | key_find | 232 | const,const | 1 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | user | eq_ref | phid | phid | 194 | phabricator2_user.appsearch_0.objectPHID | 1 | |
+----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
2 rows in set (0.00 sec)
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T418, T1703, T2625, T3794
Differential Revision: https://secure.phabricator.com/D6992
Summary:
Currently, these events don't fire for Conduit updates, which makes them sort of silly.
This will get proper treatment after T2222.
Test Plan: Installed a `throw new Exception(...)` event listener. Performed Conduit and web updates of revisions, saw event listener fire.
Reviewers: btrahan, guywarner
Reviewed By: guywarner
CC: aran
Differential Revision: https://secure.phabricator.com/D7004
Summary: Fixes T3833. Serious business was seriously disrupted.
Test Plan: Looked at button in both seriousness modes.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T3833
Differential Revision: https://secure.phabricator.com/D7003
Summary: This class is no longer used. It has no callsites.
Test Plan: `grep`
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D6996
Summary: A few things link to old URIs for Maniphest, update them.
Test Plan: Clicked all the things.
Reviewers: chad, btrahan
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D6989
Summary: Deploy on paste and macro for create stories, 'cuz those are boring emails. Fixes T3808.
Test Plan: made a paste and a macro. commented on 'em. verified i got mail on comments only.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3808
Differential Revision: https://secure.phabricator.com/D6988
Summary: Pagers in Maniphest (and, to some degree, apps like Pholio) get lost a bit. Put them in a little box.
Test Plan: Looked at Maniphest and Pholio, pager was more obvious and less un-designed-looking.
Reviewers: chad, btrahan
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D6987
Summary: reported by csilvers in irc
Test Plan: ran a bum query with --trace and verified table scan not run
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6986
Summary:
Removes a bunch of dead stuff:
- Old side nav with hard-coded filters.
- Old edit/list/delete/update interfaces for those filters.
- Old `buildStandardPageResponse()`.
- Some other junk with no callsites.
- Reduce the number of places where the "Create Task" button is built.
Test Plan: `grep`; used list view, batch editor, reports.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6985
Summary: Drive these purely out of configuration after removing behavioral hardcodes in D6981.
Test Plan:
Mucked around with them:
{F58128} {F58129} {F58130}
Reviewers: btrahan
Reviewed By: btrahan
CC: chad, aran
Differential Revision: https://secure.phabricator.com/D6984
Summary: Accidentally lost this in the melee. Put it back.
Test Plan: Saw link, then clicked it. Great success!
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6982
Summary:
Ref T3583. Currently, we have some hard-coded behaviors associated with the "Unbreak Now" and "Needs Triage" priorities. Remove them:
- Users seem somewhat confused by these on occasion, and never seem to think they're cool/useful (that I've seen, at least).
- I think they have low utility in general, see T3583.
- Saves three queries on the home page, which can no longer use row counting since they must be policy filtered.
- Primarily, this paves the way for allowing installs to customize priorities, which is an occasional request.
Also deletes a lot of code with no callsites.
Test Plan: Mostly `grep`. Loaded home page. Viewed reports and task list.
Reviewers: btrahan
Reviewed By: btrahan
CC: chad, aran
Maniphest Tasks: T3583
Differential Revision: https://secure.phabricator.com/D6981
Summary: This marks the first time in history that "Pro" has been removed.
Test Plan: `grep`
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6980
Summary: Point these at the new data and URIs.
Test Plan:
- Batch edited some tasks.
- Exported some tasks to excel.
{F58112}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6978
Summary: Ref T2625. EVERYONE LOVES MIGRATIONS!!!
Test Plan:
- Created and migrated a query with every field, verified results were preserved.
- Created and migrated a query using "noproject" and "upforgrabs" magic, verified results were preserved.
Here's the pre-migration "everything" query:
{F58110}
Here it is after migration:
{F58111}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6977
Summary:
Current page size is `1000`. This is nice to have in some cases, but makes pages slower than necessary in others. Task lists are generally dominated by rendering costs.
For example, my default is "recent tasks", which just lists all tasks ordered by date created. Showing 100 tasks here instead of 1000 makes this several times faster without compromising utility.
I don't want to force the default to 100, though, since sometimes listing everything is quite useful and I think an advantage of Maniphest is that it generally deals reasonably well with large task sets.
(This `limit` property is actually read by the default implementation of `getPageSize()` in the parent class.)
Test Plan: Made queries with page sizes 1, 100, 12, 9, 3000, etc.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6976
Summary: Ref T2625. Fixes user and project paging. Adds visibility-aware project group filtering.
Test Plan: Set page size very small and paged forward and backward in Maniphest, particularly with "Assigned" and "Project" group-by filters.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6973
Summary:
Ref T2625. Depends on D6971. Maniphest is complicated to implement cursor paging for. Builds on D6971 to do so.
This is //almost// complete. Paging on projects and authors doesn't quite work, I'll clean that up shortly. Left some TODOs.
Test Plan: Set page size to `3`, paged forward and backward in a bunch of group/order modes. Results seemed to be as expected.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6972
Summary:
We currently have two giant messes for paging across multiple columns (usually because one column is not unique), and I'm about to add a third for Maniphest.
Provide a more structured way to build these `A > a OR (A = a AND B > b)` clauses.
Test Plan: Set page size to `2` for Differential and Diffusion and paged forward and backward with a bunch of different orders set. Pages worked as expected.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6971
Summary: This allows administrative overreach. Administrators can enable `javascript:` and then XSS things if this isn't locked.
Test Plan: Viewed value on web UI, verified it was locked.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6975
Summary: Fixes T3825. See that task for details.
Test Plan: Verified that `#\herp` no longer matches project `#herp`, but `#herp` still works fine.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3825
Differential Revision: https://secure.phabricator.com/D6970
Summary:
Fixes T3807. Several issues:
- Currently, we split config of type `list<string>` on commas, which makes it impossible to enter a regex with a comma in it.
- Split on newlines only.
- Some of the examples are confusing (provided in JSON instead of the format you actually have to enter them).
- Show examples in the same format you should enter text.
- We didn't validate regexps.
- Introduce `list<regex>` to validate regexes.
@hlau: Note that the old config format for the bugtraq stuff implied the delimiters on the regular expression. They are no longer implied. The examples show the correct format.
Test Plan: Viewed and edited affected config, hitting error and success cases.
Reviewers: btrahan
Reviewed By: btrahan
CC: hlau, aran
Maniphest Tasks: T3807
Differential Revision: https://secure.phabricator.com/D6969
Summary:
Fixes T3821. Maybe. The existing code seemed to have a bug and actually return the //commit phid//. Judging by the function name this is not intended.
Also, sorry to step on toes here -- I thought no one was assigned and was curious about loadRelativeEdges and here we are...
Test Plan: lots of logic here as I have no idea how to use Releeph.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3821
Differential Revision: https://secure.phabricator.com/D6967
Summary: Adds the small caret to differential. Cleans up dropdown frame.
Test Plan: Test caret in differential.
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6983
Summary: Consilidate some of the bar colors, used in Releeph?
Test Plan: UIExamples
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6974
Summary: Swaps the rendering over to the current rendering. This is mostly copy/paste out of TaskListController, which is going to get nuked, with some cleanup.
Test Plan:
{F58064}
- Ran a bunch of queries.
- Viewed empty states.
- Drag-and-dropped stuff.
- (Batch editor / excel export need a tweak to run the new-style queries.)
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6961
Summary: Fixes T1485.
Test Plan: made a herald rule for "not exists". committed to master with no diff. audit was triggered
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T1485
Differential Revision: https://secure.phabricator.com/D6964
Summary: Followup to D6924. Fixes T3824.
Test Plan: deleted a file in a diff. was able to view file content without JS errors
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3824
Differential Revision: https://secure.phabricator.com/D6963
Summary: This is the last missing filter.
Test Plan: Grouped results by a bunch of stuff.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6960
Summary: I think the old thing did this, but this makes queries a bit less ridiculous. For example, `secure.phabricator.com` currently issues a query for 664 handles on my task list, but only 73 of them are unique (basically, all the projects plus all the authors). This proably is slightly good for performance, but mostly makes the "Services" tab manageable.
Test Plan: Looked at Maniphest and some other pages, saw handles and objects where they were expected to be.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6959
Summary:
See discussion in D6955. Currently, the logic for "Group by: Project" is roughly:
- Load every possible result.
- Lots of in-process garbage.
Instead, use the new local project name index (from D6957) to service this query more reasonably. Basically:
- Join a table which has keyed project names.
- Order by that table.
Test Plan: {F58033}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6958
Summary: See discussion in D6955. This provides a table we can JOIN against to (effectively) "ORDER BY project name", populates it intially, and keeps it up to date as projects are edited.
Test Plan:
- Ran storage upgrade, verified projects populated into the table.
- Edited a project, verified its entry updated.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6957
Summary:
See discussion in D6955. Provide an event for applications and users to update secondary search indexes.
Facebook: I don't recall exactly how all the search stuff is rigged up, but this might provide a more practical / less fragile alternative. I think it publishes into ElasticSearch now, and then intern somehow handles the result merge at display time, implictly relying on Phabricator's storage format? A cleaner approach might be to publish a secondary "intern" index in a standard format.
Test Plan: Ran `bin/search index --type proj --trace`, saw events fire.
Reviewers: btrahan
Reviewed By: btrahan
CC: FacebookPOC, aran
Differential Revision: https://secure.phabricator.com/D6956
Summary:
Part one of a large and complicated plot:
- The last filter for Maniphest "pro" queries is "Group By".
- This is currently executed in a convoluted and ridiculous way, loading massive amounts of data.
- The primary reason it works like it does is that we don't have a project name index available in Maniphest, so we can't sort in the DB.
- So, I want to provide a name index to Maniphest and push this work to the DB.
To do that, my plan is:
- Index projects in Search.
- Add a "did update index" event.
- Have Maniphest listen for it.
- When projects are updated, update their indexes in Maniphest.
- Rewrite the giant mess of "group by: project" to be somewhat reasonable.
- This may also extend to some future "group by: assignee".
This is the first small step down this path, which just indexes projects in search.
Test Plan: Ran `bin/search index --type project`, then searched for projects.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6955
Summary: Depends on D6952. Unpunts there since I'm rolling into a swamp full of schema changes.
Test Plan: Issued date-constrained query and saw key as a candidate.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6954
Summary: Noticed this in the schema. "Touches" were an idea that never really got off the ground, as we built out more/better notification channels instead. Essentially, they recorded any object you'd ever interacted with. Maybe this will be useful some day, but for now it does nothing and can't be interacted with. Nuke it.
Test Plan: `grep`, loaded Maniphest.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6953
Summary: Adds date created filtering. There's a task for this somewhere that I can't immediately find.
Test Plan: Filtered tasks.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6952
Summary: Restores any/all/user/exclude project filters to the new search.
Test Plan: Filtered stuff by projects.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6951
Summary: Restores this field to the new ApplicationSearch-based search.
Test Plan: Used fulltext search to find tasks.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6950
Summary: Move this into a more consistent location.
Test Plan: Loaded Maniphest.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6949
Summary: I'm going to just delete all this code at some point, but fixing it now means piles of single gets, so unbreak it first. I'll file something.
Test Plan: Releeph is less fataley.
Reviewers: andrewjcg, btrahan
Reviewed By: andrewjcg
CC: aran
Differential Revision: https://secure.phabricator.com/D6945
Summary: Ref T603. Killing this class is cool because the classes that replace it are policy-aware. Tried to keep my wits about me as I did this and fixed a few random things along the way. (Ones I remember right now are pulling a query outside of a foreach loop in Releeph and fixing the text in UIExample to note that the ace of hearts if "a powerful" card and not the "most powerful" card (Q of spades gets that honor IMO))
Test Plan: tested the first few changes (execute, executeOne X handle, object) then got real mechanical / careful with the other changes.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran, FacebookPOC
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D6941
