Summary: Ref T2625. EVERYONE LOVES MIGRATIONS!!!
Test Plan:
- Created and migrated a query with every field, verified results were preserved.
- Created and migrated a query using "noproject" and "upforgrabs" magic, verified results were preserved.
Here's the pre-migration "everything" query:
{F58110}
Here it is after migration:
{F58111}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6977
Summary:
Current page size is `1000`. This is nice to have in some cases, but makes pages slower than necessary in others. Task lists are generally dominated by rendering costs.
For example, my default is "recent tasks", which just lists all tasks ordered by date created. Showing 100 tasks here instead of 1000 makes this several times faster without compromising utility.
I don't want to force the default to 100, though, since sometimes listing everything is quite useful and I think an advantage of Maniphest is that it generally deals reasonably well with large task sets.
(This `limit` property is actually read by the default implementation of `getPageSize()` in the parent class.)
Test Plan: Made queries with page sizes 1, 100, 12, 9, 3000, etc.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6976
Summary: Ref T2625. Fixes user and project paging. Adds visibility-aware project group filtering.
Test Plan: Set page size very small and paged forward and backward in Maniphest, particularly with "Assigned" and "Project" group-by filters.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6973
Summary:
Ref T2625. Depends on D6971. Maniphest is complicated to implement cursor paging for. Builds on D6971 to do so.
This is //almost// complete. Paging on projects and authors doesn't quite work, I'll clean that up shortly. Left some TODOs.
Test Plan: Set page size to `3`, paged forward and backward in a bunch of group/order modes. Results seemed to be as expected.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6972
Summary:
We currently have two giant messes for paging across multiple columns (usually because one column is not unique), and I'm about to add a third for Maniphest.
Provide a more structured way to build these `A > a OR (A = a AND B > b)` clauses.
Test Plan: Set page size to `2` for Differential and Diffusion and paged forward and backward with a bunch of different orders set. Pages worked as expected.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6971
Summary: This allows administrative overreach. Administrators can enable `javascript:` and then XSS things if this isn't locked.
Test Plan: Viewed value on web UI, verified it was locked.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6975
Summary: Fixes T3825. See that task for details.
Test Plan: Verified that `#\herp` no longer matches project `#herp`, but `#herp` still works fine.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3825
Differential Revision: https://secure.phabricator.com/D6970
Summary:
Fixes T3807. Several issues:
- Currently, we split config of type `list<string>` on commas, which makes it impossible to enter a regex with a comma in it.
- Split on newlines only.
- Some of the examples are confusing (provided in JSON instead of the format you actually have to enter them).
- Show examples in the same format you should enter text.
- We didn't validate regexps.
- Introduce `list<regex>` to validate regexes.
@hlau: Note that the old config format for the bugtraq stuff implied the delimiters on the regular expression. They are no longer implied. The examples show the correct format.
Test Plan: Viewed and edited affected config, hitting error and success cases.
Reviewers: btrahan
Reviewed By: btrahan
CC: hlau, aran
Maniphest Tasks: T3807
Differential Revision: https://secure.phabricator.com/D6969
Summary:
Fixes T3821. Maybe. The existing code seemed to have a bug and actually return the //commit phid//. Judging by the function name this is not intended.
Also, sorry to step on toes here -- I thought no one was assigned and was curious about loadRelativeEdges and here we are...
Test Plan: lots of logic here as I have no idea how to use Releeph.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3821
Differential Revision: https://secure.phabricator.com/D6967
Summary: Adds the small caret to differential. Cleans up dropdown frame.
Test Plan: Test caret in differential.
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6983
Summary: Consilidate some of the bar colors, used in Releeph?
Test Plan: UIExamples
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6974
Summary: Swaps the rendering over to the current rendering. This is mostly copy/paste out of TaskListController, which is going to get nuked, with some cleanup.
Test Plan:
{F58064}
- Ran a bunch of queries.
- Viewed empty states.
- Drag-and-dropped stuff.
- (Batch editor / excel export need a tweak to run the new-style queries.)
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6961
Summary: Fixes T1485.
Test Plan: made a herald rule for "not exists". committed to master with no diff. audit was triggered
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T1485
Differential Revision: https://secure.phabricator.com/D6964
Summary: Followup to D6924. Fixes T3824.
Test Plan: deleted a file in a diff. was able to view file content without JS errors
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3824
Differential Revision: https://secure.phabricator.com/D6963
Summary: This is the last missing filter.
Test Plan: Grouped results by a bunch of stuff.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6960
Summary: I think the old thing did this, but this makes queries a bit less ridiculous. For example, `secure.phabricator.com` currently issues a query for 664 handles on my task list, but only 73 of them are unique (basically, all the projects plus all the authors). This proably is slightly good for performance, but mostly makes the "Services" tab manageable.
Test Plan: Looked at Maniphest and some other pages, saw handles and objects where they were expected to be.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6959
Summary:
See discussion in D6955. Currently, the logic for "Group by: Project" is roughly:
- Load every possible result.
- Lots of in-process garbage.
Instead, use the new local project name index (from D6957) to service this query more reasonably. Basically:
- Join a table which has keyed project names.
- Order by that table.
Test Plan: {F58033}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6958
Summary: See discussion in D6955. This provides a table we can JOIN against to (effectively) "ORDER BY project name", populates it intially, and keeps it up to date as projects are edited.
Test Plan:
- Ran storage upgrade, verified projects populated into the table.
- Edited a project, verified its entry updated.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6957
Summary:
See discussion in D6955. Provide an event for applications and users to update secondary search indexes.
Facebook: I don't recall exactly how all the search stuff is rigged up, but this might provide a more practical / less fragile alternative. I think it publishes into ElasticSearch now, and then intern somehow handles the result merge at display time, implictly relying on Phabricator's storage format? A cleaner approach might be to publish a secondary "intern" index in a standard format.
Test Plan: Ran `bin/search index --type proj --trace`, saw events fire.
Reviewers: btrahan
Reviewed By: btrahan
CC: FacebookPOC, aran
Differential Revision: https://secure.phabricator.com/D6956
Summary:
Part one of a large and complicated plot:
- The last filter for Maniphest "pro" queries is "Group By".
- This is currently executed in a convoluted and ridiculous way, loading massive amounts of data.
- The primary reason it works like it does is that we don't have a project name index available in Maniphest, so we can't sort in the DB.
- So, I want to provide a name index to Maniphest and push this work to the DB.
To do that, my plan is:
- Index projects in Search.
- Add a "did update index" event.
- Have Maniphest listen for it.
- When projects are updated, update their indexes in Maniphest.
- Rewrite the giant mess of "group by: project" to be somewhat reasonable.
- This may also extend to some future "group by: assignee".
This is the first small step down this path, which just indexes projects in search.
Test Plan: Ran `bin/search index --type project`, then searched for projects.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6955
Summary: Depends on D6952. Unpunts there since I'm rolling into a swamp full of schema changes.
Test Plan: Issued date-constrained query and saw key as a candidate.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6954
Summary: Noticed this in the schema. "Touches" were an idea that never really got off the ground, as we built out more/better notification channels instead. Essentially, they recorded any object you'd ever interacted with. Maybe this will be useful some day, but for now it does nothing and can't be interacted with. Nuke it.
Test Plan: `grep`, loaded Maniphest.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6953
Summary: Adds date created filtering. There's a task for this somewhere that I can't immediately find.
Test Plan: Filtered tasks.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6952
Summary: Restores any/all/user/exclude project filters to the new search.
Test Plan: Filtered stuff by projects.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6951
Summary: Restores this field to the new ApplicationSearch-based search.
Test Plan: Used fulltext search to find tasks.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6950
Summary: Move this into a more consistent location.
Test Plan: Loaded Maniphest.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6949
Summary: I'm going to just delete all this code at some point, but fixing it now means piles of single gets, so unbreak it first. I'll file something.
Test Plan: Releeph is less fataley.
Reviewers: andrewjcg, btrahan
Reviewed By: andrewjcg
CC: aran
Differential Revision: https://secure.phabricator.com/D6945
Summary: Ref T603. Killing this class is cool because the classes that replace it are policy-aware. Tried to keep my wits about me as I did this and fixed a few random things along the way. (Ones I remember right now are pulling a query outside of a foreach loop in Releeph and fixing the text in UIExample to note that the ace of hearts if "a powerful" card and not the "most powerful" card (Q of spades gets that honor IMO))
Test Plan: tested the first few changes (execute, executeOne X handle, object) then got real mechanical / careful with the other changes.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran, FacebookPOC
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D6941
Summary: Ref T603. Ref D6941.
Test Plan: Clicked around all over - looked good. I plan to re-test D6941 to make sure the executeOne case works now as intended
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D6944
Summary: Less space
Test Plan: Checked out audit page.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6940
Summary: Ref T2625. Further expands the "pro" search.
Test Plan: Used new options to query tasks.
Reviewers: btrahan, garoevans
Reviewed By: garoevans
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6935
Summary: Ref T2625. Moves this a step toward being able to replace the current search.
Test Plan: Used search interface.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2625
Differential Revision: https://secure.phabricator.com/D6934
Summary:
Ref T603. Ref T2625. Cutting this over is tricky because of Maniphest's existing saved queries. Plan here is:
- Build out the "pro" controller at `/maniphest/query/`.
- Once it's at parity, migrate custom queries.
- Nuke the old UI.
This provides a minimal implementation with no filter support.
Test Plan: Looked at `/maniphest/query/`, saw results technically available.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603, T2625
Differential Revision: https://secure.phabricator.com/D6933
Summary:
Ref T603. Moves to detangle and optimize how we apply policies to filtering objects. Notably:
- Add a short circuit for omnipotent users.
- When performing project filtering, do a stricter check for user membership. We don't actually care if the user can see the project or not according to other policy constraints, and checking if they can may be complicated.
- When performing project filtering, do a local check to see if we're filtering the project itself. This is a common case (a project editable by members of itself, for example) and we can skip queries when it is satisfied.
- Don't perform policy filtering in ObjectQuery. All the data it aggregates is already filtered correctly.
- Clean up a little bit of stuff in Feed.
Test Plan: Pages like the Maniphest task list and Project profile pages now issue dramatically fewer queries.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D6931
Summary: Ref T603. These were deprecated some time ago in favor of the more standard withIDs() / withPHIDs().
Test Plan: `grep`, loaded some interfaces.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D6929
Summary: Ref T603. Prepare for conversion to a policy-aware query.
Test Plan: Browsed various interfaces which use this stuff.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D6928
Summary:
Ref T418. These implementations share no method names, so we can safely just move Maniphest fields into the `PhabricatorCustomField` hierarchy.
Replaces two Maniphest-specific custom field exceptions which nothing catches.
Test Plan: Viewed Maniphest, edited/altered custom fields.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T418
Differential Revision: https://secure.phabricator.com/D6927
Summary:
Fixes T3814. Broadly, remarkup tables in inline comments did not work properly. I ran into several messes here and cleaned up some of them:
- Some of this code is doing `JX.$N('div', {}, JX.$H(response.markup))`, to turn an HTML response into a node, passing that around, and then doing junk with it. This is super old and gross.
- The slightly more modern pattern is `JX.$H(response.markup).getFragment().firstChild`, but this is kind of yuck too and not as safe as it could be.
- Introduce `JX.$H(response.markup).getNode()`, which actually expresses intent here. We have a bunch of `getFragment().firstChild` callsites which should switch to this, but I didn't clean those up yet because I don't want to test them all.
- Switch the `JX.$N('div', {}, JX.$H(response.markup))`-style callsites to `JX.$H(response.markup).getNode()`.
- `copyRows()` is too aggressive in finding `<tr />` tags. This actually causes the bug in T3814. We only want to find these tags at top level, not all tags. Don't copy `<tr />` tags which belong to some deeper table.
- Once this is fixed, there's another bug with mousing over the cells in tables in inline comments. We select the nearest `<td />`, but that's the cell in the remarkup table. Instead, select the correct `<td />`.
- At this point, these last two callsites were looking ugly. I provided `findAbove()` to clean them up.
Test Plan: Created, edited, deleted, moused over, and reloaded a revision with inline comments including remarkup tables. Used "Show more context" links.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3814
Differential Revision: https://secure.phabricator.com/D6924
