1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-28 09:42:41 +01:00
Commit graph

88 commits

Author SHA1 Message Date
Austin McKinley
dd6e82698a More-robust search for task assignees
Summary: See discussion in D19415.

Test Plan: Searched for some owners, found tasks as expected.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19417
2018-04-30 12:18:09 -07:00
epriestley
c46be2a70b Allow Maniphest tasks to be queried by workboard Column PHID via SearchEngine
Summary:
Ref T13120. See PHI571. Fixes T5024. This adds a "View as Query" action to workboard columns, which builds a query in Maniphest that has the current query constraints plus an additional constraint to select only tasks in the specified column.

This is a normal query and can be turned into a dashboard panel, added to a menu, edited, saved as a link, etc.

Much of the complexity here is that finding tasks in a given column isn't entirely straightforward because of how board layout works: when you create a task, it isn't immediately placed in columns. It's only actually added to the "Backlog" column on any boards when someone looks at the board.

To get the right behavior, we must do "board layout" for any queried columns before we can constrain results. This isn't enormously efficient, but should be OK for reasonable boards.

Test Plan:
  - Used "View as Query" for normal columns and milestome columns, got appropriate queries in Maniphest.
  - Applied filters to the board (e.g., "Priorities: wishlist"), then used "View As Query" and had my custom filters respected.
  - Queried some large boards/columns with more than a thousand tasks, got results back within a second or so.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T5024

Differential Revision: https://secure.phabricator.com/D19366
2018-04-13 16:07:44 -07:00
epriestley
e26a784dcf Allow tasks to be filtered and ordered by closed date
Summary: Depends on D19038. Fixes T4434. Updates the SearchEngine and Query to handle these fields.

Test Plan: Filtered and ordered by date and closer.

Maniphest Tasks: T4434

Differential Revision: https://secure.phabricator.com/D19039
2018-02-08 15:42:26 -08:00
epriestley
495ab7363b Remove "Contains Words" constraint from Maniphest
Summary:
Ref T12819. Obsoleted by the Ferret engine, which is unprototyping shortly.

This breaks compatibility in two ways:

  - `maniphest.query` no longer supports "fullText" (now throws an explicit exception).
  - Existing saved searches with a "Contains Words" constraint will no longer have that constraint.

It seems unlikely (?) that either of these are seeing too much use, and they should be easy to fix. I'll note them in the changelog.

Test Plan: Viewed Maniphest, no more "Contains Words" field. Called `maniphest.query` with "fullText", got explicit exception.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12819

Differential Revision: https://secure.phabricator.com/D18586
2017-09-11 18:04:12 -07:00
epriestley
a2a2b3f7f4 Sort global fulltext results by overall relevance
Summary:
Ref T12819. Currently, under the Ferret engine, we query each application's index separately and then aggregate the results.

At the moment, results are aggregated by type first, then by actual rank. For example, all the revisions appear first, then all the tasks.

Instead, surface the internal ranking data from the underlying query and sort by it.

Test Plan: Searched for "A B" with a task named "A B" and a revision named "A". Saw task first. Broadly, saw mixed task and revision order in result sets.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12819

Differential Revision: https://secure.phabricator.com/D18551
2017-09-07 13:21:58 -07:00
epriestley
1644b45050 Disperse task subpriorities in blocks
Summary:
Ref T7664. The current algorithm for moving task subpriorities can end up stuck in a real sticky swamp in some unusual situations.

Instead, use an algorithm which works like this:

  - When we notice two tasks are too close together, look at the area around those tasks (just a few paces).
  - If things look pretty empty, we can just spread the tasks out a little bit.
  - But, if things are still real crowded, take another look further.
  - Keep doing that until we're looking at a real nice big spot which doesn't have too many tasks in it in total, even if they're all in one place right now.
  - Then, move 'em out!

Also:

  - Just swallow our pride and do the gross `INSERT INTO ... "", "", "", "", "", "", ... ON DUPLICATE KEY UPDATE` to bulk update.
  - Fix an issue where a single move could cause two different subpriority recalculations.

Test Plan:
  - Changed `ManiphesTaskTestCase->testTaskAdjacentBlocks()` to insert 1,000 tasks with identical subpriorities, saw them spread out in 11 queries instead of >1,000.
  - Dragged tons of tasks around on workboards.
  - Ran unit tests.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7664

Differential Revision: https://secure.phabricator.com/D17959
2017-05-19 15:45:14 -07:00
epriestley
a9e2732a5c Spell "Elasticsearch" correctly, not "ElasticSearch"
Summary: Ref T12450. These are like 95% my fault, but Elastic appears to spell the name "Elasticsearch" consistently in their branding.

Test Plan: `grep ElasticSearch`

Reviewers: chad, 20after4

Maniphest Tasks: T12450

Differential Revision: https://secure.phabricator.com/D17601
2017-04-02 14:58:59 -07:00
Mukunda Modell
e41c25de50 Support multiple fulltext search clusters with 'cluster.search' config
Summary:
The goal is to make fulltext search back-ends more extensible, configurable and robust.

When this is finished it will be possible to have multiple search storage back-ends and
potentially multiple instances of each.

Individual instances can be configured with roles such as 'read', 'write' which control
which hosts will receive writes to the index and which hosts will respond to queries.

These two roles make it possible to have any combination of:

* read-only
* write-only
* read-write
* disabled

This 'roles' mechanism is extensible to add new roles should that be needed in the future.

In addition to supporting multiple elasticsearch and mysql search instances, this refactors
the connection health monitoring infrastructure from PhabricatorDatabaseHealthRecord and
utilizes the same system for monitoring the health of elasticsearch nodes. This will
allow Wikimedia's phabricator to be redundant across data centers (mysql already is,
elasticsearch should be as well).

The real-world use-case I have in mind here is writing to two indexes (two elasticsearch clusters
in different data centers) but reading from only one. Then toggling the 'read' property when
we want to migrate to the other data center (and when we migrate from elasticsearch 2.x to 5.x)

Hopefully this is useful in the upstream as well.

Remaining TODO:

* test cases
* documentation

Test Plan:
(WARNING) This will most likely require the elasticsearch index to be deleted and re-created due to schema changes.

Tested with elasticsearch versions 2.4 and 5.2 using the following config:

```lang=json
  "cluster.search": [
    {
      "type": "elasticsearch",
      "hosts": [
        {
          "host": "localhost",
          "roles": { "read": true, "write": true }
        }
      ],
      "port": 9200,
      "protocol": "http",
      "path": "/phabricator",
      "version": 5
    },
    {
      "type": "mysql",
      "roles": { "write": true }
     }
  ]

Also deployed the same changes to Wikimedia's production Phabricator instance without any issues whatsoever.
```

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Tags: #elasticsearch, #clusters, #wikimedia

Differential Revision: https://secure.phabricator.com/D17384
2017-03-26 08:16:47 +00:00
epriestley
dc7ecf5875 Add "subtype" storage to Maniphest tasks
Summary: Ref T12314. Provides a field on tasks for storing subtypes. Does nothing interesting yet.

Test Plan:
  - Ran storage upgrade.
  - Created some tasks.
  - Looked in the database.
  - Used Conduit to query some tasks.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12314

Differential Revision: https://secure.phabricator.com/D17441
2017-03-02 04:17:47 -08:00
epriestley
a372627fcd Provide URI/API support for querying subtasks/parents of a particular task
Summary:
Ref T8126. Ref T4788. This adds a way to query by parent or subtask.

I plan to link to this from the task graph (e.g., {nav View > Search Subtasks} or similar, in a dropdown on the "Task Graph" element) as a way to let us bail out if tasks have 300 subtasks and send the user to a big query result list. That'll give us more flexibility to tailor the UI for reasonable numbers of tasks.

There's no UI for this unless you specify a query yourself, so the only ways to get to it are:

  - Manually put `?parentIDs=...` into the URI.
  - Use the API.
  - Future link from task graphs.

It doesn't seem too useful to me on its own, outside of the context of links from tasks.

Test Plan:
  - Manually put `?parentIDs=...` and `?subtaskIDs=...` into Maniphest query UI, got expected results.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4788, T8126

Differential Revision: https://secure.phabricator.com/D16341
2016-07-28 11:01:31 -07:00
epriestley
c715b42f36 Fix "Blocked" task queries with multiple subtasks, and update language
Summary:
Ref T8126. See that task for discussion. This change:

  - Updates language to be more consistent ("Parents", "Subtasks") since I moved us away from the often-confusing "Block" language in T4788.
  - Fixes bugs with finding the wrong set of tasks if tasks have a mixture of open and closed subtasks or parents.

Test Plan:
  - Created four tasks: no subtasks, one closed subtask, one open subtask, mixture of open and closed subtasks.
  - Created four more tasks: no parents, one closed parent, one open parent, mixture of open and closed parents.
  - Searched for all this stuff, got the proper results:

{F1740683}

{F1740684}

{F1740685}

{F1740686}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8126

Differential Revision: https://secure.phabricator.com/D16340
2016-07-28 11:00:41 -07:00
epriestley
47dedfb152 Introduce "bridged" objects
Summary:
Ref T10537. These are objects which are bound to some external object, like a Maniphest task which is a representation of a GitHub issue.

This doesn't do much yet and may change, but my thinking is:

  - I'm putting these on-object instead of on edges because I think we want to actively change the UI for them (e.g., clearly call out that the object is bridged) but don't want every page to need to do extra queries in the common case where zero bridged objects exist anywhere in the system.
  - I'm making these one-to-one, more or less: an issue can't be bridged to a bunch of tasks, nor can a bunch of tasks be bridged to a single issue. Pretty sure this makes sense? I can't come up with any reasonable, realistic cases where you want a single GitHub issue to publish to multiple different tasks in Maniphest.
  - Technically, one type of each bridgable object could be bridged, but I expect this to never actually occur. Hopefully.

Test Plan: Ran storage upgrade, loaded some pages.

Reviewers: chad

Reviewed By: chad

Subscribers: Luke081515.2

Maniphest Tasks: T10537

Differential Revision: https://secure.phabricator.com/D15502
2016-03-22 15:06:57 -07:00
epriestley
cb1c3424a8 Use "tag" more consistenty when referring to associating a project with an object
Summary:
Ref T10144. This isn't comprehensive, but we can give it a try and see how it feels?

  - EditEngine forms now say "Tags" instead of "Projects".
  - Modern SearchEngine forms now say "Tags" instead of "Projects".
  - For clarity, replaced as much "in project" language as I could find with "tagged with project" language.

Test Plan: reading / grepping + used "not tagged with any project" token

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10144

Differential Revision: https://secure.phabricator.com/D15108
2016-01-24 10:02:42 -08:00
epriestley
23c42486e4 Rename "SearchEngine" to "FulltextStorageEngine"
Summary:
Ref T9979. I picked this name long before the advent of modern "Engine" architecture and it ended up being pretty confusing.

Rename "SearchEngine" (currently: mysql or elasticsearch, used to store and query fulltext indexes) to "FulltextStorageEngine" to make it more clear what it does and disambituate it from ApplicationSearch, which also has a bunch of stuff called "SearchEngine", "SearchEngineExtension", etc.

Test Plan: Grepped for `phabricatorsearchengine`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14843
2015-12-21 17:26:19 -08:00
epriestley
487e12101a Make ManiphestTaskQuery more modern and safe
Summary: Ref T8637. This class has some really old parameter handling which can send `withIDs(array())` down a "fetch everything" pathway. Clean up most of it.

Test Plan: Issued every ApplicationSearch query.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8637

Differential Revision: https://secure.phabricator.com/D13390
2015-06-22 11:54:51 -07:00
Paul Kassianik
72fc3da416 Add a "Date Updated (Oldest First)" sort order to Maniphest
Summary: Closes T8484

Test Plan: Create sample tasks then navigate to Maniphest > Advanced Search and choose choose "Date Updated (Oldest First)" in the "Order By" select box.

Reviewers: lpriestley, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T8484

Differential Revision: https://secure.phabricator.com/D13237
2015-06-10 12:21:12 -07:00
epriestley
1a091e5260 Drive Maniphest grouping and ordering through standard infrastructure
Summary: Ref T8441. Ref T7715. Ref T7909. Clean up all the ordering and grouping hacks in Maniphest so we can drive it through normal infrastructure, move it to SearchField, introduce Spaces, and eventually modernize the Conduit API.

Test Plan:
  - Executed all grouping/ordering queries, including custom queries.
  - Forced execution with old aliases; got modern results.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7909, T7715, T8441

Differential Revision: https://secure.phabricator.com/D13197
2015-06-08 12:23:13 -07:00
epriestley
bf87976d25 Support ordering in SearchField
Summary:
Ref T8441. Ref T7715. Automatically generate a modern "Order" control in ApplicationSearch for engines which fully support SearchField.

Notably, this allows the standard "Order" control to automatically support custom field orders. We do this in Maniphest today, but in an ad-hoc way.

Test Plan: Performed order-by queries in Almanac (Services), Pholio, Files, People, Projects, and Paste.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13193
2015-06-08 12:21:48 -07:00
Joshua Spence
36e2d02d6e phtize all the things
Summary: `pht`ize a whole bunch of strings in rP.

Test Plan: Intense eyeballing.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12797
2015-05-22 21:16:39 +10:00
Joshua Spence
16a8ed72bd Modernize search engine selection
Summary: Remove the `PhabricatorDefaultSearchEngineSelector` class. This is quite similar to D12053.

Test Plan: Went to `/view/PhabricatorSearchApplication/` and saw the storage engine configuration. Set `search.elastic.host` and saw the highlighted storage engine change.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12670
2015-05-20 06:59:59 +10:00
Joshua Spence
acb45968d8 Use __CLASS__ instead of hard-coding class names
Summary: Use `__CLASS__` instead of hard-coding class names. Depends on D12605.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12806
2015-05-14 07:21:13 +10:00
Povilas Balzaravicius Pawka
7196d29c0c Fix query types on maniphest fulltext search.
Summary:
Parameter `type` is not used anywhere. Instead param `types` is used.
Due this bug search was performed over whole index instead on tasks only.

Test Plan:
- Setup phabricator to run with Elasticsearch.
- Open http://yourphab.com/conduit/method/maniphest.query/
- Fill `fullText` field only.
- Expect get results. Query should be performed on `/phabricator/TASK/` only (not on whole `/phabricator/` index).

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12542
2015-04-24 11:16:40 -07:00
epriestley
3ba56ceaf0 Add an anyone() typeahead function for matching any document owner
Summary: Fixes T5733.

Test Plan: Ran search and Maniphest queries.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5733

Differential Revision: https://secure.phabricator.com/D12535
2015-04-23 15:44:27 -07:00
epriestley
22e3e35418 Move ManiphestTaskQuery to EdgeLogic
Summary:
Ref T4100. Share all edge logic code across applications.

  - Internalizes the "check that the viewer can see projects" check into edge logic.
  - Adds some convenience functions. Some of these aren't really all that convenient, but it's rare that we actually apply project constraints to queries in the applications -- and most of these callsites will go away in the long term -- so I didn't go too crazy with providing a simpler `withProjectPHIDs()` universal API or anything.

Test Plan:
  - Grepped for all affected symbols.
  - Tried to violate policies.
  - Used workboards.
  - Used normal Maniphest queries.
  - Used `maniphest.query`.
  - Verified the special grouping behavior works as expected.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100

Differential Revision: https://secure.phabricator.com/D12526
2015-04-23 11:49:33 -07:00
epriestley
7ab025eef5 Use a function typeahead for "projects" in Maniphest
Summary: Ref T4100. Collapse the five inputs into one.

Test Plan:
  - Searched for a bunch of stuff.
  - Used "Group By: Project", which is a bit of a special case and possibly tricky.
  - Created an old query with all the fields, then updated; verified it was preserved/transformed correctly.

{F379971}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100

Differential Revision: https://secure.phabricator.com/D12525
2015-04-23 11:49:33 -07:00
epriestley
135c8799e6 Refine global search UI for tokenizer functions
Summary:
Ref T4100.

  - Removes the "with unowned" checkbox in favor of the "no owners" function.
  - Support functions in "Authors" and "Owners".

Test Plan:
  - Ran various global search and Maniphest queries.

{F379931}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100

Differential Revision: https://secure.phabricator.com/D12523
2015-04-23 11:49:33 -07:00
epriestley
f044c1e999 Possibly fix issue with subpriority recursion
Summary:
Ref T7664. Currently, when spreading subpriorities we may recurse deeply in certain conditions. Make sure we never recurse more than one level.

To try to mitigate issues with floating point precision, be more aggressive about selecting tasks to reorder.

I wasn't really able to come up with a realistic test case here, and the test cases I found which sort of approximated the behavior took way too long to generate data to actually commit.

This approach is inherently somewhat fragile but hopefully this is approximately good enough. We don't have a durable storage engine which can meaningfully represent double-linked lists right now.

Test Plan:
  - Wrote some (slow) tests which kind of approximately hit the issue.
  - Verified they maxed out at stack depth 2 after the change.
  - Unit tests still pass.
  - Dragged some tasks around.
  - Couldn't come up with any pathological issues here by thinking about it?

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7664

Differential Revision: https://secure.phabricator.com/D12511
2015-04-22 16:48:56 -07:00
epriestley
6fa9b10914 Collapse "Show unassigned tasks" in Maniphest into a parameterized owners typeahead
Summary: Ref T4100. Support viewer(), members(), and add a new none().

Test Plan:
  - Used all new functions.
  - Batch edited tasks with unassign action.
  - Saved a query from master, upgrade it to this patch, checkbox migrated cleanly into a "no one" token.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100

Differential Revision: https://secure.phabricator.com/D12470
2015-04-20 10:07:30 -07:00
epriestley
e27c0b416d Add "Edge Logic" support to PolicyAwareQuery
Summary:
Ref T4100. Ref T5595. This allows PolicyAwareQuery to write all the logic for AND, OR, NOT, and NULL (i.e., "not in any projects") queries against any edge type.

It accepts an edge type and a list of constraints (which are basically just operator-value pairs, like `<NOT, PHID-X-Y>`, meaning the results must not have an edge connecting them to `PHID-X-Y`).

This doesn't actually do anything yet; see future diffs.

Test Plan: `arc unit --everything`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100, T5595

Differential Revision: https://secure.phabricator.com/D12455
2015-04-20 10:06:12 -07:00
epriestley
8efdc4aabf Replace getPagingValue() with cursor methods
Summary:
Ref T7803. Prior to this change sequence, Query classes conflated paging values (the actual thing that goes in a "x > 3" clause) with cursor values (arbitrary identifiers which track where the user is in a result list).

Although the two can sometimes be the same, the vast majority of implementations are simpler and better when object IDs are used as cursors and paging values are derived from them.

The new stuff handles this in a consistent way, so we're free to separate getPagingValue() from paging. The new method is essentially getResultCursor().

This also implements getPageCursors(), which allows queries to return directional cursors. The inability to do this was a practical limitation blocking the implementation of T7803.

Test Plan:
  - Browsed a bunch of results and paged through queries.
  - Grepped for removed methods.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12383
2015-04-13 11:58:38 -07:00
epriestley
bdd1edea7a Modernize ManiphestTask paging and ordering
Summary:
Ref T7803. The ApplicationSearch integration is still a little rough here, but it seems to have the correct behavior.

The rest of this is now at least relatively sane, cohesive, and properly behaved.

Test Plan:
  - Used all grouping and ordering queries in Maniphest. Pagingated results.
  - Used custom field ordering in Maniphest. Paginated results.
  - Paginated through the `null` section of "Assigned" and "Projects" group-by queries. Pagingation now works correctly (it does not work at HEAD).
  - Ran unit tests covering priority changes.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12372
2015-04-13 11:58:31 -07:00
epriestley
4114560844 Modernize more paging/order queries
Summary:
Ref T7803. Removes some getReversePaging().

This also fixes `null` column handling, by adding an explicit `'null'` key with possible values "head" (put NULL before other values) or "tail" (put NULL after other values).

Maniphest has some glitchiness in paging through NULLs right now, but I believe it's all pre-existing and will be resolved when it fully converts. Diffusion is fully converted and pages through NULL correctly.

Test Plan:
  - Failed to identify any reason for ChangesetQuery to reverse paging.
  - Paged thorugh Diffusion.
  - Paged through Maniphest.
    - Maniphest has some issues when paging inside a NULL section, but these issues are preexisting and will be resolved later in this change sequence.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12371
2015-04-13 11:58:30 -07:00
epriestley
a4a198342e Modernize ReleephProjectQuery ordering/paging
Summary: Ref T7803. Continue removing implementations of getPagingColumn() and getReversePaging().

Test Plan: Browsed and paged through Releeph projects, Maniphest tasks, Diffusion repositories.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12358
2015-04-13 11:58:21 -07:00
epriestley
4fba6e7730 Remove trivial implementations of getPagingColumn()
Summary:
Ref T7803. Some Query subclasses implement getPagingColumn() in a trivial way, usually to provide a table alias.

Formalize the concept of a primary table alias, and remove obsoleted getPagingColumn() implementations.

Test Plan: Issued affected queries.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12356
2015-04-13 11:58:19 -07:00
epriestley
604d1409f1 Make buildPagingClauseFromMultipleColumns() safer
Summary: Ref T7803. Reduce the amount of code we're trusting to build SQL queries.

Test Plan:
  - Paged through results in Maniphest, Differential and Diffusion.
  - Some of the NULLable groups in Maniphest are a bit funky but this was preexisting.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12353
2015-04-13 11:58:15 -07:00
epriestley
731404445f Improve task subpriority movement algorithm for homogenous blocks
Summary:
Fixes T7664. When there are a large number of tasks (400+) with the same subpriority (which can happen if the subpriority features are rarely used), it may take more than 30 seconds to rebalance them.

Make the algorithm more aggressive about rebalancing homogenous blocks of tasks.

This may need to get even fancier, but I'd guess it can process blocks 1-2 orders of magnitude larger, which should be ~all installs.

(If someone still hits issues with this, I'll make it fancier.)

Once a block is rebalanced, it doesn't need to be rebalanced again (at least, not as a whole block) so we basically just need to get over the initial hurdle here and then we're good.

In the worst case, we can provide `bin/maniphest rebalance` or similar and do the rebalance step offline.

And, in any case, we have more test coverage here now.

Test Plan:
  - Existing tests.
  - New tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7664

Differential Revision: https://secure.phabricator.com/D12166
2015-03-26 11:11:23 -07:00
epriestley
5aca529980 Fix literally thousands of drag-to-reorder priority bugs
Summary:
Fixes T7563. Fixes T5201. Reframe this as two separate operations:

  - Move before or after a task.
  - Move to the beginning or end of a priority.

Then:

  - Make all the order queries unambiguous and properly reversible, with an explicit `id` order.
  - Just reuse `ManiphestTask` to get results in the correct order.
  - Simplify the actual transaction apply logic.
  - Detect and recover from cases where tasks have identical or similar subpriorities.

Test Plan:
  - Wrote and executed unit tests.
  - Dragged and dropped tasks within priorities and between priorities in the main Maniphest view.
  - Dragged and dropped tasks within priorities in the workboard view, when ordered by priority.
  - Also poked at the "natural" order, but that shouldn't be affected.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: chad, epriestley

Maniphest Tasks: T5201, T7563

Differential Revision: https://secure.phabricator.com/D12121
2015-03-20 17:38:25 -07:00
epriestley
106ca70acb Fix an issue where subpriority paging could be truncated
Ref T7548. Subpriority is a float, but we're truncating it to an int, which can cause reselection of the same row while paging.
2015-03-14 13:42:06 -07:00
epriestley
77e0a4abba Guarantee that Maniphest paging clauses strictly progress
Ref T7548. Some of these clauses are not guaranteed to select only rows following the cursor.
2015-03-14 13:29:05 -07:00
epriestley
c431d91f58 Fail explicitly instead of looping if we can't build a paging clause
Ref T7548. This isn't a complete fix but hopefully will give me more debugging information.
2015-03-14 13:23:48 -07:00
Bob Trahan
05510aa41f Maniphest - fix blocking / blocked task queries
Summary: Fixes T7392. I kind of stink at SQL so my approach here was to "start over" conceptually and this way makes the most sense to me - we basically do one join on the dependency table and then a second join back from the dependency table to the main task table. In the where clause we filter the resulting rows, first checking the data from dependency join for existence as appropros and then checking the second join for main task table for the proper "open" task values.

Test Plan: made a task X be blocked by task Y. closed task y. search for "not blocked" tasks and saw task X.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7392

Differential Revision: https://secure.phabricator.com/D11962
2015-03-03 15:53:08 -08:00
epriestley
a76049abd4 Fix not blocking / not blocked queries
Summary: Fixes T7434. We need to LEFT JOIN, not JOIN here, because we still want result rows where the value is `null`.

Test Plan: Issued blocked/not-blocked + blocking/not-blocking queries, got results in all cases.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7434

Differential Revision: https://secure.phabricator.com/D11939
2015-03-02 17:00:20 -08:00
epriestley
bb6eb9bfcf Fix ambiguous column names in Maniphest blocking queries
Summary: Fixes T7421. Now that we join the task table again to ignore //closed// blockers, all the column names are ambiguous. Make them unambiguous.

Test Plan: Issued some searches with various different parameters.

Reviewers: btrahan, joshuaspence, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T7421

Differential Revision: https://secure.phabricator.com/D11922
2015-03-02 06:43:52 -08:00
Joshua Spence
130aa755bb Exclude closed tasks from blocked/blocking filter
Summary: Fixes T7392. When filtering blocked/blocking Maniphest tasks, don't consider closed tasks.

Test Plan:
# Created `T1` and `T2` with `T2 depends on T1`.
# Marked `T1` as resolved.
# Searched for tasks "blocked by other tasks" and noted that `T2` wasn't in the result set.

Reviewers: btrahan, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7392

Differential Revision: https://secure.phabricator.com/D11911
2015-03-01 09:39:34 +11:00
Bob Trahan
da1531f219 Policy - make ManiphestTaskQuery verify project visibility first thing
Summary: Fixes T7094 (last of many revisions). Its important to do this filtering ASAP so that users can't deduce the identify of an unknown / invisible project.

Test Plan: executed a query for tasks in project foo using user bar. using user foo, lock user bar out of project foo. reissued the query and saw "no data" as well as "restricted project" in the project typeahead.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7094

Differential Revision: https://secure.phabricator.com/D11660
2015-02-03 13:53:35 -08:00
Joshua Spence
463d094f96 Fix method visibility for PhabricatorPolicyAwareQuery subclasses
Summary: Ref T6822.

Test Plan:
`grep` for the following:

  - `->willFilterPage(`
  - `->loadPage(`
  - `->didFilterPage(`
  - `->getReversePaging(`
  - `->didFilterPage(`
  - `->willExecute(`
  - `->nextPage(`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11367
2015-01-14 07:01:16 +11:00
Bob Trahan
1eb8b6a1b8 Maniphest - allow for searching for tasks based on dependency relationships
Summary:
Fixes T5352. This is very useful for finding things that should be easy to do ("not blocked") as well as things that are important to do ("blocking"). I have wanted to check out the latter case in our installation, though no promises on what I would end up actually doing from that search result list. =D

I also think supporting something like T6638 is reasonable but the UI seems trickier to me; its some sort of task tokenizer, which I don't think we've done before?

Test Plan: toggled various search options and got reasonable results. When i clicked conflicting things like "blocking" and "not blocking" verified it was like I had not clicked anything at all.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5352

Differential Revision: https://secure.phabricator.com/D11306
2015-01-12 13:42:37 -08:00
Joshua Spence
7c2a7d0365 Modernize remaining edge types
Summary: Modernize remaining edges to subclass `PhabricatorEdgeType`. Largely based on D11045.

Test Plan: Browsed around and performed various actions include subscribing, unsubscribing and watching.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11116
2015-01-03 10:58:20 +11:00
Bob Trahan
83db5965ab Maniphest - introduce needProjectPHIDs
Summary: Ref T5245. This is some of the associated cleanup there.

Test Plan:
foreach ManiphestTaskQuery site, I made the change (or not) and tested as follows:

=== Call sites where added needProjectPHIDs ===

- PhabricatorHomeMainController - loaded the home page
- ManiphestBatchEditController - batch edited some tasks (added a project)
- ManiphestConduitAPIMethod - tested implicitly when tested ManiphestUpdateConduitAPIMethod
- ManiphestInfoConduitAPIMethod - used the method via conduit console with input id : 1
- ManiphestQueryConduitAPIMethod - used the method via conduit console with input ids : [1, 2]
- ManiphestUpdateConduitAPIMethod - used the method via conduit with input id : 1 and comment : “asdasds"
- ManiphestReportController - viewed “By User” and “By Project”
- ManiphestSubpriorityController - changed the priority of a task via a drag on manphest home
- ManiphestTaskMailReceiver - updated Task 1 via bin/mail receive-test with a comment that is the README
- ManiphestTaskSearchEngine - loaded Manifest home page
- ManiphestTaskEditController - edited a task
- ManiphestTransactionEditor - closed a blocking task
- ManiphestTransactionSaveController - commented on a task
- PhabricatorProjectProfileController - viewed project with id of 1 that has a few tasks in it
- PhabricatorSearchAttachController - merged tasks together
- DifferentialTransactionEditor - submit a diff that references a task; commit the diff (thus closing the diff) and the task gets updated
- PhabricatorRepositoryCommitMessageParserWorker - submit a diff that references a task; commit the diff (thus closing the diff) and the task gets updated

=== Calls sites where *did not* add needProjectPHIDs (they do not appear in this revision) ===

- PhabricatorManiphestApplication - loaded the home page
- ManiphestGetTaskTransactionsConduitAPIMethod - used the method via conduit console with input ids : [1, 2] ManiphestTaskDetailController - viewed a task with and without associated projects; finished workflow creating a task with a parent
- ManiphestTransactionPreviewController - verified transaction preview showed up properly
- PhabricatorProjectBoardViewController - viewed a board
- PhabricatorProjectMoveController - moved a task around
- ManiphestRemarkupRule - made a task reference like {T123}
- ManiphestTaskQuery - executed a custom query for all tasks with page size of 2 and paginated through some tasks
- ManiphestTaskPHIDType - nothing random seems broken? =D

=== Call sites where had to do something funky ===

- ManiphestHovercardEventListener - loaded hover cards from task mentions

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D11004
2014-12-18 13:53:45 -08:00
Bob Trahan
7d96870570 Maniphest - use subscribers framework properly
Summary: Fixes T5604. This should fix some random bugs, lets us move forward more easily, and all that good stuff about killing code debt.

Test Plan:
- Conduit method maniphest.createtask
  - verified creating user subscribed
  - verified subscription transaction
- Conduit method maniphest.update
  - verified subscribers set as specified to ccPHIDs parameter
  - verified subscription transaction
- Herald
  - verified herald rule to add subscriber worked
  - verified no subscribers removed accidentally
- edit controller
  - test create and verify author gets added IFF they put themselves in subscribers control box
  - test update gets set to exactly what user enters
- lipsum generator'd tasks work
- bulk add subscribers works
- bulk remove subscriber works
- detail controller
  - added myself by leaving a comment
  - added another user via explicit action
  - added another user via implicit mention
- task merge via search attach controller
- mail reply handler
  - add subscriber via ./bin/mail receive-test
  - unsubscribe via ./bin/mail receive-test

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5604

Differential Revision: https://secure.phabricator.com/D10965
2014-12-10 16:27:30 -08:00