Summary: Ref T8441. Ref T7715. Let SearchFields do some value massaging before buiding queries so we don't have to work as hard in each SearchEngine. Particularly, they can handle evaluateTokens() from Tokenizers.
Test Plan: Paste automatically gained access to `viewer()`, etc.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7715, T8441
Differential Revision: https://secure.phabricator.com/D13170
Summary:
Ref T8441. Ref T7715. Add a layer of indirection to fields in search engines. This will allow us to:
- Simplify SearchEngine code, which has collected a lot of duplication around expressing what is effectively field types.
- Automatically add fields like "Spaces" and "Projects" (primary driver for T8441).
- Reorder or hide fields (not sure if we really want to do this, but it seems plausible, and this will let us play around with it, at least).
- Drive Conduit Query methods via SearchEngines, so the same code specifies both the search UI and the `application.query` endpoint (primary driver in T7715).
Test Plan:
- Searched for stuff in Paste, everything behaved exaclty like it used to (except that I removed the "no language" checkbox, which seemed like fluff from a bygone era).
- Searched for stuff in other applications, saw no changes.
- Hit date field errors.
- Used query strings to specify values.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7715, T8441
Differential Revision: https://secure.phabricator.com/D13169
Summary: Ref T8387. This is now completely obsoleted by mailing list users.
Test Plan: Grepped for `mailinglist` and related symbols.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: eadler, epriestley
Maniphest Tasks: T8387
Differential Revision: https://secure.phabricator.com/D13129
Summary: Remove the `PhabricatorDefaultSearchEngineSelector` class. This is quite similar to D12053.
Test Plan: Went to `/view/PhabricatorSearchApplication/` and saw the storage engine configuration. Set `search.elastic.host` and saw the highlighted storage engine change.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12670
Summary: Ref T8024. Allow `DateControlValue` to manage enabled/disabled state, so we can eventually delete the copy of this logic in `DateControl`.
Test Plan:
- Used Calendar ApplicationSearch queries to observe improved behaviors:
- Error for invalid start date, if enabled.
- Error for invalid end date, if enabled.
- Error for invalid date range, if both enabled.
- When submitting an invalid date (for example, with the time "Tea Time"), form retains invalid date verbatim instead of discarding information.
- Created an event, using existing date controls to check that I didn't break anything.
Reviewers: chad, lpriestley, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8024
Differential Revision: https://secure.phabricator.com/D12673
Summary:
Ref T4100. This just makes the "specify stuff in query parameters" workflow a little better:
- You can now do `?projects=differential,diffusion`.
- You can now do `?projects=projects(alincoln)`.
Test Plan: Did that stuff ^^^^
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4100
Differential Revision: https://secure.phabricator.com/D12468
Summary:
Ref T4100. Ref T5595. This implements these fields in one mega-field:
- Projects
- Not in projects
- In any project
- Include results in no projects
- In users' projects
Hopefully, this is a step in the right direction.
Test Plan: {F375555}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, chad, epriestley
Maniphest Tasks: T4100, T5595
Differential Revision: https://secure.phabricator.com/D12463
Summary:
Ref T4100. This is still a bit rough around the edges, but mostly does what we're after.
- Implements viewer() and members(...) functions.
- The new browse workflow makes these discoverable.
Test Plan: {F374201}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: chad, epriestley
Maniphest Tasks: T4100
Differential Revision: https://secure.phabricator.com/D12444
Summary: Ref T5750. Update the Almanac service query to be browsable.
Test Plan:
- Browsed and reordered Diffusion.
- Browsed and reordered services in Almanac.
{F373735}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5750
Differential Revision: https://secure.phabricator.com/D12433
Summary: If match is found on document title, boost result.
Test Plan:
- Setup phabricator with elasticsearch.
- Index documents with `bin/search index --all`
- Use search and expect query matching titles appear on the top of results list.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: WikiChad, Korvin, epriestley
Maniphest Tasks: T7827
Differential Revision: https://secure.phabricator.com/D12432
Summary:
Ref T7803. Currently, available high-level orders are spread across Query and SearchEngine classes and implemented separately for each application.
Lift the concept of "builtin" (high-level, user-facing, named) orders (similar to "builtin" queries in ApplicationSearch) into the root Query class, and let it drive the SearchEngine implementation. This allows you to define a new order in one place and have it automatically work across the entire stack.
This will also let Conduit expose this information in a straightforward way.
Test Plan:
- Used ApplicationSearch in Diffusion.
- Used all result orderings.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7803
Differential Revision: https://secure.phabricator.com/D12379
Summary: Fixes T7425. Overall, this is surprising and confusing after jump nav was merged with global search.
Test Plan: Searched for "help", got documents matching the word "help".
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: chasemp, epriestley
Maniphest Tasks: T7425
Differential Revision: https://secure.phabricator.com/D11936
Summary: Fixes T7335. "help" gets you to a specific diviner doc which is an external link, so make sure the code sets is external for the redirect response in this case.
Test Plan: typed "help" and got some
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7335
Differential Revision: https://secure.phabricator.com/D11830
Summary:
Ref T7234. Turns out some search engines are context specific such that they can't be bubbled up to a dashboard panel generically. The example in question is an Instance Members search, where the instance must be specified and is done so in normal codepaths but the dashboard panel stuff has no way of doing that. Ergo, just turn off these sorts of panels.
Note this code just makes it so we can turn off these sorts of panels but does not do any of that.
Test Plan:
made sure all the queries still showed up
otherwise, next diff
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7234
Differential Revision: https://secure.phabricator.com/D11750
Summary: Fixes T7118. This does the basic "filter the list" thing, though it ends up being a little manual since I guess this hasn't come up before? There is also potential weird behavior if the user was using an app and lost access to it - they will have nothing selected on edit - but I think this is actually correct behavior in this circumstance.
Test Plan:
used a user who couldn't get access to the "quick create" apps and noted that the dropdown list on dashboard panel create was missing the expected engines
ran `arc unit --everything` to verify abstract method implemented everywhere
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7118
Differential Revision: https://secure.phabricator.com/D11687
Summary:
Since the default query will sort on this when no query string is
attached we want to make sure the property at least exists.
Otherwise Elasticsearch yells at you: "No mapping found for [dateCreated]
in order to sort on" when you try to search for documents that haven't
been indexed yet.
Test Plan:
Searched for Mocks and Initiatives (no such documents exist in my index)
and got the error. After patching and reinitializing the index, the error
during querying went away.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D11056
Summary:
When the index does not exist and auto_create_index isn't
enabled, running ./bin/index results in a failure. That's
T5990
Instead create an index properly. This also allows us to do
nice things like do a proper mapping and analysis like for
substring matching like outlined by @fabe in T6552.
Test Plan:
Deleted and created index multiple times to verify
proper index creation and usage.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, manybubbles, chasemp, fabe, epriestley
Differential Revision: https://secure.phabricator.com/D10955
Summary:
It's like query_string but fails a little nicer on bad
input. It also allows for limited Lucene syntax; notably
exact string matches with quotation marks.
Fixes T6780
Test Plan:
Tested multiple query constructions, including exact string
matching.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley
Maniphest Tasks: T6780
Differential Revision: https://secure.phabricator.com/D11011
Summary:
Resolves T4659. This implements support for sorting tasks by custom fields.
Some of this feels hacky in the way it's hooked up to the Maniphest search engine and task query.
Test Plan: Queryed on a custom date field, with a small page size, and moved back and forth through the result set.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T4659
Differential Revision: https://secure.phabricator.com/D10106
Summary:
Similar to storage.default-namespace sometimes during development you'll want
to handle multiple indexes alongside one another. Rather than hardcoding the
/phabricator/ index make this exposed in new search.elastic.index setting,
defaulting to the existing "phabricator"
Test Plan:
Existing installations should be unaffected by this change. Changing the new
setting will result in new indexes being created when someone runs
`./bin/search index` again
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: 20after4, rush898, epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D9798
Summary:
Switch to the `match` query. The operator is set to `and` because it defaults to `or` which is likely to annoy users. We might want to consider using `query_string` to get booleans, wildcards, and other features. The only problem with `query_string` is that it can allow querying on other fields in the json document, and we may want to prevent that. That might even expose information we don't want to expose. Another option would be to parse booleans ourselves and translate them to the ES query DSL.
fixes T5488
Test Plan: Try the `vpn`/`VPN` test case described in T5488.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: WikiChad, epriestley, Korvin
Maniphest Tasks: T5488
Differential Revision: https://secure.phabricator.com/D9785
Summary:
ElasticSearch silently removed the long-deprecated `text` query in favor of the `match` query. `match` works just like `text`, so the fix is simple.
fixes T5507
Test Plan: see if the breakage is fixed
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: WikiChad, epriestley, Korvin
Maniphest Tasks: T5507
Differential Revision: https://secure.phabricator.com/D9784
Summary: I'm pretty sure that `@group` annotations are useless now... see D9855. Also fixed various other minor issues.
Test Plan: Eye-ball it.
Reviewers: #blessed_reviewers, epriestley, chad
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin, hach-que
Differential Revision: https://secure.phabricator.com/D9859
Summary:
Fixes T5467.
- Let search engines figure out if they're rendering for a panel or not.
- If Maniphest is rendering a panel, turn off the grips and batch selection.
Test Plan:
- Viewed task panels (no grips).
- Viewed non-panel query results (grips).
Reviewers: chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T5467
Differential Revision: https://secure.phabricator.com/D9714
Summary: Ref T4986. Instead of requiring users to know the name of an application search engine class, let them select from a list.
Test Plan:
Created a new panel.
{F165468}
Reviewers: chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T4986
Differential Revision: https://secure.phabricator.com/D9500
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.
Test Plan: Eyeballed it.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin, hach-que
Differential Revision: https://secure.phabricator.com/D9431
Summary:
Elasticsearch 1.0 deprecated the "filter" top-level
parameter in favor of "post_filter" which is applied
after scores and so forth are calculated.
Instead search field.corpus with a term query.
Test Plan:
Tested against Elasticsearch 1.1.1, able to perform
basic queries without query parse errors.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Maniphest Tasks: T4446
Differential Revision: https://secure.phabricator.com/D9321
Summary:
Ref T4673.
IMPORTANT: I had to break one thing (see TODO) to get this working. Not sure how you want to deal with that. I might be able to put the element //inside// the workboard, or I could write some JS. But I figured I'd get feedback first.
General areas for improvement:
- It would be nice to give you some feedback that you have a filter applied.
- It would be nice to let you save and quickly select common filters.
- These would probably both be covered by a dropdown menu instead of a button, but that's more JS than I want to sign up for right now.
- Managing custom filters is also a significant amount of extra UI to build.
- Also, maybe these filters should be sticky per-board? Or across all boards? Or have a "make this my default view"? I tend to dislike implicit stickiness.
Test Plan:
Before:
{F157543}
Apply Filter:
{F157544}
Filtered:
{F157545}
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: qgil, swisspol, epriestley
Maniphest Tasks: T4673
Differential Revision: https://secure.phabricator.com/D9211
Summary: Ref T4986. These are mostly mechanical now, I skipped a couple of slightly tricky ones. Still a bunch to go.
Test Plan:
For each engine:
- Viewed the application;
- created a panel to issue the query.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4986
Differential Revision: https://secure.phabricator.com/D9017
Summary:
Ref T4986. This one needs `getApplicationURI()` so make it a little beefier to deal with that.
(It would be vaguely nice to somehow share the handle and application stuff between Controllers and Engine classes like this, but I don't immediately see a clean way to do it without traits. Not a big deal, in any case.)
Test Plan:
- Viewed Calendar.
- Made a Calendar panel.
- Viewed feed.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4986
Differential Revision: https://secure.phabricator.com/D9012
Summary:
Ref T4986. Updates audit.
Slightly tweaks on method visibility.
Just used a HandleQuery since we have to rebuild the whole view thing otherwise; this is an unusual case.
Test Plan:
- Checked Audit.
- Checked Feed.
- Checked Slowvote.
{F151555}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4986
Differential Revision: https://secure.phabricator.com/D9011
Summary:
Ref T4986. This adds a bit of structure for handles, since we used to have Controller utilities but no longer do.
Hopefully these will start going faster soon...
Test Plan:
- Checked feed for collateral damage.
- Checked slowvote for collateral damage.
- Made a slowvote panel.
{F151550}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4986
Differential Revision: https://secure.phabricator.com/D9010
Summary:
Ref T4986. We need to introduce alternate views to make this more pleasant, but let rendering move to engines so it can be shared between panels and controllers.
I also moved some of the pagination logic in to avoid duplicating that.
So far, only Feed works. I'm going to do these gradually since we have ~40-50 of them.
Test Plan:
- Used global search to check for collateral damage.
- Used not-global search too.
- Used normal feed.
{F151541}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: chad, epriestley
Maniphest Tasks: T4986
Differential Revision: https://secure.phabricator.com/D9008
Summary: Ref T4986. This isn't pretty/usable yet (I need to move rendering out of ListController classes and into SearchEngine classes, I think) but does pull the correct results.
Test Plan: {F151537}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4986
Differential Revision: https://secure.phabricator.com/D9007
Summary:
- Point them at the new Diviner.
- Make them a little less cumbersome to write.
Test Plan: Found almost all of these links in the UI and clicked them.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D8553
Summary:
Ref T2222. Currently, Differential has a fairly hairy piece of logic to parse object lists, like `Reviewers: alincoln, htaft`. Extract, generalize, and cover this.
- Some of the logic can be simplified with modern ObjectQuery stuff.
- Make `@username` the formal monogram for users.
- Make `list@domain.com` the formal monogram for mailing lists.
- Add test coverage.
Test Plan:
- Ran unit tests.
- Called `differential.parsecommitmessage` with a bunch of real-world inputs and got sensible results.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2222
Differential Revision: https://secure.phabricator.com/D8445
Summary: Ref T4446. Some discussion in IRC. Prior to hitting the 1.0.0 issue, we hit and resolved this issue; this is a leftover call from bringing ApplicationSearch to main search.
Test Plan: User confirmed this fixes the issue.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4446
Differential Revision: https://secure.phabricator.com/D8260
Summary: Ref T4375. Basic ApplicationSearch integration to power this more flexibly.
Test Plan: {F108762}
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4375
Differential Revision: https://secure.phabricator.com/D8148
Summary:
Ref T4365. Drive primary search through ApplicationSearch instead of through a bunch of custom nonsense. Notably, this allows you to save searches, notably.
The one thing this doesn't do -- which I'd like it to -- is carry your query text across searches. When you search for "quack", I want to overwrite the query in your default filter and give you those results, so you can turn the search into an "Open Tasks" search by default by reordering the queries. I'll probably do that next. It feels a little hacky but I want to try it out.
Test Plan: {F106932}
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran, bigo
Maniphest Tasks: T4365
Differential Revision: https://secure.phabricator.com/D8123
