1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-26 15:30:58 +01:00
Commit graph

422 commits

Author SHA1 Message Date
epriestley
d1f1d3ec33 Implement a basic project.search third-generation API method
Summary: Ref T10010. This still needs support for attachments (to get members) and more constraints (like slugs), but mostly works.

Test Plan: Ran query, saw basically sensible results.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D14889
2015-12-27 09:21:13 -08:00
epriestley
bdc517485c Modernize Hovercard implementation
Summary:
Ref T8980. Move away from events to EngineExtensions.

This also simplifies hovercards a bit:

  - Removes tasks from revision cards.
  - Removes blockers/blocked from task cards.
  - Removes "Send Message" from user cards.

These mostly felt cluttery to me. Open to arguments to retain them. I think we can make better use of the space, though (e.g., flags, projects + board columns).

Test Plan:
  - Viewed people, task, revision, commit and project hovercards.

{F1043256}

{F1043257}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8980

Differential Revision: https://secure.phabricator.com/D14878
2015-12-24 12:18:28 -08:00
epriestley
96fe8c0b83 Implement basic ngram search for Owners Package names
Summary:
Ref T9979. This uses ngrams (specifically, trigrams) to build a reasonably efficient index for substring matching. Specifically, for a package like "Example", with ID 123, we store rows like this:

```
< ex, 123>
<exa, 123>
<xam, 123>
<amp, 123>
<mpl, 123>
<ple, 123>
<le , 123>
```

When the user searches for `exam`, we join this table for packages with tokens `exa` and `xam`. MySQL can do this a lot more efficiently than it can process a `LIKE "%exam%"` query against a huge table.

When the user searches for a one-letter or two-letter string, we only search the beginnings of words. This is probably what they want, the only thing we can do quickly, and a reasonable/expected behavior for typeaheads.

Test Plan:
  - Ran storage upgrades and search indexer.
  - Searched for stuff with "name contains".
  - Used typehaead and got sensible results.
  - Searched for `aabbccddeeffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz` and saw only 16 joins.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14846
2015-12-22 08:00:33 -08:00
epriestley
a761f73384 Allow index extensions to skip indexing if the object has not changed
Summary:
Fixes T9890. This allows IndexExtensions to emit an object version.

Before we build indexes, we check if the indexed version is the same as the current version. If it is, we just don't call that extension.

T9890 has a case where this is useful: a script went crazy and posted thousands of comments to a single task.

Without versioning, that results in the same comments being indexed over and over again. With versioning, most of the queue could just exit without doing any work.

Test Plan:
  - Added a `sleep(1)` to the actual indexing, used `bin/search index --background` to queue up a lot of tasks, ran them with `bin/phd debug task`, saw them complete very quickly with only one actual index operation performed.
  - Used `bin/search index --trace` and `bin/search index --trace --background` to observe the behavior of queries against the index version store, which looked sensible.
  - Made comments/transactions, saw versions update.
  - Used `bin/remove destroy`, verified index versions were purged.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9890

Differential Revision: https://secure.phabricator.com/D14845
2015-12-21 17:27:14 -08:00
epriestley
23c42486e4 Rename "SearchEngine" to "FulltextStorageEngine"
Summary:
Ref T9979. I picked this name long before the advent of modern "Engine" architecture and it ended up being pretty confusing.

Rename "SearchEngine" (currently: mysql or elasticsearch, used to store and query fulltext indexes) to "FulltextStorageEngine" to make it more clear what it does and disambituate it from ApplicationSearch, which also has a bunch of stuff called "SearchEngine", "SearchEngineExtension", etc.

Test Plan: Grepped for `phabricatorsearchengine`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14843
2015-12-21 17:26:19 -08:00
epriestley
99c9df96b4 Convert all "DocumentIndexers" into "FulltextEngines"
Summary: Ref T9979. This simplifies/standardizes the code a bit, but mostly gives us more consistent class names and structure.

Test Plan:
  - Used `bin/search index --type ...` to index documents of every indexable type.
  - Searched for documents by unique text, found them.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14842
2015-12-21 17:25:23 -08:00
epriestley
99bd12b98d Lift Conpherence indexing up out of the Fulltext index
Summary:
Ref T9979. There are currently some hacks around Conpherence indexing: it does not really use the fulltext index, but its own specialized index. However, it's kind of hacked up so it can get reindexed by the normal indexing pipeline.

Lift it up into IndexEngine, instead of FulltextEngine. Specifically, the new stuff is going to look like this:

  - IndexEngine: Rebuild all indexes.
    - ConpherenceIndexExtension: Rebuild thread indexes.
    - ProjectMemberIndexExtension: Rebuild project membership views.
    - NgramIndexExtension: Rebuild ngram indexes.
    - FulltextIndexExtension / FulltextEngine: Rebuild fulltext indexes, a special type of index.
      - FulltextCommentExtension: Rebuild comment fulltext indexes.
      - FulltextProjectExtension: Rebuild project fulltext indexes.
      - etc.

Most of this is at least sort-of-in-place as of this diff, although some of the part in the middle is still pretty rough.

Test Plan:
  - Made a unique comment in a Conpherence thread.
  - Used `bin/search index --force` to rebuild the index.
  - Searched for the comment.
  - Found the thread.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14841
2015-12-21 17:25:05 -08:00
epriestley
ecc3314a25 Modularize transaction/comment indexing in the FulltextEngine
Summary:
Ref T9979. This is currently hard-coded but can be done in a generic way.

This has one minor behavioral changes: answer text is no longer included in the question text index in Ponder. I'm not planning to accommodate that for now since I don't want to dig this hole any deeper than I already have. This behavior should be different anyway (e.g., index the answer, then show the question in the results or something).

Test Plan:
  - Put a unique word in a Maniphest comment.
  - Searched for the word.
  - Found the task.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14837
2015-12-21 17:24:40 -08:00
epriestley
aab1574e33 Remove TYPE_SEARCH_DIDUPDATEINDEX event
Summary:
Ref T9979. This event had one weird callsite and no known third-party callers. It can be done more cleanly as an extension, now.

This index is used to allow us to "Group By: Project" in Maniphest without joining into the Projects database.

Test Plan:
  - Ran a query with "Group By: Project" in Maniphest.
  - Renamed project "Apples" to "Zebras".
  - Reloaded page.
  - UI properly moved "Zebras" tasks to the bottom of the list.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14836
2015-12-21 17:23:59 -08:00
epriestley
02f82c2af5 Modularize fulltext indexing of Projects, Subscriptions and Custom Fields
Summary: Ref T9979. This is going to become `FulltextEngine`, but pave the way for that by pulling extensions out of it.

Test Plan:
{F1036624}

  - Used `bin/search index Txxx`, saw projects, subscribers and custom fields rebuild in the index.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14835
2015-12-21 17:04:25 -08:00
epriestley
2447d9bdf2 Begin improving modularity of IndexEngine, add locks
Summary:
Ref T9890. Ref T9979. Several adjacent goals:

  - The `SearchEngine` vs `ApplicationSearchEngine` thing is really confusing. There are also a bunch of confusing class names and class relationships within the fulltext indexing. I want to rename these classes to be more standard (`IndexEngine`, `IndexEngineExtension`, etc). Rename `SearchIndexer` to `IndexEngine`. A future change will rename `SearchEngine`.
  - Add the index locks described in T9890.
  - Structure things a little more normally so future diffs can do the "EngineExtension" thing more cleanly.

Test Plan:
Indexing:

  - Renamed a task to have a unique word in the title.
  - Ran `bin/search index Txxx`.
  - Searched for unique word.
  - Found task.

Locking:

  - Added a `sleep(10)` after the `lock()` call.
  - Ran `bin/search index Txxx` in two windows.
  - Saw first one lock, sleep 10 seconds, index.
  - Saw second one give up temporarily after failing to grab the lock.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9890, T9979

Differential Revision: https://secure.phabricator.com/D14834
2015-12-21 17:04:10 -08:00
epriestley
674388ce6a Prepare DestructionEngine to be modularized
Summary:
Ref T9979. The general shape of "engine" code feels pretty good, and I plan to move indexing to be more in line with other modern engines, with the ultimate goal of supporting subprojects (T10010) and several intermediate goals.

Before moving indexing, clean up Destruction, since some of the new indexes will need destruction hooks and destruction currently has a lot of `instanceof` stuff that should be easy to fix by applying more modern approaches.

Test Plan:
  - Used `bin/remove destroy` to destory an Almanac device.
  - Verified that properties for the device were destroyed.
  - Viewed module panel in UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14831
2015-12-21 17:03:32 -08:00
Chad Little
c6f3a03209 Basic NUX blank states
Summary: Implement in Badges

Test Plan:
Test with nux=true.

{F1033431}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: johnny-bit, Korvin

Differential Revision: https://secure.phabricator.com/D14833
2015-12-21 07:39:46 -08:00
epriestley
dbb84f1ddc Add basic NUX support to SearchEngines
Summary:
This is just putting a hook in that pretty much works. Behavior:

  - If you visit `/maniphest/?nux=true`, it always shows NUX for testing.
  - Otherwise, it shows NUX if there are no objects in the application yet.

Test Plan: {F1031846}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14828
2015-12-19 12:57:13 -08:00
epriestley
57cc30d0c4 Continue hammering new *.search / *.edit documentation into shape
Summary: Ref T9964. Create some docuemntation for this stuff, and clean up the *.edit endpoints a bit.

Test Plan: Read documentation.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14798
2015-12-16 08:46:05 -08:00
epriestley
0a50219f1b Formalize custom Conduit fields on objects
Summary: Ref T9964. This just adds more structure to application fields, to make it harder to make typos and easier to validate them later.

Test Plan: Viewed APIs, called some APIs, saw good documentation and correct results.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14776
2015-12-14 11:54:13 -08:00
epriestley
3db175f79d Add a "content" attachment for Pastes for Conduit API
Summary: Ref T9964. Builds on D14772. Allows callers to get the raw content of pastes as an attachment.

Test Plan:
  - Read docs.
  - Executed attachment query.
  - Saw raw paste content.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14774
2015-12-14 11:53:32 -08:00
epriestley
2160c45619 Implement an "Attachments" behavior for Conduit Search APIs
Summary:
Ref T9964. We have various kinds of secondary data on objects (like subscribers, projects, paste content, Owners paths, file attachments, etc) which is somewhat slow, or somewhat large, or both.

Some approaches to handling this in the API include:

  - Always return all of it (very easy, but slow).
  - Require users to make separate API calls to get each piece of data (very simple, but inefficient and really cumbersome to use).
  - Implement a hierarchical query language like GraphQL (powerful, but very complex).
  - Kind of mix-and-match a half-power query language and some extra calls? (fairly simple, not too terrible?)

We currently mix-and-match internally, with `->needStuff(true)`. This is not a general-purpose, full-power graph query language like GraphQL, and it occasionally does limit us.

For example, there is no way to do this sort of thing:

  $conpherence_thread_query = id(new ConpherenceThreadQuery())
    ->setViewer($viewer)
    // ...
    ->setNeedMessages(true)
    ->setWhenYouLoadTheMessagesTheyNeedProfilePictures(true);

However, we almost never actually need to do this and when we do want to do it we usually don't //really// want to do it, so I don't think this is a major limit to the practical power of the system for the kinds of things we really want to do with it.

Put another way, we have a lot of 1-level hierarchical queries (get pictures or repositories or projects or files or content for these objects) but few-to-no 2+ level queries (get files for these objects, then get all the projects for those files).

So even though 1-level hierarchies are not a beautiful, general-purpose, fully-abstract system, they've worked well so far in practice and I'm comfortable moving forward with them in the API.

If we do need N-level queries in the future, there is no technical reason we can't put GraphQL (or something similar) on top of this eventually, and this would represent a solid step toward that. However, I suspect we'll never need them.

Upshot: I'm pretty happy with "->needX()" for all practical purposes, so this is just adding a way to say "->needX()" to the API.

Specifically, you say:

```
{
  "attachments": {
    "subscribers": true,
  }
}
```

...and get back subscriber data. In the future (or for certain attachments), `true` might become a dictionary of extra parameters, if necessary, and could do so without breaking the API.

Test Plan:
- Ran queries to get attachments.

{F1025449}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14772
2015-12-14 11:53:00 -08:00
epriestley
8ec413b972 Clean up "ids" and "phids" handling in SearchEngines
Summary:
Ref T9964. I added several hacks to get these working. Clean them up and pull this into a proper extension.

The behavior in the web UI is:

  - they work in all applications; but
  - they only show up in the UI if a value is specified.

So if you visit `/view/?ids=1,2` you get the field, but normally it's not present. We could refine this later. I'm going to add documentation about how to prefill these forms regardless, which should make this discoverable by reading the documentation.

There's one teensey weensey hack: in the API, I push these fields to the top of the table. That one feels OK, since it's purely a convenience/display adjustment.

Test Plan: Queried by IDs, reviewed docs.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14769
2015-12-14 04:24:54 -08:00
epriestley
fdd2d802d2 Clean up "*.search" API method documentation pages
Summary:
Ref T9964. Building tables in Remarkup is kind of neat-ish but ends up feeling kind of hacky, and requires weird workarounds if any of the values have `|` in them.

Switch to normal elements instead.

Also move the magic "ids" and "phids" to be more like real fields. I'll clean this up fully in a diff or two, it's just a little tricky because Maniphest has an "ids" field.

Test Plan: {F1024294}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14768
2015-12-14 04:24:39 -08:00
epriestley
99ade500bc Flesh out Conduit parmeter types for maniphest.search
Summary: Ref T9964. I left a couple of these unsupported for now since they're weird in some way.

Test Plan: {F1024031}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14767
2015-12-14 04:24:01 -08:00
epriestley
663dce5029 Flesh out Conduit parameter types for Owners + CustomFields
Summary:
Ref T9964. Fill in more parameter types and descriptions.

(No date support yet since it's a bit more involved.)

Test Plan: {F1024022}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14766
2015-12-14 04:23:44 -08:00
epriestley
0282ce74ab Flesh out Conduit types for Paste search fields
Summary: Ref T9964. This fills in types and descriptions for ApplicationSearch fields in Paste.

Test Plan:
Got this nice table now:

{F1023999}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14765
2015-12-14 04:23:28 -08:00
epriestley
1b325a0a89 Modularize SearchEngine extensions
Summary:
Ref T9964. ApplicationSearch currently has a bunch of hard-coded `if ($object instanceof thing)` stuff.

Pull that out so it can live in extensions.

Test Plan:
 - Searched by spaces, subscribers, projects.

{F1023921}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14764
2015-12-14 04:23:02 -08:00
epriestley
05a798e3ac Add basic typechecking support to Conduit
Summary:
Ref T9964. I want to show users what we're expecting in "constraints", and let constraints like "authors=epriestley" work to make things easier.

I'm generally very happy with the "HTTPParameterType" stuff from EditEngine, so add a parallel set of "ConduitParameterType" classes. These are a little simpler than the HTTP ones, but have a little more validation logic.

Test Plan:
This is really just a proof of concept; some of these fields are now filled in:

{F1023845}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14763
2015-12-14 04:21:39 -08:00
epriestley
d1a1d48001 Give ConduitAPIMethod->getMethodDescription() access to a real Viewer
Summary:
Ref T9964. The new `*.search` and `*.edit` methods generate documentation which depends on the viewer.

For example, the `*.search` methods show a reference table of the keys for all your saved queries.

Give them a real viewer to work with.

During normal execution, just populate this viewer with the request's viewer, so `$request->getViewer()` and `$this->getViewer()` both work and mean the same thing.

Test Plan: {F1023780}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14761
2015-12-14 04:20:11 -08:00
epriestley
9499987cfe Add "owners.search" Conduit API endpoint, with CustomField support
Summary:
Ref T9964. Adds a new-style "owners.search" endpoint, and an extension for customfields.

Puts enough indirection in place to give us nice, consistent "custom.key" user-facing keys instead of "std:custom:owners:na0shf9a8dfdsafl" junk.

Test Plan:
  - Searched Owners via API.
  - Searched by ID.
  - Ordered by custom fields.
  - Reviewed API docs.
  - Used normal search with ordering.
  - Viewed custom field values in search results.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9964

Differential Revision: https://secure.phabricator.com/D14758
2015-12-13 02:11:59 -08:00
epriestley
4ec6990ca7 Implement a rough initial version of ApplicationSearch-driven Conduit read endpoints
Summary:
Ref T9964. See that task for some context and discussion.

Ref T7715, which has the bigger picture here.

Basically, I want Conduit read endpoints to be full-power, ApplicationSearch-driven endpoints, so that applications can:

  - Write one EditEngine and get web + conduit writes for free.
  - Write one SearchEngine and get web + conduit reads for free.

I previously made some steps toward this, but this puts more of the structure in place.

Test Plan:
Viewed API console endpoint and read 20 pages of docs:

{F1021961}

Made various calls: with query keys, constraints, pagination, and limits.

Viewed new {nav Config > Modules} page.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7715, T9964

Differential Revision: https://secure.phabricator.com/D14743
2015-12-11 15:27:06 -08:00
epriestley
bce83bf844 Delete old Maniphest edit controller
Summary:
Ref T9908. No more callsites. Also:

  - Phurl a couple of documentation URIs.
  - Get rid of "task:" in the global search since it doesn't really make sense anymore.

Test Plan: `grep`, edited/created tasks.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9908

Differential Revision: https://secure.phabricator.com/D14716
2015-12-08 17:56:31 -08:00
epriestley
eef2572508 Replace workboard task creation with EditEngine
Summary: Ref T9908. This is the last of the things that need to swap over.

Test Plan:
  - Created tasks from a workboard.
  - Created tasks in different columns.
  - Edited tasks.
  - Used `?parent=..`.
  - Verified that default edit form config now affects comment actions.
  - No more weird comment thing on forms, at least for now.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9908

Differential Revision: https://secure.phabricator.com/D14715
2015-12-08 17:56:11 -08:00
epriestley
59ae0d6fff Allow EditEngine create and edit forms to be reordered
Summary:
Ref T9132. Ref T9908. Puts reordering UI in place:

  - For create forms, this just lets you pick a UI display order other than alphabetical. Seems nice to have.
  - For edit forms, this lets you create a hierarchy of advanced-to-basic forms and give them different visibility policies, if you want.

Test Plan:
{F1017842}

  - Verified that "Edit Thing" now takes me to the highest-ranked edit form.
  - Verified that create menu and quick create menu reflect application order.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9132, T9908

Differential Revision: https://secure.phabricator.com/D14704
2015-12-08 13:00:54 -08:00
epriestley
1b00ef08a0 Remove some low-hanging buildStandardPageResponse() methods
Summary: Ref T9690. I wanted to do an example of how to do these but it looks like most of them are trivial (no callsites) and the rest are a little tricky (weird interaction with frames, or in Releeph).

Test Plan:
  - Used `grep` to look for callsites.
  - Hit all applications locally, everything worked.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9690

Differential Revision: https://secure.phabricator.com/D14385
2015-11-03 10:11:36 -08:00
epriestley
300c74c49d Make mobile navigation work properly by default in more cases
Summary:
Fixes T5752. This obsoletes a bunch of old patterns and I'll follow up on those with a big "go do a bunch of mechanical code changes" task. Major goals are:

  - Don't load named queries multiple times on search pages.
  - Don't require extra code to get standard navigation right on mobile.
  - Reduce the amount of boilerplate in ListControllers.
  - Reduce the amount of boilerplate around navigation/menus in all controllers.

Specifically, here's what this does:

  - The StandardPage is now a smarter/more structured object with `setNavigation()` and `setCrumbs()` methods. More rendering decisions are delayed until the last possible moment.
    - It uses this to automatically add crumb actions to the application menu.
    - It uses this to automatically reuse one SearchEngine instead of running queries multiple times.
  - The new preferred way to build responses is `$this->newPage()` (like `$this->newDialog()`), which has structured methods for adding stuff (`setTitle()`, etc).
  - SearchEngine exposes a new convenience method so you don't have to do all the controller delegation stuff.
  - Building menus is generally simpler.

Test Plan:
  - Tested paste list, view, edit, comment, raw controllers for functionality, mobile menu, crumbs, navigation menu.
  - Edited saved queries.
  - Tested Differential, Maniphest (no changes).
  - Verified the paste pages don't run any duplicate NamedQuery queries.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5752

Differential Revision: https://secure.phabricator.com/D14382
2015-11-03 10:11:24 -08:00
epriestley
4e112537b2 Probable fix for ElasticSearch 2.0 type strictness
Summary: Fixes T9670.

Test Plan: Will follow up on task.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9670

Differential Revision: https://secure.phabricator.com/D14370
2015-11-02 16:21:43 +00:00
Chad Little
09ab82faef Update Search for handleRequest
Summary: Ref T8628. Updates Search.

Test Plan: Did various searches, saved new queries, reordered, ran new queries.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T8628

Differential Revision: https://secure.phabricator.com/D14268
2015-10-20 09:02:55 -07:00
epriestley
24845c70b9 Refine error behavior of bin/search index
Summary: Fixes T5991. If //all requested documents// failed to index, consider this a catastrophic failure and exit with an error code.

Test Plan:
  - Ran `bin/search index --type TASK`, observed successful exit despite a small number of un-indexable documents.
  - Ran `bin/search index PHID-TASK-xxx` for an invalid task, observed exception on exit after complete failure.
  - Ran normal indexing through daemons.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5991

Differential Revision: https://secure.phabricator.com/D14174
2015-09-27 13:11:11 -07:00
epriestley
1c45a7d8e2 Revert "Allow search results to be snippeted, roughly"
Summary:
This reverts commit 1583738842.

See T8646 for discussion. This version of the feature feels terrible on real data.

Test Plan: Strict revert.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14097
2015-09-10 20:57:26 -07:00
epriestley
1583738842 Allow search results to be snippeted, roughly
Summary:
Ref T8646. This is fairly rough:

This interface is very niche, and not really flexible enough to accommodate other result customization (but I don't think we have any plans here)?

I'm just //summarizing// the content of documents, basically showing the first paragraph of their content, summary, etc. This isn't what Google does: it shows snippets surrounding the actual search terms. However, this is more involved and might be less useful in structured data: for example, I'd imagine that the first line of most phriciton documents, maniphest tasks and Differential revisions really might be the best machine-generatable summary of them. The actual contextual snippeting in Google doesn't often seem hugely useful to me. But this might also not be very useful.

There's not much design, not sure if you had any ideas.

I only implemented this for tasks, revisions and the wiki since those seem most useful.

I'm generally on the fence about this, but it's not a ton of work to swap out for something else later. Maybe we can see how it feels? But happy to toss it or rethink the approach.

Test Plan: {F788026}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8646

Differential Revision: https://secure.phabricator.com/D14095
2015-09-10 19:06:36 -07:00
Joshua Spence
368f359114 Use PhutilClassMapQuery instead of PhutilSymbolLoader
Summary: Use `PhutilClassMaQuery` instead of `PhutilSymbolLoader`, mostly for consistency. Depends on D13588.

Test Plan: Poked around a bunch of pages.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13589
2015-08-14 07:49:01 +10:00
Joshua Spence
2cf9ded878 Various linter fixes
Summary: Self explanatory.

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D13863
2015-08-11 22:36:55 +10:00
epriestley
ae81f86a67 Fix some Releeph jankiness in interacting with the redesign
Summary: Releeph does some really old sketchy stuff in result rendering. Modernize it, remove the holdover/oldschool interface (these were the last two implementors) and make errors a little softer.

Test Plan: Viewed Releeph products and branches.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D13604
2015-07-08 08:10:58 -07:00
epriestley
1a001ca033 Render ApplicationSearch errors correctly
Summary:
Fixes T8774. When an ApplicationSearch page returned an error (e.g., using a "viewer()" query while logged out), we would try to add action links to a box without a header. Instead:

  - If a box has no header, but has show/hide actions, just create an empty header so the view renders. This is a little silly looking but does what the caller asks.
  - Always set the title on the result box, so we get a header.

Test Plan: Did an invalid (logged out, with viewer()) search. Did a valid search.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T8774

Differential Revision: https://secure.phabricator.com/D13576
2015-07-07 11:52:06 -07:00
Joshua Spence
f695dcea9e Use PhutilClassMapQuery
Summary: Use `PhutilClassMapQuery` where appropriate.

Test Plan: Browsed around the UI to verify things seemed somewhat working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13429
2015-07-07 22:51:57 +10:00
Joshua Spence
f2435fd1d0 Return $this from setter methods
Summary: Return `$this` from setter methods for consistency. I started writing a [[https://secure.phabricator.com/differential/diff/32506/ | linter rule]] to detect this, but I don't think it is trivial to do this properly.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13422
2015-07-06 22:53:43 +10:00
Joshua Spence
acb1eb81cc Move some PhabricatorSearchField subclasses
Summary: Move some `PhabricatorSearchField` subclasses to be adjacent to the application to which they belong. This seems generally better to me than lumping them all together in the `src/applications/search/field/` directory. I was also wondering if it makes sense to rename these subclasses as `PhabricatorXSearchField` rather than `PhabricatorSearchXField` (as per T5655), but wasn't really sure if these objects are meant to be search-fields, or just fields belonging to the #search application.

Test Plan: N/A.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13374
2015-07-06 22:52:05 +10:00
Chad Little
08ec0f1704 [Redesign] Remove AphrontTable restriction on Search Engine
Summary: Ref T8099, functionally I prefer to be able to set anything 'table-like' with `setTable` for design consistency. This looses the restriction and did some light grepping for other missed cases.

Test Plan: Test new UI, grep for other missing cases.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8099

Differential Revision: https://secure.phabricator.com/D13471
2015-06-29 12:28:55 -07:00
epriestley
729606ba93 Update BulkJob and MetaMTA search engines for redesign-2015 2015-06-23 13:39:27 -07:00
epriestley
1ccdb941b9 Merge branch 'master' into redesign-2015 2015-06-22 15:52:40 -07:00
Chad Little
dbdeca58fc [Redesign] Fix People Log Query
Summary: Fixes T8642, This is a table, but not returned as one. Set it to Content for now with a Collapsed layout.

Test Plan: Test /people/logs/

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8642

Differential Revision: https://secure.phabricator.com/D13395
2015-06-22 22:10:07 +01:00
lkassianik
c3efa261f9 Refactor Calendar Search, and implement Projects on events
Summary: Ref T7950, Refactor Calendar Search, and implement Projects on events

Test Plan: Verify that all queries in Calendar search still work, and that events can now have associated Projects that you can search by in Calendar Search.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T7950

Differential Revision: https://secure.phabricator.com/D13393
2015-06-22 13:27:37 -07:00