Summary:
Ref T6740. Currently, we issue fulltext queries with an "ORDER BY <score>" on the entire result set.
For very large result sets, this can require MySQL to do a lot of work. However, this work is generally useless: if you search for some common word like "diff" or "internet" or whatever and match 4,000 documents, the chance that we can score whatever thing you were thinking of at the top of the result set is nearly nothing. It's more useful to return quickly, and let the user see that they need to narrow their query to get useful results.
Instead of doing all that work, let MySQL find up to 1,000 results, then pick the best ones out of those.
This actual change is a little flimsy, since our index isn't really big enough to suffer indexing issues. However, searching for common terms on my local install (where I have some large repositories imported and indexed) drops from ~40ms to ~10ms.
My hope is to improve downstream performance for queries like "translatewiki" here, particularly:
<https://phabricator.wikimedia.org/T143863>
That query matches about 300 trillion documents but there's a ~0% chance that the one the user wants is at the top. It takes a couple of seconds to execute, for me. Better to return quickly and let the user refine their results.
I think this will also make some other changes related to stemming easier.
This also removes the "list users first" ordering on the query, which made performance more complicated and seems irrelevant now that we have the typeahead.
Test Plan:
- Searched for some common terms like "code" locally, saw similar results with better performance.
- Searched for useful queries (e.g., small result set), got identical results.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T6740
Differential Revision: https://secure.phabricator.com/D16944
Summary:
Ref T11741. This makes everything work if we switch to InnoDB, but never actually switches yet.
Since the default minimum word length (3) and stopword list (36 common English words) in InnoDB are generally pretty reasonable, I just didn't add any setup advice for them. I figure we're better off with simpler setup until we identify some real problem that the builtin stopwords create.
Test Plan: Swapped the `false` to `true`, ran `storage adjust`, got InnoDB fulltext indexes, searched for stuff, got default "AND" behavior.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11741
Differential Revision: https://secure.phabricator.com/D16942
Summary:
Ref T11741. InnoDB uses a stopwords table instead of a stopwords file.
During `storage upgrade`, synchronize the table from the stopwords file on disk.
Test Plan:
- Ran `storage upgrade`.
- Ran `select * from stopwords`, saw stopwords.
- Added some garbage to the table.
- Ran `storage upgrade`, saw it remove it.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11741
Differential Revision: https://secure.phabricator.com/D16940
Summary:
Ref T11741. Fixes T10642. Parse and compile user queries with a consistent ruleset, then submit queries to the backend using whatever ruleset MySQL is configured with.
This means that `ft_boolean_syntax` no longer needs to be configured (we'll just do the right thing in all cases).
This should improve behavior with RDS immediately (T10642), and allow us to improve behavior with InnoDB in the future (T11741).
Test Plan:
- Ran various queries in the UI, saw the expected results.
- Ran bad queries, got useful errors.
- Searched threads in Conpherence.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10642, T11741
Differential Revision: https://secure.phabricator.com/D16939
Summary:
This has been replaced by `PolicyCodex` after D16830. Also:
- Rebuild Celerity map to fix grumpy unit test.
- Fix one issue on the policy exception workflow to accommodate the new code.
Test Plan:
- `arc unit --everything`
- Viewed policy explanations.
- Viewed policy errors.
Reviewers: chad
Reviewed By: chad
Subscribers: hach-que, PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D16831
Summary: Ref T11773. Not committed to this implementation, but adds some "Developer" query actions to jump to the nux/overheated states without needing to know secret magic URL variables.
Test Plan: {F1878984}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11773
Differential Revision: https://secure.phabricator.com/D16736
Summary: Ref T11773. This is an initial first step toward a more complete solution, but should make the worst case much less bad: prior to this change, the worst case was "30 second exeuction timeout". After this patch, the worst case is "no results + explanatory message", which is strictly better.
Test Plan:
Made all feed stories fail policy checks, loaded home page.
- Before adding overheating: 9,600 queries / 20 seconds
- After adding overheating: 376 queries / 800ms
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11773
Differential Revision: https://secure.phabricator.com/D16735
Summary:
Ref T10747. When a user drops a ".ics" file or a bunch of ".ics" files into a calendar view, import the events.
(Possibly we should just do this if you drop ".ics" files into any application, but we can look at that later.)
Test Plan: Dropped some .ics files into calendar views, got imports.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10747
Differential Revision: https://secure.phabricator.com/D16722
Summary: Accept Conduit parameter values as strings (e.g. from `curl`) and convert to required type.
Test Plan:
Call conduit method with int/bool parameter iusing `curl` and make sure it does not result in validation error, e.g.
```
$ curl http://$PHABRICATOR_HOST/api/maniphest.search -d api.token=$CONDUIT_TOKEN -d constraints[modifiedEnd]=$(date +%s) -d constraints[hasParents]=true -d limit=1
```
Fixes T10456.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T10456
Differential Revision: https://secure.phabricator.com/D16694
Summary: Background is now always white, spacing in header is more consistent
Test Plan: test mobile, table, desktop application search apps.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16691
Summary:
Ref T10747.
- Adds a "Use Results..." dropdown to query result pages, with actions you can take with search results (today: create export; in future: bulk edit, export as excel, make dashboard panel, etc).
- Allows you to create an export against a query key.
- I'm just using a text edit field for this for now.
- Fleshes out export modes. I plan to support: public (as though you were logged out), privileged (as though you were logged in) and availability (event times, but not details).
This does not actually export stuff yet.
Test Plan: Created some exports. Viewed and listed exports.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10747
Differential Revision: https://secure.phabricator.com/D16676
Summary: Creates a background that renders inside the Quicksand frame, through sorcery.
Test Plan: Turn on Quicksand, visit lots of pages. See correct background colors. This probably blows something up I'm not testing.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16642
Summary:
Ref T11593. When you call a `*.search` method like `maniphest.search`, we don't currently validate that all the constraints you pass are recognized.
I think there were two very weak arguments for not doing this:
- It makes compatibility in `arc` across versions slightly easier: if we add a new constraint, we could add it to `arc` but also do client-side filtering for a while.
- Conduit parameter types //could//, in theory, accept multiple inputs or optional/alias inputs.
These reasons are pretty fluff and T11593 is a concrete issue caused by not validating. Just validate instead.
Test Plan:
- Made a `maniphest.search` call with a bogus constraint, got an explicit error about the bad constraint.
- Made a `maniphest.search` call with a valid constraint (`"ids"`).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11593
Differential Revision: https://secure.phabricator.com/D16507
Summary: Previously we collapsed all table search results, but the new UI doesn't need it. Remove unused methods and fix CSS.
Test Plan: Legalpad Signatures, Phortune Accounts.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16469
Summary: Ref T11132, significantly cleans up the Config app, new layout, icons, spacing, etc. Some minor todos around re-designing "issues", mobile support, and maybe another pass at actual Group pages.
Test Plan: Visit and test every page in the config app, set new items, resolve setup issues, etc.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam, Korvin
Maniphest Tasks: T11132
Differential Revision: https://secure.phabricator.com/D16468
Summary: Ref T10951. This adds the application name as an attribute below the document type in the UI for doc type search.
Test Plan: Verify that the application name appears as an attribute on the document type results.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Maniphest Tasks: T10951
Differential Revision: https://secure.phabricator.com/D16446
Summary: Ref T10951. This diff removes uninstalled applications from the result set for DocumentType restults
Test Plan: Uninstall an application (diviner for example), then go to the document type search menu and ensure that the uninstalled application doesn't show up.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley
Maniphest Tasks: T10951
Differential Revision: https://secure.phabricator.com/D16445
Summary: Added a brand new shiny cat fact
Test Plan: Pulled up a project with motivator installed and nothing broke
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D16430
Summary: Switches over to new property UI boxes, splits core and apps into separate pages. Move Versions into "All Settings". I think there is some docs I likely need to update here as well.
Test Plan: Click on each item in the sidebar, see new headers.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16429
Summary:
Ref T11473. If you write a method like `get_stuff(ids)` and then call it with an empty list of IDs, you can end up passing an empty constraint to Conduit.
If you run a `*.search` method with such a constraint, like this one:
```
{
"ids": []
}
```
...we have three possible beahviors:
# Treat it like the user passed no constraint (basically, ignore the constraint).
# Respect the constraint (return no results).
# Error.
Currently, we do (1). However, this is pretty confusing and I think clearly the worst option, since it means `get_stuff(array())` in client code will often tend to return a ton of results.
We could do (2) instead, but this is also sort of confusing (it may not be obvious why nothing matched, even though it's an application bug) and I think most reasonable client code should be doing an `if ($ids)` test: this test makes clients a little more complicated, but they can save a network call, and I think they often need to do this test anyway (for example, to show the user a different message).
This implements (3), and just considers these to be errors: this is the least tricky behavior, it's consistent with what we do in PHP, makes fairly good sense, and the only cost for this is that client code may need to be slightly more complex, but this slightly more complex code is usually better code.
Test Plan: Ran Conduit `*.search` queries with `"ids":[]` and `"phids":[]`, got sensible errors instead of runaway result sets.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11473
Differential Revision: https://secure.phabricator.com/D16396
Summary: I find this fact very useful for understanding my feline companion
Test Plan: Added "Motivator: Cat Facts" to the project, nothing broke
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D16374
Summary: This moves aphront-side-nav to use same table css display as profile nav. Slightly less code to support. Cleans up AppSearch UI, think I've gotten all the edge cases here, but bang on it, can hold until after release cut.
Test Plan: Config, Maniphest, Differential, Diffusion, Home.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16346
Summary:
Ref T11404. Currently, SearchEngineAttachments can bulk-load data but SearchEngineExtensions can not.
This leads to poor performance of custom fields. See T11404 for discussion.
This changes the API to support a bulk load + format pattern like the one Attachments use. The next change will use it to bulk-load custom field data.
Test Plan:
- Ran `differential.query`, `differential.revision.search` as a sanity check.
- No behavioral changes are expected
- See next revision.
Reviewers: yelirekim, chad
Reviewed By: chad
Maniphest Tasks: T11404
Differential Revision: https://secure.phabricator.com/D16350
Summary:
Ref T11326. This isn't perfect, but should be a little easier to use and less weird/confusing.
Generally, provide a "Query > Month > Day" crumb on day views, and a "Wed, July 3" header.
Generally, provide a "Query > Month" crumb on month views, and a "July 2019" header.
Also try to fix a bit of padding/spacing on the day view.
Test Plan: {F1739128}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11326
Differential Revision: https://secure.phabricator.com/D16338
Summary:
Ref T11326. This doesn't go quite as far as the mock in T11326#185932, but gets rid of the easy margins.
Also cleans up some of the border rules so they're simpler and more consistent (no weird ragged edges on the far right).
Test Plan: {F1738951}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11326
Differential Revision: https://secure.phabricator.com/D16335
Summary:
Fixes T11375. Some validation code was mishandling raw epoch timestamps.
For numeric values larger than 29999999 (e.g., 2999-12-25, christmas 2999), assume the value is a timestamp.
Test Plan: Used `maniphest.search` to query for `modifiedStart`, got a better result set and saw the `dateModified` constraint in the query.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11375
Differential Revision: https://secure.phabricator.com/D16326
Summary:
Ref T11326. These are last-generation and neither of these have callsites anymore.
(I nuked these since I'm trying to simplify date handling.)
Test Plan: Grepped for callsites.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11326
Differential Revision: https://secure.phabricator.com/D16301
Summary:
Fixes T11304. Prior to this change, we did an unnecessary write on every "*.search" call (this write didn't always actually write a row, since we only save //unique// saved queries, but still doesn't do anything useful ever, currently).
Instead, change this to not-write by default. We could add an "oh, and also I want you to do a write" option later, which would let us implement something like `arc query-stuff` which says "To see more results, view this URI in your browser: ...".
(It's possible to run one of these methods with an existing SavedQuery by using the key, so we still sometimes have a queryKey to return.)
Test Plan: Ran `almanac.service.search`, used DarkConsole to verify that no serachengine writes occurred.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11304
Differential Revision: https://secure.phabricator.com/D16263
Summary:
Fixes T11285. We can end up loading panel handles while processing edits (e.g., disabling menu items on a project). We probably started loading these after the modular transaction changes in T9789, which load the handle for the transaction object unconditionally.
The handles aren't too useful, but they currently fail to load/build because panels don't have a URI. We could give them some sort of method here, but just nuke it for now since they don't appear anywhere and this unclogs the daemon queue.
Test Plan:
- Disabled a menu item on a project.
- Ran publish task with `bin/worker execute --id <id>`.
- Before patch: fatal on getURI() with stack trace similar to T11285.
- After patch: clean execution.
Reviewers: chad, avivey
Reviewed By: avivey
Maniphest Tasks: T11285
Differential Revision: https://secure.phabricator.com/D16249
Summary: Ref T4788. This seems reasonable locally, but not sure how it will feel on real data. Might need some tweaks, or might just be a terrible idea.
Test Plan: {F1708059}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4788
Differential Revision: https://secure.phabricator.com/D16214
Summary:
Ref T4788. When closing a task as a duplicate of another task, you can only select one task, since it doesn't really make sense to merge one task into several other tasks (this operation is //possible//, but probably not what anyone ever wants to do, I think?).
Make the UI understand this: after you select a task, disable all of the "select" buttons in the UI to make this clear.
Test Plan:
- Used "Close as Duplicate", only allowed to select 1 task.
- Used other editors like "Merge Duplicates In", allowed to select lots of tasks.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4788
Differential Revision: https://secure.phabricator.com/D16203
Summary:
Ref T4788. Fixes T10703.
In the longer term I want to put this on top of ApplicationSearch, but that's somewhat complex and we're at a fairly good point to pause this feature for feedback.
Inch toward that instead: provide more appropriate filters and defaults without rebuilding the underlying engine. Specifically:
- No "assigned" for commits (barely makes sense).
- No "assigned" for mocks (does not make sense).
- Default to "open" for parent tasks, subtasks, close as duplicate, and merge into.
Also, add a key to the `search_document` table to improve the performance of the "all open stuff of type X" query. "All Open Tasks" is about 100x faster on my machine with this key.
Test Plan:
- Clicked all object relationships, saw more sensible filters and defaults.
- Saw "open" query about 100x faster locally (300ms to 3ms).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4788, T10703
Differential Revision: https://secure.phabricator.com/D16202
Summary: Fixes T11240. Also simplify things a little and share a bit more code.
Test Plan:
- Viewed revisions and tasks, opened submenu.
- Viewed as a user without edit permission, saw the menus greyed out.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11240
Differential Revision: https://secure.phabricator.com/D16201
Summary:
Ref T4788. Fixes T7820. This updates the "Merge Duplicates In" interaction, and adds a "Close as Duplicate" action.
These are the last interactions that were using the old code, so it removes that code.
Merges are now recorded as real edges, so we can show them in the UI later on (originally from T9390, etc).
Also provides more general support for relationships which need EDIT permission, not-undoable relationships like merges, preventing relating an object to itself, and relationship side effects like merges.
Finally, fixes a couple of behaviors around typing an exact object name (like `T123`) to find the related object.
Test Plan:
- Merged tasks into the current task.
- Closed the current task as a duplicate of another task.
- Edited other relationships.
- Searched for tasks, commits, etc., by object monogram.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4788, T7820
Differential Revision: https://secure.phabricator.com/D16196
Summary: Ref T4788. This moves everything except "merge" to the new code.
Test Plan:
- Edited relationships in Differential, Diffusion, and Pholio.
- Uninstalled Pholio, made sure "Edit Mocks..." actions vanished.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4788
Differential Revision: https://secure.phabricator.com/D16193
Summary:
Ref T4788. Fixes T9232. This moves the "search for stuff to attach to this object" flow away from hard-coding and legacy constants and toward something more modular and flexible.
It also adds an "Edit Commits..." action to Maniphest, resolving T9232. The behavior of the search for commits isn't great right now, but it will improve once these use real ApplicationSearch.
Test Plan: Edited a tasks' related commits, mocks, tasks, etc.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4788, T9232
Differential Revision: https://secure.phabricator.com/D16189
Summary:
Ref T11179. This splits "Edit Blocking Tasks" into two options now that we have more room ("Edit Parent Tasks", "Edit Subtasks").
This also renames "Blocking" tasks to "Subtasks", and "Blocked" tasks to "Parent" tasks. My goals here are:
- Make the relationship direction more clear: it's more clear which way is up with "parent" and "subtask" at a glance than with "blocking" and "blocked" or "dependent" and "dependency".
- Align language with "Create Subtask".
- To some small degree, use more flexible/general-purpose language, although I haven't seen any real confusion here.
Fixes T6815. I think I narrowed this down to two issues:
- Just throwing a bare exeception (we now return a dialog explicitly).
- Not killing open transactions when the cyclec check fails (we now kill them).
Test Plan:
- Edited parent tasks.
- Edited subtasks.
- Tried to introduce graph cycles, got a nice error dialog.
{F1697087}
{F1697088}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T6815, T11179
Differential Revision: https://secure.phabricator.com/D16166
Summary:
Ref T11179. Ref T4768. Currently, on `master`, if two users open "Edit Revisions" at the same time, then add revisions A and B, only the last state wins (just "B").
Instead, apply these as "add A" and "add B" so they merge in a natural way.
Test Plan:
- Opened edit dialog in two windows.
- Added "A" in one, "B" in the other.
- Saved both.
- Saw "Added A" and "Added B" transactions, instead of "Added A" and "Removed A, added B".
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4768, T11179
Differential Revision: https://secure.phabricator.com/D16164
Summary: Ref T11179. This is basically a "pro" controller to replace the SearchAttach controller. It does basically the same stuff, just in a (mostly) more modern and modular way.
Test Plan:
- Added and removed mocks.
- Added and removed revisions.
- Everything worked just like it did before.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11179
Differential Revision: https://secure.phabricator.com/D16163
Summary:
Ref T11179. This generates the Maniphest menu items in a modular way. It doesn't change any of the underlying code yet.
Searching for commits doesn't work particularly well so I've just hidden that for now, but the item itself works fine.
Test Plan: {F1696849}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11179
Differential Revision: https://secure.phabricator.com/D16162
Summary: Fixes T11193. Assume this is the correct place to check for permissions before attaching edges.
Test Plan: Create a task and set edit policy to Admins, log into test account. Try to Edit Subtasks, Merge Duplicates, Attach a Diff, or Attach a Mock, get a Policy Dialog explaing why.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T11193
Differential Revision: https://secure.phabricator.com/D16161
Summary:
Ref T11034. This seems a little more promising. Two problems at the moment:
- This doesn't actually provide any useful information at all right now.
- Many object types have no profile images.
Test Plan:
{F1695254}
{F1695255}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11034
Differential Revision: https://secure.phabricator.com/D16155
Summary: Ref T4103. Fully modernize the filetree show/hide, durable column show/hide, and profile menu collapse/wide settings.
Test Plan:
- Toggled filetree on/off, reloaded page, setting stuck.
- Same with conpherence column and profile menus.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16034
Summary:
Ref T4103. This doesn't get everything, but takes care of most of the easy stuff.
The tricky-ish bit here is that I need to move timezones, pronouns and translations to proper settings. I expect to pursue that next.
Test Plan:
- Grepped for `loadPreferences` to identify callsites.
- Changed start-of-week setting, loaded Calendar, saw correct start.
- Visited welcome page, read "Adjust Settings" point.
- Loaded Conpherence -- I changed behavior here slightly (switching threads drops the title glyph) but it wasn't consistent to start with and this seems like a good thing to push to the next version of Conpherence.
- Enabled Filetree, toggled in Differential.
- Disabled Filetree, no longer visible in Differential.
- Changed "Unified Diffs" preference to "Small Screens" vs "Always".
- Toggled filetree in Diffusion.
- Edited a task, saw sensible projects in policy dropdown.
- Viewed user profile, uncollapsed/collapsed side nav, reloaded page, sticky'd.
- Toggled "monospaced textareas", used a comment box, got appropriate fonts.
- Toggled durable column.
- Disabled title glyphs.
- Changed monospaced font to 18px/36px impact.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16004
Test Plan: click new link, get to the right page
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15996
Summary:
Ref T4103. This removes these options:
{F1660585}
The jump nav option came from T916, when we had a separate jump nav on the home page. Essentially no one has ever been confused by the behavior of search or disabled this feature. Here are the stats for this install:
| Total Users | 36656 |
| Have Set Any Preference | 3084 |
| Have Disabled Jump | 6
| Are Not "Security Researchers" | 2
| Any Account Activity | 0
The "/" option came in the same change, but the preference came from T989. This keystroke conflicts with a default Firefox keystroke. Almost no one cares about this either, but I count 6 real users who have disabled the behavior. I suspect the number of real users who //use// it may be smaller.
In Safari and Firefox, the "tab" key does the same thing.
In Chrome, the "tab" key does the same thing if {nav Preferences > Web Content > "Pressing Tab highlights..."} is disabled.
Upshot: jump nav is great, bulk of the change in T989 was clearly great, specific preferences that came out of it seem not-so-great and now is a good time to kill them as we head into T4103.
Test Plan:
- Grepped for removed constants.
- Pressed "/".
- Searched for `T123`.
- Viewed settings.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D15976
Summary:
Fixes T10381. When we converted to `PHUIRemarkupView`, some instructional text got linebreaks added when it shouldn't have them (the source is written in PHP and wrapped at 80 characters, but the output should flow naturally).
Fix this so we don't preserve linebreaks.
This also makes `PHUIRemarkupView` a little more powerful and inches us toward fixing Phame/CORGI remarkup issues, getting rid of `PhabricatorMarkupInterface` / `PhabricatorMarkupOneOff`, and dropping all the application hard-coding in `PhabricatorMarkupEngine`.
Test Plan:
- Grepped for all callsites, looking for callsites which accept remarkup written in `<<<HEREDOC` format.
- Viewed form instructions, Conduit API methods, HTTP parameter edit instructions.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10381
Differential Revision: https://secure.phabricator.com/D15963
Summary: Ref T10512. This is fairly bare-bones but appears to work.
Test Plan: Queried all users, queried some stuff by constraints.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10512
Differential Revision: https://secure.phabricator.com/D15959
Summary:
Ref T10939. If you haven't installed a dashboard, we show an "Active Revisions" panel on the homepage by default. I waited a bit to update this, but the new buckets don't seem to have caused any major problems so far.
Update this to use the new logic. I'm just showing "must review" + "should review", which is similar to the old beahvior.
Also replace the notification count with this same number. This is a little different from the old behavior, but simpler, and I think we should probably move toward getting rid of these counts completely.
Test Plan:
- Viewed homepage as logged-in user, saw my revisions (including revisions I have authority over only because of project membership).
- Saw consistent notification count.
- Grepped for removed method.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10939
Differential Revision: https://secure.phabricator.com/D15950
Summary:
Ref T10917. This primarily prepares these for transactions by giving us a place to:
- review old deactivated keys; and
- review changes to keys.
Future changes will add transactions and a timeline so key changes are recorded exhaustively and can be more easily audited.
Test Plan:
{F1652089}
{F1652090}
{F1652091}
{F1652092}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10917
Differential Revision: https://secure.phabricator.com/D15946
Summary:
Ref T10939. Fixes T9263. Ref T4144.
First, this resolves users (converting users into all packages and projects they are responsible for) earlier, so bucketing can act on that data correctly. Previously, your own blocking reviews would appear in "Must Review" but your packages/projects' would not. Now, all of them will.
Second, this adds `exact(username)` to mean "just me, not my packages/projects". You can use this along with "Bucket: By Required Action" to create a personal view of "Active Revisions" if you'd like, and ignore all your project/package reviews.
Test Plan: Queried by "me" and "exact(me)", got reasonable looking results.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4144, T9263, T10939
Differential Revision: https://secure.phabricator.com/D15925
Summary:
Ref T10939. Ref T4144. This splits the existing buckets ("Blocking Others", "Action Required", "Waiting on Others") into 6-7 buckets with a stronger focus on what the next action you need to take is.
See T10939#175423 for some discussion.
Overall, I think some of the root problems here are caused by reviewer laziness and shotgun review workflows (where a ton of people get automatically added to everything, probably unnecessarily), but these buckets haven't been updated since the introduction of blocking reviewers or project/package reviewers and I think splitting the 3 buckets into 6 buckets isn't unreasonable, even though it's kind of a lot of buckets and the root problem here is approximately "I want to ignore a bunch of stuff on my dashboard".
I didn't remove the old bucketing code yet since it's still in use on the default homepage.
This also isn't quite right until I fix the tokenizer to work properly, since it won't bucket project/package reviewers accurately.
Test Plan: {F1395972}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4144, T10939
Differential Revision: https://secure.phabricator.com/D15924
Summary:
Ref T10939. Currently, Differential hard-codes some behaviors for the "active" filter. This introduces "buckets" to make this grouping behavior more general/flexible.
The buckets don't actually do any grouping yet, this just gets rid of the `$query === 'active'` stuff so far.
These buckets change the page size to a large value, becuase pagination won't currently work with bucketing.
The problem is that we normally paginate by selecting one more result than we need: so if we're building a page of size 10, we'll select 11 results. This is fast, and if we get 11 back, we know there's a next page with at least one result on it.
With buckets, we can't do this, since our 11 results might come back in these buckets:
- A, B, C, A, C, C, A, A, B, B, (B)
So we know there are more results, and we know that bucket B has more results, but we have no clue if bucket A and bucket C have more results or not (or if there's anything in bucket D, etc).
We might need to select a thousand more results to get the first (D) or the next (A).
So we could render something like "Some buckets have more results, click here to go to the next page", but users would normally expect to be able to see "This specific bucket, A, has more results.", and we can't do that without a lot more work.
It doesn't really matter for revisions, because almost no one has 1K of them, but this may need to be resolved eventually.
(I have some OK-ish ideas for resolving it but nothing I'm particularly happy with.)
Test Plan: {F1376542}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10939
Differential Revision: https://secure.phabricator.com/D15923
Summary:
Fixes T10778. This is a result of T10262: when we save a form configuration and adjust the policy, we try to scramble attached file secrets.
There aren't going to be any attached files, but there's also no edge table, so we fail.
We could skip this code, but we'll likely need an edge table here sooner or later so it's probably simpler in the long run to just add an empty one.
Test Plan:
- Ran `bin/storage upgrade`, got a clean bill of health.
- Saved a form configuration after making a policy edit, no more `edge` exception.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10778
Differential Revision: https://secure.phabricator.com/D15803
Summary: This is completely obsoleted by `owners.search`. See D15472.
Test Plan: Viewed API method in UI console.
Reviewers: avivey, chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15769
Summary: Updates to use new UI
Test Plan: Save a custom query, edit a custom query
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15618
Summary: View various conduit pages and update to new UI and add calls to newPage
Test Plan: View list, view method, make a call.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15613
Summary: Cleans up EditEngine, adds new layout to EditEngine and descendents
Test Plan: Test creating a new form, reordering, marking and unmarking defaults. View new forms.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15531
Summary:
Fixes T10011.
- Modernize searching for buildables.
- Prepare for `harbormaster.buildable.search`.
- Allow users to query by status (see T10011).
- Collapse the four weird "commit / diff / revision / repository" fields into two slightly less weird fields with more UI hinting?
Test Plan: {F1131918}
Reviewers: chad
Reviewed By: chad
Subscribers: Luke081515.2
Maniphest Tasks: T10011
Differential Revision: https://secure.phabricator.com/D15356
Summary: Fixes T10411. Ref T10246. There are probably still some rough edges with this, but replace the old-school endpoints with modern ones so we don't unprototype with deprecated stuff.
Test Plan:
- Made a bunch of calls to the new endpoints with various constraints/attachments.
- Created and edited services, devices, interfaces, bindings, and properties on everything.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10246, T10411
Differential Revision: https://secure.phabricator.com/D15329
Summary:
Ref T10010. Milestones can't have subprojects, so this item isn't very useful.
I think there is also an argument for disabling "Members", but that panel is a little less useless and explains the membership rule, so I'm less certain about removing it. I do generally lean toward removing it at some point, though.
Test Plan:
- Viewed a milestone, no "Subprojects" menu item.
- Viewed a normal project, saw item.
- Edited both menus, saw consistent UI.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10010
Differential Revision: https://secure.phabricator.com/D15200
Summary: No UI changes, just some search and replace for UI consistency.
Test Plan: Test person and object hovercards still work. UIExamples too.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15172
Summary: Mostly for consistency, we're not using other forms of icons and this makes all classes that use an icon call it in the same way.
Test Plan: tested uiexamples, lots of other random pages.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15125
Summary: Fixes T10213. I think the "Edit" item was originally conditional (or maybe I just forgot to add that part), but that got dropped when we swapped how it worked. This is all stable now anyway and can be available without needing prototypes enabled.
Test Plan: Edited a project menu.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10213
Differential Revision: https://secure.phabricator.com/D15117
Summary: Ref T10054. Prevent users from removing this item and locking themselves out of the system unless they can guess the URI.
Test Plan: Tried to disable "Manage", wasn't permitted to.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15113
Summary:
Ref T10144. This isn't comprehensive, but we can give it a try and see how it feels?
- EditEngine forms now say "Tags" instead of "Projects".
- Modern SearchEngine forms now say "Tags" instead of "Projects".
- For clarity, replaced as much "in project" language as I could find with "tagged with project" language.
Test Plan: reading / grepping + used "not tagged with any project" token
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10144
Differential Revision: https://secure.phabricator.com/D15108
Summary:
First pass at a new Project Home page. This is starting to sprawl, so punting this up now before it gets too large.
- Project homes now have "large header"
- Custom Fields / Descriptions are in the main column
- Feed is simpler visually
- new "Background" option for PHUIObjectBoxView
- move header buttons globally to "Grey" instead of "Simple"
- New color and hover states for "Grey"
- Transitions on Buttons haha
- Edit Icon on Nav is now under "Manage" panel
- New "Manage" Panel
TODO:
- More testing of bad cases of Custom Fields
- Members Page in flux, needs design
- Um still not sure how to make Custom Field not show UI
Test Plan:
Lots of random Project page visits. Save project, watch project, edit project, etc.
{F1068191}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15097
Summary: Ref T10054. Since we no longer have the "workboard default if it exists" rule, provide a quick way to make it the default.
Test Plan: Created a new workboard with the box checked, saw menu change appropriately.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15092
Summary: racooons → racoons
Test Plan: Read again the sentence.
Reviewers: #blessed_reviewers, chad
Reviewed By: #blessed_reviewers, chad
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D15094
Summary: Ref T10054. This is all pretty straightforward. Also include some project-specific examples in the project documentation.
Test Plan: Read documentation.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15090
Summary:
Ref T10054.
- Just let users delete non-builtin items.
- Let users choose a default item explicitly.
- Do a better job of cleaning up items which no longer exist or belong to uninstalled applications.
(NOTE) This has one user-facing change: workboards are no longer the default on projects with workboards. I think this is probably OK since we're giving users a ton of new toys at the same time, but I'll write some docs at least.
Test Plan:
- Deleted custom items.
- Disabled/enabled builtin items.
- Made various things defaults.
- Uninstalled Maniphest, saw Workboards tab disappear entirely.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15089
Summary: `alterClass()` is strict about true/false but we set 0/1 elsewhere.
Test Plan: Collapsed/expanded menu, reloaded expanded menu, clicked collapse, got immediate collapse.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15087
Summary:
Fixes T10196. This is a weird interaction and this might not be the best long-term fix, but just get it working OK for now.
General problem is that Quicksand doesn't currently use GET for requests. This is a very unusual case where the method is relevant. In the future, I might change Quicksand to use GET.
Test Plan: Clicked "Open Tasks" with Quicksand active, got a results list.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10196
Differential Revision: https://secure.phabricator.com/D15082
Summary: Mostly a visual spacing pass, also adds in circle icons for edit, collapse. For now removing the fixed position on the icons for simplicity while the basics are being polished.
Test Plan: Projects, Profiles, wide and narrow.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15081
Summary:
Ref T10054. I think this gets everything except:
- circles on icons;
- I spent ~15 minutes poking at animations but wasn't able to get anything that looked reasonable whatsoever.
Test Plan:
- Collapsed menus.
- Expanded menus.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15056
Summary: Ref T10054. The daemons look for this but currently can't find it.
Test Plan: Ran daemons, clean exit on profile menu edits instead of permanent failure.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15053
Summary: Motivate your employees with inspirational quotes. A new quote every day!
Test Plan: So inspirational.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15026
Summary:
Ref T10054. Primary goal is to be able to remove IconNav from the codebase.
I've made these non-editable so users can't customize them yet. We //might// want administrators to customize these globally instead? In any case, we avoid a bunch of product questions by just locking these down for now.
Test Plan: {F1061348}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15020
Summary: Ref T10054. Take specialization off the objects and put it on Engine subclasses instead. One reason for this is that certain objects (like users) might have multiple different sets of panels in the future (e.g., their user profile and their home page).
Test Plan:
- No user-visible changes.
- PanelEngine no longer has any hardcoded "project" stuff.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15018
Summary:
Ref T10054. I haven't done any of the big-picture layout stuff yet, but this should get look-and-feel somewhere in the ballpark of reasonablness, I think.
Major missing stuff:
- No "collapse" state or action yet.
- Menu is not full-height (requires changes to the rendering pipeline).
Test Plan: {F1060941}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15016
Summary:
Ref T10054.
I made this a dropdown (currently: "Visible" or "Disabled") since I imagine we //miiiight// want to add a "Hidden, but click 'More' to reveal" state or do other special stuff in this vein. Not 100% sold on that but seemed within the realm of plausibility.
Test Plan: {F1060759}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15012
Summary: Ref T10054. Allows users to drag menu items to reorder them.
Test Plan: Reordered a project menu.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15011
Summary:
Ref T10054. This does a big chunk of the legwork to let users reconfigure profile menus (currently, just project menus).
This includes:
- Editing builtin items (e.g., you can rename the default items).
- Creating new items (for now, only links are available).
This does not yet include:
- Hiding items.
- Reordering items.
- Lots of fancy types of items (dashboards, etc).
- Any UI changes.
- Documentation (does feature: TODO link for documentation).
Test Plan:
{F1060695}
{F1060696}
{F1060697}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15010
Summary: Ref T10054. This has no product impact, but prepares us for customizable side nav on "profiles" (today, projects; probably users some day; and maybe other stuff down the road).
Test Plan: Clicked all links on a profile, everything was exactly the same as before.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15007
Summary:
If you try to pull the hovercard of something you can no longer see (maybe you loaded the page, then the policy changed) there won't be a value in the array here.
(The rest of the code anticipates this possibility.)
Test Plan: Hovered some stuff.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D14987
Summary: Ref T4245. Before doing any hard work here, we can dramatically reduce the number of things that make calls to `getCallsign()` to make navigating things easier. Almost all of them only care about a monogram, URI, or display name.
Test Plan:
- Searched for `r uniquename` in jump nav.
- Ran `bin/repository reparse --change rXXXyyyyy --trace`, observed query against bad commit table.
- Ran `bin/search index rXXXyyyy --trace --force`, observed proper title when indexing commit.
- Browed repository list, saw proper `rXXX` and appropriate link targets.
- Mentioned `rXXX` in Remarkup, got a link to the right place.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4245
Differential Revision: https://secure.phabricator.com/D14923
Summary: Ref T9897. Adds basic blog query support.
Test Plan: Ran some queries.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9897
Differential Revision: https://secure.phabricator.com/D14900
Summary: Ref T10010. This still needs support for attachments (to get members) and more constraints (like slugs), but mostly works.
Test Plan: Ran query, saw basically sensible results.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10010
Differential Revision: https://secure.phabricator.com/D14889
Summary:
Ref T8980. Move away from events to EngineExtensions.
This also simplifies hovercards a bit:
- Removes tasks from revision cards.
- Removes blockers/blocked from task cards.
- Removes "Send Message" from user cards.
These mostly felt cluttery to me. Open to arguments to retain them. I think we can make better use of the space, though (e.g., flags, projects + board columns).
Test Plan:
- Viewed people, task, revision, commit and project hovercards.
{F1043256}
{F1043257}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T8980
Differential Revision: https://secure.phabricator.com/D14878
Summary:
Ref T9979. This uses ngrams (specifically, trigrams) to build a reasonably efficient index for substring matching. Specifically, for a package like "Example", with ID 123, we store rows like this:
```
< ex, 123>
<exa, 123>
<xam, 123>
<amp, 123>
<mpl, 123>
<ple, 123>
<le , 123>
```
When the user searches for `exam`, we join this table for packages with tokens `exa` and `xam`. MySQL can do this a lot more efficiently than it can process a `LIKE "%exam%"` query against a huge table.
When the user searches for a one-letter or two-letter string, we only search the beginnings of words. This is probably what they want, the only thing we can do quickly, and a reasonable/expected behavior for typeaheads.
Test Plan:
- Ran storage upgrades and search indexer.
- Searched for stuff with "name contains".
- Used typehaead and got sensible results.
- Searched for `aabbccddeeffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz` and saw only 16 joins.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14846
Summary:
Fixes T9890. This allows IndexExtensions to emit an object version.
Before we build indexes, we check if the indexed version is the same as the current version. If it is, we just don't call that extension.
T9890 has a case where this is useful: a script went crazy and posted thousands of comments to a single task.
Without versioning, that results in the same comments being indexed over and over again. With versioning, most of the queue could just exit without doing any work.
Test Plan:
- Added a `sleep(1)` to the actual indexing, used `bin/search index --background` to queue up a lot of tasks, ran them with `bin/phd debug task`, saw them complete very quickly with only one actual index operation performed.
- Used `bin/search index --trace` and `bin/search index --trace --background` to observe the behavior of queries against the index version store, which looked sensible.
- Made comments/transactions, saw versions update.
- Used `bin/remove destroy`, verified index versions were purged.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9890
Differential Revision: https://secure.phabricator.com/D14845
Summary:
Ref T9979. I picked this name long before the advent of modern "Engine" architecture and it ended up being pretty confusing.
Rename "SearchEngine" (currently: mysql or elasticsearch, used to store and query fulltext indexes) to "FulltextStorageEngine" to make it more clear what it does and disambituate it from ApplicationSearch, which also has a bunch of stuff called "SearchEngine", "SearchEngineExtension", etc.
Test Plan: Grepped for `phabricatorsearchengine`.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14843
Summary: Ref T9979. This simplifies/standardizes the code a bit, but mostly gives us more consistent class names and structure.
Test Plan:
- Used `bin/search index --type ...` to index documents of every indexable type.
- Searched for documents by unique text, found them.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14842
Summary:
Ref T9979. There are currently some hacks around Conpherence indexing: it does not really use the fulltext index, but its own specialized index. However, it's kind of hacked up so it can get reindexed by the normal indexing pipeline.
Lift it up into IndexEngine, instead of FulltextEngine. Specifically, the new stuff is going to look like this:
- IndexEngine: Rebuild all indexes.
- ConpherenceIndexExtension: Rebuild thread indexes.
- ProjectMemberIndexExtension: Rebuild project membership views.
- NgramIndexExtension: Rebuild ngram indexes.
- FulltextIndexExtension / FulltextEngine: Rebuild fulltext indexes, a special type of index.
- FulltextCommentExtension: Rebuild comment fulltext indexes.
- FulltextProjectExtension: Rebuild project fulltext indexes.
- etc.
Most of this is at least sort-of-in-place as of this diff, although some of the part in the middle is still pretty rough.
Test Plan:
- Made a unique comment in a Conpherence thread.
- Used `bin/search index --force` to rebuild the index.
- Searched for the comment.
- Found the thread.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14841
Summary:
Ref T9979. This is currently hard-coded but can be done in a generic way.
This has one minor behavioral changes: answer text is no longer included in the question text index in Ponder. I'm not planning to accommodate that for now since I don't want to dig this hole any deeper than I already have. This behavior should be different anyway (e.g., index the answer, then show the question in the results or something).
Test Plan:
- Put a unique word in a Maniphest comment.
- Searched for the word.
- Found the task.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14837
Summary:
Ref T9979. This event had one weird callsite and no known third-party callers. It can be done more cleanly as an extension, now.
This index is used to allow us to "Group By: Project" in Maniphest without joining into the Projects database.
Test Plan:
- Ran a query with "Group By: Project" in Maniphest.
- Renamed project "Apples" to "Zebras".
- Reloaded page.
- UI properly moved "Zebras" tasks to the bottom of the list.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14836
Summary: Ref T9979. This is going to become `FulltextEngine`, but pave the way for that by pulling extensions out of it.
Test Plan:
{F1036624}
- Used `bin/search index Txxx`, saw projects, subscribers and custom fields rebuild in the index.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14835
Summary:
Ref T9890. Ref T9979. Several adjacent goals:
- The `SearchEngine` vs `ApplicationSearchEngine` thing is really confusing. There are also a bunch of confusing class names and class relationships within the fulltext indexing. I want to rename these classes to be more standard (`IndexEngine`, `IndexEngineExtension`, etc). Rename `SearchIndexer` to `IndexEngine`. A future change will rename `SearchEngine`.
- Add the index locks described in T9890.
- Structure things a little more normally so future diffs can do the "EngineExtension" thing more cleanly.
Test Plan:
Indexing:
- Renamed a task to have a unique word in the title.
- Ran `bin/search index Txxx`.
- Searched for unique word.
- Found task.
Locking:
- Added a `sleep(10)` after the `lock()` call.
- Ran `bin/search index Txxx` in two windows.
- Saw first one lock, sleep 10 seconds, index.
- Saw second one give up temporarily after failing to grab the lock.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9890, T9979
Differential Revision: https://secure.phabricator.com/D14834
Summary:
Ref T9979. The general shape of "engine" code feels pretty good, and I plan to move indexing to be more in line with other modern engines, with the ultimate goal of supporting subprojects (T10010) and several intermediate goals.
Before moving indexing, clean up Destruction, since some of the new indexes will need destruction hooks and destruction currently has a lot of `instanceof` stuff that should be easy to fix by applying more modern approaches.
Test Plan:
- Used `bin/remove destroy` to destory an Almanac device.
- Verified that properties for the device were destroyed.
- Viewed module panel in UI.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9979
Differential Revision: https://secure.phabricator.com/D14831
Summary:
This is just putting a hook in that pretty much works. Behavior:
- If you visit `/maniphest/?nux=true`, it always shows NUX for testing.
- Otherwise, it shows NUX if there are no objects in the application yet.
Test Plan: {F1031846}
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D14828
Summary: Ref T9964. Create some docuemntation for this stuff, and clean up the *.edit endpoints a bit.
Test Plan: Read documentation.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14798
Summary: Ref T9964. This just adds more structure to application fields, to make it harder to make typos and easier to validate them later.
Test Plan: Viewed APIs, called some APIs, saw good documentation and correct results.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14776
Summary: Ref T9964. Builds on D14772. Allows callers to get the raw content of pastes as an attachment.
Test Plan:
- Read docs.
- Executed attachment query.
- Saw raw paste content.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14774
Summary:
Ref T9964. We have various kinds of secondary data on objects (like subscribers, projects, paste content, Owners paths, file attachments, etc) which is somewhat slow, or somewhat large, or both.
Some approaches to handling this in the API include:
- Always return all of it (very easy, but slow).
- Require users to make separate API calls to get each piece of data (very simple, but inefficient and really cumbersome to use).
- Implement a hierarchical query language like GraphQL (powerful, but very complex).
- Kind of mix-and-match a half-power query language and some extra calls? (fairly simple, not too terrible?)
We currently mix-and-match internally, with `->needStuff(true)`. This is not a general-purpose, full-power graph query language like GraphQL, and it occasionally does limit us.
For example, there is no way to do this sort of thing:
$conpherence_thread_query = id(new ConpherenceThreadQuery())
->setViewer($viewer)
// ...
->setNeedMessages(true)
->setWhenYouLoadTheMessagesTheyNeedProfilePictures(true);
However, we almost never actually need to do this and when we do want to do it we usually don't //really// want to do it, so I don't think this is a major limit to the practical power of the system for the kinds of things we really want to do with it.
Put another way, we have a lot of 1-level hierarchical queries (get pictures or repositories or projects or files or content for these objects) but few-to-no 2+ level queries (get files for these objects, then get all the projects for those files).
So even though 1-level hierarchies are not a beautiful, general-purpose, fully-abstract system, they've worked well so far in practice and I'm comfortable moving forward with them in the API.
If we do need N-level queries in the future, there is no technical reason we can't put GraphQL (or something similar) on top of this eventually, and this would represent a solid step toward that. However, I suspect we'll never need them.
Upshot: I'm pretty happy with "->needX()" for all practical purposes, so this is just adding a way to say "->needX()" to the API.
Specifically, you say:
```
{
"attachments": {
"subscribers": true,
}
}
```
...and get back subscriber data. In the future (or for certain attachments), `true` might become a dictionary of extra parameters, if necessary, and could do so without breaking the API.
Test Plan:
- Ran queries to get attachments.
{F1025449}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14772
Summary:
Ref T9964. I added several hacks to get these working. Clean them up and pull this into a proper extension.
The behavior in the web UI is:
- they work in all applications; but
- they only show up in the UI if a value is specified.
So if you visit `/view/?ids=1,2` you get the field, but normally it's not present. We could refine this later. I'm going to add documentation about how to prefill these forms regardless, which should make this discoverable by reading the documentation.
There's one teensey weensey hack: in the API, I push these fields to the top of the table. That one feels OK, since it's purely a convenience/display adjustment.
Test Plan: Queried by IDs, reviewed docs.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14769
Summary:
Ref T9964. Building tables in Remarkup is kind of neat-ish but ends up feeling kind of hacky, and requires weird workarounds if any of the values have `|` in them.
Switch to normal elements instead.
Also move the magic "ids" and "phids" to be more like real fields. I'll clean this up fully in a diff or two, it's just a little tricky because Maniphest has an "ids" field.
Test Plan: {F1024294}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14768
Summary: Ref T9964. I left a couple of these unsupported for now since they're weird in some way.
Test Plan: {F1024031}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14767
Summary:
Ref T9964. Fill in more parameter types and descriptions.
(No date support yet since it's a bit more involved.)
Test Plan: {F1024022}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14766
Summary: Ref T9964. This fills in types and descriptions for ApplicationSearch fields in Paste.
Test Plan:
Got this nice table now:
{F1023999}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14765
Summary:
Ref T9964. ApplicationSearch currently has a bunch of hard-coded `if ($object instanceof thing)` stuff.
Pull that out so it can live in extensions.
Test Plan:
- Searched by spaces, subscribers, projects.
{F1023921}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14764
Summary:
Ref T9964. I want to show users what we're expecting in "constraints", and let constraints like "authors=epriestley" work to make things easier.
I'm generally very happy with the "HTTPParameterType" stuff from EditEngine, so add a parallel set of "ConduitParameterType" classes. These are a little simpler than the HTTP ones, but have a little more validation logic.
Test Plan:
This is really just a proof of concept; some of these fields are now filled in:
{F1023845}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14763
Summary:
Ref T9964. The new `*.search` and `*.edit` methods generate documentation which depends on the viewer.
For example, the `*.search` methods show a reference table of the keys for all your saved queries.
Give them a real viewer to work with.
During normal execution, just populate this viewer with the request's viewer, so `$request->getViewer()` and `$this->getViewer()` both work and mean the same thing.
Test Plan: {F1023780}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14761
Summary:
Ref T9964. Adds a new-style "owners.search" endpoint, and an extension for customfields.
Puts enough indirection in place to give us nice, consistent "custom.key" user-facing keys instead of "std:custom:owners:na0shf9a8dfdsafl" junk.
Test Plan:
- Searched Owners via API.
- Searched by ID.
- Ordered by custom fields.
- Reviewed API docs.
- Used normal search with ordering.
- Viewed custom field values in search results.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9964
Differential Revision: https://secure.phabricator.com/D14758
Summary:
Ref T9964. See that task for some context and discussion.
Ref T7715, which has the bigger picture here.
Basically, I want Conduit read endpoints to be full-power, ApplicationSearch-driven endpoints, so that applications can:
- Write one EditEngine and get web + conduit writes for free.
- Write one SearchEngine and get web + conduit reads for free.
I previously made some steps toward this, but this puts more of the structure in place.
Test Plan:
Viewed API console endpoint and read 20 pages of docs:
{F1021961}
Made various calls: with query keys, constraints, pagination, and limits.
Viewed new {nav Config > Modules} page.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7715, T9964
Differential Revision: https://secure.phabricator.com/D14743
Summary:
Ref T9908. No more callsites. Also:
- Phurl a couple of documentation URIs.
- Get rid of "task:" in the global search since it doesn't really make sense anymore.
Test Plan: `grep`, edited/created tasks.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9908
Differential Revision: https://secure.phabricator.com/D14716
Summary: Ref T9908. This is the last of the things that need to swap over.
Test Plan:
- Created tasks from a workboard.
- Created tasks in different columns.
- Edited tasks.
- Used `?parent=..`.
- Verified that default edit form config now affects comment actions.
- No more weird comment thing on forms, at least for now.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9908
Differential Revision: https://secure.phabricator.com/D14715
Summary:
Ref T9132. Ref T9908. Puts reordering UI in place:
- For create forms, this just lets you pick a UI display order other than alphabetical. Seems nice to have.
- For edit forms, this lets you create a hierarchy of advanced-to-basic forms and give them different visibility policies, if you want.
Test Plan:
{F1017842}
- Verified that "Edit Thing" now takes me to the highest-ranked edit form.
- Verified that create menu and quick create menu reflect application order.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9132, T9908
Differential Revision: https://secure.phabricator.com/D14704
Summary: Ref T9690. I wanted to do an example of how to do these but it looks like most of them are trivial (no callsites) and the rest are a little tricky (weird interaction with frames, or in Releeph).
Test Plan:
- Used `grep` to look for callsites.
- Hit all applications locally, everything worked.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9690
Differential Revision: https://secure.phabricator.com/D14385
Summary:
Fixes T5752. This obsoletes a bunch of old patterns and I'll follow up on those with a big "go do a bunch of mechanical code changes" task. Major goals are:
- Don't load named queries multiple times on search pages.
- Don't require extra code to get standard navigation right on mobile.
- Reduce the amount of boilerplate in ListControllers.
- Reduce the amount of boilerplate around navigation/menus in all controllers.
Specifically, here's what this does:
- The StandardPage is now a smarter/more structured object with `setNavigation()` and `setCrumbs()` methods. More rendering decisions are delayed until the last possible moment.
- It uses this to automatically add crumb actions to the application menu.
- It uses this to automatically reuse one SearchEngine instead of running queries multiple times.
- The new preferred way to build responses is `$this->newPage()` (like `$this->newDialog()`), which has structured methods for adding stuff (`setTitle()`, etc).
- SearchEngine exposes a new convenience method so you don't have to do all the controller delegation stuff.
- Building menus is generally simpler.
Test Plan:
- Tested paste list, view, edit, comment, raw controllers for functionality, mobile menu, crumbs, navigation menu.
- Edited saved queries.
- Tested Differential, Maniphest (no changes).
- Verified the paste pages don't run any duplicate NamedQuery queries.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T5752
Differential Revision: https://secure.phabricator.com/D14382
Summary: Ref T8628. Updates Search.
Test Plan: Did various searches, saved new queries, reordered, ran new queries.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T8628
Differential Revision: https://secure.phabricator.com/D14268
Summary: Fixes T5991. If //all requested documents// failed to index, consider this a catastrophic failure and exit with an error code.
Test Plan:
- Ran `bin/search index --type TASK`, observed successful exit despite a small number of un-indexable documents.
- Ran `bin/search index PHID-TASK-xxx` for an invalid task, observed exception on exit after complete failure.
- Ran normal indexing through daemons.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T5991
Differential Revision: https://secure.phabricator.com/D14174
Summary:
This reverts commit 1583738842.
See T8646 for discussion. This version of the feature feels terrible on real data.
Test Plan: Strict revert.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D14097
Summary:
Ref T8646. This is fairly rough:
This interface is very niche, and not really flexible enough to accommodate other result customization (but I don't think we have any plans here)?
I'm just //summarizing// the content of documents, basically showing the first paragraph of their content, summary, etc. This isn't what Google does: it shows snippets surrounding the actual search terms. However, this is more involved and might be less useful in structured data: for example, I'd imagine that the first line of most phriciton documents, maniphest tasks and Differential revisions really might be the best machine-generatable summary of them. The actual contextual snippeting in Google doesn't often seem hugely useful to me. But this might also not be very useful.
There's not much design, not sure if you had any ideas.
I only implemented this for tasks, revisions and the wiki since those seem most useful.
I'm generally on the fence about this, but it's not a ton of work to swap out for something else later. Maybe we can see how it feels? But happy to toss it or rethink the approach.
Test Plan: {F788026}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T8646
Differential Revision: https://secure.phabricator.com/D14095
Summary: Use `PhutilClassMaQuery` instead of `PhutilSymbolLoader`, mostly for consistency. Depends on D13588.
Test Plan: Poked around a bunch of pages.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13589
Summary: Releeph does some really old sketchy stuff in result rendering. Modernize it, remove the holdover/oldschool interface (these were the last two implementors) and make errors a little softer.
Test Plan: Viewed Releeph products and branches.
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D13604
Summary:
Fixes T8774. When an ApplicationSearch page returned an error (e.g., using a "viewer()" query while logged out), we would try to add action links to a box without a header. Instead:
- If a box has no header, but has show/hide actions, just create an empty header so the view renders. This is a little silly looking but does what the caller asks.
- Always set the title on the result box, so we get a header.
Test Plan: Did an invalid (logged out, with viewer()) search. Did a valid search.
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T8774
Differential Revision: https://secure.phabricator.com/D13576
Summary: Use `PhutilClassMapQuery` where appropriate.
Test Plan: Browsed around the UI to verify things seemed somewhat working.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13429
Summary: Return `$this` from setter methods for consistency. I started writing a [[https://secure.phabricator.com/differential/diff/32506/ | linter rule]] to detect this, but I don't think it is trivial to do this properly.
Test Plan: Eyeball it.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13422
Summary: Move some `PhabricatorSearchField` subclasses to be adjacent to the application to which they belong. This seems generally better to me than lumping them all together in the `src/applications/search/field/` directory. I was also wondering if it makes sense to rename these subclasses as `PhabricatorXSearchField` rather than `PhabricatorSearchXField` (as per T5655), but wasn't really sure if these objects are meant to be search-fields, or just fields belonging to the #search application.
Test Plan: N/A.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13374
Summary: Ref T8099, functionally I prefer to be able to set anything 'table-like' with `setTable` for design consistency. This looses the restriction and did some light grepping for other missed cases.
Test Plan: Test new UI, grep for other missing cases.
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T8099
Differential Revision: https://secure.phabricator.com/D13471
Summary: Fixes T8642, This is a table, but not returned as one. Set it to Content for now with a Collapsed layout.
Test Plan: Test /people/logs/
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T8642
Differential Revision: https://secure.phabricator.com/D13395
Summary: Ref T7950, Refactor Calendar Search, and implement Projects on events
Test Plan: Verify that all queries in Calendar search still work, and that events can now have associated Projects that you can search by in Calendar Search.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Maniphest Tasks: T7950
Differential Revision: https://secure.phabricator.com/D13393
Summary: Ref T8099. This adds a new class which all search engines return for layout. I thought about this a number of ways, and I think this is the cleanest path. Each Engine can return whatever UI bits they needs, and AppSearch or Dashboard picks and lays the bits out as needed. In the AppSearch case, interfaces like Notifications, Calendar, Legalpad all need more custom layouts. I think this also leaves a resonable path forward for NUX as well. Also, not sure I implemented the class correctly, but assume thats easy to fix?
Test Plan: Review and do a search in each application changed. Grep for all call sites.
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T8099
Differential Revision: https://secure.phabricator.com/D13332
Summary: Ref T8099, Adds ability to set hidden form open, which leaves form open on searches and moves anchor point.
Test Plan: Run saved searches, advanced searches, and searches from main search ui.
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T8099
Differential Revision: https://secure.phabricator.com/D13308
Summary: All classes should extend from some other class. See D13275 for some explanation.
Test Plan: `arc unit`
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13283
