1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-11 16:16:14 +01:00
Commit graph

718 commits

Author SHA1 Message Date
epriestley
719dd6d3f4 Remove the "search_documentfield" table
Summary: Ref T11741. See PHI1276. After the switch to "Ferret", this table has no remaining readers or writers.

Test Plan:
  - Ran `bin/storage upgrade -f`, no warnings.
  - Grepped for class name, table name, `stemmedCorpus` column; got no relevant hits.
  - Did a fulltext search.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T11741

Differential Revision: https://secure.phabricator.com/D20549
2019-05-23 19:11:38 -07:00
epriestley
f2feb0378f Remove ancient "PhabricatorQuickSearchEngineExtension" compatibility class
Summary: Ref T5378. This class was renamed more than a year ago, in D19087. Remove the leftover compatiblity layer.

Test Plan: `grep`

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5378

Differential Revision: https://secure.phabricator.com/D20509
2019-05-16 11:14:49 -07:00
epriestley
9d716379a3 Add a "Customize Query" action to query panels to make it easier to make minor query adjustments
Summary:
Depends on D20473. Ref T13272. Fixes T7216. If you want to tweak the query a panel uses, you currently have to complete 7 Great Labors.

Instead, add a "Customize Query" action which lets you update the query inline.

Test Plan: {F6402171}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T7216

Differential Revision: https://secure.phabricator.com/D20474
2019-05-01 11:01:09 -07:00
epriestley
382e2c32fc When an invalid "constraints" parameter is provided to a "*.search" method, raise a more tailored error
Summary:
See <https://discourse.phabricator-community.org/t/constraints-parameter-type-handling-in-maniphest-search/2636>.

(The caller provided `constraints={...}` via cURL, but you can't mix JSON and HTTP parameters like that.)

Test Plan: Called `curl 'http://local.phacility.com/api/maniphest.search?api.token=...&constraints=X'` (with a valid API token), got a more sensible error.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20429
2019-04-17 13:17:07 -07:00
epriestley
4eab3c4c8d Reindex dashboards and panels (allow migrations to queue a job to queue other indexing jobs)
Summary:
Depends on D20411. Ref T13272. Dashboards and panels have new indexes (Ferret and usage edges) that need a rebuild.

For large datasets like commits we have the "activity" flow in T11932, but realistically these rebuilds won't take more than a few minutes on any realistic install so we should be able to just queue them up as migrations.

Let migrations insert a job to basically run `bin/search index --type SomeObjectType`, then do that for dashboards and panels.

(I'll do Herald rules in a followup too, but I want to tweak one indexing thing there.)

Test Plan: Ran the migration, ran `bin/phd debug task`, saw everything get indexed with no manual intervention.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20412
2019-04-17 12:05:49 -07:00
epriestley
a3c43c473b Convert dashboard read/display pathways to the new panel storage format
Summary:
Depends on D20406. Ref T13272. This gets about half of Dashboards working with the new "duplicate panel friendly" storage format. Followups will fix the write pathways.

Collateral damage here includes:

  - Remove the old Dashboard/Panel edge type. We have a new, more general edge type for "container X uses panel Y", and we don't need this edge type for anything else.
  - Remove "attachPanels()" from Dashboard. Only rendering actually needs this, and it can just load the panels.
  - Remove "attachPanelPHIDs()" from Dashboard. We can look at the panel refs to figure this out.
  - Remove "attachProjects()" from Dashboard. Nothing uses this and it's not a very modern approach.
  - `getPanelPHIDs()` just looks at the config now.
  - Deleted some `LayoutConfig`-related code which is broken/obsolete.

Test Plan:
  - Viewed various dashboards which were created before the changes, saw them render correctly.
  - Viewed a dashboard with two of the same panel! AMAZING!

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20407
2019-04-14 10:23:42 -07:00
epriestley
ce723f999d Move Dashboards main edit flow to EditEngine
Summary:
Depends on D20402. Ref T13272. Replaces an old-school hard-coded "EditController" with a more modern one.

The actual panel stuff is still using a weird mix of legacy manual `save()` calls, but that's up next.

Test Plan: Created Dashboards, edited all dashboard fields via "Edit Dashboard".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20403
2019-04-12 06:15:15 -07:00
epriestley
51f2ed498d On panel pages, show where panels are used
Summary:
Depends on D20398. Ref T13272. Fixes T6018. Previously, panels showed "used on dashboards: x, y", but this did not include cases where a panel was used by another container panel (today, a tab panel).

Do edge indexing when a dashboard or panel is saved, then pull the edges on the Panel page so we can provide a full list of uses.

Test Plan: {F6369289}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T6018

Differential Revision: https://secure.phabricator.com/D20399
2019-04-12 06:14:21 -07:00
epriestley
d62f4dbfc9 Index and surface usage sites for Dashboards
Summary:
Depends on D20397. Ref T13272. Similar to the recent "where are Herald rules used" stuff, show which menus Dashboards are installed in.

This is mostly straightforward, except that I pulled some of the Herald logic into a parent class so it could be shared.

Test Plan: {F6369164}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20398
2019-04-12 06:13:44 -07:00
epriestley
4d0904ef95 Make "Favorites" work more like other customizable menus
Summary:
Depends on D20373. Ref T13272. When you put Dashboards in Favorites, the render without a navigation menu, which is kind of weird.

Instead, make Favorites display a navigation menu. This effectively turns it into a weird cross between the home page and a portal, but so be it.

Also change the icon from "star" to "bookmark" since I think that a clearer hint about how it works.

Test Plan: Viewed dashboards in favorites, got a navigation menu. Edited items from portals, home, favorites.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20374
2019-04-09 13:59:35 -07:00
epriestley
12b9224387 Make the "Install Dashboard" flow smoother
Summary:
Depends on D20362. Ref T13272. Currently, Dashboards have an "Install Dashboard" flow which is pretty janky and only allows you to install things to the home page.

Instead, allow users to install things to any valid target (home, favorites, portals, projects). This also provides URIs like `dashboard/install/1/home/personal/` which allow you to link users to an "install a dashboard" page; this may or may not get used.

Test Plan: Installed dashboards on home, favorites, projects, and portals.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20364
2019-04-09 13:34:09 -07:00
epriestley
eea093bec8 Combine the "View", "Arrange", and "Manage" modes of Dashboards into a single mode
Summary:
Depends on D20361. Ref T13272. Currently, Dashboards have three separate modes: view, arrange, manage.

With the advent of Portals, I think we can simplify this, and make the dashboard view a combined view/edit/manage page. To view it in a cleaner standalone way, you can add it to a portal/home/project. I'll also improve the "Install" workflow.

Test Plan:
Viewed a dashboard page, clicked through all the actions, grepped for affected URIs.

{F6327027}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20362
2019-04-09 13:32:36 -07:00
epriestley
dfe47157d3 When picking a default menu item to render, don't pick disabled items
Summary:
Depends on D20358. Fixes T12871. After refactoring, we can now tell when a "storage" menu item generated only disabled "display" menu items, and not pick any of them as the default rendering.

This means that if you're looking at a portal/menu with several dashboards, but can't see some at the top, you'll get the first one you can see.

Also clean up a lot of minor issues with less-common states.

Test Plan:
  - Created a portal with two private dashboards and a public dashboard.
  - Viewed it as another user, saw the default view show the dashboard I can actually see.
  - Minor fix: Disabled and enabled the hard-coded "Home" item, now worked cleanly with the right menu state.
  - Minor fix: added a motivator panel.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12871

Differential Revision: https://secure.phabricator.com/D20359
2019-04-02 15:21:27 -07:00
epriestley
5192ae4750 Update all existing ProfileMenuItems for the more-structured API
Summary:
Depends on D20357. Ref T13275. Now that there's a stronger layer between "stuff in the database" and "stuff on the screen", these subclasses all need to emit intermediate objects instead of raw, HTML-producing view objects.

This update is mostly mechanical.

Test Plan:
  - Viewed Home, Favorites, Portals, User Profiles, Project Profiles.
  - Clicked each item on each menu/profile type.
  - Added every (I think?) type of item to a menu and clicked them all.
  - Grepped for obsolete symbols (`newNavigationMenuItems`, `willBuildNavigationItems`).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20358
2019-04-02 15:20:39 -07:00
epriestley
950e9d085b In ProfileMenu, put more structure between "stored/configured items" and "display items"
Summary:
Depends on D20356. Ref T13275. See also T12871 and T12949.

Currently, the whole "ProfileMenu" API operates around //stored// items. However, stored items are allowed to produce zero or more //display// items, and we sometimes want to highlight display item X but render stored item Y (as is the case with "Link" items pointing at `?filter=xyz` on Workboards).

For the most part, this either: doesn't work; or works by chance; or is kind of glued together with hope and prayer (as in D20353).

Put an actual structural layer in place between "stored/configured item" and "display item" that can link them together more clearly. Now:

  - The list of `ItemConfiguration` objects (stored/configured items) is used to build an `ItemViewList`.
  - This handles the selection/highlighting/default state, and knows which display items are related to which stored items.
  - When we're all done figuring out what we're going to select and what we're going to highlight, it pops out an actual View which can build the HTML.

This requires API changes which are not included in this change, see next change.

This doesn't really do anything on its own, but builds toward a more satisfying fix for T12871. I'd hoped to avoid doing this for now, but wasn't able to get a patch I felt good about for T12871 built without fixing this first.

Test Plan: See next change.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20357
2019-04-02 15:19:59 -07:00
epriestley
971a272bf6 Automatically build mobile menus from navigation, and clean up external ProfileMenu API
Summary:
Depends on D20355. Ref T13275. Ref T13247. Currently, "Hamburger" menus are not automatically built from navigation menus. However, this is (I'm almost completely sure?) a reasonable and appropriate default behavior, and saves us some code around profile menus.

With this rule in place, we can remove `setApplicationMenu()` and `getApplicationMenu()` from `StandardPageView`, since they have no callers.

This also updates a lot of profile menu callsites to a new API which is added in the next change.

Test Plan: See the next two changes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275, T13247

Differential Revision: https://secure.phabricator.com/D20356
2019-04-02 15:17:44 -07:00
epriestley
47bf382435 Allow profile menu items to be locked to the top or bottom of the menu
Summary:
Depends on D20353. Ref T13275. This is just some small quality-of-life fixes:

  - When you add items to menus, they currently go below the "Edit Menu/Manage Menu" links by default. This isn't a very good place for them. Instead, lock "edit" items to the bottom of the menu.
  - Lock profile pictures to the top of the menu. This just simplifies things a little.
  - Show more iconography hints on the "edit menu items" UI.
  - Add a "drag stuff to do things" hint if some stuff can be dragged.

Test Plan:
  - Added new items to a Portal, they didn't go to the very bottom. Instead, they went above the "Edit/Manage" links; a sensible place for them.
  - Viewed the "edit menu items" screen, saw more hints and visual richness.
  - Viewed/edited Home, Projects, Portals, Favorites

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20355
2019-04-02 15:08:20 -07:00
epriestley
36a8b4ea17 When a ProfileMenu has a link item that adds URI parameters, highlight it when clicked
Summary:
Depends on D20352. Fixes T12949. If a user adds a link (for example, to a workboard) that takes you to the same page but with some URI parameters, we'd prefer to highlight the "link" item instead of the default "workboard" item when you click it.

For example, you add a `/query/assigned/` "link" item to a workboard, called "Click This To Show Tasks Assigned To Me On This Workboard", i.e. filter the current view.

This is a pretty reasonable thing to want to do. When you click it, we'd like to highlight that item to show that you've activated the "Assigned to Me" filter you added.

However, we currently highlight the thing actually serving the content, i.e. the "Workboard" item.

Instead:

  - When picking what to highlight, look through all the items for one with a link to the current request URI.
  - If we find one or more, pick the one that would be the default.
  - Otherwise, pick the first one.

This means that you can have several items like "?a=1", "?a=2", etc., and we will highlight them correctly when you click them.

This actual patch has some questionable bits (see some discussion in T13275), but I'd like to wait for stronger motivation to refactor it more extensively.

Test Plan:
  - On a portal, added a `?a=1` link. Saw it highlight properly when clikced.
  - On a workboard, added a link to the board itself with a different filter. Saw it highlight appropriately when clicked.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12949

Differential Revision: https://secure.phabricator.com/D20353
2019-04-02 14:45:33 -07:00
epriestley
408cbd633c On portals, make the "selected" / "default" logic more straightforward
Summary:
Depends on D20349. Ref T13275. Currently, a default item is selected as a side effect of generating the full list of items, for absolutely no reason.

The logic to pick the currently selected item can also be separated out pretty easily.

(And fix a bug in with a weird edge case in projects.)

This doesn't really change anything, but it will probably make T12949 a bit easier to fix.

Test Plan: Viewed Home / projects / portals, clicked various links, got same default/selection behavior as before.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20352
2019-04-02 14:44:26 -07:00
epriestley
d0d49d1efd Allow Portals to be edited, and improve empty/blank states
Summary:
Depends on D20348. Ref T13275. Portals are mostly just a "ProfileMenuEngine" menu, and that code is already relatively modular/flexible, so set that up to start with.

The stuff it gets wrong right now is mostly around empty/no-permission states, since the original use cases (project menus) didn't have any of these states: it's not possible to have a project menu with no content.

Let the engine render an "empty" state (when there are no items that can render a content page) and try to make some of the empty behavior a little more user-friendly.

This mostly makes portals work, more or less.

Test Plan: {F6322284}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13275

Differential Revision: https://secure.phabricator.com/D20349
2019-04-02 14:43:05 -07:00
epriestley
6bb9d3ac67 Allow users to add "ProfileMenu" items on mobile
Summary:
Depends on D20337. Fixes T12167. Ref T13272. On this page ("Favorites > Edit Favorites > Personal", for example) the curtain actions aren't available on mobile.

Normally, curtains are built with `Controller->newCurtainView()`, which sets an ID on the action list, which populates the header button. This curtain is built directly because there's no `Controller` handy.

To fix the issue, just set an ID. This could probably be cleaner, but that's likely a much more involved change.

Test Plan: Edited my favorites, narrowed the window, saw an "Actions" button.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T12167

Differential Revision: https://secure.phabricator.com/D20338
2019-03-27 16:05:55 -07:00
epriestley
6648942bc8 Don't allow "Conpherence" menu items to be added to editable menus if Conpherence is not installed
Summary: Depends on D20336. Ref T13272. Fixes T12745. If you uninstall Conpherence, "Edit Favorites" and other editable menu interfaces still allow you to add "Conpherence" items. Prevent this.

Test Plan:
  - Installed Conpherence: Saw option to add a "Conpherence" link when editing favorites menu.
  - Uninstalled Conpherence: No more option.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T12745

Differential Revision: https://secure.phabricator.com/D20337
2019-03-27 16:01:16 -07:00
epriestley
2c184bd4cd When query panels overheat, degrade them and show a warning
Summary:
Depends on D20334. Ref T13272. After recent changes to make overheating queries throw by default, dashboard panels now fail into an error state when they overheat.

This is a big step up from the hard-coded homepage panels removed by D20333, but can be improved. Let these panels render partial results when they overheat and show a human-readable warning.

Test Plan: {F6314114}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20335
2019-03-27 15:54:28 -07:00
epriestley
47856dc93f Track how many columns use a particular trigger
Summary:
Ref T5474. In 99% of cases, a separate "archived/active" status for triggers probably doesn't make much sense: there's not much reason to ever disable/archive a trigger explcitly, and the archival rule is really just "is this trigger used by anything?".

(The one reason I can think of to disable a trigger manually is because you want to put something in a column and skip trigger rules, but you can already do this from the task detail page anyway, and disabling the trigger globally is a bad way to accomplish this if it's in use by other columns.)

Instead of adding a separate "status", just track how many columns a trigger is used by and consider it "inactive" if it is not used by any active columns.

Test Plan: This is slightly hard to test exhaustively since you can't share a trigger across multiple columns right now, but: rebuild indexes, poked around the trigger list and trigger details, added/removed triggers.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20308
2019-03-25 14:04:55 -07:00
epriestley
252b6f2260 Provide basic scaffolding for workboard column triggers
Summary:
Depends on D20278. Ref T5474. This change creates some new empty objects that do nothing, and some new views for looking at those objects. There's no actual useful behavior yet.

The "Edit" controller is custom instead of being driven by "EditEngine" because I expect it to be a Herald-style "add new rules" UI, and EditEngine isn't a clean match for those today (although maybe I'll try to move it over).

The general idea here is:

  - Triggers are "real" objects with a real PHID.
  - Each trigger has a name and a collection of rules, like "Change status to: X" or "Play sound: Y".
  - Each column may be bound to a trigger.
  - Multiple columns may share the same trigger.
  - Later UI refinements will make the cases around "copy trigger" vs "reference the same trigger" vs "create a new ad-hoc trigger" more clear.
  - Triggers have their own edit policy.
  - Triggers are always world-visible, like Herald rules.

Test Plan: Poked around, created some empty trigger objects, and nothing exploded. This doesn't actually do anything useful yet since triggers can't have any rule behavior and columns can't actually be bound to triggers.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T5474

Differential Revision: https://secure.phabricator.com/D20279
2019-03-25 13:13:58 -07:00
epriestley
3940c8e1f4 Make the UI when you use an invalid cursor ("?after=19874189471232892") a little nicer
Summary:
Ref T13259. Currently, visiting a page that executes a query with an invalid cursor raises a bare exception that escapes to top level.

Catch this a little sooner and tailor the page a bit.

Test Plan: Visited `/maniphest/?after=335234234223`, saw a nicer exception page.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20295
2019-03-19 13:04:07 -07:00
epriestley
18b444e427 When queries overheat, raise an exception
Summary:
Ref T13259. Currently, queries set a flag and return a partial result set when they overheat. This is mostly okay:

  - It's very unusual for queries to overheat if they don't have a real viewer.
  - Overheating is rare in general.
  - In most cases where queries can overheat, the context is a SearchEngine UI, which handles this properly.

In T13259, we hit a case where a query with an omnipotent viewer can overheat: if you have more than 1,000 consecutive commits in the database with invalid `repositoryID` values, we'll overheat and bail out. This is pretty bad, since we don't process everything.

Change this beahvior:

  - Throw by default, so this stuff doesn't slip through the cracks.
  - Handle the SearchEngine case explicitly ("it's okay to overheat, we'll handle it").
  - Make `QueryIterator` disable overheating behavior: if we're iterating over all objects, we want to hit the whole table even if most of it is garbage.

There are some cases where this might cause new exception behavior that we don't necessarily want. For example, in Owners, each package shows "recent commits in this package". If you can't see the first 1,000 recent commits, you'd previously get a slow page with no results. Now you'll probably get an exception.

If these crop up, I think the best approach for now is to deal with them on a case-by-case basis and see how far we get. In the "Owners" case, it might be good to query by repositories you can see first, then query by commits in the package in those repositories. That should give us a better outcome than any generic behavior we could implement.

Test Plan:
  - Added 100000 to all repositoryID values for commits on my local install.
  - Before making changes, ran `bin/repository rebuild-identities --all --trace`. Saw the script process 1,000 rows and exit silently.
  - Applied the first part ("throw by default") and ran `bin/repository rebuild-identities`. Saw the script process 1,000 rows, then raise an exception.
  - Applied the second part ("disable for queryiterator") and ran the script again. Saw the script process all 15,000 rows without issues (although none are valid and none actually load).
  - Viewed Diffusion, saw appropriate NUX / "overheated" UIs.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20294
2019-03-19 13:02:59 -07:00
epriestley
46ab71f834 Fix "abou" typo of "about" in SearchEngine API methods
Summary: See <https://discourse.phabricator-community.org/t/minor-typo-abou-in-conduit-help/2498/2>.

Test Plan: Read carefully.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20280
2019-03-12 12:02:05 -07:00
epriestley
d1546209c5 Expand documentation for "transaction.search"
Summary: Depends on D20209. Ref T13255. It would probably be nice to make this into a "real" `*.search` API method some day, but at least document the features for now.

Test Plan: Read documentation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13255

Differential Revision: https://secure.phabricator.com/D20211
2019-02-25 10:52:29 -08:00
epriestley
312ba30714 Don't report search indexing errors to the daemon log except from "bin/search index"
Summary:
Depends on D20177. Fixes T12425. See <https://discourse.phabricator-community.org/t/importing-libphutil-repository-on-fresh-phabricator-triggers-an-error/2391/>.

Search indexing currently reports failures to load objects to the log. This log is noisy, not concerning, not actionable, and not locally debuggable (it depends on the reporting user's entire state).

I think one common, fully legitimate case of this is indexing temporary files: they may fully legitimately be deleted by the time the indexer runs.

Instead of sending these errors to the log, eat them. If users don't notice the indexes aren't working: no harm, no foul. If users do notice, we'll run or have them run `bin/search index` as a first diagnostic step anyway, which will now report an actionable/reproducible error.

Test Plan:
  - Faked errors in both cases (initial load, re-load inside the locked section).
  - Ran indexes in strict/non-strict mode.
  - Got exception reports from both branches in strict mode.
  - Got task success without errors in both cases in non-strict mode.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12425

Differential Revision: https://secure.phabricator.com/D20178
2019-02-19 11:17:11 -08:00
epriestley
454a762562 Queue search indexing tasks at a new PRIORITY_INDEX, not PRIORITY_IMPORT
Summary:
Depends on D20175. Ref T12425. Ref T13253. Currently, importing commits can stall search index rebuilds, since index rebuilds use an older priority from before T11677 and weren't really updated for D16585.

In general, we'd like to complete all indexing tasks before continuing repository imports. A possible exception is if you rebuild an entire index with `bin/search index --rebuild-the-world`, but we could queue those at a separate lower priority if issues arise.

Test Plan: Ran some search indexing through the queue.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13253, T12425

Differential Revision: https://secure.phabricator.com/D20177
2019-02-15 14:16:28 -08:00
epriestley
5892c78986 Replace all "setQueryParam()" calls with "remove/replaceQueryParam()"
Summary: Ref T13250. See D20149. Mostly: clarify semantics. Partly: remove magic "null" behavior.

Test Plan: Poked around, but mostly just inspection since these are pretty much one-for-one.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20154
2019-02-14 11:56:39 -08:00
epriestley
9f5e6bee90 Make the default behavior of getApplicationTransactionCommentObject() "return null" instead of "throw"
Summary:
Depends on D20115. See <https://discourse.phabricator-community.org/t/transaction-search-endpoint-does-not-work-on-differential-diffs/2369/>.

Currently, `getApplicationTransactionCommentObject()` throws by default. Subclasses must override it to `return null` to indicate that they don't support comments.

This is silly, and leads to a bunch of code that does a `try / catch` around it, and at least some code (here, `transaction.search`) which doesn't `try / catch` and gets the wrong behavior as a result.

Just make it `return null` by default, meaning "no support for comments". Then remove the `try / catch` stuff and all the `return null` implementations.

Test Plan:
  - Grepped for `getApplicationTransactionCommentObject()`, fixed each callsite / definition.
  - Called `transaction.search` on a diff with transactions (i.e., not a sourced-from-commit diff).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jbrownEP

Differential Revision: https://secure.phabricator.com/D20121
2019-02-07 14:56:38 -08:00
epriestley
f0c6ee4823 Add "Contact Numbers" so we can send users SMS mesages
Summary:
Ref T920. To send you SMS messages, we need to know your phone number.

This adds bare-bone basics (transactions, storage, editor, etc).

From here:

**Disabling Numbers**: I'll let you disable numbers in an upcoming diff.

**Primary Number**: I think I'm just going to let you pick a number as "primary", similar to how email works. We could imagine a world where you have one "MFA" number and one "notifications" number, but this seems unlikely-ish?

**Publishing Numbers (Profile / API)**: At some point, we could let you say that a number is public / "show on my profile" and provide API access / directory features. Not planning to touch this for now.

**Non-Phone Numbers**: Eventually this could be a list of other similar contact mechanisms (APNS/GCM devices, Whatsapp numbers, ICQ number, twitter handle so MFA can slide into your DM's?). Not planning to touch this for now, but the path should be straightforward when we get there. This is why it's called "Contact Number", not "Phone Number".

**MFA-Required + SMS**: Right now, if the only MFA provider is SMS and MFA is required on the install, you can't actually get into Settings to add a contact number to configure SMS. I'll look at the best way to deal with this in an upcoming diff -- likely, giving you partial access to more of Setings before you get thorugh the MFA gate. Conceptually, it seems reasonable to let you adjust some other settings, like "Language" and "Accessibility", before you set up MFA, so if the "you need to add MFA" portal was more like a partial Settings screen, maybe that's pretty reasonable.

**Verifying Numbers**: We'll probably need to tackle this eventually, but I'm not planning to worry about it for now.

Test Plan: {F6137174}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: avivey, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D19988
2019-01-23 13:39:56 -08:00
epriestley
11cf8f05b1 Remove "getApplicationTransactionObject()" from ApplicationTransactionInterface
Summary:
Depends on D19919. Ref T11351. This method appeared in D8802 (note that "get...Object" was renamed to "get...Transaction" there, so this method was actually "new" even though a method of the same name had existed before).

The goal at the time was to let Harbormaster post build results to Diffs and have them end up on Revisions, but this eventually got a better implementation (see below) where the Harbormaster-specific code can just specify a "publishable object" where build results should go.

The new `get...Object` semantics ultimately broke some stuff, and the actual implementation in Differential was removed in D10911, so this method hasn't really served a purpose since December 2014. I think that broke the Harbormaster thing by accident and we just lived with it for a bit, then Harbormaster got some more work and D17139 introduced "publishable" objects which was a better approach. This was later refined by D19281.

So: the original problem (sending build results to the right place) has a good solution now, this method hasn't done anything for 4 years, and it was probably a bad idea in the first place since it's pretty weird/surprising/fragile.

Note that `Comment` objects still have an unrelated method with the same name. In that case, the method ties the `Comment` storage object to the related `Transaction` storage object.

Test Plan: Grepped for `getApplicationTransactionObject`, verified that all remaining callsites are related to `Comment` objects.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T11351

Differential Revision: https://secure.phabricator.com/D19920
2018-12-20 15:16:19 -08:00
epriestley
937e88c399 Remove obsolete, no-op implementations of "willRenderTimeline()"
Summary:
Depends on D19918. Ref T11351. In D19918, I removed all calls to this method. Now, remove all implementations.

All of these implementations just `return $timeline`, only the three sites in D19918 did anything interesting.

Test Plan: Used `grep willRenderTimeline` to find callsites, found none.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T11351

Differential Revision: https://secure.phabricator.com/D19919
2018-12-20 15:04:49 -08:00
epriestley
43cf4edfb1 When waiting for long-running Harbormaster futures to resolve, close idle database connections
Summary:
Ref T13216. See PHI916. Harbormaster builds may be long-running, particularly if they effectively wrap `ssh ... ./run-huge-build.sh`. If we spend more than a few seconds waiting for futures to resolve, close idle database connections.

The general goal here is to reduce the held connection load for installs with a very large number of test runners.

Test Plan: Added debugging code to `phlog()` closures, saw connections closed while running builds.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13216

Differential Revision: https://secure.phabricator.com/D19824
2018-11-21 07:53:40 -08:00
epriestley
98690ee326 Update many Phabricator queries for new %Q query semantics
Summary: Depends on D19785. Ref T13217. This converts many of the most common clause construction pathways to the new %Q / %LQ / %LO / %LA / %LJ semantics.

Test Plan: Browsed around a bunch, saw fewer warnings and no obvious behavioral errors. The transformations here are generally mechanical (although I did them by hand).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: hach-que

Maniphest Tasks: T13217

Differential Revision: https://secure.phabricator.com/D19789
2018-11-15 03:48:10 -08:00
epriestley
da40f80741 Update PhabricatorLiskDAO::chunkSQL() for new %Q semantics
Summary:
Ref T13217. This method is slightly tricky:

  - We can't safely return a string: return an array instead.
  - It no longer makes sense to accept glue. All callers use `', '` as glue anyway, so hard-code that.

Then convert all callsites.

Test Plan: Browsed around, saw fewer "unsafe" errors in error log.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13217

Differential Revision: https://secure.phabricator.com/D19784
2018-11-13 08:59:18 -08:00
epriestley
5d4970d6b2 Fix a bug where "View as Query" could replace a saved query row by ID, causing workboard 404s
Summary:
Fixes T13208. See that task for details.

The `clone $query` line is safe if `$query` is a builtin query (like "open").

However, if it's a saved query we clone not only the query parameters but the ID, too. Then when we `save()` the query later, we overwrite the original query.

So this would happen in the database. First, you run a query and save it as the workboard default (query key "abc123"):

| 123 | abc123 | {"...xxx..."} |

Then we `clone` it and change the parameters, and `save()` it. But that causes an `UPDATE ... WHERE id = 123` and the table now looks like this:

| 123 | def456 | {"...yyy..."} |

What we want is to create a new query instead, with an `INSERT ...`:

| 123 | abc123 | {"...xxx..."} |
| 124 | def456 | {"...yyy..."} |

Test Plan:
  - Followed reproduction steps from above.
    - With just the new `save()` guard, hit the guard error.
    - With the `newCopy()`, got a new copy of the query and "View as Query" remained functional without overwriting the original query row.
  - Ran migration, saw an affected board get fixed.

Reviewers: amckinley, joshuaspence

Reviewed By: joshuaspence

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13208

Differential Revision: https://secure.phabricator.com/D19768
2018-11-01 05:44:49 -07:00
epriestley
8eb8e8e1d8 Make DiffusionCommitSearch accept modern (string) constants
Summary:
Depends on D19650. Ref T13197. Allow `SearchCheckboxesField` to have a "deprecated" map of older aliases, then convert them to modern values.

On the API method page, show all the values.

This technically resolves the issue in PHI841, although I still plan to migrate behind this.

Test Plan:
{F5875363}

- Queried audits, fiddled with `?status=1,audited`, etc.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13197

Differential Revision: https://secure.phabricator.com/D19651
2018-09-10 16:25:42 -07:00
epriestley
5a38b75f16 In Conduit, let checkbox constraints self-document
Summary:
Ref T13195. Ref PHI851. Currently, checkbox constraints don't tell you what values are supported on the API page.

You can figure this out with "Inspect Element" or by viewing the source code, but just render a nice table instead.

Test Plan: {F5862969}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19641
2018-09-06 08:44:16 -07:00
epriestley
a20f0674a9 Improve some documentation for "diffusion.commit.search"
Summary:
Ref T13195. See PHI851. Start by making some minor improvements here:

  - Clarify that the example of what constraints look like is just an example.
  - Add descriptions for parameters missing descriptions.

Test Plan: Looked at API method page, saw more helpful/complete instructions.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13195

Differential Revision: https://secure.phabricator.com/D19640
2018-09-06 08:39:21 -07:00
epriestley
77d7bb7af0 Document the Ferret "=" operator and improve related documentation
Summary:
Depends on D19529. See PHI778.

  - Document the "name" constraint as deprecated. All callers are likely better served by the "query" constraint.
  - Guide users toward the "query" constraint a little better.
  - Document the `=` syntax.

Test Plan: Read various new documentation.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19531
2018-07-23 12:44:43 -07:00
epriestley
71d4fa41c9 Support the Ferret "=" (exact match) operator in the actual query engine
Summary:
Ref PHI778. In D18492, I added support for parsing this operator, but did not actually implement it in the query engine.

Implementation is fairly straightforward. This supports querying for objects by exact title with `title:="exact title"`. This is probably a bad idea, but sometimes maybe useful anyway.

Test Plan: Queried for `title:="xxx"`, found only exact matches.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: ahoffer2

Differential Revision: https://secure.phabricator.com/D19529
2018-07-23 12:44:00 -07:00
epriestley
d84f866ca0 When search indexers contend for a lock, just yield
Summary:
Depends on D19503. Ref T13151. See PHI719. If you have something like a script which updates an object in a loop, we can end up queueing many search reindex tasks.

These tasks may reasonably contend for the lock, especially if the object is larger (lots of text and/or lots of comments) and indexing takes a few seconds.

This isn't concerning, and the indexers should converge to good behavior quickly once the updates stop.

Today, they'll spew a bunch of serious-looking lock exceptions into the log. Instead, just yield so it's more clear that there's (normally) no cause for concern here.

Test Plan: Ran `bin/search index Txxx --force` on a large object in multiple windows with a 0 second lock, saw an explicit yield instead of a lock exception.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13151

Differential Revision: https://secure.phabricator.com/D19504
2018-06-22 17:41:45 -07:00
epriestley
cbc71e75fa When queueing search index tasks, include the "objectPHID" in the task metadata
Summary:
Ref T13151. See PHI719. One minor hiccup in debugging the issue (which ended up being "revision has 100K comments") was that the `SearchWorker` did not show which object it was indexing.

Add `'objectPHID'` to the queue call so you can see which object is affected from the web UI.

Test Plan:
  - Stopped daemons.
  - Used `bin/search index D123 --background` to queue a search task.
  - Viewed task details in web UI from `/daemon/`.
    - Before change: no indication of which object was being indexed.
    - After change: page helpfully shows that the task is indexing D123.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13151

Differential Revision: https://secure.phabricator.com/D19502
2018-06-22 17:40:32 -07:00
epriestley
2951e0c86b Include owners packages in the MailableFunction datasource
Summary:
Ref T13151. See PHI684. Currently, the `MailableFunction` datasource does not include Owners packages, but they are valid subscribers and the `Mailable` datasource includes them.

Include them in the `MailableFunction` datasource, too.

Test Plan: Searched for revisions with particular package subscribers, got expected results in the UI (tokenizer knew about packages) and response.

Reviewers: amckinley, jmeador

Reviewed By: jmeador

Maniphest Tasks: T13151

Differential Revision: https://secure.phabricator.com/D19476
2018-06-07 12:02:50 -07:00
epriestley
9bb338c038 Revert the alternate menu names for applications
Summary: This reverts D18524. See that revision for discussion.

Test Plan: Viewed home menu, saw application names as menu items.

Differential Revision: https://secure.phabricator.com/D19308
2018-04-08 10:20:24 -07:00
epriestley
c5e4bd8187 Fix some minor errors (DarkConsole warning, unstable Ferret sort)
Summary:
DarkConsole could warn when "Analyze Query Plans" was not active.

`msort()` is not stable, so Ferret results with similar relevance could be returned out-of-order.

Test Plan: Saw fewer traces and more-stable result ordering.

Differential Revision: https://secure.phabricator.com/D19236
2018-03-18 15:12:25 -07:00