Summary:
Ref T12819. I gave this stuff a sweet code name because all the terms related to "fulltext" and "search" already mean 5 different things. It, uh, ferrets out documents for you?
I'm building this to work a lot like the existing ngram index, which seems to work pretty well. If this sticks, it will auto-resolve the join issue (in T12443) by letting us do the entire thing locally in a JOIN and thus dodge a lot of mess.
This index gets built alongside other indexes, but only shows up in the UI if you have prototypes enabled. If you do, it appears under the existing fulltext field in Maniphest. No existing functionality is affected or disrupted.
NOTE: The query engine half of this is still EXTREMELY primitive, and this probably performs worse than the existing field for now. If this doesn't show obvious signs of being awful on `secure` I'll improve that in followup changes.
Test Plan:
Indexed my tasks, ran some simple queries, got the results I wanted, even for queries "ko", "k", "v0.1".
{F5147746}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12819, T12443
Differential Revision: https://secure.phabricator.com/D18484
Summary:
See PHI42. Currently, `maniphest.search` incorrectly applies this default (group by priority) to all queries via Conduit.
The correct behavior is to apply no grouping constraint.
I think this is also a reasonable general behavior, and the current code seems to date from D6960 in 2013 and didn't seem particularly carefully considered.
This is a minor compatibility break -- saved queries which are more than 4 years old might change their group behavior. I'll note this in the change logs but expect essentially no one to be affected.
Test Plan: Ran a `maniphest.search` Conduit call and observed the underlying query. Before this change, it executed `ORDER BY priority, id`. After this change, it correctly executed `ORDER BY id` only.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D18459
Summary:
See PHI39. This adds support for editing parents and subtasks of a task via Conduit.
It might be nice to tie this into the `PhabricatorObjectRelationship` stuff eventually, but I think we'd effectively end up in the same place anyway in terms of what the API looks like.
Test Plan: {F5116163}
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D18456
Summary: Fixes T12929. Sets a create transaction if new.
Test Plan: test a new task over email via command line
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T12929
Differential Revision: https://secure.phabricator.com/D18279
Summary:
Fixes T12870. Ref T12845.
Technically, this addresses the core issue in T12845 too, but I'm going to convert the rest of the `custom:...` types before closing that.
In particular, for T12870:
- Validates that keywords are unique across priorities.
- Fixes missing newline in documentation.
- Updates documentation to note that keywords are now mandatory and must be unique across priorities.
Test Plan: Edited, deleted and mangled all the Maniphest custom options (priorities, statuses, points, subtypes).
Reviewers: chad, amckinley
Reviewed By: amckinley
Maniphest Tasks: T12870, T12845
Differential Revision: https://secure.phabricator.com/D18165
Summary:
Ref T12124. This is a fairly narrow fix for existing saved EditEngine forms with a default priority value.
These saved forms have a numeric (or probably "string-numeric") default value, like "50". They lost their meaning after D18111, when "50" no longer appears in the dropdown. Instead, these forms all select the highest available priority.
At time of writing, this form was broken on this install, for example:
> https://secure.phabricator.com/transactions/editengine/maniphest.task/view/13/
Additionally, `/task/edit/form/123/?priority=...` (for templating forms) stopped working with `priority=50`. This isn't nearly as important, but a larger and more sudden compatiblity break than we need to make.
Add support for an "alias map" on `<select />` controls, so if the value comes in with something we don't recognize we'll treat it like some other value. Then alias all the numeric constants -- and other keywords -- to the right constants.
This ended up only affecting the `<select />` control in the web UI.
Test Plan:
- On `stable`, created a form with "Priority: Low".
- Before patch: form has "Priority: Unbreak Now!" on `master`.
- After patch: form has "Priority: Low" again.
- Used `?priority=25`, `?priority=wish`, `?priority=wishlist` to template forms: all forms worked.
Reviewers: amckinley, chad
Reviewed By: amckinley
Maniphest Tasks: T12124
Differential Revision: https://secure.phabricator.com/D18134
Summary:
Ref T12124. Currently, Conduit provides a fairly rough error message if you provide an invalid priority.
Instead, provide a more tailored message. Also, block `!!unknown!!` except from web edits.
Test Plan:
Before:
{F5007964}
After:
{F5007965}
Also, changed a priority to `999` in the database, edited it with the normal web UI form, it let me make the edit without being forced to adjust the priority.
Reviewers: amckinley, chad
Reviewed By: amckinley
Maniphest Tasks: T12124
Differential Revision: https://secure.phabricator.com/D18135
Summary: Fixes T12124. Changes `ManiphestEditEngine` to populate the select using priority keywords instead of the integer value. Marks `maniphest.querystatuses` as frozen. Adds a new Conduit method for fetching potential task statuses.
Test Plan: Created tasks and changed their priorities, observed that transactions in the DB still have the same type (integers as strings). Invoked `maniphest.update` with `priority => '90'` and observed that it still works. Invoked `maniphest.edit` with `priority => 'unbreak'` and observed that it now works.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T12124
Differential Revision: https://secure.phabricator.com/D18111
Summary: Does the UI work that's part of T12234 and adds migrations for both of the old-style duplicate transactions.
Test Plan:
- Started with a clean DB.
- Checked out really old code that marks tasks as dupes using comments.
- Made a bunch of tasks and closed some as dupes. Made a bunch of additional comments.
- Checked out D10427 and did a `storage upgrade`.
- Made a bunch more new tasks and dupes.
- Snapshotted DB.
- Ran migration repeatedly until all expected edges showed up in the `phabricator_maniphest.edge`table.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T12234
Differential Revision: https://secure.phabricator.com/D18037
Summary: Try to dis-ambiguate various button types and colors. Moves `simple` to `phui-button-simple` and moves colors to `button-color`.
Test Plan: Grep for buttons still inline, UIExamples, PHUIX, Herald, and Email Preferences.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D18077
Summary:
See D18018. Ref T12787. This doesn't actually work; we started publishing these stories as a side effect of converting to ModularTransactions, then I fixed the rendering.
This mechanism has very few callsites and I suspect we may want to get rid of it (see T12787) so just keep publishing these stories for now.
Test Plan: Changed the point value of a task, saw a feed story both before and after the patch.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12787
Differential Revision: https://secure.phabricator.com/D18059
Summary: Fixes T12787. Modular Transactions don't actually support `shouldHideForFeed()`. I'll add some discussion to the task.
Test Plan: Created a subtask, saw no more "X reopened Y, a subtask of P" feed story.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12787
Differential Revision: https://secure.phabricator.com/D18058
Summary: Start on plan outlined in T12124. Adds a new Conduit method for querying information about task priorities.
Test Plan: Ran locally; observed expected output: {F4979109}
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D18035
Summary:
Ref T12314. Open to counterdiffs / iterating / suggestions / skipping most or all of this, mostly just throwing this out there as a maybe-reasonable first pass.
When a task has a subtype (like "Plant" or "Animal"), provide some hints on the task list, workboards, and task detail.
To make these hints more useful, allow subtypes to have icons and colors.
Also use these icons and colors in the typeahead tokens.
The current rule is that we show the subtype if it's not the default subtype. Another rule we could use is "show the subtype if there's more than one subtype defined", but my guess is that most installs will mostly have something like "normal task" as the default subtype.
Test Plan:
The interfaces this affects are: task detail view, task list view, workboard cards, subtype typeahead.
{F3539128}
{F3539144}
{F3539167}
{F3539185}
Reviewers: chad
Reviewed By: chad
Subscribers: johnny-bit, bbrdaric, benwick, fooishbar
Maniphest Tasks: T12314
Differential Revision: https://secure.phabricator.com/D17451
Summary: Fixes T12744. Unclear why `null` doesn't work here but does for the title, but `!strlen` seems to work fine in both cases.
Test Plan: Create a new task, check mail folder, see [Created]
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T12744
Differential Revision: https://secure.phabricator.com/D18002
Summary: Grep for phui-tag-shade and verify we're no longer calling shade-color directly.
Test Plan: Search, workboard, story points, etc.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D17993
Summary:
Ref T7664. The current algorithm for moving task subpriorities can end up stuck in a real sticky swamp in some unusual situations.
Instead, use an algorithm which works like this:
- When we notice two tasks are too close together, look at the area around those tasks (just a few paces).
- If things look pretty empty, we can just spread the tasks out a little bit.
- But, if things are still real crowded, take another look further.
- Keep doing that until we're looking at a real nice big spot which doesn't have too many tasks in it in total, even if they're all in one place right now.
- Then, move 'em out!
Also:
- Just swallow our pride and do the gross `INSERT INTO ... "", "", "", "", "", "", ... ON DUPLICATE KEY UPDATE` to bulk update.
- Fix an issue where a single move could cause two different subpriority recalculations.
Test Plan:
- Changed `ManiphesTaskTestCase->testTaskAdjacentBlocks()` to insert 1,000 tasks with identical subpriorities, saw them spread out in 11 queries instead of >1,000.
- Dragged tons of tasks around on workboards.
- Ran unit tests.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7664
Differential Revision: https://secure.phabricator.com/D17959
Summary: It's an icon. For story points.
Test Plan: Set some points, see icon.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D17915
Summary: I think this is the correct fix, sets a consistent value for transactions, old and new, for Maniphest point values.
Test Plan:
Edit title, see no point feed story, set points, see point story, set points to same value, see no story, remove points, see remove point story.
{F4958233}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D17885
Summary: Swaps out hovercard boring view for super cool workboard card view. Will have more diffs to add additional information down the road.
Test Plan: {F4921092}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D17769
Summary:
Ref T12509. Many of the calls to HMAC+SHA1 are just to compute cachekeys for remarkup objects.
Make these use HMAC+SHA256 instead. There is no downside to swapping these since they just cause a cache miss in the worst case.
I also plan to get rid of `PhabricatorMarkupInterface` eventually, but this doesn't go that far.
Test Plan: Browsed some different types of documents (tasks, legalpad documents, phame blogs / posts, pholio mocks, etc).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12509
Differential Revision: https://secure.phabricator.com/D17631
Summary:
Fixes T12461. This returns the field as a dictionary with a `"raw"` value, so we could eventually do this if we want without breaking the API:
```
{
"type": "remarkup",
"raw": "**raw**",
"html": "<strong>raw</strong>",
"text": "raw"
}
```
Test Plan: Called `maniphest.search`, reviewed output.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12461
Differential Revision: https://secure.phabricator.com/D17603
Summary: Ref T12450. These are like 95% my fault, but Elastic appears to spell the name "Elasticsearch" consistently in their branding.
Test Plan: `grep ElasticSearch`
Reviewers: chad, 20after4
Maniphest Tasks: T12450
Differential Revision: https://secure.phabricator.com/D17601
Summary:
The goal is to make fulltext search back-ends more extensible, configurable and robust.
When this is finished it will be possible to have multiple search storage back-ends and
potentially multiple instances of each.
Individual instances can be configured with roles such as 'read', 'write' which control
which hosts will receive writes to the index and which hosts will respond to queries.
These two roles make it possible to have any combination of:
* read-only
* write-only
* read-write
* disabled
This 'roles' mechanism is extensible to add new roles should that be needed in the future.
In addition to supporting multiple elasticsearch and mysql search instances, this refactors
the connection health monitoring infrastructure from PhabricatorDatabaseHealthRecord and
utilizes the same system for monitoring the health of elasticsearch nodes. This will
allow Wikimedia's phabricator to be redundant across data centers (mysql already is,
elasticsearch should be as well).
The real-world use-case I have in mind here is writing to two indexes (two elasticsearch clusters
in different data centers) but reading from only one. Then toggling the 'read' property when
we want to migrate to the other data center (and when we migrate from elasticsearch 2.x to 5.x)
Hopefully this is useful in the upstream as well.
Remaining TODO:
* test cases
* documentation
Test Plan:
(WARNING) This will most likely require the elasticsearch index to be deleted and re-created due to schema changes.
Tested with elasticsearch versions 2.4 and 5.2 using the following config:
```lang=json
"cluster.search": [
{
"type": "elasticsearch",
"hosts": [
{
"host": "localhost",
"roles": { "read": true, "write": true }
}
],
"port": 9200,
"protocol": "http",
"path": "/phabricator",
"version": 5
},
{
"type": "mysql",
"roles": { "write": true }
}
]
Also deployed the same changes to Wikimedia's production Phabricator instance without any issues whatsoever.
```
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Tags: #elasticsearch, #clusters, #wikimedia
Differential Revision: https://secure.phabricator.com/D17384
Summary: These were once ordered, but I think we switched to being defined in the Engine and never implemented the sorts there. This adds sort ordering to Tasks, Projects, and Repositories.
Test Plan: Review Favorites Menu in local install, see order is now set per the engine. Click Edit Favorites, and re-order. See order sticks.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D17490
Summary:
Fixes T12378. Two minor issues here:
- CAN_INTERACT on tasks uses "USER", but should just use the view policy, which may be more permissive ("PUBLIC").
- CAN_INTERACT is currently prevented from being "PUBLIC" by additional safeguards. Define an explicit capability object for the permission which returns `true` from `shouldAllowPublicPolicySetting()`.
Test Plan:
- Viewed an unlocked task as a logged-out user, saw "login to comment" instead of "locked".
- Viewed a locked task as a logged-out user, saw "locked".
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12378
Differential Revision: https://secure.phabricator.com/D17485
Summary: Ref T12337. This just fills out a couple more task relationships.
Test Plan: Viewed the edges in the Conduit console, queried for them.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12337
Differential Revision: https://secure.phabricator.com/D17465
Summary:
Ref T12335. See that task for discussion. Here are the behavioral changes:
- Statuses can be flagged with `locked`, which means that tasks in that status are locked to further discussion and interaction.
- A new "CAN_INTERACT" permission facilitates this. For most objects, "CAN_INTERACT" is just the same as "CAN_VIEW".
- For tasks, "CAN_INTERACT" is everyone if the status is a normal status, and no one if the status is a locked status.
- If a user doesn't have "Interact" permission:
- They can not submit the comment form.
- The comment form is replaced with text indicating "This thing is locked.".
- The "Edit" workflow prompts them.
This is a mixture of advisory and hard policy checks but sholuld represent a reasonable starting point.
Test Plan: Created a new "Locked" status, locked a task. Couldn't comment, saw lock warning, saw lock prompt on edit. Unlocked a task.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12335
Differential Revision: https://secure.phabricator.com/D17453
Summary:
Ref T12314. When we pick an "Edit" form for a subtyped object, only consider forms with the same subtype.
For example, editing an "Animal" uses the forms with subtype "animal" which are marked as edit forms.
This also makes "Create Subtask" carry the parent task's type.
Test Plan:
- Edited an Animal, got an animal edit form.
- Edited a normal task, got a normal task form.
- Edited a paste, got the normal workflow.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12314
Differential Revision: https://secure.phabricator.com/D17445
Summary:
Ref T12314. Allow tasks to be queried by subtype using a typeahead.
Open to a better default icon. I'll probably let you configure them later.
Just hide this constraint if there's only one subtype.
Test Plan:
- Searched for subtypes.
- Verified that the control hides if there is only one subtype.
{F3492293}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12314
Differential Revision: https://secure.phabricator.com/D17444
Summary:
Ref T12314. If you set a form to have the "plant" subtype, then create a task with it, save "plant" as the task subtype.
For Conduit, the default subtype is used by default, but a new "subtype" transaction is exposed. You can apply this transaction at create time to create an object of a certain subtype, or at any later time to change the subtype of an object.
This still doesn't do anything particularly useful or interesting.
Test Plan:
- Created a non-subtyped object (a Paste).
- Created "task" and "plant" tasks via different forms.
- Created "default" and "plant" tasks via Conduit.
- Changed the subtype of a task via Conduit.
- Tried to set a bad subtype.
{F3492061}
{F3492066}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12314
Differential Revision: https://secure.phabricator.com/D17443
Summary:
Ref T12314. This adds a "Change Form Subtype" workflow to the EditEngine form configuration screen, for forms that edit/create objects which support subtyping (for now, only tasks).
For example, this allows you to switch a form from being a "task" form to a "plant" or "animal" form.
Doing this doesn't yet do anything useful or interesting. I'm also not showing it in the UI yet since I'm not sure what we should make that look like (presumably, we should just echo whatever UI we end up with on tasks).
Test Plan:
- Changed the subtype of a task form.
- Verified that the "Change Subtype" action doesn't appear on other forms (for example, those for Pastes).
{F3491374}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12314
Differential Revision: https://secure.phabricator.com/D17442
Summary: Ref T12314. Provides a field on tasks for storing subtypes. Does nothing interesting yet.
Test Plan:
- Ran storage upgrade.
- Created some tasks.
- Looked in the database.
- Used Conduit to query some tasks.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12314
Differential Revision: https://secure.phabricator.com/D17441
Summary:
Ref T12314. Builds toward letting you define "animal" and "plant" tasks.
This just adds some configuration. I'll probably add some more quality-of-life options (like "icon") later but these are the only bits I'm sure I'll need.
Test Plan:
- Configured sensible subtypes.
- Tried to configure bad subtypes: bad key, missing "default", duplicate keys. Got sensible error messages.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12314
Differential Revision: https://secure.phabricator.com/D17440
Summary:
Ref T12319. Currently, `bin/lipsum` uses substring matches against human-readable text to chose which objects to generate.
Instead:
- Use separate selector keys which are guaranteed to be unique.
- When a match is exact, select only that generator.
- When a match is ambiguous, fail and warn the user.
Test Plan: Generated several types of objects, tried to generate ambiguous objects like "e".
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12319
Differential Revision: https://secure.phabricator.com/D17420
Summary: Ref T12319. The product name is misspelled in some methods, and a few places in the documentation.
Test Plan: `grep`
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12319
Differential Revision: https://secure.phabricator.com/D17419
Summary: Ref T10390. Simplifies dropdown by rolling out canUseInPanel in useless panels
Test Plan: Add a query panel, see less options.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T10390
Differential Revision: https://secure.phabricator.com/D17341
Summary: Ref T12136. This just yanks the band-aid off. Fundamentally these were useful well before Dashboards and advanced bucketing, but not so much any more. They also have some performance hit.
Test Plan: Add some tasks and diffs onto a new instance, see there is no count on the home menu bar.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T12136
Differential Revision: https://secure.phabricator.com/D17238
Summary: Ref T5867. Instead of hard-coding projects, tasks and repositories, let EditEngines say "I want a quick create item" so third-party code can also hook into the menu without upstream changes.
Test Plan: Saw same default items in menu.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T5867
Differential Revision: https://secure.phabricator.com/D17215
Summary:
Fixes T7076. This could probably use some tweaking but should get the basics in place.
This shows overall object state (e.g., "Needs Review"), not individual viewer state (e.g., "you need to review this"). After the bucketing changes it seems like we're mostly in a reasonable place on showing global state instead of viewer state. This makes the overall change much easier than it might otherwise have been.
Test Plan: {F2351867}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7076
Differential Revision: https://secure.phabricator.com/D17193
Summary:
Ref T12074. The "v3" API methods (`*.search`, `*.edit`) are currently marked as "unstable", but they're pretty stable and essentially all new code should be using them.
Although these methods are seeing some changes, almost all changes are additive (support for new constraints or attachemnts) and do not break backward compatibility. We have no major, compatibility-breaking changes planned.
I don't want to mark the older methods "deprecated" yet since `arc` still uses a lot of them and there are some capabilities not yet available on the v3 methods, but introduce a new "frozen" status with pointers to the new methods.
Overall, this should gently push users toward the newer methods.
Test Plan: {F2325323}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12074
Differential Revision: https://secure.phabricator.com/D17158
Summary:
Ref T12074. This allows callers to identify which columns an object appears in (currently, always tasks).
There are a few major cases:
- Object is in a normal column: we return column information.
- Object is in a proxy column (subproject or milestone). For example, when you look at the board for "Some Parent Project", the task might show up in a milestone column. I've chosen to not return anything in this case: you can figure out that the task is there by looking at the project structure, and this is kind of an internal artifact of the implementation and probably not useful to callers.
- Project does not have a workboard: we return nothing.
These seem fairly reasonable, I think?
Test Plan:
- Queried for tasks, using the "columns" attachment.
- Dragged a task across a board, querying it repeatedly. Got expected results for normal column (the column), subprojects with no board (nothing), milestones with no board (nothing) and mielstones/subprojects with a board (the column on //that// board, only, not the proxy column on the parent).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12074
Differential Revision: https://secure.phabricator.com/D17156
