1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-25 08:12:40 +01:00
Commit graph

440 commits

Author SHA1 Message Date
epriestley
50debecf52 Allow Almanac namespaces to be searched by ngram index
Summary: Ref T6741. Ref T10246. This is largely modernization, but will partially support namespace locking in Almanac.

Test Plan:
Searched for Almanac networks by name substring.

{F1121740}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6741, T10246

Differential Revision: https://secure.phabricator.com/D15322
2016-02-22 04:58:18 -08:00
epriestley
959bb16d0f Allow Almanac services to be searched by substring
Summary: Ref T10246. Build an ngram index for Almanac services, and use it to support improved search.

Test Plan: {F1121725}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10246

Differential Revision: https://secure.phabricator.com/D15321
2016-02-22 04:58:03 -08:00
epriestley
1b6ddae6b2 Allow Almanac devices to be queried and sorted by name
Summary:
Ref T10205. Ref T10246. This is general modernization, but also supports fixing the interface datasource in T10205.

  - Update Query.
  - Update SearchEngine.
  - Use an ngrams index for searching names efficiently.

Test Plan:
  - Ran migrations.
  - Searched Almanac devices by name.
  - Created a new device, searched for it by name.

{F1121303}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10205, T10246

Differential Revision: https://secure.phabricator.com/D15319
2016-02-21 10:44:46 -08:00
epriestley
dc7d0b4a56 Make repository callsigns optional
Summary:
Ref T4245. This could still use a little UI smoothing, but:

  - Don't require a callsign on the create flow (you can add one later in "Edit Basic Information" if you want).
  - Allow existing callsigns to be removed.

Test Plan:
  - Created a new repository with no callsign.
  - Cloned it; pushed to it.
  - Browsed around Diffusion a bunch.
  - Visited a commit URI.
  - Added a callsign to it.
  - Removed the callsign again.
  - Referenced it with `R22` in remarkup.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4245

Differential Revision: https://secure.phabricator.com/D15305
2016-02-18 10:36:13 -08:00
epriestley
71ee97d74f Give Owners real view and edit policies
Summary: Fixes T10360. In modern code, most of the meat is automatic.

Test Plan:
  - Edited view policy and edit policy from web UI.
  - Viewed package, saw policy badge in header.
  - Tried to edit a package as a user without permission, got appropriate disabled states and errors.
  - Changed policies via Conduit.
  - Tried to view a package as a user without permission.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10360

Differential Revision: https://secure.phabricator.com/D15275
2016-02-15 11:56:35 -08:00
epriestley
de379c8b61 Allow workboard sorting and filtering to be saved as defaults
Summary:
Fixes T6641. This allows users who have permission to edit a project to use "Save as Default" to save the current order and filter as defaults for the project.

These are per-board defaults, and apply to all users. The rationale is that I think the best default ordering/filtering depends mostly on the board, not the viewer.

This seems to align with most requests in the task, although rationale is a bit light. But, for example, it seems reasonable you might want to change the default filter to "All Tasks" on a sprint board, so you can see what's in the "Done" column.

This also fixes some minor issues I ran into:

  - Herald could hit an issue while checking permissions if the project was a subproject and a non-member had a triggering rule.
  - "Advanced filter..." did not prefill with the current filter.

Test Plan:
  - Set default order and filter on a workboard.
  - Reloaded board, saw settings stick.
  - Tried to edit a board as an unprivileged user (disabled menu items, error).
  - Reviewed transaction log.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6641

Differential Revision: https://secure.phabricator.com/D15260
2016-02-12 07:47:23 -08:00
epriestley
e9f3807cf5 Add a "points" field to tasks
Summary:
Currently never read or written.

Supports fractions.

There's no such thing as an unsigned double so this also supports negative values, technically, although I'll eventually prevent this in the UI.

Test Plan: `bin/storage upgrade`, then created and edited a task. Nothing was different.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15218
2016-02-08 15:28:00 -08:00
epriestley
32225d1dd0 Remove three ancient columns from Maniphest tasks: attached, projectPHIDs, ccPHIDs
Summary:
Before edges, we stored some of this stuff directly on tasks.

  - `attached` was migrated to edges in Jan 2013.
  - `projectPHIDs` was never used, as far as I can tell?
  - `ccPHIDs` was migrated away and dropped more than a year ago.

None of these columns are used in modern code (instead, modern code uses edges).

Test Plan: `grep`, browsed around, `bin/storage upgrade`, unit tests.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15216
2016-02-08 14:10:02 -08:00
epriestley
b6a38b403c Add storage and read logic for workboard card cover photos
Summary:
No way to set photos yet, but if you magic them in they work.

Primarily, this consolidates rendering logic so the move + edit + view controllers all run the same code to do tags / cover photos.

Test Plan: {F1095870}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15201
2016-02-06 15:34:41 -08:00
epriestley
90a0459821 Roughly implement milestone columns on workboards
Summary:
Ref T10010. These aren't perfect but I think (?) they aren't horribly broken.

  - When a project is a parent project, destroy (as far as the user can tell) any custom columns.
  - When a project has milestones, automatically generate columns on the project's workboard (if it has a workboard).
  - When you move tasks between milestones, add the proper milestone tag.
  - When you move tasks out of milestones back into the backlog, add the proper parent project tag.
  - (Plenty of UI / design stuff to adjust.)

Test Plan:
  - Dragged stuff between milestone columns.
  - Used a normal workboard.
  - Wasn't able to find any egregiously bad cases that did anything terrible.

{F1088224}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D15171
2016-02-03 16:37:59 -08:00
epriestley
9d125b459e Use large text columns to store IP addresses
Summary: Fixes T10259. There was no real reason to do this `ip2long()` stuff in the first place -- it's very slightly smaller, but won't work with ipv6 and the savings are miniscule.

Test Plan:
  - Ran migration.
  - Viewed logs in web UI.
  - Pulled and pushed.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10259

Differential Revision: https://secure.phabricator.com/D15165
2016-02-02 10:13:14 -08:00
epriestley
30473549ac Add a basic pull event log for debugging repository cloning
Summary:
Ref T10228. This is currently quite limited:

  - No UI.
  - No SSH support.

My primary goal is to debug the issue in T10228. In the long run we can expand this to be a bit fancier.

Test Plan:
Made various valid and invalid clones, got sucess responses and not-so-successful responses, viewed the log table for general corresponding messages and broad sanity.

Ran GC via `bin/phd debug trigger`, no issues.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10228

Differential Revision: https://secure.phabricator.com/D15127
2016-01-28 08:18:34 -08:00
epriestley
06aa207960 Allow users to have profile icons
Summary: Ref T10054. This primarily improves aesthetics and consistency for member/wathcher lists in projects.

Test Plan:
{F1068873}

{F1068874}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10054

Differential Revision: https://secure.phabricator.com/D15103
2016-01-24 09:58:01 -08:00
epriestley
9f56a014e2 Migrate existing projects to retain "Workboard" as default item
Summary:
Ref T10054. Ref T6961.

  - Existing projects with workboards had "Workboard" as the default menu item. Retain this behavior.
  - Populate the recently-added `hasWorkboard` flag so we can do a couple of things a little faster (see T6961).

Test Plan:
  - Ran migration.
  - Verified a bunch of projects looked sensible/correct after the migration.
  - Created a workboard, verified `hasWorkboard` got set properly.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6961, T10054

Differential Revision: https://secure.phabricator.com/D15093
2016-01-22 09:44:43 -08:00
epriestley
8463ad2659 Replace subscribe/unsubscribe for projects with explicit mail setting
Summary:
Ref T10054. Ref T6113. Users can currently subscribe to projects, which causes them to receive:

  # mail about project membership changes, description changes, etc; and
  # mail to the project, e.g. when the project is added as a subscriber on a task, or a reviewer on a revision.

Almost no one cares about (1), and after D15061 you can use Herald to get this stuff if you really want it. (It will get progressively more annoying in the future with external membership sources causing automated project membership updates.)

A lot of users are confused about (2) and how it relates to membership, watching, etc, and most users who want (2) don't want (1).

Instead, add an explicit option for this and explain what it does.

This is fairly verbose but I've hidden it on the member/watch screen, which is now the "explain how projects work" screen, I guess.

Test Plan:
{F1064929}

{F1064930}

{F1064931}

  - Disabled/enabled mail for a project.
  - Sent mail to a project with mail disabled, verified I didn't get a copy.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6113, T10054

Differential Revision: https://secure.phabricator.com/D15065
2016-01-19 19:39:02 -08:00
epriestley
f24318f308 Make "profile menu" configuration mostly work
Summary:
Ref T10054. This does a big chunk of the legwork to let users reconfigure profile menus (currently, just project menus).

This includes:

  - Editing builtin items (e.g., you can rename the default items).
  - Creating new items (for now, only links are available).

This does not yet include:

  - Hiding items.
  - Reordering items.
  - Lots of fancy types of items (dashboards, etc).
  - Any UI changes.
  - Documentation (does feature: TODO link for documentation).

Test Plan:
{F1060695}

{F1060696}

{F1060697}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10054

Differential Revision: https://secure.phabricator.com/D15010
2016-01-13 11:45:31 -08:00
epriestley
1eab16c395 Move repository URIs to a dedicated index
Summary:
Ref T4705 (there are also some other adjacent related tasks dealing with URIs).

Currently, we issue a "get repositories matching URIs: ..." query by loading every possible repository and then checking their URIs in PHP.

Instead, put URIs in a separate table. I plan for each repository to potentially have multiple URIs soon, so this prepares for that.

Test Plan:
  - Ran migrations.
  - Looked at index table, made sure it appeared sensible.
  - Ran some queries by `uri` to find repositories, found the repositories I expected.
  - Updated the remote URI of a repository, saw queries / index update appropriately.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4705

Differential Revision: https://secure.phabricator.com/D15005
2016-01-13 09:34:31 -08:00
epriestley
96ebd35824 Change repository "Clone/Checkout As" to "Short Name"
Summary:
Ref T4245.

  - Rename "Clone/Checkout As" to "Short Name" in the UI.
  - Allow any repository to have a short name, not just hosted repositories.

Test Plan:
  - Ran migration.
  - Reviewed old transactions, saw they looked good.
  - Edited an existing repository's short name.
  - Gave an imported repository a new short name.
  - Removed a repository's short name.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4245

Differential Revision: https://secure.phabricator.com/D14989
2016-01-11 09:17:32 -08:00
epriestley
0b3d10c3da Enforce sensible, unique clone/checkout names for repositories
Summary:
Fixes T7938.

  - Primarily, users can currently shoot themselves in the foot by putting `../../etc/passwd` and other similar nonsense in these fields (this is not dangerous, but also does not work). Require sensible names.
  - Enforce uniqueness so these names can be used in URIs and as identifiers in the future.
  - (This doesn't start actually using them for anything fancy yet.)

Test Plan:
  - Gave several repositories clone names: a valid name, two duplicate names, an invalid, name, some with no names.
  - Ran migrations.
  - Got clean conversion for valid names, appropriate errors for invalid/duplicate names.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7938

Differential Revision: https://secure.phabricator.com/D14986
2016-01-11 02:06:44 -08:00
epriestley
9ab22e21b3 Allow installs to customize project icons
Summary:
Ref T10010. Ref T5819. General alignment of the stars:

  - There were some hacks in Conduit around stripping `fa-...` off icons when reading and writing that I wanted to get rid of.
  - We probably have room for a subtitle in the new heavy nav, and using the icon name is a good starting point (and maybe good enough on its own?)
  - The project list was real bad looking with redundant tag/names, now it is very slightly less bad looking with non-redundant types?
  - Some installs will want to call Milestones something else, and this gets us a big part of the way there.
  - This may slightly help to reinforce "tag" vs "policy" vs "group" stuff?

---

I'm letting installs have enough rope to shoot themselves in the foot (e.g., define 100 icons). It isn't the end of the world if they reuse icons, and is clearly their fault.

I think the cases where 100 icons will break down are:

  - Icon selector dialog may get very unwieldy.
  - Query UI will be pretty iffy/huge with 100 icons.

We could improve these fairly easily if an install comes up with a reasonable use case for having 100 icons.

---

The UI on the icon itself in the list views is a little iffy -- mostly, it's too saturated/bold.

I'd ideally like to try either:

  - rendering a "shade" version (i.e. lighter, less-saturated color); or
  - rendering a "shade" tag with just the icon in it.

However, there didn't seem to be a way to do the first one right now (`fa-example sh-blue` doesn't work) and the second one had weird margins/padding, so I left it like this for now. I figure we can clean it up once we build the thick nav, since that will probably also want an identical element.

(I don't want to render a full tag with the icon + name since I think that's confusing -- it looks like a project/object tag, but is not.)

Test Plan:
{F1049905}

{F1049906}

Reviewers: chad

Reviewed By: chad

Subscribers: 20after4, Luke081515.2

Maniphest Tasks: T5819, T10010

Differential Revision: https://secure.phabricator.com/D14918
2016-01-08 14:01:53 -08:00
epriestley
373ff7f9d4 Read materialized project members instead of real members
Summary:
Ref T10010. This will allow us to find superprojects with `withMemberPHIDs(...)` queries.

  - Copy all the current real member edges to materialized member edges.
  - Redirect all reads to look at materialized members.
  - This table is already kept in sync by earlier work with indexing.

Basically, flow is:

  - Writes (joining, leaving, adding/removing members) write to the real member edge type.
  - After a project's members change, they're copied to the materialized member edge type for that project and all of its superprojects.
  - Reads look at materialized members, so "Parent" sees the members of "Child" and "Grandchild" as its own members, but we still have the "real members" edge type to keep track of "natural" or "direct" members.

Test Plan:
  - Ran migration.
  - Ran unit tests.
  - Saw the same projects as projects I was a member of.
  - Added some `var_dump()` stuff to verify the Owners changed.
  - Used `grep` to look for other readers of this edge type.
  - Made some project updates.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D14893
2015-12-27 09:26:27 -08:00
epriestley
367955f3fd Improve UX and messaging for certain errors when landing revisions
Summary:
Ref T9994.

  - Allow errors to be dismissed.
  - Tailor messaging for closed/abandoned revisions.
  - Reduce scare messaging on land dialog, since it's not really that scary anymore.

Test Plan:
  - Dismissed errors.
  - Hit new warnings.
  - Wasn't as scared when landing.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9994

Differential Revision: https://secure.phabricator.com/D14886
2015-12-27 08:01:20 -08:00
epriestley
70f6bf306f Implement child/descendant query rules in Projects
Summary:
Ref T10010. This adds infrastructure for querying projects by type, depth, parent and ancestor.

I needed to revise the "extended policy check" cycle detection rules. When, e.g., querying a grandchild, they incorrectly detected a cycle because both the child and grandchild needed to check the policy of the grandparent.

Instead, simplify it to just do a basic runaway calldepth check. There are many other safety mechanisms to make it so this can't ever occur.

(Cycle detection does have existing test coverage, and those tests still pass, it just takes a little longer to detect the cycle internally.)

There is still no way to create subprojects in the UI.

Test Plan: Added and executed unit tests.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D14862
2015-12-23 14:38:51 -08:00
epriestley
3068639ccf Implement query and policy rules for subprojects
Summary:
Ref T10010. This implements technical groundwork for subprojects. Specifically, it implements policy rules like Phriction:

  - to see a project, you must be able to see all of its parents (and the project itself).
  - you can edit a project if you can edit any of its parents (or the project itself).

To facilitiate this, we load all project ancestors when querying projects so we can do the view/edit checks.

This does NOT yet implement:

  - proper membership rules for these projects (up next);
  - any kind of UI to let users create subprojects.

Test Plan:
  - Added unit tests.
  - Executed unit tests.
  - Browsed Projects (no change in behavior is expected).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D14861
2015-12-23 14:38:35 -08:00
epriestley
96fe8c0b83 Implement basic ngram search for Owners Package names
Summary:
Ref T9979. This uses ngrams (specifically, trigrams) to build a reasonably efficient index for substring matching. Specifically, for a package like "Example", with ID 123, we store rows like this:

```
< ex, 123>
<exa, 123>
<xam, 123>
<amp, 123>
<mpl, 123>
<ple, 123>
<le , 123>
```

When the user searches for `exam`, we join this table for packages with tokens `exa` and `xam`. MySQL can do this a lot more efficiently than it can process a `LIKE "%exam%"` query against a huge table.

When the user searches for a one-letter or two-letter string, we only search the beginnings of words. This is probably what they want, the only thing we can do quickly, and a reasonable/expected behavior for typeaheads.

Test Plan:
  - Ran storage upgrades and search indexer.
  - Searched for stuff with "name contains".
  - Used typehaead and got sensible results.
  - Searched for `aabbccddeeffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz` and saw only 16 joins.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9979

Differential Revision: https://secure.phabricator.com/D14846
2015-12-22 08:00:33 -08:00
epriestley
a761f73384 Allow index extensions to skip indexing if the object has not changed
Summary:
Fixes T9890. This allows IndexExtensions to emit an object version.

Before we build indexes, we check if the indexed version is the same as the current version. If it is, we just don't call that extension.

T9890 has a case where this is useful: a script went crazy and posted thousands of comments to a single task.

Without versioning, that results in the same comments being indexed over and over again. With versioning, most of the queue could just exit without doing any work.

Test Plan:
  - Added a `sleep(1)` to the actual indexing, used `bin/search index --background` to queue up a lot of tasks, ran them with `bin/phd debug task`, saw them complete very quickly with only one actual index operation performed.
  - Used `bin/search index --trace` and `bin/search index --trace --background` to observe the behavior of queries against the index version store, which looked sensible.
  - Made comments/transactions, saw versions update.
  - Used `bin/remove destroy`, verified index versions were purged.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9890

Differential Revision: https://secure.phabricator.com/D14845
2015-12-21 17:27:14 -08:00
Nick Zheng
8eec9e2c0e Provide a more straightforward way to revoke SSH keys by finding and destroying the objects
Summary: Ref T9967

Test Plan:
Ran migrations.
Verified database populated properly with PHIDs (SELECT * FROM auth_sshkey;).
Ran auth.querypublickeys conduit method to see phids show up
Ran bin/remove destroy <phid>.
Viewed the test key was gone.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T9967

Differential Revision: https://secure.phabricator.com/D14823
2015-12-19 11:52:26 -08:00
epriestley
aeae0e7028 Prepare Projects schema for subprojects
Summary:
Ref T10010. This does some cleanups on the schema:

  - `viewPolicy`, `editPolicy` and `joinPolicy` were nullable, but should never be `null`. Set them to defaults if they're null, then make the column non-nullable.
  - Rename `phrictionSlug` to `primarySlug` and stop adding and removing trailing slashes from it.
  - Add new columns to support milestones and non-milestone subprojects.
  - Drop very old subprojectPHIDs column. This hasn't done anything in the UI for years and years, and isn't particularly realistic to migrate forward.

The new columns aren't reachable from the UI.

Test Plan:
  - Applied patches.
  - Grepped for `phrictionSlug`.
  - Grepped for `subprojectPHIDs`.
  - Created tasks.
  - Edited tasks.
  - Verified existing tasks still had primary slugs.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10010

Differential Revision: https://secure.phabricator.com/D14825
2015-12-19 09:21:36 -08:00
epriestley
2d588715bc Always automatically generate Phame slugs
Summary:
Fixes T9995. I think letting users customize slugs is not a hugely compelling as a product feature, and this fixes the issue with slugs that have "/" characters in them and makes the move to EditEngine easier since I don't have to deal with the weird JS thing.

Instead, just generate slugs automatically. No more JS, no more separate field, things automatically update if you rename a blog, and now that URIs have IDs in them the old URI will still work after a rename.

Test Plan:
  - Applied migration.
  - Created new posts.
  - Edited existing posts.
  - Visited various posts.
  - Created a post with a bunch of "/" in the title, things still worked fine.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9995

Differential Revision: https://secure.phabricator.com/D14792
2015-12-15 14:18:56 -08:00
epriestley
2a203fbab1 Add proper PHIDs to RefCursors
Summary: Ref T9952. See discussion there. This change is primarily aimed at letting me build a typeahead of branches in a repository so that we can land to arbitrary branches a few diffs from now.

Test Plan:
  - Ran migrations.
  - Verified database populated properly with PHIDs (`SELECT * FROM repository_refcursor;`).
  - Ran `bin/repository update`.
  - Viewed a Git repository in Diffusion.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9952

Differential Revision: https://secure.phabricator.com/D14731
2015-12-10 14:21:08 -08:00
epriestley
82e67e6bb9 Clean up some EditEngine meta-policies
Summary:
Ref T9908. Simplify some of the policies here:

  - If you can edit an application (currently, always "Administrators"), you can view and edit all of its forms.
  - You must be able to edit an application to create new forms.
  - Improve some error messages.
  - Get about halfway through letting users reorder forms in the "Create" menu if they want to sort by something weird since it'll need schema changes and I can do them all in one go here.

Test Plan:
  - Tried to create and edit forms as an unprivileged user.
  - Created and edited forms as an administrator.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9908

Differential Revision: https://secure.phabricator.com/D14700
2015-12-07 15:40:31 -08:00
epriestley
b82863d972 Implement versioned drafts in EditEngine comment forms
Summary:
Ref T9132. Fixes T5031. This approximately implements the plan described in T5031#67988:

When we recieve a preview request, don't write a draft if the form is from a version of the object before the last update the viewer made.

This should fix the race-related (?) zombie draft comments that sometimes show up.

I just added a new object for this stuff to make it easier to do stacked actions (or whatever we end up with) a little later, since I needed to do some schema adjustments anyway.

Test Plan:
  - Typed some text.
  - Reloaded page.
  - Draft stayed there.
  - Tried real hard to get it to ghost by submitting stuff in multiple windows and typing a lot and couldn't, although I didn't bother specifically narrowing down the race condition.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5031, T9132

Differential Revision: https://secure.phabricator.com/D14640
2015-12-03 07:07:29 -08:00
lkassianik
47a5ebb4fe Correctly implementing mailkey for Phurl
Summary: Re T6049, Correctly implementing mailkey for Phurl

Test Plan: Edit Phurl URL, receive email.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14605
2015-11-30 10:44:54 -08:00
Chad Little
5eada3d89c Add Profile Images to PhameBlog
Summary: Will use these more in the upcoming unbeta design of PhameBlog, likely. Also curious how this works.

Test Plan: Add an image to a blog, remove an image from a blog.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14587
2015-11-28 13:39:08 -08:00
Chad Little
62e129d7a6 Allow Phame Blogs to be archived instead of deleted
Summary: Removes "delete" and uses "archive/activate" instead for Phame Blogs. Ref T9756

Test Plan: Archive a blog, see in search, activate blog, see in other search.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: joshuaspence, Korvin

Maniphest Tasks: T9756

Differential Revision: https://secure.phabricator.com/D14465
2015-11-21 08:54:22 -08:00
epriestley
5aae89babb Fix file PHID extraction in Owners and Differential
Summary:
Ref T9787. To fix this, I want to change how file PHIDs are extracted slightly: specifically, I'm going to extract them later in the editing process.

Before doing this, clean up a couple of bad implementations:

  - Owners extracts its description as a file PHID. This is an error.
    - Extract the description as a remarkup block instead.
    - Add an edge table so stuff like file attachment works properly.
  - Differential has a no-op extract method. This is presumably just a copy/paste issue from long ago.

Test Plan:
  - Edited a revision in Differential.
  - Dropped a file into the description of an Owners package.
    - Before change: this did not attach the file.
    - After change: the file now attaches properly and shows up as "Attached" in the file details.

Reviewers: chad, joshuaspence

Reviewed By: joshuaspence

Subscribers: joshuaspence

Maniphest Tasks: T9787

Differential Revision: https://secure.phabricator.com/D14493
2015-11-17 08:36:50 -08:00
Joshua Spence
26a235ab8a Make Herald rules subscribable
Summary: Fixes T9757.

Test Plan: Created a Herald rule and then subscribed to it with a different account.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T9757

Differential Revision: https://secure.phabricator.com/D14468
2015-11-17 06:27:13 +11:00
Joshua Spence
ca0b36c174 Rename XHPAST database
Summary: Rename the XHPAST database from `{$NAMESPACE}_xpastview` to `{$NAMESPACE}_xhpast`.

Test Plan: Ran `./bin/storage --namespace test upgrade --no-quickstart`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14442
2015-11-14 21:41:28 +11:00
Joshua Spence
321c61a853 Remove daemon envHash and envInfo
Summary: Ref T7053. Remove the `envHash` and `envInfo` fields, which are no longer used now that the daemons restart automagically. Depends on D14458.

Test Plan: Saw no more setup issues.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: tycho.tatitscheff, epriestley

Maniphest Tasks: T7053

Differential Revision: https://secure.phabricator.com/D14446
2015-11-11 08:54:45 +11:00
epriestley
0398097498 Allow ApplicationEditor forms to be reconfigured
Summary:
Ref T9132. This diff doesn't do anything interesting, it just lays the groundwork for more interesting future diffs.

Broadly, the idea here is to let you create multiple views of each edit form. For example, we might create several different "Create Task" forms, like:

  - "New Bug Report"
  - "New Feature Request"

These would be views of the "Create Task" form, but with various adjustments:

  - A form might have additional instructions ("how to file a good bug report").
  - A form might have prefilled values for some fields (like particular projects, subscribers, or policies).
  - A form might have some fields locked (so they can not be edited) or hidden.
  - A form might have a different field order.
  - A form might have a limited visibility policy, so only some users can access it.

This diff adds a new storage object (`EditEngineConfiguration`) to keep track of all those customizations and represent "a form which has been configured to look and work a certain way".

This doesn't let these configurations do anything useful/interesting, and you can't access them directly yet, it's just all the boring plumbing to enable more interesting behavior in the future.

Test Plan:
ApplicationEditor forms now let you manage available forms and edit the current form:

{F959025}

There's a new (bare bones) list of all available engines:

{F959030}

And if you jump into an engine, you can see all the forms for it:

{F959038}

The actual form configurations have standard detail/edit pages. The edit pages are themselves driven by ApplicationEditor, of course, so you can edit the form for editing forms.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9132

Differential Revision: https://secure.phabricator.com/D14453
2015-11-10 10:24:40 -08:00
Chad Little
b315f61f49 Add comments to internal Phame Posts
Summary: Adds commenting to Phame Posts, also testing a new "document comment style". Unsure about it but Phame is a prototype so good place to explore.

Test Plan: Leave some comments, see some comments, test show/hide.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9746

Differential Revision: https://secure.phabricator.com/D14451
2015-11-10 08:19:38 -08:00
David Fisher
c589af51e8 add "update" mode to Diffusion coverage Conduit
Summary:
This diff adds a new mode argument to the Diffusion Conduit API with two options:
- "overwrite": the default, maintains the current behavior of deleting all coverage
  in the specified branch before uploading the new coverage
- "update": does not delete old coverage, but will overwrite previous
  coverage information if it's for the same file and commit

`DiffusionRequest::loadCoverage` already loads a file's coverage from the
latest available commit, so uploading coverage for different files in different
commits with "update" will result in seeing the latest uploaded coverage in
Diffusion.

Test Plan: manual local verification

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14428
2015-11-09 16:52:34 -08:00
Chad Little
df23d893f7 Remove Join Policy from Phame
Summary: Drops Join Policy, uses Edit Policy where needed. Allows anyone with Blog Edit permissions to post and edit any post on that blog. Fixes T5371

Test Plan: Draft Post as chad, see post, log in with notchad, edit that post and publish it.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T5371

Differential Revision: https://secure.phabricator.com/D14444
2015-11-09 08:52:58 -08:00
Joshua Spence
a2f909f0bd Improve XHPAST handling of syntax errors
Summary: Currently, a bunch of developers are using #xhpast for writing custom linter rules. As such, we end up with a fair few `XHPASTSyntaxErrorException` in our PHP error logs. I think that throwing an exception is not quite correct in this case because it is somewhat expected that invalid PHP may be entered. Instead, catch the exception and show the user a helpful message.

Test Plan: This doesn't quite work yet... the stream and tree views render as blank but the exceptions still propogate to the error logs. Mostly, I'm not sure how the exception should be rendered for display.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14028
2015-11-09 07:03:32 +11:00
Chad Little
c3ecea9788 Add mail support to PhameBlog
Summary: Add some mailkeys, allow feed stories to be published.

Test Plan: New Blog, Edit Blog

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14434
2015-11-08 08:11:47 -08:00
Chad Little
6fe2377cc2 Add mail/feed support to PhamePost
Summary: Allows feed stories and mail for new Phame Posts.

Test Plan: Write Post, Get Mail

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14426
2015-11-06 17:43:46 -08:00
Aviv Eyal
1898864b6c add initiator.phid parameter to HM builds
Summary:
Fix T9662.

Record who initiated the build, and allow this information as a parameter.

In this implementation, a 're-run' keeps the original initiator, which we maybe not desired?

Test Plan:
Make a HTTP step with initiator.phid, trigger manually, via HM, via ./bin/harbormaster build.
Look at requests made.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T9662

Differential Revision: https://secure.phabricator.com/D14380
2015-11-04 18:32:18 +00:00
Joshua Spence
3a046384e9 Drop the metamta_mailinglist table
Summary: We haven't seen any issues here, remove the table and schema spec.

Test Plan: Not yet tested.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14067
2015-10-31 11:20:59 +00:00
epriestley
5ee4a1a306 Give Harbormaster Build Plans real policies
Summary:
Ref T9614. Currently, a lot of Build Plan behavior is covered by a global "can manage" policy.

One install in particular is experiencing difficulty with warring factions within engineering aborting one another's builds.

As a first step to remedy this, and also generally make Harbormaster more flexible and bring it in line with other applications in terms of policy power:

  - Give Build Plans normal view/edit policies.
  - Require "Can Edit" to run a plan manually.

Having "Can View" on plans may be a little weird in some cases (the status of a Buildable might be bad because of a build you can't see) but we can cross that bridge when we come to it.

Next change here will require "Can Edit" to abort a build. This will reasonably allow installs to reserve pause/abort for administrators/adults. (I might let anyone restart a plan, though?)

Test Plan:
  - Created a new build plan.
  - Verified defaults were inherited from application defaults (swapped them around, too).
  - Saved build plan.
  - Edited policies.
  - Verified autoplans get the right policies.
  - Verified old plans got migrated properly.
  - Tried to run a plan I couldn't edit (denied).
  - Ran a plan from CLI with `bin/harbormaster`.
  - Tried to create a plan with an unprivileged user.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9614

Differential Revision: https://secure.phabricator.com/D14321
2015-10-26 12:38:21 -07:00
epriestley
ad53e7b878 Record how long storage patches took to apply
Summary:
It's hard for us to predict how long patches and migrations will take in the general case since it varies a lot from install to install, but we can give installs some kind of rough heads up about longer patches. I'm planning to just put a sort of hint for things in the changelog, something like this:

{F905579}

To make this easier, start storing how long stuff took. I'll write a little script to dump this into a table for the changelog.

Test Plan:
Ran `bin/storage status`:

{F905580}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14320
2015-10-24 05:58:44 -07:00
epriestley
b4af57ec51 Rough cut of DrydockRepositoryOperation
Summary:
Ref T182. This doesn't do anything interesting yet and is mostly scaffolding, but here's roughly the workflow. From previous revision, you can configure "Repository Automation" for a repository:

{F875741}

If it's configured, a new "Land Revision" button shows up:

{F875743}

Once you click it you get a big warning dialog that it won't work, and then this shows up at the top of the revision (completely temporary/placeholder UI, some day a nice progress bar or whatever):

{F875747}

If you're lucky, the operation eventually sort of works:

{F875750}

It only runs `git show` right now, doesn't actually do any writes or anything.

Test Plan:
  - Clicked "Land Revision".
  - Watched `phd debug task`.
  - Saw it log `git show` to output.
  - Verified operation success in UI (by fiddling URL, no way to get there normally yet).

Reviewers: chad

Reviewed By: chad

Subscribers: revi

Maniphest Tasks: T182

Differential Revision: https://secure.phabricator.com/D14266
2015-10-13 15:46:12 -07:00
epriestley
cd8be8106b Improve ruleset for generating project hashtags
Summary:
Ref T9551. We currently use the same logic for generating project hashtags and Phriction slugs, but should be a little more conservative with project hashtags.

Stop them from generating with stuff that won't parse in a "Reviewers:" field or generally in commments (commas, colons, etc).

Test Plan:
Created a bunch of projects with nonsense in them and saw them generate pretty reasonable hashtags.

{F873456}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9551

Differential Revision: https://secure.phabricator.com/D14261
2015-10-12 17:02:58 -07:00
epriestley
1bdf225354 Use Drydock authorizations when acquiring leases
Summary:
Ref T9519. When acquiring leases on resources:

  - Only consider resources created by authorized blueprints.
  - Only consider authorized blueprints when creating new resources.
  - Fail with a tailored error if no blueprints are allowed.
  - Fail with a tailored error if missing authorizations are causing acquisition failure.

One somewhat-substantial issue with this is that it's pretty hard to figure out from the Harbormaster side. Specifically, the Build step UI does not show field value anywhere, so the presence of unapproved blueprints is not communicated. This is much more clear in Drydock. I'll plan to address this in future changes to Harbormaster, since there are other related/similar issues anyway.

Test Plan: {F872527}

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9519

Differential Revision: https://secure.phabricator.com/D14254
2015-10-12 17:02:35 -07:00
epriestley
2f6d3119f5 Rough cut of "Blueprint Authorizations"
Summary:
Ref T9519. This is like 80% of the way there and doesn't fully work yet, but roughly shows the shape of things to come. Here's how it works:

First, there's a new custom field type for blueprints which works like a normal typeahead but has some extra logic. It's implemented this way to make it easy to add to Blueprints in Drydock and Build Plans in Harbormaster. Here, I've added a "Use Blueprints" field to the "WorkingCopy" blueprint, so you can control which hosts the working copies are permitted to allocate on:

{F869865}

This control has a bit of custom rendering logic. Instead of rendering a normal list of PHIDs, it renders an annotated list with icons:

{F869866}

These icons show whether the blueprint on the other size of the authorization has approved this object. Once you have a green checkmark, you're good to go.

On the blueprint side, things look like this:

{F869867}

This table shows all the objects which have asked for access to this blueprint. In this case it's showing that one object is approved to use the blueprint since I already approved it, but by default new requests come in here as "Authorization Requested" and someone has to go approve them.

You approve them from within the authorization detail screen:

{F869868}

You can use the "Approve" or "Decline" buttons to allow or prevent use of the blueprint.

This doesn't actually do anything yet -- objects don't need to be authorized in order to use blueprints quite yet. That will come in the next diff, I just wanted to get the UI in reasonable shape first.

The authorization also has a second piece of state, which is whether the request from the object is active or inactive. We use this to keep track of the authorization if the blueprint is (maybe temporarily) deleted.

For example, you might have a Build Plan that uses Blueprints A and B. For a couple days, you only want to use A, so you remove B from the "Use Blueprints: ..." field. Later, you can add B back and it will connect to its old authorization again, so you don't need to go re-approve things (and if you're declined, you stay declined instead of being able to request authorization over and over again). This should make working with authorizations a little easier and less labor intensive.

Stuff not in this diff:

  - Actually preventing any allocations (next diff).
  - Probably should have transactions for approve/decline, at least, at some point, so there's a log of who did approvals and when.
  - Maybe should have a more clear/loud error state when no blueprints are approved?
  - Should probably restrict the typeahead to specific blueprint types.

Test Plan:
  - Added the field.
  - Typed some stuff into it.
  - Saw the UI update properly.
  - Approved an authorization.
  - Declined an authorization.
  - Saw active authorizations on a blueprint page.
  - Didn't see any inactive authroizations there.
  - Clicked "View All Authorizations", saw all authorizations.

Reviewers: chad, hach-que

Reviewed By: chad

Maniphest Tasks: T9519

Differential Revision: https://secure.phabricator.com/D14251
2015-10-10 07:15:25 -07:00
epriestley
3c4b05bcd4 Correct a Dashboard status constant in a migration
Summary:
Fixes T9500. All the code is fine in D13836, but the value of the constant got updated (from "open" to "active") and the migration still used the old value.

Correct any affected dashboards to use the proper constant.

This only affected old dashboards: newly created ones use the right constant.

Test Plan: Ran migration, verified that all active dashboards appeared on "Active Dashboards".

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9500

Differential Revision: https://secure.phabricator.com/D14223
2015-10-02 09:17:43 -07:00
epriestley
2728a9f964 Allow builds to have parameters
Summary: Ref T9352. See D13635. Build targets can have variables already, but let builds have them too. This mostly enables future use cases (sub-builds, more sophisticated build triggers).

Test Plan: With a custom Herald rule + action like the one in T9352, updated a revision and saw it generate multiple builds with varying parameters.

Reviewers: chad, hach-que

Reviewed By: hach-que

Maniphest Tasks: T9352

Differential Revision: https://secure.phabricator.com/D14222
2015-10-02 06:32:08 -07:00
epriestley
d4a0b1c870 Remove names from Drydock resources
Summary:
Ref T9252. Long ago you sometimes manually created resources, so they had human-enterable names. However, users never make resources manually any more, so this field isn't really useful any more.

In particular, it means we write a lot of untranslatable strings like "Working Copy" to the database in the default locale. Instead, do the call at runtime so resource names are translatable.

Also clean up a few minor things I hit while kicking the tires here.

It's possible we might eventually want to introduce a human-choosable label so you can rename your favorite resources and this would just be a default name. I don't really have much of a use case for that yet, though, and I'm not sure there will ever be one.

Test Plan:
  - Restarted a Harbormaster build, got a clean build.
  - Released all leases/resources, restarted build, got a clean build with proper resource names.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14213
2015-10-01 08:13:43 -07:00
epriestley
2ef5b5321d Move Drydock logs to PHIDs and increased structure
Summary:
Ref T9252. Several general changes here:

  - Moves logs to use PHIDs instead of IDs. This generally improves flexibility (for example, it's a lot easier to render handles).
  - Adds `blueprintPHID` to logs. Although you can usually figure this out from the leasePHID or resourcePHID, it lets us query relevant logs on Blueprint views.
  - Instead of making logs a top-level object, make them strictly a sub-object of Blueprints, Resources and Leases. So you go Drydock > Lease > Logs, etc., to get to logs.
    - I might restore the "everything" view eventually, but it doesn't interact well with policies and I'm not sure it's very useful. A policy-violating `bin/drydock log` might be cleaner.
  - Policy-wise, we always show you that logs exist, we just don't show you log content if it's about something you can't see. This is similar to seeing restricted handles in other applications.
  - Instead of just having a message, give logs "type" + "data". This will let logs be more structured and translatable. This is similar to recent changes to Herald which seem to have worked well.

Test Plan:
Added some placeholder log writes, viewed those logs in the UI.

{F855199}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14196
2015-10-01 08:06:23 -07:00
epriestley
ec6d69e74d Give Drydock resources a proper expiry mechanism
Summary:
Fixes T6569. This implements an expiry mechanism for Drydock resources which parallels the mechanism for leases.

A few things are missing that we'll probably need in the future:

  - An "EXPIRES" command to update the expiration time. This would let resources be permanent while leased, then expire after, say, 24 hours without any leases.
  - A callback like `shouldActuallyExpireRightNow()` for resources and leases that lets them decide not to expire at the last second.
  - A callback like `didAcquireLease()` for resource blueprints, to parallel `didReleaseLease()`, letting them clear or extend their timer.

However, this stuff would mostly just let us tune behaviors, not really open up new capabilities.

Test Plan: Changed host resources to expire after 60 seconds, leased one, saw it vanish 60 seconds later.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T6569

Differential Revision: https://secure.phabricator.com/D14176
2015-09-28 09:35:14 -07:00
epriestley
b441e8b81e Allow Drydock blueprints to be disabled
Summary: Ref T9252. If you have a blueprint and you do not like that blueprint very much, you can disable it.

Test Plan: Disabled / enabled some blueprints.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14156
2015-09-24 10:18:17 -07:00
epriestley
e117ace8c7 Convert Drydock lease and resource constants to strings
Summary:
Ref T9252. Drydock currently uses integer statuses, but there's no reason for this (they don't need to be ordered) and it makes debugging them, working with them, future APIs, etc., more cumbersome.

Switch to string instead.

Also rename `STATUS_OPEN` to `STATUS_ACTIVE` and `STATUS_CLOSED` to `STATUS_RELEASED` for consistency. This makes resources and leases have more similar states, and gives resource states more accurate names.

Test Plan: Browsed web UI, grepped for changed constants, applied patch, inspected database.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14153
2015-09-24 07:57:05 -07:00
epriestley
c6aade4392 Give Drydock leases a resourcePHID instead of a resourceID
Summary:
Ref T9252. Leases currently have a `resourceID`, but this is a bit nonstandard and generally less flexible than giving them a `resourcePHID`.

In particular, a `resourcePHID` is easier to use when rendering interfaces, since you can get handles out of a PHID.

Add a PHID column, copy over all the PHIDs that correspond to existing IDs, then drop the ID column.

Test Plan:
  - Browsed web UIs.
  - Inspected database during/after migration.
  - Grepped for `resourceID`.
  - Allocated a new lease with `bin/drydock lease`.

Reviewers: chad, hach-que

Reviewed By: hach-que

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14151
2015-09-24 04:19:27 -07:00
epriestley
99e4472447 Soften checks on a very old Maniphest transactionmigration
Summary:
Ref T9464. If an ancient transaction doesn't have array values for whatever reason, we fail here.

Instead, just recover as gracefully as we can. We may get the transaction "wrong" in some sense, but this only impacts what is rendered in the transaction log.

Test Plan: This is nearly a year old and there's no real way to test it.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9464

Differential Revision: https://secure.phabricator.com/D14149
2015-09-23 19:18:44 -07:00
epriestley
fcb6d1e2fa Strip some obsolete code out of Drydock
Summary:
Ref T9252. This simplifies some Drydock code.

Most of this code relates to the old notion of Drydock being able to enumerate all the tasks it needs to complete in order to acquire a lease. The code has stepped back from this, since it's unnecessary, the queue is more powerful than it used to be, and it would be a lot of work to keep track of.

The ~only thing that should ever wait for leases in modern code is `bin/drydock lease`, and it's fine for it to just sit there sleeping, so this just does that.

This reduces the granularity of logging, but I'll address that separately in future logging-focused changes.

Test Plan: Used `bin/drydock lease` to acquire a lease, saw it acquire cleanly.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14147
2015-09-23 13:21:41 -07:00
epriestley
789df89c84 Add a command queue to Drydock to manage lease/resource release
Summary:
Ref T9252. Broadly, Drydock currently races on releasing objects from the "active" state. To reproduce this:

  - Scatter some sleep()s pretty much anywhere in the release code.
  - Release several times from web UI or CLI in quick succession.

Resources or leases will execute some release code twice or otherwise do inconsistent things.

(I didn't chase down a detailed reproduction scenario for this since inspection of the code makes it clear that there are no meaningful locks or mechanisms preventing this.)

Instead, add a Harbormaster-style command queue to resources and leases. When something wants to do a release, it adds a command to the queue and schedules a worker. The workers acquire a lock, then try to consume commands from the queue.

This guarantees that only one process is responsible for writes to active resource/leases.

This is the last major step to giving resources and leases a single writer during all states:

  - Resource, Unsaved: AllocatorWorker
  - Resource, Pending: ResourceWorker (Possible rename to "Allocated?")
  - Resource, Open: This diff, ResourceUpdateWorker. (Likely rename to "Active").
  - Resource, Closed/Broken: Future destruction worker. (Likely rename to "Released" / "Broken"; maybe remove "Broken").
  - Resource, Destroyed: No writes.
  - Lease, Unsaved: Whatever wants the lease.
  - Lease, Pending: AllocatorWorker
  - Lease, Acquired: LeaseWorker
  - Lease, Active: This diff, LeaseUpdateWorker.
  - Lease, Released/Broken: Future destruction worker (Maybe remove "Broken"?)
  - Lease, Expired: No writes. (Likely rename to "Destroyed").

In most phases, we can already guarantee that there is a single writer without doing any extra work. This is more complicated in the "Active" case because the release buttons on the web UI, the release tools on the CLI, the lease requestor itself, the garbage collector, and any other release process cleaning up related objects may try to effect a release. All of these could race one another (and, in many cases, race other processes from other phases because all of these get to act immediately) as this code is currently written. Using a queue here lets us make sure there's only a single writer in this phase.

One thing which is notable is that whatever acquires a lease **can not write to it**! It is never the writer once it queues the lease for activation. It can not write to any resources, either. And, likewise, Blueprints can not write to resources while acquiring or releasing leases.

We may need to provide a mechinism so that blueprints and/or resource/lease holders get to attach some storage to resources/leases for bookkeeping. For example, a blueprint might need to keep some kind of cache on a resource to help it manage state. But I think we can cross that bridge when we come to it, and nothing else would need to write to this storage so it's technically straightforward to introduce such a mechanism if we need one.

Test Plan:
  - Viewed buttons in web UI, checked enabled/disabled states.
  - Clicked the buttons.
  - Saw commands show up in the command queue.
  - Saw some daemon stuff get scheduled.
  - Ran CLI tools, saw commands get consumed and resources/leases release.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14143
2015-09-23 07:42:08 -07:00
epriestley
3ac99006bf Implement optimistic "slot locks" in Drydock
Summary:
See discussion in D10304. There's a lot of context there, but the general idea is:

  - Blueprints should manage locks in a granular way during the actual allocation/acquisition phase.
  - Optimistic "slot locks" might a pretty good primitive to make that easy to implement and reason about in most cases.

The way these locks work is that you just pick some name for the lock (like the PHID of a resource) and say that it needs to be acquired for the allocation/acquisition to work:

```
...
->needSlotLock("mylock(PHID-XYZQ-...)")
...
```

When you fire off the acquisition or allocation, it fails unless it could acquire the slot with that name. This is really simple (no explicit lock management) and a pretty good fit for most of the locking that blueprints and leases need to do.

If you need to do limit-based locks (e.g., maximum of 3 locks) you could acquire a lock like this:

```
mylock(whatever).slot(2)
```

Blueprints generally only contend with themselves, so it's normally OK for them to pick whatever strategy works best for them in naming locks.

This may not work as well if you have a huge number of slots (e.g., 100TB you want to give out in 1MB chunks), or other complex needs for locks (like you have to synchronize access to some external resource), but slot locks don't need to be the only mechanism that blueprints use. If they run into a problem that slot locks aren't a good fit for, they can use something else instead. For now, slot locks seem like a good fit for the problems we currently face and most of the problems I anticipate facing.

(The release workflows have other race issues which I'm not addressing here. They work fine if nothing races, but aren't race-safe.)

Test Plan:
To create a race where the same binding is allocated as a resource twice:

  - Add `sleep(10)` near the beginning of `allocateResource()`, after the free bindings are loaded but before resources are allocated.
  - (Comment out slot lock acquisition if you have this patch.)
  - Run `bin/drydock lease ...` in two windows, within 10 seconds of one another.

This will reliably double-allocate the binding because both blueprints see a view of the world where the binding is free.

To verify the lock works, un-comment it (or apply this patch) and run the same test again. Now, the lock fails in one process and only one resource is allocated.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Differential Revision: https://secure.phabricator.com/D14118
2015-09-21 04:45:25 -07:00
epriestley
f8080ce931 Add CustomField support to Owners
Summary: Fixes T9351. This is straightforward since this application is now relatively modern and doesn't have any bizarre craziness.

Test Plan:
{F787981}

{F787982}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9351

Differential Revision: https://secure.phabricator.com/D14093
2015-09-10 13:32:31 -07:00
epriestley
76665f725b Correct a bad Herald migration
Summary:
One of the migrations in rPa335004a91 (`20150730.herald.5.sql`) incorrectly swapped "add" and "add blocking" Differential Herald rules.

Swap any rules last modified before this patch was applied back. This is the best we can do without possibly overwriting more recent, intentional data. I'll issue some guidance on this in the changelog.

Test Plan:
  - Made a rule, ran patch, no change.
  - Changed rule modified time to a few months ago, ran patch, saw swap from non-blocking to blocking.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14061
2015-09-04 15:11:25 -07:00
Chad Little
2665970762 Basic Answer Wiki for Ponder
Summary: Adds an additional field for questions, an answer wiki, should should usually be community editable.

Test Plan: New question, edit question, no wiki, lots of wiki.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14003
2015-08-29 13:59:33 -07:00
Chad Little
d718415868 Swap duplicate close status on Ponder for invalid
Summary: Until we have a proper close as duplicate workflow for Ponder, remove the option with something more sensible.

Test Plan: Closed a question as invalid, saw it closed and in feed.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14007
2015-08-29 08:29:23 -07:00
epriestley
c612579854 Add very basic routing to Nuance
Summary:
Ref T8783. Sort out some relationships and fields:

  - Make Items 1:1 with Queues: each item is always in exactly one queue. Minor discussion on T8783. I think this is easier to understand and reason about (and implement!) and can't come up with any real cases where it isn't powerful enough.
  - Remove "QueueItem", which allowed items to be in multiple queues at once.
  - Remove "dateNuanced", which is equivalent to "dateCreated" in all cases.

Then add really basic routing:

  - Add "Default Queue" for Sources. New items from the source route into that queue.
  - (Some day there will be routing rules, but for now the rule is "always route into the default queue".)
  - Show queue on items.
  - Show more / more useful edit history and transactions in several UIs.

Test Plan:
{F749445}

{F749446}

{F749447}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8783

Differential Revision: https://secure.phabricator.com/D13988
2015-08-24 09:21:56 -07:00
Chad Little
e0faa66772 Allow Owners Packages to be archived
Summary: Fixes T8428. Adds status to packages, allows setting and application search. I presume though these need checked elsewhere?

Test Plan: New package, edit package, archive package, run search queries.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T8428

Differential Revision: https://secure.phabricator.com/D13925
2015-08-18 13:36:05 -07:00
epriestley
57b0353034 Add harbormaster.createartifact
Summary:
Ref T8659. In the general case, this eventually allows build processes to do things like:

  - Upload build results (like a ".app" or ".exe" or other binary).
  - Pass complex results between build steps (e.g., build step A does something hard and build step B uses it to do something else).

Today, we're a long way away from having the infrastructure for that. However, it is useful to let third party build processes (like Jenkins) upload URIs that link back to the external build results.

This adds `harbormaster.createartifact` so they can do that. The only useful thing to do with this method today is have your Jenkins build do this:

  params = array(
    "uri": "https://jenkins.mycompany.com/build/23923/details/",
    "name": "View Build Results in Jenkins",
    "ui.external": true,
  );
  harbormaster.createartifact(target, 'uri', params);

Then (after the next diff) we'll show a link in Differential and a prominent link in Harbormaster. I didn't actually do the UI stuff in this diff since it's already pretty big.

This change moves a lot of code around, too:

  - Adds PHIDs to artifacts.
  - It modularizes build artifact types (currently "file", "host" and "URI").
  - It formalizes build artifact parameters and construction:
    - This lets me generate usable documentation about how to create artifacts.
    - This prevents users from doing dangerous or policy-violating things.
  - It does some other general modernization.

Test Plan:
{F715633}

{F715634}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8659

Differential Revision: https://secure.phabricator.com/D13900
2015-08-15 07:28:56 -07:00
Chad Little
603c91e08a Add ability to hide answers in Ponder
Summary: Ref T9173, adds basic hide support for answers. Answer authors and Moderators can hide answers, unhide them.

Test Plan: Hide answer, log into other account, see hidden message. Mark as visible, see answer again.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9173

Differential Revision: https://secure.phabricator.com/D13894
2015-08-14 09:25:02 -07:00
Chad Little
7e7e38e9c0 Remove VotableInterface from PonderQuestion
Summary: Ref T6920, This removes the PonderVotableInterface from PonderQuestion and assocaited code. Also... never used?

Test Plan: Visit Ponder, See List, New Question, Add Answer.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T6920

Differential Revision: https://secure.phabricator.com/D13833
2015-08-08 20:29:37 -07:00
Chad Little
f98f5a081e Add a default moderation policy to Ponder
Summary: This allows installs to essentially set a "moderator" for Ponder, who can clean up answers. Fixes T9098

Test Plan: Edit an answer I don't own.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9098

Differential Revision: https://secure.phabricator.com/D13818
2015-08-08 12:20:01 -07:00
Chad Little
736bda7081 Fix SQL issue with Ponder migration
Summary: This fails to apply on my second sandbox with incorrect DOUBLE value. Reran SQL, works as expected.

Test Plan: Rerun new SQL on ponder_question table

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D13830
2015-08-08 11:07:42 -07:00
Chad Little
d2ef273ecd Add additional statuses to Ponder
Summary: Ref T9096. This is a first cut at adding additional statuses, happy to add or subtract as needed... maybe even configurable? Also, the dialog doesn't seem to fire, I'll keep debugging.

Test Plan: Close and Reopen many questions. Test applicationSearch params by seeing resolved questions, all questions.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9096

Differential Revision: https://secure.phabricator.com/D13826
2015-08-08 10:23:33 -07:00
Chad Little
5eeda6040e Allow setting of Archive/Active on Paste
Summary: Ref T9076, adds basic plumbing for setting the state of a Paste.

Test Plan: Archive Paste, Activate Paste, New Paste

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9076

Differential Revision: https://secure.phabricator.com/D13801
2015-08-05 13:44:55 -07:00
Chad Little
a3b955f948 Add Spaces to Ponder
Summary: Ref T8493, Ref T3578. Adds spaces support to ponder.

Test Plan: Ask a question in a new space, see new question.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T3578, T8493

Differential Revision: https://secure.phabricator.com/D13792
2015-08-05 09:38:14 -07:00
Chad Little
135d0c9ee7 Add Edit/View Policy to Ponder Questions
Summary: Ref T3578, adds ability to set a default edit and view policy for questions. Not sure what to set viewPolicy to ?

Test Plan: Test an old question, edit policy still on myself. Test a new question, see new default.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T3578

Differential Revision: https://secure.phabricator.com/D13791
2015-08-04 15:41:09 -07:00
Chad Little
4e7b5defc3 Add mailKeys to Ponder Answer
Summary: Ref T3846. Adds mailkey generation and migration.

Test Plan: Ran the migration, see keys in mysql.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T3846

Differential Revision: https://secure.phabricator.com/D13785
2015-08-04 10:49:15 -07:00
epriestley
a3e2f655eb Modularize the Diffusion/Differential "Block" Herald actions
Also removes HeraldCustomAction. This completes action modularization.

Ref T8726.
2015-08-03 14:33:28 -07:00
epriestley
6f6d88794b Modularize the Diffusion "Add Auditors" Herald action
Ref T8726.
2015-08-03 14:33:27 -07:00
epriestley
776caa507b Modularize the Harbormaster "Run build plan" Herald action
Ref T8726. Modularizes "Run build plan" in Differential and Diffusion.
2015-08-03 14:33:26 -07:00
epriestley
fdd379a026 Modularize the Legalpad "Require Signature" Herald Action
Ref T8726. Modularizes "Require Signatures" for Legalpad.
2015-08-03 14:33:26 -07:00
epriestley
a335004a91 Modularize Differential Reviewer actions in Herald
Ref T8726. Modularizes the "Add Reviewers" and "Add Blocking Reviewers" Herald actions.
2015-08-03 14:33:25 -07:00
epriestley
8d8ee18ce0 Modularize Maniphest Herald "Assign Task" action
Summary: Ref T8726. No surprises here.

Test Plan:
  - Created rules using this action.
  - Applied migration.
  - Verified rules still work.

{F659324}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13706
2015-08-03 14:33:25 -07:00
epriestley
3782992670 Modularize "add projects" and "remove projects" Herald actions
Summary: Ref T8726. Convert these to be modular.

Test Plan:
  - Created rules using these actions.
  - Upgraded them.
  - Verified they still work.

{F659266}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13705
2015-08-03 14:33:24 -07:00
epriestley
51fead17cf Modularize "Send an Email" Herald actions
Summary: Ref T8726. No surprises.

Test Plan:
Created rules using both action variants, applied upgrade, saw rules still work correctly.

{F658842}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13701
2015-08-03 14:33:23 -07:00
epriestley
8ae08a3de7 Make "Add Subscribers" and "Remove Subscribers" Herald actions modular
Summary: Ref T8726. Converts these actions to be modular. No real surprises in this change.

Test Plan:
{F658709}

  - Wrote some rules.
  - Migrated them forward.
  - Used a bunch of these rules.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T8726

Differential Revision: https://secure.phabricator.com/D13699
2015-08-03 14:33:22 -07:00
Chad Little
42c0dd2b8e Add mail/feed to Slowvote
Summary: Adds mailkeys, basic structure for publishing to feed, sending mail.

Test Plan: New Poll, vote, comment, etc.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13749
2015-08-01 15:41:08 -07:00
epriestley
8d53f15115 Put the HeraldActionRecord table back on the shelf
Summary:
Fixes T8958. I renamed this class and table (in D13644), but the migration `20150606.mlist.1.php` needs to load records from the table before the table rename applies.

Put the table back and accept a tiny bit of cruft here to fix this problem. We can no-op the migration after a while.

I'll cherry-pick this to `stable`.

Test Plan: Applied migration. Created Herald rules. Reviewed existing Herald rules.

Reviewers: btrahan

Subscribers: epriestley

Maniphest Tasks: T8958

Differential Revision: https://secure.phabricator.com/D13736
2015-07-27 15:08:01 -07:00
Chad Little
c8977d580e Remove viewPolicy from Badges
Summary: Drops the sql column.

Test Plan: View logged in and logged out badges, edited some too.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13715
2015-07-25 16:41:41 -07:00
Chad Little
a962aeaf85 Add mailKeys to Countdown
Summary: Adds mailkeys and reply handler support

Test Plan: Edit Countdown, New Countdown, no errors.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13713
2015-07-25 15:47:07 -07:00
Chad Little
82edde8876 Add MAILTAGs to Badges
Summary: Still doesn't mail yet, but the settings now show up.

Test Plan: View email settings, see Badges options.

Reviewers: eadler, epriestley

Reviewed By: eadler, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13712
2015-07-25 13:34:43 -07:00
Chad Little
17fe8511d7 Add Comments to Countdown
Summary: 3, 2, 1, Liftoff! (Adds basic commenting to countdown)

Test Plan: asdf asdf asdf

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8895

Differential Revision: https://secure.phabricator.com/D13707
2015-07-24 15:40:55 -07:00
lkassianik
930b6fec25 DRAFT - throw together Phurl skeleton.
Summary: DRAFT - throw together Phurl skeleton.

Test Plan: The idea is that `some/long/url` will become `install/Udet4d` and can be viewed and edited at `install/Udet4d/view` and `install/Udet4d/edit`, respectively?

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: joshuaspence, chad, epriestley, Korvin

Maniphest Tasks: T6049

Differential Revision: https://secure.phabricator.com/D13681
2015-07-24 11:06:27 -07:00
Chad Little
10fb011a49 Add commenting to Badges
Summary: Fixes T8949. Adds the ability to render honors on those who have fought and received badges of distinction and honor.

Test Plan: Write 'asdf'. See 'asdf'.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8949

Differential Revision: https://secure.phabricator.com/D13704
2015-07-24 10:56:08 -07:00
Chad Little
c22d6c7beb Add ability to archive a Dashboard
Summary: Let's people archive unused Dashboards (if they have permission).

Test Plan: Archive and Unarchive a dashboard, view history, view search, build new filters.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T6443

Differential Revision: https://secure.phabricator.com/D13683
2015-07-23 14:22:56 -07:00