1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-30 18:52:42 +01:00
Commit graph

773 commits

Author SHA1 Message Date
Chad Little
62e129d7a6 Allow Phame Blogs to be archived instead of deleted
Summary: Removes "delete" and uses "archive/activate" instead for Phame Blogs. Ref T9756

Test Plan: Archive a blog, see in search, activate blog, see in other search.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: joshuaspence, Korvin

Maniphest Tasks: T9756

Differential Revision: https://secure.phabricator.com/D14465
2015-11-21 08:54:22 -08:00
epriestley
5aae89babb Fix file PHID extraction in Owners and Differential
Summary:
Ref T9787. To fix this, I want to change how file PHIDs are extracted slightly: specifically, I'm going to extract them later in the editing process.

Before doing this, clean up a couple of bad implementations:

  - Owners extracts its description as a file PHID. This is an error.
    - Extract the description as a remarkup block instead.
    - Add an edge table so stuff like file attachment works properly.
  - Differential has a no-op extract method. This is presumably just a copy/paste issue from long ago.

Test Plan:
  - Edited a revision in Differential.
  - Dropped a file into the description of an Owners package.
    - Before change: this did not attach the file.
    - After change: the file now attaches properly and shows up as "Attached" in the file details.

Reviewers: chad, joshuaspence

Reviewed By: joshuaspence

Subscribers: joshuaspence

Maniphest Tasks: T9787

Differential Revision: https://secure.phabricator.com/D14493
2015-11-17 08:36:50 -08:00
Joshua Spence
26a235ab8a Make Herald rules subscribable
Summary: Fixes T9757.

Test Plan: Created a Herald rule and then subscribed to it with a different account.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T9757

Differential Revision: https://secure.phabricator.com/D14468
2015-11-17 06:27:13 +11:00
Joshua Spence
ca0b36c174 Rename XHPAST database
Summary: Rename the XHPAST database from `{$NAMESPACE}_xpastview` to `{$NAMESPACE}_xhpast`.

Test Plan: Ran `./bin/storage --namespace test upgrade --no-quickstart`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14442
2015-11-14 21:41:28 +11:00
Joshua Spence
321c61a853 Remove daemon envHash and envInfo
Summary: Ref T7053. Remove the `envHash` and `envInfo` fields, which are no longer used now that the daemons restart automagically. Depends on D14458.

Test Plan: Saw no more setup issues.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: tycho.tatitscheff, epriestley

Maniphest Tasks: T7053

Differential Revision: https://secure.phabricator.com/D14446
2015-11-11 08:54:45 +11:00
epriestley
0398097498 Allow ApplicationEditor forms to be reconfigured
Summary:
Ref T9132. This diff doesn't do anything interesting, it just lays the groundwork for more interesting future diffs.

Broadly, the idea here is to let you create multiple views of each edit form. For example, we might create several different "Create Task" forms, like:

  - "New Bug Report"
  - "New Feature Request"

These would be views of the "Create Task" form, but with various adjustments:

  - A form might have additional instructions ("how to file a good bug report").
  - A form might have prefilled values for some fields (like particular projects, subscribers, or policies).
  - A form might have some fields locked (so they can not be edited) or hidden.
  - A form might have a different field order.
  - A form might have a limited visibility policy, so only some users can access it.

This diff adds a new storage object (`EditEngineConfiguration`) to keep track of all those customizations and represent "a form which has been configured to look and work a certain way".

This doesn't let these configurations do anything useful/interesting, and you can't access them directly yet, it's just all the boring plumbing to enable more interesting behavior in the future.

Test Plan:
ApplicationEditor forms now let you manage available forms and edit the current form:

{F959025}

There's a new (bare bones) list of all available engines:

{F959030}

And if you jump into an engine, you can see all the forms for it:

{F959038}

The actual form configurations have standard detail/edit pages. The edit pages are themselves driven by ApplicationEditor, of course, so you can edit the form for editing forms.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9132

Differential Revision: https://secure.phabricator.com/D14453
2015-11-10 10:24:40 -08:00
Chad Little
b315f61f49 Add comments to internal Phame Posts
Summary: Adds commenting to Phame Posts, also testing a new "document comment style". Unsure about it but Phame is a prototype so good place to explore.

Test Plan: Leave some comments, see some comments, test show/hide.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9746

Differential Revision: https://secure.phabricator.com/D14451
2015-11-10 08:19:38 -08:00
David Fisher
c589af51e8 add "update" mode to Diffusion coverage Conduit
Summary:
This diff adds a new mode argument to the Diffusion Conduit API with two options:
- "overwrite": the default, maintains the current behavior of deleting all coverage
  in the specified branch before uploading the new coverage
- "update": does not delete old coverage, but will overwrite previous
  coverage information if it's for the same file and commit

`DiffusionRequest::loadCoverage` already loads a file's coverage from the
latest available commit, so uploading coverage for different files in different
commits with "update" will result in seeing the latest uploaded coverage in
Diffusion.

Test Plan: manual local verification

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14428
2015-11-09 16:52:34 -08:00
Chad Little
df23d893f7 Remove Join Policy from Phame
Summary: Drops Join Policy, uses Edit Policy where needed. Allows anyone with Blog Edit permissions to post and edit any post on that blog. Fixes T5371

Test Plan: Draft Post as chad, see post, log in with notchad, edit that post and publish it.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T5371

Differential Revision: https://secure.phabricator.com/D14444
2015-11-09 08:52:58 -08:00
Joshua Spence
a2f909f0bd Improve XHPAST handling of syntax errors
Summary: Currently, a bunch of developers are using #xhpast for writing custom linter rules. As such, we end up with a fair few `XHPASTSyntaxErrorException` in our PHP error logs. I think that throwing an exception is not quite correct in this case because it is somewhat expected that invalid PHP may be entered. Instead, catch the exception and show the user a helpful message.

Test Plan: This doesn't quite work yet... the stream and tree views render as blank but the exceptions still propogate to the error logs. Mostly, I'm not sure how the exception should be rendered for display.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14028
2015-11-09 07:03:32 +11:00
Chad Little
c3ecea9788 Add mail support to PhameBlog
Summary: Add some mailkeys, allow feed stories to be published.

Test Plan: New Blog, Edit Blog

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14434
2015-11-08 08:11:47 -08:00
Chad Little
6fe2377cc2 Add mail/feed support to PhamePost
Summary: Allows feed stories and mail for new Phame Posts.

Test Plan: Write Post, Get Mail

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14426
2015-11-06 17:43:46 -08:00
Aviv Eyal
1898864b6c add initiator.phid parameter to HM builds
Summary:
Fix T9662.

Record who initiated the build, and allow this information as a parameter.

In this implementation, a 're-run' keeps the original initiator, which we maybe not desired?

Test Plan:
Make a HTTP step with initiator.phid, trigger manually, via HM, via ./bin/harbormaster build.
Look at requests made.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T9662

Differential Revision: https://secure.phabricator.com/D14380
2015-11-04 18:32:18 +00:00
Joshua Spence
3a046384e9 Drop the metamta_mailinglist table
Summary: We haven't seen any issues here, remove the table and schema spec.

Test Plan: Not yet tested.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14067
2015-10-31 11:20:59 +00:00
epriestley
5ee4a1a306 Give Harbormaster Build Plans real policies
Summary:
Ref T9614. Currently, a lot of Build Plan behavior is covered by a global "can manage" policy.

One install in particular is experiencing difficulty with warring factions within engineering aborting one another's builds.

As a first step to remedy this, and also generally make Harbormaster more flexible and bring it in line with other applications in terms of policy power:

  - Give Build Plans normal view/edit policies.
  - Require "Can Edit" to run a plan manually.

Having "Can View" on plans may be a little weird in some cases (the status of a Buildable might be bad because of a build you can't see) but we can cross that bridge when we come to it.

Next change here will require "Can Edit" to abort a build. This will reasonably allow installs to reserve pause/abort for administrators/adults. (I might let anyone restart a plan, though?)

Test Plan:
  - Created a new build plan.
  - Verified defaults were inherited from application defaults (swapped them around, too).
  - Saved build plan.
  - Edited policies.
  - Verified autoplans get the right policies.
  - Verified old plans got migrated properly.
  - Tried to run a plan I couldn't edit (denied).
  - Ran a plan from CLI with `bin/harbormaster`.
  - Tried to create a plan with an unprivileged user.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9614

Differential Revision: https://secure.phabricator.com/D14321
2015-10-26 12:38:21 -07:00
epriestley
ad53e7b878 Record how long storage patches took to apply
Summary:
It's hard for us to predict how long patches and migrations will take in the general case since it varies a lot from install to install, but we can give installs some kind of rough heads up about longer patches. I'm planning to just put a sort of hint for things in the changelog, something like this:

{F905579}

To make this easier, start storing how long stuff took. I'll write a little script to dump this into a table for the changelog.

Test Plan:
Ran `bin/storage status`:

{F905580}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14320
2015-10-24 05:58:44 -07:00
epriestley
b4af57ec51 Rough cut of DrydockRepositoryOperation
Summary:
Ref T182. This doesn't do anything interesting yet and is mostly scaffolding, but here's roughly the workflow. From previous revision, you can configure "Repository Automation" for a repository:

{F875741}

If it's configured, a new "Land Revision" button shows up:

{F875743}

Once you click it you get a big warning dialog that it won't work, and then this shows up at the top of the revision (completely temporary/placeholder UI, some day a nice progress bar or whatever):

{F875747}

If you're lucky, the operation eventually sort of works:

{F875750}

It only runs `git show` right now, doesn't actually do any writes or anything.

Test Plan:
  - Clicked "Land Revision".
  - Watched `phd debug task`.
  - Saw it log `git show` to output.
  - Verified operation success in UI (by fiddling URL, no way to get there normally yet).

Reviewers: chad

Reviewed By: chad

Subscribers: revi

Maniphest Tasks: T182

Differential Revision: https://secure.phabricator.com/D14266
2015-10-13 15:46:12 -07:00
epriestley
cd8be8106b Improve ruleset for generating project hashtags
Summary:
Ref T9551. We currently use the same logic for generating project hashtags and Phriction slugs, but should be a little more conservative with project hashtags.

Stop them from generating with stuff that won't parse in a "Reviewers:" field or generally in commments (commas, colons, etc).

Test Plan:
Created a bunch of projects with nonsense in them and saw them generate pretty reasonable hashtags.

{F873456}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9551

Differential Revision: https://secure.phabricator.com/D14261
2015-10-12 17:02:58 -07:00
epriestley
1bdf225354 Use Drydock authorizations when acquiring leases
Summary:
Ref T9519. When acquiring leases on resources:

  - Only consider resources created by authorized blueprints.
  - Only consider authorized blueprints when creating new resources.
  - Fail with a tailored error if no blueprints are allowed.
  - Fail with a tailored error if missing authorizations are causing acquisition failure.

One somewhat-substantial issue with this is that it's pretty hard to figure out from the Harbormaster side. Specifically, the Build step UI does not show field value anywhere, so the presence of unapproved blueprints is not communicated. This is much more clear in Drydock. I'll plan to address this in future changes to Harbormaster, since there are other related/similar issues anyway.

Test Plan: {F872527}

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9519

Differential Revision: https://secure.phabricator.com/D14254
2015-10-12 17:02:35 -07:00
epriestley
2f6d3119f5 Rough cut of "Blueprint Authorizations"
Summary:
Ref T9519. This is like 80% of the way there and doesn't fully work yet, but roughly shows the shape of things to come. Here's how it works:

First, there's a new custom field type for blueprints which works like a normal typeahead but has some extra logic. It's implemented this way to make it easy to add to Blueprints in Drydock and Build Plans in Harbormaster. Here, I've added a "Use Blueprints" field to the "WorkingCopy" blueprint, so you can control which hosts the working copies are permitted to allocate on:

{F869865}

This control has a bit of custom rendering logic. Instead of rendering a normal list of PHIDs, it renders an annotated list with icons:

{F869866}

These icons show whether the blueprint on the other size of the authorization has approved this object. Once you have a green checkmark, you're good to go.

On the blueprint side, things look like this:

{F869867}

This table shows all the objects which have asked for access to this blueprint. In this case it's showing that one object is approved to use the blueprint since I already approved it, but by default new requests come in here as "Authorization Requested" and someone has to go approve them.

You approve them from within the authorization detail screen:

{F869868}

You can use the "Approve" or "Decline" buttons to allow or prevent use of the blueprint.

This doesn't actually do anything yet -- objects don't need to be authorized in order to use blueprints quite yet. That will come in the next diff, I just wanted to get the UI in reasonable shape first.

The authorization also has a second piece of state, which is whether the request from the object is active or inactive. We use this to keep track of the authorization if the blueprint is (maybe temporarily) deleted.

For example, you might have a Build Plan that uses Blueprints A and B. For a couple days, you only want to use A, so you remove B from the "Use Blueprints: ..." field. Later, you can add B back and it will connect to its old authorization again, so you don't need to go re-approve things (and if you're declined, you stay declined instead of being able to request authorization over and over again). This should make working with authorizations a little easier and less labor intensive.

Stuff not in this diff:

  - Actually preventing any allocations (next diff).
  - Probably should have transactions for approve/decline, at least, at some point, so there's a log of who did approvals and when.
  - Maybe should have a more clear/loud error state when no blueprints are approved?
  - Should probably restrict the typeahead to specific blueprint types.

Test Plan:
  - Added the field.
  - Typed some stuff into it.
  - Saw the UI update properly.
  - Approved an authorization.
  - Declined an authorization.
  - Saw active authorizations on a blueprint page.
  - Didn't see any inactive authroizations there.
  - Clicked "View All Authorizations", saw all authorizations.

Reviewers: chad, hach-que

Reviewed By: chad

Maniphest Tasks: T9519

Differential Revision: https://secure.phabricator.com/D14251
2015-10-10 07:15:25 -07:00
epriestley
3c4b05bcd4 Correct a Dashboard status constant in a migration
Summary:
Fixes T9500. All the code is fine in D13836, but the value of the constant got updated (from "open" to "active") and the migration still used the old value.

Correct any affected dashboards to use the proper constant.

This only affected old dashboards: newly created ones use the right constant.

Test Plan: Ran migration, verified that all active dashboards appeared on "Active Dashboards".

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9500

Differential Revision: https://secure.phabricator.com/D14223
2015-10-02 09:17:43 -07:00
epriestley
2728a9f964 Allow builds to have parameters
Summary: Ref T9352. See D13635. Build targets can have variables already, but let builds have them too. This mostly enables future use cases (sub-builds, more sophisticated build triggers).

Test Plan: With a custom Herald rule + action like the one in T9352, updated a revision and saw it generate multiple builds with varying parameters.

Reviewers: chad, hach-que

Reviewed By: hach-que

Maniphest Tasks: T9352

Differential Revision: https://secure.phabricator.com/D14222
2015-10-02 06:32:08 -07:00
epriestley
d4a0b1c870 Remove names from Drydock resources
Summary:
Ref T9252. Long ago you sometimes manually created resources, so they had human-enterable names. However, users never make resources manually any more, so this field isn't really useful any more.

In particular, it means we write a lot of untranslatable strings like "Working Copy" to the database in the default locale. Instead, do the call at runtime so resource names are translatable.

Also clean up a few minor things I hit while kicking the tires here.

It's possible we might eventually want to introduce a human-choosable label so you can rename your favorite resources and this would just be a default name. I don't really have much of a use case for that yet, though, and I'm not sure there will ever be one.

Test Plan:
  - Restarted a Harbormaster build, got a clean build.
  - Released all leases/resources, restarted build, got a clean build with proper resource names.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14213
2015-10-01 08:13:43 -07:00
epriestley
2ef5b5321d Move Drydock logs to PHIDs and increased structure
Summary:
Ref T9252. Several general changes here:

  - Moves logs to use PHIDs instead of IDs. This generally improves flexibility (for example, it's a lot easier to render handles).
  - Adds `blueprintPHID` to logs. Although you can usually figure this out from the leasePHID or resourcePHID, it lets us query relevant logs on Blueprint views.
  - Instead of making logs a top-level object, make them strictly a sub-object of Blueprints, Resources and Leases. So you go Drydock > Lease > Logs, etc., to get to logs.
    - I might restore the "everything" view eventually, but it doesn't interact well with policies and I'm not sure it's very useful. A policy-violating `bin/drydock log` might be cleaner.
  - Policy-wise, we always show you that logs exist, we just don't show you log content if it's about something you can't see. This is similar to seeing restricted handles in other applications.
  - Instead of just having a message, give logs "type" + "data". This will let logs be more structured and translatable. This is similar to recent changes to Herald which seem to have worked well.

Test Plan:
Added some placeholder log writes, viewed those logs in the UI.

{F855199}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14196
2015-10-01 08:06:23 -07:00
epriestley
52040bc9e4 Update quickstart.sql
Summary: I haven't regenerated this for a while and it makes instances and unit tests a little faster.

Test Plan:
  - Manually reviewed changes for sanity.
  - Ran `arc unit --everything`.
  - Observed runtime drop from ~15-16 seconds to ~12-13 seconds.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14192
2015-09-29 13:27:33 -07:00
epriestley
ec6d69e74d Give Drydock resources a proper expiry mechanism
Summary:
Fixes T6569. This implements an expiry mechanism for Drydock resources which parallels the mechanism for leases.

A few things are missing that we'll probably need in the future:

  - An "EXPIRES" command to update the expiration time. This would let resources be permanent while leased, then expire after, say, 24 hours without any leases.
  - A callback like `shouldActuallyExpireRightNow()` for resources and leases that lets them decide not to expire at the last second.
  - A callback like `didAcquireLease()` for resource blueprints, to parallel `didReleaseLease()`, letting them clear or extend their timer.

However, this stuff would mostly just let us tune behaviors, not really open up new capabilities.

Test Plan: Changed host resources to expire after 60 seconds, leased one, saw it vanish 60 seconds later.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T6569

Differential Revision: https://secure.phabricator.com/D14176
2015-09-28 09:35:14 -07:00
epriestley
b441e8b81e Allow Drydock blueprints to be disabled
Summary: Ref T9252. If you have a blueprint and you do not like that blueprint very much, you can disable it.

Test Plan: Disabled / enabled some blueprints.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14156
2015-09-24 10:18:17 -07:00
epriestley
e117ace8c7 Convert Drydock lease and resource constants to strings
Summary:
Ref T9252. Drydock currently uses integer statuses, but there's no reason for this (they don't need to be ordered) and it makes debugging them, working with them, future APIs, etc., more cumbersome.

Switch to string instead.

Also rename `STATUS_OPEN` to `STATUS_ACTIVE` and `STATUS_CLOSED` to `STATUS_RELEASED` for consistency. This makes resources and leases have more similar states, and gives resource states more accurate names.

Test Plan: Browsed web UI, grepped for changed constants, applied patch, inspected database.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14153
2015-09-24 07:57:05 -07:00
epriestley
c6aade4392 Give Drydock leases a resourcePHID instead of a resourceID
Summary:
Ref T9252. Leases currently have a `resourceID`, but this is a bit nonstandard and generally less flexible than giving them a `resourcePHID`.

In particular, a `resourcePHID` is easier to use when rendering interfaces, since you can get handles out of a PHID.

Add a PHID column, copy over all the PHIDs that correspond to existing IDs, then drop the ID column.

Test Plan:
  - Browsed web UIs.
  - Inspected database during/after migration.
  - Grepped for `resourceID`.
  - Allocated a new lease with `bin/drydock lease`.

Reviewers: chad, hach-que

Reviewed By: hach-que

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14151
2015-09-24 04:19:27 -07:00
epriestley
99e4472447 Soften checks on a very old Maniphest transactionmigration
Summary:
Ref T9464. If an ancient transaction doesn't have array values for whatever reason, we fail here.

Instead, just recover as gracefully as we can. We may get the transaction "wrong" in some sense, but this only impacts what is rendered in the transaction log.

Test Plan: This is nearly a year old and there's no real way to test it.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9464

Differential Revision: https://secure.phabricator.com/D14149
2015-09-23 19:18:44 -07:00
epriestley
fcb6d1e2fa Strip some obsolete code out of Drydock
Summary:
Ref T9252. This simplifies some Drydock code.

Most of this code relates to the old notion of Drydock being able to enumerate all the tasks it needs to complete in order to acquire a lease. The code has stepped back from this, since it's unnecessary, the queue is more powerful than it used to be, and it would be a lot of work to keep track of.

The ~only thing that should ever wait for leases in modern code is `bin/drydock lease`, and it's fine for it to just sit there sleeping, so this just does that.

This reduces the granularity of logging, but I'll address that separately in future logging-focused changes.

Test Plan: Used `bin/drydock lease` to acquire a lease, saw it acquire cleanly.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14147
2015-09-23 13:21:41 -07:00
epriestley
789df89c84 Add a command queue to Drydock to manage lease/resource release
Summary:
Ref T9252. Broadly, Drydock currently races on releasing objects from the "active" state. To reproduce this:

  - Scatter some sleep()s pretty much anywhere in the release code.
  - Release several times from web UI or CLI in quick succession.

Resources or leases will execute some release code twice or otherwise do inconsistent things.

(I didn't chase down a detailed reproduction scenario for this since inspection of the code makes it clear that there are no meaningful locks or mechanisms preventing this.)

Instead, add a Harbormaster-style command queue to resources and leases. When something wants to do a release, it adds a command to the queue and schedules a worker. The workers acquire a lock, then try to consume commands from the queue.

This guarantees that only one process is responsible for writes to active resource/leases.

This is the last major step to giving resources and leases a single writer during all states:

  - Resource, Unsaved: AllocatorWorker
  - Resource, Pending: ResourceWorker (Possible rename to "Allocated?")
  - Resource, Open: This diff, ResourceUpdateWorker. (Likely rename to "Active").
  - Resource, Closed/Broken: Future destruction worker. (Likely rename to "Released" / "Broken"; maybe remove "Broken").
  - Resource, Destroyed: No writes.
  - Lease, Unsaved: Whatever wants the lease.
  - Lease, Pending: AllocatorWorker
  - Lease, Acquired: LeaseWorker
  - Lease, Active: This diff, LeaseUpdateWorker.
  - Lease, Released/Broken: Future destruction worker (Maybe remove "Broken"?)
  - Lease, Expired: No writes. (Likely rename to "Destroyed").

In most phases, we can already guarantee that there is a single writer without doing any extra work. This is more complicated in the "Active" case because the release buttons on the web UI, the release tools on the CLI, the lease requestor itself, the garbage collector, and any other release process cleaning up related objects may try to effect a release. All of these could race one another (and, in many cases, race other processes from other phases because all of these get to act immediately) as this code is currently written. Using a queue here lets us make sure there's only a single writer in this phase.

One thing which is notable is that whatever acquires a lease **can not write to it**! It is never the writer once it queues the lease for activation. It can not write to any resources, either. And, likewise, Blueprints can not write to resources while acquiring or releasing leases.

We may need to provide a mechinism so that blueprints and/or resource/lease holders get to attach some storage to resources/leases for bookkeeping. For example, a blueprint might need to keep some kind of cache on a resource to help it manage state. But I think we can cross that bridge when we come to it, and nothing else would need to write to this storage so it's technically straightforward to introduce such a mechanism if we need one.

Test Plan:
  - Viewed buttons in web UI, checked enabled/disabled states.
  - Clicked the buttons.
  - Saw commands show up in the command queue.
  - Saw some daemon stuff get scheduled.
  - Ran CLI tools, saw commands get consumed and resources/leases release.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14143
2015-09-23 07:42:08 -07:00
epriestley
3ac99006bf Implement optimistic "slot locks" in Drydock
Summary:
See discussion in D10304. There's a lot of context there, but the general idea is:

  - Blueprints should manage locks in a granular way during the actual allocation/acquisition phase.
  - Optimistic "slot locks" might a pretty good primitive to make that easy to implement and reason about in most cases.

The way these locks work is that you just pick some name for the lock (like the PHID of a resource) and say that it needs to be acquired for the allocation/acquisition to work:

```
...
->needSlotLock("mylock(PHID-XYZQ-...)")
...
```

When you fire off the acquisition or allocation, it fails unless it could acquire the slot with that name. This is really simple (no explicit lock management) and a pretty good fit for most of the locking that blueprints and leases need to do.

If you need to do limit-based locks (e.g., maximum of 3 locks) you could acquire a lock like this:

```
mylock(whatever).slot(2)
```

Blueprints generally only contend with themselves, so it's normally OK for them to pick whatever strategy works best for them in naming locks.

This may not work as well if you have a huge number of slots (e.g., 100TB you want to give out in 1MB chunks), or other complex needs for locks (like you have to synchronize access to some external resource), but slot locks don't need to be the only mechanism that blueprints use. If they run into a problem that slot locks aren't a good fit for, they can use something else instead. For now, slot locks seem like a good fit for the problems we currently face and most of the problems I anticipate facing.

(The release workflows have other race issues which I'm not addressing here. They work fine if nothing races, but aren't race-safe.)

Test Plan:
To create a race where the same binding is allocated as a resource twice:

  - Add `sleep(10)` near the beginning of `allocateResource()`, after the free bindings are loaded but before resources are allocated.
  - (Comment out slot lock acquisition if you have this patch.)
  - Run `bin/drydock lease ...` in two windows, within 10 seconds of one another.

This will reliably double-allocate the binding because both blueprints see a view of the world where the binding is free.

To verify the lock works, un-comment it (or apply this patch) and run the same test again. Now, the lock fails in one process and only one resource is allocated.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Differential Revision: https://secure.phabricator.com/D14118
2015-09-21 04:45:25 -07:00
epriestley
f8080ce931 Add CustomField support to Owners
Summary: Fixes T9351. This is straightforward since this application is now relatively modern and doesn't have any bizarre craziness.

Test Plan:
{F787981}

{F787982}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9351

Differential Revision: https://secure.phabricator.com/D14093
2015-09-10 13:32:31 -07:00
epriestley
76665f725b Correct a bad Herald migration
Summary:
One of the migrations in rPa335004a91 (`20150730.herald.5.sql`) incorrectly swapped "add" and "add blocking" Differential Herald rules.

Swap any rules last modified before this patch was applied back. This is the best we can do without possibly overwriting more recent, intentional data. I'll issue some guidance on this in the changelog.

Test Plan:
  - Made a rule, ran patch, no change.
  - Changed rule modified time to a few months ago, ran patch, saw swap from non-blocking to blocking.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14061
2015-09-04 15:11:25 -07:00
Chad Little
2665970762 Basic Answer Wiki for Ponder
Summary: Adds an additional field for questions, an answer wiki, should should usually be community editable.

Test Plan: New question, edit question, no wiki, lots of wiki.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14003
2015-08-29 13:59:33 -07:00
Chad Little
d718415868 Swap duplicate close status on Ponder for invalid
Summary: Until we have a proper close as duplicate workflow for Ponder, remove the option with something more sensible.

Test Plan: Closed a question as invalid, saw it closed and in feed.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14007
2015-08-29 08:29:23 -07:00
epriestley
c612579854 Add very basic routing to Nuance
Summary:
Ref T8783. Sort out some relationships and fields:

  - Make Items 1:1 with Queues: each item is always in exactly one queue. Minor discussion on T8783. I think this is easier to understand and reason about (and implement!) and can't come up with any real cases where it isn't powerful enough.
  - Remove "QueueItem", which allowed items to be in multiple queues at once.
  - Remove "dateNuanced", which is equivalent to "dateCreated" in all cases.

Then add really basic routing:

  - Add "Default Queue" for Sources. New items from the source route into that queue.
  - (Some day there will be routing rules, but for now the rule is "always route into the default queue".)
  - Show queue on items.
  - Show more / more useful edit history and transactions in several UIs.

Test Plan:
{F749445}

{F749446}

{F749447}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8783

Differential Revision: https://secure.phabricator.com/D13988
2015-08-24 09:21:56 -07:00
Chad Little
e0faa66772 Allow Owners Packages to be archived
Summary: Fixes T8428. Adds status to packages, allows setting and application search. I presume though these need checked elsewhere?

Test Plan: New package, edit package, archive package, run search queries.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T8428

Differential Revision: https://secure.phabricator.com/D13925
2015-08-18 13:36:05 -07:00
epriestley
57b0353034 Add harbormaster.createartifact
Summary:
Ref T8659. In the general case, this eventually allows build processes to do things like:

  - Upload build results (like a ".app" or ".exe" or other binary).
  - Pass complex results between build steps (e.g., build step A does something hard and build step B uses it to do something else).

Today, we're a long way away from having the infrastructure for that. However, it is useful to let third party build processes (like Jenkins) upload URIs that link back to the external build results.

This adds `harbormaster.createartifact` so they can do that. The only useful thing to do with this method today is have your Jenkins build do this:

  params = array(
    "uri": "https://jenkins.mycompany.com/build/23923/details/",
    "name": "View Build Results in Jenkins",
    "ui.external": true,
  );
  harbormaster.createartifact(target, 'uri', params);

Then (after the next diff) we'll show a link in Differential and a prominent link in Harbormaster. I didn't actually do the UI stuff in this diff since it's already pretty big.

This change moves a lot of code around, too:

  - Adds PHIDs to artifacts.
  - It modularizes build artifact types (currently "file", "host" and "URI").
  - It formalizes build artifact parameters and construction:
    - This lets me generate usable documentation about how to create artifacts.
    - This prevents users from doing dangerous or policy-violating things.
  - It does some other general modernization.

Test Plan:
{F715633}

{F715634}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8659

Differential Revision: https://secure.phabricator.com/D13900
2015-08-15 07:28:56 -07:00
Chad Little
603c91e08a Add ability to hide answers in Ponder
Summary: Ref T9173, adds basic hide support for answers. Answer authors and Moderators can hide answers, unhide them.

Test Plan: Hide answer, log into other account, see hidden message. Mark as visible, see answer again.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9173

Differential Revision: https://secure.phabricator.com/D13894
2015-08-14 09:25:02 -07:00
Chad Little
7e7e38e9c0 Remove VotableInterface from PonderQuestion
Summary: Ref T6920, This removes the PonderVotableInterface from PonderQuestion and assocaited code. Also... never used?

Test Plan: Visit Ponder, See List, New Question, Add Answer.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T6920

Differential Revision: https://secure.phabricator.com/D13833
2015-08-08 20:29:37 -07:00
Chad Little
f98f5a081e Add a default moderation policy to Ponder
Summary: This allows installs to essentially set a "moderator" for Ponder, who can clean up answers. Fixes T9098

Test Plan: Edit an answer I don't own.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9098

Differential Revision: https://secure.phabricator.com/D13818
2015-08-08 12:20:01 -07:00
Chad Little
736bda7081 Fix SQL issue with Ponder migration
Summary: This fails to apply on my second sandbox with incorrect DOUBLE value. Reran SQL, works as expected.

Test Plan: Rerun new SQL on ponder_question table

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D13830
2015-08-08 11:07:42 -07:00
Chad Little
d2ef273ecd Add additional statuses to Ponder
Summary: Ref T9096. This is a first cut at adding additional statuses, happy to add or subtract as needed... maybe even configurable? Also, the dialog doesn't seem to fire, I'll keep debugging.

Test Plan: Close and Reopen many questions. Test applicationSearch params by seeing resolved questions, all questions.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9096

Differential Revision: https://secure.phabricator.com/D13826
2015-08-08 10:23:33 -07:00
Chad Little
5eeda6040e Allow setting of Archive/Active on Paste
Summary: Ref T9076, adds basic plumbing for setting the state of a Paste.

Test Plan: Archive Paste, Activate Paste, New Paste

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9076

Differential Revision: https://secure.phabricator.com/D13801
2015-08-05 13:44:55 -07:00
Chad Little
a3b955f948 Add Spaces to Ponder
Summary: Ref T8493, Ref T3578. Adds spaces support to ponder.

Test Plan: Ask a question in a new space, see new question.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T3578, T8493

Differential Revision: https://secure.phabricator.com/D13792
2015-08-05 09:38:14 -07:00
Chad Little
135d0c9ee7 Add Edit/View Policy to Ponder Questions
Summary: Ref T3578, adds ability to set a default edit and view policy for questions. Not sure what to set viewPolicy to ?

Test Plan: Test an old question, edit policy still on myself. Test a new question, see new default.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T3578

Differential Revision: https://secure.phabricator.com/D13791
2015-08-04 15:41:09 -07:00
Chad Little
4e7b5defc3 Add mailKeys to Ponder Answer
Summary: Ref T3846. Adds mailkey generation and migration.

Test Plan: Ran the migration, see keys in mysql.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T3846

Differential Revision: https://secure.phabricator.com/D13785
2015-08-04 10:49:15 -07:00
epriestley
a3e2f655eb Modularize the Diffusion/Differential "Block" Herald actions
Also removes HeraldCustomAction. This completes action modularization.

Ref T8726.
2015-08-03 14:33:28 -07:00