1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-14 10:52:41 +01:00
Commit graph

436 commits

Author SHA1 Message Date
epriestley
1786093c6e Replace "Cancel Build" with "Stop", "Resume" and "Restart"
Summary:
Ref T1049. Currently you can cancel a build, but now that we're tracking a lot more state we can stop, resume, and restart builds.

When the user issues a command against a build, I'm writing it into an auxiliary queue (`HarbormasterBuildCommand`) and then reading them out in the worker. This is mostly to avoid race messes where we try to `save()` the object in multiple places: basically, the BuildEngine is the //only// thing that writes to Build objects, and it holds a lock while it does it.

Test Plan:
  - Created a plan which runs "sleep 2" a bunch of times in a row.
  - Stopped, resumed, and restarted it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7892
2014-01-06 12:32:20 -08:00
epriestley
6abe65bfdc Add mailKey to macros
Summary:
If you have private replies on and a Macro reply handler set, we try to access `getMailKey()` and fail. See P1039 for a trace.

(Thanks to @Korvin for picking this up.)

Test Plan: Set configuration, repro'd the exception, applied the patch, then disabled/enabled a macro.

Reviewers: btrahan

Reviewed By: btrahan

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7896
2014-01-06 12:17:23 -08:00
epriestley
09341be10f Remove repository shortcuts
Summary:
Repositories currently have a no-UI "shortcut" feature which is only used by Facebook (and I'm not sure it's even used). As implemented, this feature is policy-oblivious and kind of nonsensical. Throw it away.

I'm open to reimplementing this, but I want to see some level of interest in it before I do. The new implementation would add shortcuts to each repository, similar to how mirrors work. My original plan was to follow this up with such an implementation (it's half-implemented in my sandbox), but as I worked through it I'm not sure it's really valuable.

Test Plan: Browsed repository list, grep.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Differential Revision: https://secure.phabricator.com/D7862
2014-01-02 11:59:27 -08:00
epriestley
f5fb3f05dc Lay most groundwork for Herald object rules
Summary:
Ref T4264. This gets most of the plumbing in for "object" rules, which will bind to a specific object, like a repository or project.

It does not yet let you actually create these rules.

Test Plan: Ran `storage upgrade`, created/edited rules, browsed Herald.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4264

Differential Revision: https://secure.phabricator.com/D7847
2013-12-27 13:17:10 -08:00
epriestley
ac19c55822 Formalize "manual" buildables in Harbormaster
Summary:
Ref T1049. Generally, it's useful to separate test/trial/manual runs from production/automatic runs.

For example, you don't want to email a bunch of people that the build is broken just because you messed something up when writing a new build plan. You'd rather try it first, then promote it into production once you have some good runs.

Similarly, test runs generally should not affect the outside world, etc. Finally, some build steps (like "wait for other buildables") may want to behave differently when run in production/automation than when run in a testing environment (where they should probably continue immediately).

So, formalize the distinction between automatic buildables (those created passively by the system in response to events) and manual buildables (those created explicitly by users). Add filtering, and stop the automated parts of the system from interacting with the manual parts (for example, we won't show manual results on revisions).

This also moves the "Apply Build Plan" to a third, new home: instead of the sidebar or Buildables, it's now on plans. I think this generally makes more sense given how things have developed. Broadly, this improves isolation of test environments.

Test Plan: Created some builds, browsed around, used filters, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7824
2013-12-26 10:40:43 -08:00
epriestley
c179ce6279 (Later...) Drop legacy Project transaction table
Summary: Ref T4010. I'll hold this for a bit, but we should eventually drop this table once the dust has settled.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7372
2013-12-19 07:03:51 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
James Rhodes
86ec4d6021 Implement policies in Phragment
Summary: This implements support for enforcing and setting policies in Phragment.

Test Plan: Set policies and ensured they were enforced successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7751
2013-12-13 14:42:12 +11:00
James Rhodes
8bb6e807f0 Implement snapshots in Phragment
Summary:
Ref T4212.  This implements snapshots in Phragment, which allows you to take a snapshot of a fragment at a given point in time, and download a ZIP of the snapshot as it was in this state.

There's also functionality for deleting and promoting snapshots.  You can promote a snapshot to either the latest version or any other snapshot of the fragment.

Test Plan: Clicked around, took some snapshots, promoted them to different points and deleted snapshots.  Also downloaded ZIPs of the snapshots and saw the right versions coming through for all the files downloaded.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205, T4212

Differential Revision: https://secure.phabricator.com/D7741
2013-12-09 08:24:50 +11:00
James Rhodes
25e7b7d53c Implement support for creating and updating fragments from ZIPs
Summary:
This implements support for creating and updating fragments from ZIP files.  It allows you to upload a ZIP via the Files application, create a fragment from it, and have it recursively imported into Phragment.  Updating that folder with another ZIP will recursively create, update and delete files as appropriate.

The logic for creating and updating fragments from files has also been centralized into the PhragmentFragment class.  Directories are also now supported; a directory fragment is simply a fragment that has no patches; thus a directory fragment can be converted to a file fragment by uploading a first patch for it.

Test Plan: Uploaded ZIP files through the interface and saw all of the fragments get created and updated as expected.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7729
2013-12-07 13:37:18 +11:00
James Rhodes
4c143ad3b2 Phragment v0
Summary: Ref T4205.  This is an initial implementation of Phragment.  You can create and browse fragments in the system (but you can't yet view a fragment's patches / history).

Test Plan: Clicked around and created fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7726
2013-12-07 12:43:49 +11:00
James Rhodes
79d153b85d Implement explicit build step ordering in Harbormaster
Summary: This implements support for explicitly marking the sequence of build steps.  Users can now drag and re-order build steps in plans, and artifact dependencies are re-calculated so that if you move "Run Command" before "Lease Host", the "Run Command" step has it's artifact setting cleared and thus the step becomes invalid.

Test Plan: Re-ordered build steps and observed dependencies being correctly recalculated.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7715
2013-12-06 14:12:15 +11:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
James Rhodes
53250d84df Introduce HarbormasterBuildTarget to snapshot build steps through a build
Summary: This implements build targets as outlined in D7582.  Build targets represent an instance of a build step particular to the build.  Logs and artifacts have been adjusted to attach to build targets instead of build / build step pairs.

Test Plan: Ran builds and clicked around the interface.  Everything seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7703
2013-12-05 12:01:12 +11:00
James Rhodes
ba16df0fed Restructure Drydock so that blueprints are instances in the DB
Summary:
//(this diff used to be about applying policies to blueprints)//

This restructures Drydock so that blueprints are instances in the DB, with an associated implementation class.  Thus resources now have a `blueprintPHID` instead of `blueprintClass` and DrydockBlueprint becomes a DAO.  The old DrydockBlueprint is renamed to DrydockBlueprintImplementation, and the DrydockBlueprint DAO has a `blueprintClass` column on it.

This now just implements CAN_VIEW and CAN_EDIT policies for blueprints, although they are probably not enforced in all of the places they could be.

Test Plan: Used the `create-resource` and `lease` commands.  Closed resources and leases in the UI.  Clicked around the new and old lists to make sure everything is still working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T2015

Differential Revision: https://secure.phabricator.com/D7638
2013-12-03 11:09:07 +11:00
epriestley
95c2b50974 Shorten extremely long credential names when migrating them
Summary: Fixes T4183. If you have too many repositories sharing the same credential and MySQL is in strict mode, we'll fail a query when trying to write a credential with a name longer than 255 characters. Instead, shorten the variable-length part to 128 characters.

Test Plan: Wiped credentials column and successfully re-ran migration with `storage upgrade --force --apply phabricator:20131121.repocredentials.2.mig.php`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4183

Differential Revision: https://secure.phabricator.com/D7677
2013-12-02 11:26:12 -08:00
epriestley
04b099d9ba Fix an issue with repository credential migration for older repositories
Summary: See IRC. It's possible to have a functional repository with the SSH username only in the URL. Look there if the username property isn't set. These should all be older repostiories.

Test Plan: Did a `--force --apply` upgrade.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D7665
2013-11-27 11:09:07 -08:00
epriestley
4b91c4f7ae Add UI for defining repository mirrors
Summary:
Ref T4038. This adds everything except the actual pushing part for mirrors.

This isn't the most beautiful or sophisticated UI, but I want get the authoritative repositories self-hosted and get users beta-ing hosting as soon as possible. We can do transactions, etc., later on.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7632
2013-11-22 15:23:50 -08:00
epriestley
51fb1ca16d Migrate repositories to use Passphrase for credential management
Summary: Fixes T4122. Ref T2230. Instead of storing credentials on each repository, store them in Passphrase. This allows easy creation/management of many repositories which share credentials.

Test Plan:
  - Upgraded repositories.
  - Created and edited repositories.
  - Pulled HTTP and SSH repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230, T4122

Differential Revision: https://secure.phabricator.com/D7629
2013-11-22 15:23:33 -08:00
epriestley
819f899013 Add an edge between Passphrase credentials and objects which use them
Summary: Ref T4122. Add an edge to keep track of where a credential is used, and show it in the UI.

Test Plan:
See "Used By":

{F84099}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7628
2013-11-22 15:23:23 -08:00
Bob Trahan
7b718bb033 Nuance - federate out the design of NuanceSource via NuanceSourceDefinition
Summary: ...and get the basic edit flow "working" for a new NuanceSourceDefinition - the Phabricator Form. ...and fix a dumb bug in the query class so when you redirect to the view page / try to edit an existing NuanceSource you don't fatal.

Test Plan: played around with the edit form and it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7585
2013-11-20 13:41:19 -08:00
epriestley
91d084624b Passphrase v0
Summary:
Ref T4122. Implements a credential management application for the uses described in T4122.

@chad, this needs an icon, HA HA HAHA HA BWW HA HA HA

bwahaha

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7608
2013-11-20 09:13:35 -08:00
epriestley
d9db1d61e0 Restore population of ownerOrdering to ManiphestTasks
Summary:
Ref T4110. This denormalized field used to power "Group By: Assigned" got dropped in the T2217 migration at some point.

Restore its population, and fix all the data in the database.

Test Plan: Ran migration, verified database came out reasonable-looking. Reassigned a task, verified database. Ran a "Group By: assigned" query.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4110

Differential Revision: https://secure.phabricator.com/D7602
2013-11-19 14:10:54 -08:00
epriestley
7f11e8d740 Improve handling of email verification and "activated" accounts
Summary:
Small step forward which improves existing stuff or lays groudwork for future stuff:

  - Currently, to check for email verification, we have to single-query the email address on every page. Instead, denoramlize it into the user object.
    - Migrate all the existing users.
    - When the user verifies an email, mark them as `isEmailVerified` if the email is their primary email.
    - Just make the checks look at the `isEmailVerified` field.
  - Add a new check, `isUserActivated()`, to cover email-verified plus disabled. Currently, a non-verified-but-not-disabled user could theoretically use Conduit over SSH, if anyone deployed it. Tighten that up.
  - Add an `isApproved` flag, which is always true for now. In a future diff, I want to add a default-on admin approval queue for new accounts, to prevent configuration mistakes. The way it will work is:
    - When the queue is enabled, registering users are created with `isApproved = false`.
    - Admins are sent an email, "[Phabricator] New User Approval (alincoln)", telling them that a new user is waiting for approval.
    - They go to the web UI and approve the user.
    - Manually-created accounts are auto-approved.
    - The email will have instructions for disabling the queue.

I think this queue will be helpful for new installs and give them peace of mind, and when you go to disable it we have a better opportunity to warn you about exactly what that means.

Generally, I want to improve the default safety of registration, since if you just blindly coast through the path of least resistance right now your install ends up pretty open, and realistically few installs are on VPNs.

Test Plan:
  - Ran migration, verified `isEmailVerified` populated correctly.
  - Created a new user, checked DB for verified (not verified).
  - Verified, checked DB (now verified).
  - Used Conduit, People, Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Differential Revision: https://secure.phabricator.com/D7572
2013-11-12 14:37:04 -08:00
James Rhodes
0ac1be7094 Implemented support for build logs
Summary:
Depends on D7519.

This implements support for build logs in Harbormaster.  This includes support for appending to a log from the "Run Remote Command" build step.

It also adds the ability to cancel builds.

Currently the build view page doesn't update the logs live; I'm sure this can be achieved with Javelin, but I don't have enough experience with Javelin to actually make it poll from updates to content in the background.

{F79151}

{F79153}

{F79150}

{F79152}

Test Plan:
Tested this by setting up SSH on a Windows machine and using a Remote Command configured with:

```
C:\Windows\system32\cmd.exe /C cd C:\Build && mkdir Build_${timestamp} && cd Build_${timestamp} && git clone --recursive https://github.com/hach-que/Tychaia.git && cd Tychaia && Protobuild.exe && C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe Tychaia.Windows.sln
```

and observed the output of the build stream from the Windows machine into Phabricator.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7521
2013-11-08 18:15:07 -08:00
Bob Trahan
66ae64f7bc Nuance - get some scaffolding up there
Summary:
I updated the wiki too - https://secure.phabricator.com/w/projects/pebkac/ - with what I am thinking right now. Rough plan here is

 - next diff:
  - implement editors and transactions
  - implement "web type" for contact source
   - /pebkac/item/new/ will be the entry point for this
  - implement "actions" on a contact
  - probably some "polish" on the scaffolding laid out here; like "create" permissions maybs
 - diffs after that:
  - implement "twitter" type for source
  - implement email reply handler stuff for item and source

Probs a great time to blast huge holes in all this stuff. :D

Test Plan: these pages load and arc lint doesn't complain

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran, chad

Differential Revision: https://secure.phabricator.com/D7465
2013-11-06 17:00:09 -08:00
epriestley
4f0f95f7b5 Assign PHIDs to all diffs
Summary:
Ref T1049. Ref T2222. `DifferentialDiff` does not currently have a PHID, but we need it for Harbormaster and ApplicationTransactions. See some discussion in D7501.

(I split the SQL into two sections so we can't fail in the middle. At some point, I'd like to do a pass on the migration stuff and get this happening automatically, and also simplify the PatchList.)

Test Plan:
  - Ran `bin/storage upgrade`.
  - Checked for valid PHIDs in the database.
  - Used `phid.query` to look up a diff by PHID.
  - Created a new diff and verified it got a PHID.

Reviewers: btrahan, hach-que

Reviewed By: btrahan

CC: aran, vrana

Maniphest Tasks: T2222, T1049

Differential Revision: https://secure.phabricator.com/D7513
2013-11-06 13:59:06 -08:00
James Rhodes
c514d34b94 Add build step implementation infrastructure and sleep build step.
Summary:
Depends on D7498.

This implements support for a "build step implementation".  Build steps have an associated class name (which makes the class in PHP) and a details field, which is serialized JSON (same as PhabricatorRepository).

This also implements a SleepBuildStepImplementation which just pauses the build for a specified period of seconds.

Test Plan:
Inserted a build step with `insert into harbormaster_buildstep (phid, buildPlanPHID, className, details, dateCreated, dateModified) values ('', 'PHID-HMCP-zkh5w6czfbfpk2gxwdeo', 'SleepBuildStepImplementation', '{"seconds":5}', NOW(), NOW());` (adjusting the build plan PHID as appropriate).

Started the daemon and applied the build plan to a buildable, and saw the daemon take a 5 second delay after creating `SleepBuildStepImplementation`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, chad

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7499
2013-11-05 13:34:44 -08:00
epriestley
0278b15ceb Implementation of VCS passwords against user.
Summary: This allows users to set their HTTP access passwords via Diffusion interface.

Test Plan: Clicked the "Set HTTP Access Password" link, set a password and saw it appear in the DB.

Reviewers: #blessed_reviewers, hach-que, btrahan

Reviewed By: hach-que

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7462
2013-11-01 08:34:11 -07:00
epriestley
3a39b01233 Add "RepositoryStatusMessage" and detailed information about initilization
Summary:
`RepositoryStatusMessage` is basically a key/value table associated with a repository that I'm using to let the daemons store the most recent event of a given type, so we can easily show it on the status dashboard. I think this will be a lot easier for users to figure out than digging through logfiles.

I'm also going to write the "this needs a pull" status here eventually, for reducing the time lapse between pushes and discovery.

  - Add storage for these messages.
  - Have the pull engine populate the INIT phase. I'll do the FETCH phase next.
  - Update the status readout to show all the various states.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7461
2013-10-30 16:04:19 -07:00
epriestley
9125095587 Distinguish between empty and unparsed commits in Diffusion
Summary:
Fixes T3416. Fixes T1733.

  - Adds a flag to the commit table showing whether or not we have parsed it.
  - The flag is set to `0` initially when the commit is discovered.
  - The flag is set to `1` when the changes are parsed.
  - The UI can now use the flag to distinguish between "empty commit" and "commit which we haven't imported changes for yet".
  - Simplify rendering code a little bit.
  - Fix an issue with the Message parser for empty commits.
  - There's a key on the flag so we can do `SELECT * FROM repository_commit WHERE repositoryID = %d AND importStatus = 0 LIMIT 1` soon, to determine if a repository is fully imported or not. This will let us improve the UI (Ref T776, Ref T3217).

Test Plan:
  - Ran `bin/storage upgrade -f`.
  - Created an empty commit.
  - Without the daemons running, ran `bin/repository pull GTEST` and `bin/repository discover GTEST`.
  - Viewed web UI to get the first screenshot ("Still Importing...").
  - Ran the message and change steps with `scripts/repository/reparse.php`.
  - Viewed web UI to get the second screenshot ("Empty Commit").

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776, T1733, T3416, T3217

Differential Revision: https://secure.phabricator.com/D7428
2013-10-29 15:32:41 -07:00
epriestley
86fe020a97 Add global "push" policy to Repositories
Summary: No editing or view yet, just adds the schema and a policy default. Part of D7391.

Test Plan: `bin/storage upgrade`

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7415
2013-10-29 15:32:40 -07:00
epriestley
b5a009337f Harbormaster v(-2)
Summary:
Ref T1049. I don't really want to sink too much time into this right now, but a seemingly reasonable architecture came to me in a dream. Here's a high-level overview of how things fit together:

  - **"Build"**: In Harbormaster, "build" means any process we want to run against a working copy. It might actually be building an executable, but it might also be running lint, running unit tests, generating documentation, generating symbols, running a deploy, setting up a sandcastle, etc.
  - `HarbormasterBuildable`: A "buildable" is some piece of code which build operations can run on. Generally, this is either a Differential diff or a Diffusion commit. The Buildable class just wraps those objects and provides a layer of abstraction. Currently, you can manually create a buildable from a commit. In the future, this will be done automatically.
  - `HarbormasterBuildStep`: A "build step" is an individual build operation, like "run lint", "run unit", "build docs", etc. The step defines how to perform the operation (for example, "run unit tests by executing 'arc unit'"). In this diff, this barely exists.
  - `HarbormasterBuildPlan`: This glues together build steps into groups or sequences. For example, you might want to "run unit", and then "deploy" if the tests pass. You can create a build plan which says "run step "unit tests", then run step "deploy" on success" or whatever. In the future, these will also contain triggers/conditions ("Automatically run this build plan against every commit") and probably be able to define failure actions ("If this plan fails, send someone an email"). Because build plans will run commands, only administrators can manage them.
  - `HarbormasterBuild`: This is the concrete result of running a `BuildPlan` against a `Buildable`. It tracks the build status and collects results, so you can see if the build is running/successful/failed. A `Buildable` may have several `Build`s, because you can execute more than one `BuildPlan` against it. For example, you might have a "documentation" build plan which you run continuously against HEAD, but a "unit" build plan which you want to run against every commit.
  - `HarbormasterBuildTarget`: This is the concrete result of running a `BuildStep` against a `Buildable`. These are children of `Build`. A step might be able to produce multiple targets, but generally this is something like "Unit Tests" or "Lint" and has an overall status, so you can see at a glance that unit tests were fine but lint had some issues.
  - `HarbormasterBuildItem`: An optional subitem for a target. For lint, this might be an individual file. For unit tests, an individual test. For normal builds, an executable. For deploys, a server. For documentation generation, there might just not be subitems.
  - `HarbormasterBuildLog`: Provides extra information, like command/execution transcripts. This is where stdout/stderr will get dumped, and general details and other messages.
  - `HarbormasterBuildArtifact`: Stores side effects or results from build steps. For example, something which builds a binary might put the binary in "Files" and then put its PHID here. Unit tests might put coverage information here. Generally, any build step which produces some high-level output object can use this table to record its existence.

This diff implements almost nothing and does nothing useful, but puts most of these object relationships in place. The two major things you can't easily do with these objects are:

  1) Run arbitrary cron jobs. Jenkins does this, but it feels tacked on and I don't know of anyone using it for that. We could create fake Buildables to get a similar effect, but if we need to do this I'd rather do it elsewhere in general. Build and cron/service/monitoring feel like pretty different problems to me.
  2) Run parameterized/matrix steps (maybe?). Bamboo has this plan/stage/task/job breakdown where a build step can generate a zillion actual jobs, like "build client on x86", "build server on x86", "build client on ARM", "build server on ARM", etc. We can sort of do this by having a Step map to multiple Targets, but I haven't really thought about it too much and it may end up being not-great. I'd guess we have like an 80% chance of getting a clean implementation if/when we get there. I suspect no one actually needs this, or when they do they'll just implement a custom Step and it can be parameterized at that level. I'm not too worried about this overall.

The major difference between this and Jenkins/Bamboo/TravisCI is that all three of those are **plan-centric**: the primary object in the system is a build plan, and the dashboard shows you all your build plans and the current status. I don't think this is the right model. One disadvantage is that you basically end up with top-level messaging that says "Trunk is broken", not "Trunk was broken by commit af32f392f". Harbormaster is **buildable-centric**: the primary object in the system is stuff you can run build operations against (commits/branches/revisions), and actual build plans are secondary. The main view will be "recent commits on this branch, and whether they're good or not" -- which I think is what's most important in a larger/more complex product -- not the pass/fail status of all jobs. This also makes it easier and more natural to integrate with Differential and Diffusion, which both care about the overall status of the commit/revision, not the current status of jobs.

Test Plan: Poked around, but this doesn't really do anything yet.

Reviewers: btrahan

Reviewed By: btrahan

CC: zeeg, chad, aran, seporaitis

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7368
2013-10-22 15:01:06 -07:00
epriestley
9b89e137cf Move Project transaction storage to modern tables
Summary:
Ref T4010. Projects have a weird proto-version of ApplicationTransactions which is very similar but not quite the same.

Move the storage to a modern format, but keep all the other code for now.

Test Plan: Migrated project transactions; edited projects.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7370
2013-10-22 13:49:28 -07:00
epriestley
d7a276346f Add a secret board view to Projects
Summary:
Ref T1344. This is //very// rough. Some UI issues:

  - Empty states for the board and columns are junky.
  - Column widths are crazy. I think we need to set them to fixed-width, since we may have an arbitrarily large number of columns?
  - I don't think we have the header UI elements in M10 yet and that mock is pretty old, so I sort of very roughly approximated it.
  - What should we do when you click a task title? Popping the whole task in a dialog is possible but needs a bunch of work to actually work. Might need to build "sheets" or something.
  - Icons are slightly clipped for some reason.
  - All the backend stuff is totally faked.

Generally, my plan is just to use these to implement all of T390. Specifically:

  - "Kanban" projects will have "Backlog" on the left. You'll drag them toward the right as you make progress.
  - "Milestone" projects will have "No Milestone" on the left, then "Milestone 9", "Milestone 8", etc.
  - "Sprint" projects will have "Backlog" on the left, then "Sprint 31", "Sprint 30", etc.

So all of these things end up being pretty much exactly the same, with some minor text changes and new columns showing up on the left vs the right or whatever.

Test Plan: See screenshot.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: chad, aran, sascha-egerer

Maniphest Tasks: T1344

Differential Revision: https://secure.phabricator.com/D7374
2013-10-21 21:11:36 -07:00
epriestley
cf1c06e157 Add custom field storage to Projects
Summary: Ref T4010. Adds storage and indexes for custom fields. These tables are the same as people/maniphest/differential.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7369
2013-10-21 17:01:13 -07:00
epriestley
f010730e49 Migrate all Differential inline comments to ApplicationTransactions
Summary:
Ref T2222. This implements step (1) described there, which is moving over all the inline comments.

The old and new tables are simliar. The only real trick here is that `transactionPHID` and `legacyCommentID` mean roughly the same thing (`null` if the inline is a draft, non-null if it has been submitted) but we don't have real `transactionPHID`s yet. We just make some up -- we'll backfill them later.

Two risks here:

  - I need to take a second look at the keys on this table. I think we need to tweak them a bit, and it will be less disruptive to do that before this migration than after.
  - This will take a while for Facebook, and other large installs with tens of thousands of revisions. I'll communicate this.

I'm otherwise pretty satisfied with this, seems to work well and is pretty low risk / non-disruptive.

Test Plan:
  - Before migrating, then after migrating:
    - Made a bunch of inlines (drafts, submitted).
    - Edited and deleted inlines.
    - Verified inlines showed up in preview.
    - Verified that inlines aren't indexed when they're drafts (`bin/search index D935`).
    - Verified that inlines ARE indexed when they're not drafts.
    - Verified that drafts inlines make revisions appear as "with draft" in the revision list.
  - Made left, right, and draft inlines.
  - Migrated (`bin/storage upgrade`).
  - Verified that my inlines from before the migration still showed up.
  - (Repeated all the stuff above.)
  - Manually inspected the inline comment table.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7139
2013-10-19 05:03:25 -07:00
epriestley
460051c1d8 Drop maniphest_savedquery table
Summary: This data was migrated by D6977 and is now obsolete. I'll hold this patch for a week or two in case we get reports of migration errors.

Test Plan: Ran storage upgrade, saw the table vanish. Grepped for references to the table.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6997
2013-10-19 04:56:17 -07:00
epriestley
3410cbd53e Add application and object level policy controls to Countdown
Summary: Ref T603. Give countdowns proper UI-level policy controls, and an application-level default policy. Put policy information in the header.

Test Plan:
  - Adjusted default policy.
  - Created new countdowns.
  - Edited countdowns.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7322
2013-10-16 10:36:08 -07:00
epriestley
5899ae08b3 Add storage for custom policies
Summary: Ref T603. Allows custom policies to be saved. No integration with policy controls yet.

Test Plan:
  mysql> select * from policy where id = 3\G
  *************************** 1. row ***************************
             id: 3
           phid: PHID-PLCY-e4v2fnbyuibi4supl5tn
          rules: [{"action":"allow","rule":"PhabricatorPolicyRuleAdministrators","value":null},{"action":"allow","rule":"PhabricatorPolicyRuleProjects","value":["PHID-PROJ-cwovm5gn2ilubjehcdgd"]},{"action":"allow","rule":"PhabricatorPolicyRuleLunarPhase","value":"new"}]
  defaultAction: deny
    dateCreated: 1381437466
   dateModified: 1381437466
  1 row in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7282
2013-10-10 16:09:51 -07:00
epriestley
953ff197bf Allow Herald rules to be disabled, instead of deleted
Summary:
Ref T603. Ref T1279. Further improves transaction and policy support for Herald.

  - Instead of deleting rules (which wipes out history and can't be undone) allow them to be disabled.
  - Track disables with transactions.
  - Gate disables with policy controls.
  - Show policy and status information in the headers.
  - Show transaction history on rule detail screens.
  - Remove the delete controller.
  - Support disabled queries in the ApplicationSearch.

Test Plan:
  - Enabled and disabled rules.
  - Searched for enabled/disabled rules.
  - Verified disabled rules don't activate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279, T603

Differential Revision: https://secure.phabricator.com/D7247
2013-10-06 17:10:29 -07:00
epriestley
65ddefad8b Migrate all Differential reviewer data to edges
Summary:
Ref T1279. @champo did a lot of this work already; we've been doing double writes for a long time.

Add "double reads" (reading the edge table as both the "relationship" table and as the "reviewer status" table), and migrate all the data.

I'm not bothering to try to recover old reviewer status (e.g., we could infer from transactions who accepted old revisions) because it wold be very complicated and doesn't seem too valuable.

Test Plan:
  - Without doing the migration, used Differential. Verified that reads and writes worked. Most of the data was there anyway since we've been double-writing.
  - Performed the migration. Verified that everything was still unchanged.
  - Dropped the edge table, verified all reviweer data vanished.
  - Migrated again, verified the reviewer stuff was restored.
  - Did various cc/reviewer/subscriber queries, got consistent results.

Reviewers: btrahan

Reviewed By: btrahan

CC: champo, aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7227
2013-10-05 13:54:02 -07:00
epriestley
bd8071bb33 Add a (dst, type, src) key to Differential's edge table
Summary:
Ref T1279. This came to me in a dream.

The existing `differential_relationship` table has an `(objectPHID, type)` column, which theoretically is useful for queries like "revisions with X as a reviewer". In practice, I'm not sure it gets used much, but I can get it to show up in at least some query plans.

Add a similar index to the `edge` table. This sequences //before// D7227, which actually migrates the data.

Test Plan:
  - Ran `bin/storage upgrade`.
  - EXPLAIN'd a bunch of queries against different versions of the schema, this seemed helpful overall.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7232

Conflicts:
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-10-05 13:53:43 -07:00
epriestley
7dde01df76 Fix issues with first-time account registration
Summary: This worked originally, but the migration broke slightly after the
config was deprecated, and there was another minor issue during setup.
2013-10-05 08:02:41 -07:00
epriestley
98bf001a58 Add viewPolicy and attachedToObjectPHID to PhabricatorFile
Summary:
Ref T603. Principally, I want to implement the rule "when you upload a file to an object, users must be able to see the object in order to see the file", since I think this is strongly in line with user expectation. For example, if you attach a file to a Conpherence, it should only be visible to members of that thread.

This adds storage for policies, but doesn't do anything interesting with it yet.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7175
2013-10-01 08:45:18 -07:00
epriestley
7091dad606 Allow macros to have associated audio and audio behaviors
Summary: Ref T3887. Implements storage and editors, but not the actual audio part.

Test Plan: Edited audio, audio behaviors of macros. Transactions and email looked good. Hit error cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3887

Differential Revision: https://secure.phabricator.com/D7159
2013-09-27 16:01:37 -07:00
epriestley
3cb67480e0 Adjust keys for new Differential inline comment table
Summary:
Ref T2222. This sequences //before// D7139 and sorts out keys on the table. In particular:

  - There was a fairly silly `draft` key modeled after Pholio; drop it.
  - Add a `revisionPHID` key. This is queried mostly-transitionally on the revision view screen.
  - Add a `changesetID` key. This is queried by a bunch of interfaces that want more surgical results than `revisionPHID` provides.
  - Add an `authorPHID, transactionPHID` key. This is queried on the list interface to find pending drafts.
  - Add a `legacy` key. This is queried by the feed publisher.

Test Plan: Used the query analyzer to hit all (I think?) of the pages, saw keyed queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7140
2013-09-26 13:48:36 -07:00
epriestley
5677cd23bd Add storage and classes for CustomField in Differential
Summary: Ref T3886. Adds the storage, indexes, and storage classes for modernizing Differential custom fields.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886

Differential Revision: https://secure.phabricator.com/D7138
2013-09-26 12:37:28 -07:00
epriestley
c458517cb4 Add viewPolicy, editPolicy, repositoryPHID columns to DifferentialRevision
Summary: Ref T603. Paves the way for policy controls.

Test Plan: Ran storage upgrade, bumbled around in Differential.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7133
2013-09-26 12:36:30 -07:00
epriestley
225a38c7d3 Add viewPolicy, editPolicy storage to tasks
Summary: Ref T603. Adds storage for custom policies.

Test Plan: Ran storage upgrade. Created and edited tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7118
2013-09-25 11:18:47 -07:00
epriestley
960da82f33 Fix two migration issues with Maniphest
Summary: The new editor saves tasks before applying transactions, so fields need default values.
Also, I goofed the table name.

Auditors: btrahan
2013-09-24 12:03:47 -07:00
epriestley
6e29e809b4 Fix "edit" for Maniphest comments
Summary:
Ref T2217. Fixes T3876. We incorrectly have a unique key on `(authorPHID, transactionPHID)`, which prevents saving multiple versions of a comment.

I'm not entirely sure why this exists. I think it came from Pholio (where it works for inlines, because it has an additional component, but maybe should be adjusted) but we might need to wipe it out of more apps too.

Test Plan: Edited a comment in Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217, T3876

Differential Revision: https://secure.phabricator.com/D7102
2013-09-24 11:10:31 -07:00
epriestley
1d7cbe202f Rename "transactionpro" table to "transaction"
Summary: Ref T2217. Cleans up the table names. Moves old data to `maniphest_transaction_legacy`. We'll drop that eventually once it's more clear that I didn't break the world.

Test Plan: Did reads/writes to/from these tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7094
2013-09-24 10:49:16 -07:00
epriestley
01ecf1a236 Rename ManiphestTransactionPro -> ManiphestTransaction
Summary: Ref T2217. Pro is the new standard.

Test Plan: Lots of `grep`, made a pile of Maniphest views/edits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7093
2013-09-24 10:49:06 -07:00
epriestley
7abe9dc4c0 Migrate all Maniphest transaction data to new storage
Summary:
Ref T2217. This is the risky, hard part; everything after this should be smooth sailing. This is //mostly// clean, except:

  - The old format would opportunistically combine a comment with some other transaction type if it could. We no longer do that, so:
    - When migrating, "edit" + "comment" transactions need to be split in two.
    - When editing now, we should no longer combine these transaction types.
    - These changes are pretty straightforward and low-impact.
  - This migration promotes "auxiliary field" data to the new CustomField/StandardField format, so that's not a straight migration either. The formats are very similar, though.

Broadly, this takes the same attack that the auth migration did: proxy all the code through to the new storage. `ManiphestTransaction` is now just an API on top of `ManiphestTransactionPro`, which is the new storage format. The two formats are very similar, so this was mostly a straight copy from one table to the other.

Test Plan:
  - Without performing the migration, made a bunch of edits and comments on tasks and verified the new code works correctly.
  - Droped the test data and performed the migration.
  - Looked at the resulting data for obvious discrepancies.
  - Looked at a bunch of tasks and their transaction history.
  - Used Conduit to pull transaction data.
  - Edited task description and clicked "View Details" on transaction.
  - Used batch editor.
  - Made a bunch more edits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7068
2013-09-23 14:25:28 -07:00
epriestley
4c2bd2ca8e Add storage for new Maniphest transactions
Summary: Ref T2217. Add the tables and comment class for the new stuff. Not used yet.

Test Plan: Ran storage upgrade, browsed Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7066
2013-09-23 14:24:58 -07:00
epriestley
b8cb6ddaa5 Add some keys and policy fields to repositories
Summary:
  - Add some TODO'd keys.
  - Add policy fields.

Test Plan: Viewed repositories; created a new repository and verified it got the right default policy settings.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7056
2013-09-21 16:23:01 -07:00
epriestley
0558d53273 Convert maniphest to use standard fields
Summary: Ref T3794. Drop auxiliary field, use standard field.

Test Plan: Performed migration, field seemed to survive it intact. Edited and viewed tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3794

Differential Revision: https://secure.phabricator.com/D7036
2013-09-19 11:56:15 -07:00
epriestley
bd40e74400 Migrate auxiliary field storage to common field storage
Summary: Ref T418. Moves data from the Maniphest-specific table to the general one. This patch is a bit gross, but mostly about getting the reads and writes aimed correctly. Future patches will clean things up.

Test Plan: Migrated data across formats. Verified it survied the migration. Viewed and edited tasks' custom fields.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6999
2013-09-16 16:02:06 -07:00
epriestley
b0e145f2fe Provide generalized custom field storage and custom field indexes in Maniphest
Summary: Ref T418. Depends on D6992. This adds index and value storage for Maniphest custom fields.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6995
2013-09-16 14:06:41 -07:00
epriestley
c8574cf6fd Integrate ApplicationSearch with CustomField
Summary:
Ref T2625. Ref T3794. Ref T418. Ref T1703.

This is a more general version of D5278. It expands CustomField support to include real integration with ApplicationSearch.

Broadly, custom fields may elect to:

  - build indicies when objects are updated;
  - populate ApplicationSearch forms with new controls;
  - read inputs entered into those controls out of the request; and
  - apply constraints to search queries.

Some utility/helper stuff is provided to make this easier. This part could be cleaner, but seems reasonable for a first cut. In particular, the Query and SearchEngine must manually call all the hooks right now instead of everything happening magically. I think that's fine for the moment; they're pretty easy to get right.

Test Plan:
I added a new searchable "Company" field to People:

{F58229}

This also cleaned up the disable/reorder view a little bit:

{F58230}

As it did before, this field appears on the edit screen:

{F58231}

However, because it has `search`, it also appears on the search screen:

{F58232}

When queried, it returns the expected results:

{F58233}

And the actually good bit of all this is that the query can take advantage of indexes:

  mysql> explain SELECT * FROM `user` user JOIN `user_customfieldstringindex` `appsearch_0` ON `appsearch_0`.objectPHID = user.phid AND `appsearch_0`.indexKey = 'mk3Ndy476ge6' AND `appsearch_0`.indexValue IN ('phacility') ORDER BY user.id DESC LIMIT 101;
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  | id | select_type | table       | type   | possible_keys     | key      | key_len | ref                                      | rows | Extra                                        |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  |  1 | SIMPLE      | appsearch_0 | ref    | key_join,key_find | key_find | 232     | const,const                              |    1 | Using where; Using temporary; Using filesort |
  |  1 | SIMPLE      | user        | eq_ref | phid              | phid     | 194     | phabricator2_user.appsearch_0.objectPHID |    1 |                                              |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  2 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418, T1703, T2625, T3794

Differential Revision: https://secure.phabricator.com/D6992
2013-09-16 13:44:34 -07:00
epriestley
5e274778d1 Restore one missing "order" value to the saved query migration script
Summary: Caught this when migrating my queries in production.

Auditors: btrahan
2013-09-13 12:00:50 -07:00
epriestley
9864d562dd Migrate old Maniphest custom queries to new query infrastructure
Summary: Ref T2625. EVERYONE LOVES MIGRATIONS!!!

Test Plan:
  - Created and migrated a query with every field, verified results were preserved.
  - Created and migrated a query using "noproject" and "upforgrabs" magic, verified results were preserved.

Here's the pre-migration "everything" query:

{F58110}

Here it is after migration:

{F58111}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6977
2013-09-13 11:49:45 -07:00
epriestley
b38a688cf7 Ignore failures when rebuilding project search index in Maniphest migration
Summary: See <https://github.com/facebook/phabricator/issues/386>. Just ignore any issues here, they aren't worth complaining about.

Auditors: btrahan
2013-09-13 09:39:46 -07:00
epriestley
e50eccf109 Provide and populate an object name index for Maniphest
Summary: See discussion in D6955. This provides a table we can JOIN against to (effectively) "ORDER BY project name", populates it intially, and keeps it up to date as projects are edited.

Test Plan:
  - Ran storage upgrade, verified projects populated into the table.
  - Edited a project, verified its entry updated.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6957
2013-09-12 13:06:44 -07:00
epriestley
df86f87289 Add a "dateCreated" key to Maniphest
Summary: Depends on D6952. Unpunts there since I'm rolling into a swamp full of schema changes.

Test Plan: Issued date-constrained query and saw key as a candidate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6954
2013-09-12 13:04:31 -07:00
epriestley
8f8c61be31 Remove legacy "touched" table and indexing
Summary: Noticed this in the schema. "Touches" were an idea that never really got off the ground, as we built out more/better notification channels instead. Essentially, they recorded any object you'd ever interacted with. Maybe this will be useful some day, but for now it does nothing and can't be interacted with. Nuke it.

Test Plan: `grep`, loaded Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6953
2013-09-12 13:04:09 -07:00
epriestley
2d240142c8 Fix 20130820.file-mailkey-populate.php for databases in strict mode
Summary:
Fixes T3803. Turns out my advice was sort of terrible. :/

In strict mode, the `INSERT INTO x (y, z)` raises an error unless `(y, z, ...)` includes //all// columns without default values.

Test Plan: Ran `storage upgrade` on a strict-mode install. Verified no inserts were performed.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3803

Differential Revision: https://secure.phabricator.com/D6894
2013-09-05 15:58:28 -07:00
Bob Trahan
228496cdbe File - add transactions and editor
Summary: this ends up being a little weird since you can't actually edit files. Also, since we create files all sorts of ways, sometimes without even having a user, we don't  bother logging transactions for those events. Fixes T3651. Turns out this work is important for T3612, which is a priority of mine to help get Pholio out the door.

Test Plan: left a comment on a file. it worked! use bin/mail to verify mail content looked correct.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran, wez

Maniphest Tasks: T3651, T3612

Differential Revision: https://secure.phabricator.com/D6789
2013-09-05 13:11:02 -07:00
epriestley
41ac06959e Generate some amount of PHP class documentation
Summary:
Ref T988. This brings the class/interface atomizer over. A lot of parts of this are still varying degrees of very-rough, but most of the data ends up in approximatley the right place.

ALSO: PROGRESS BARS

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6817
2013-08-28 09:54:39 -07:00
epriestley
49e0aa203c Add storage for Releeph project and branch transactions
Summary: Ref T3663. Does what it says on the tin.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3663

Differential Revision: https://secure.phabricator.com/D6778
2013-08-21 12:31:51 -07:00
epriestley
973bb0c76e Remove ReleephRequestEvent
Summary: Ref T3663. This is obsolete code which is used only in this migration, which Facebook has already performed and which isn't relevant for any other installs.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3663

Differential Revision: https://secure.phabricator.com/D6777
2013-08-21 12:31:50 -07:00
epriestley
ca0115b361 Support configuration-driven custom fields
Summary:
Ref T1702. Ref T3718. There are a couple of things going on here:

**PhabricatorCustomFieldList**: I added `PhabricatorCustomFieldList`, which is just a convenience class for dealing with lists of fields. Often, current field code does something like this inline in a Controller:

  foreach ($fields as $field) {
    // do some junk
  }

Often, that junk has some slightly subtle implications. Move all of it to `$list->doSomeJunk()` methods (like `appendFieldsToForm()`, `loadFieldsFromStorage()`) to reduce code duplication and prevent errors. This additionally moves an existing list-convenience method there, out of `PhabricatorPropertyListView`.

**PhabricatorUserConfiguredCustomFieldStorage**: Adds `PhabricatorUserConfiguredCustomFieldStorage` for storing custom field data (like "ICQ Handle", "Phone Number", "Desk", "Favorite Flower", etc).

**Configuration-Driven Custom Fields**: Previously, I was thinking about doing these with interfaces, but as I thought about it more I started to dislike that approach. Instead, I built proxies into `PhabricatorCustomField`. Basically, this means that fields (like a custom, configuration-driven "Favorite Flower" field) can just use some other Field to actually provide their implementation (like a "standard" field which knows how to render text areas). The previous approach would have involed subclasssing the "standard" field and implementing an interface, but that would mean that every application would have at least two "base" fields and generally just seemed bleh as I worked through it.

The cost of this approach is that we need a bunch of `proxy` junk in the base class, but that's a one-time cost and I think it simplifies all the implementations and makes them a lot less magical (e.g., all of the custom fields now extend the right base field classes).

**Fixed Some Bugs**: Some of this code hadn't really been run yet and had minor bugs.

Test Plan:
{F54240}
{F54241}
{F54242}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1702, T1703, T3718

Differential Revision: https://secure.phabricator.com/D6749
2013-08-14 12:33:53 -07:00
epriestley
9f41032693 Remove "cut point commit identifier" from Releeph
Summary: Ref T3656. Releeph denormalizes branch cut point identifiers into Branch objects, but this information isn't useful or used for sorting, filtering, or enforcing unique constraints. Instead, derive it via noramlized pathways from the `cutPointCommitPHID`.

Test Plan: Ran storage upgrade. Ran `releephwork.getbranch` and `releeph.getbranches`. Grepped for `cutPointCommitIdentifier`.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3656

Differential Revision: https://secure.phabricator.com/D6636
2013-08-14 09:01:38 -07:00
epriestley
a84cc777c8 Remove "Project ID" from Releeph Projects
Summary:
Fixes T3660. Releeph Projects currently have an unused one-to-one mapping to Phabricator projects. This isn't consistent with other applications and has no integrations or uses. Get rid of it.

NOTE: Waiting for signoff from @legneato on T3660 before pulling the trigger here.

Test Plan: Created and edited Releeph projects. Grepped for references to project ID; there are a dozen or so but they're all either Releeph projects or Arcanist projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3660

Differential Revision: https://secure.phabricator.com/D6635
2013-08-14 09:00:56 -07:00
epriestley
296818e129 Remove ReleephProject->repositoryID writes
Summary: Ref T3655. Depends on D6633. This removes the writes and the column.

Test Plan: Created a project, edited a project. Verified the table doesn't have any keys including this column.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3655

Differential Revision: https://secure.phabricator.com/D6634
2013-08-14 09:00:25 -07:00
epriestley
b1c4a258c9 Add ApplicationTransactions to Herald
Summary: Ref T2769. I'm planning to keep this pretty simple, but we have this ad-hoc edit log for rules already and some other mess that we can clean up.

Test Plan: No effect yet; see future changes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6654
2013-08-07 18:03:49 -07:00
epriestley
2820fdc89b Add PHIDs to Herald Rules
Summary: Ref T2769. Precursor to various Herald-related modernizations.

Test Plan: Ran migration; loaded Herald via web.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6648
2013-08-07 18:03:37 -07:00
Bob Trahan
0e6b5073cd Paste - add support for email replies and subscribers
Summary: Email replies and subscribers seem to go hand in hand so deploy both at once.

Test Plan: played around with bin/mail. Verified replies posted comments on the paste.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3650

Differential Revision: https://secure.phabricator.com/D6682
2013-08-05 17:11:46 -07:00
epriestley
ed9edc5d3a Minor, fix Paste SQL patch for databases with all warnings turned on.
This column has no default value and throws if you have maximum warnings activated:

  EXCEPTION: (AphrontQueryException) #1364: Field 'commentVersion' doesn't have a default value...

Auditors: btrahan
2013-08-04 11:32:32 -07:00
Bob Trahan
37a5c4b11a Paste - add transactions
Summary: Ref T3650. This adds a create transaction, transactions for metadata (title, langauge, view policy), and comments. Editor is used on all create /edit paths.

Test Plan: made some pastes via web and email - yay. edited pastes - yay. verified txns showed up on pastes and in feed correctly.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3516, T3650

Differential Revision: https://secure.phabricator.com/D6645
2013-08-02 12:56:58 -07:00
epriestley
5cc3bbf721 Use application PHIDs for application transactions
Summary: Ref T2715. Ref T3578. Load application transactions through application PHID infrastructure.

Test Plan: Viewed feed, saw successful loads of application transaction objects and rendered feed stories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715, T3578

Differential Revision: https://secure.phabricator.com/D6617
2013-07-29 12:04:15 -07:00
epriestley
71841e262a Migrate Ponder comments to ApplicationTransactions
Summary: Ref T3373. Bring the storage across.

Test Plan: {F52067}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3373

Differential Revision: https://secure.phabricator.com/D6606
2013-07-29 12:04:05 -07:00
epriestley
6172f2bbd1 Require answers' authors to be unique in Ponder
Summary: Ref T3578. I forget if this was an explicit decision or not, but we currently let the same user answer questions multiple times. I think this probably causes more confusion than it provides freedom. In conjunction with other UI issues (commenting being weird, notably), we're seeing some use of answers to comment, which is undesirable. Require each answer's author to be unique. Merge existing nonunique authors' answers.

Test Plan: {F52062}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3578

Differential Revision: https://secure.phabricator.com/D6605
2013-07-29 12:04:04 -07:00
epriestley
8f0c836063 Add question and answer transactions to Ponder
Summary: Ref T3373.

Test Plan:
Ran this script, saw valid inserts:

  <?php

  $unguarded = AphrontWriteGuard::beginScopedUnguardedWrites();

  id(new PonderQuestionTransaction())
    ->setAuthorPHID('herp')
    ->setObjectPHID('derp')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setTransactionType('beep')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderAnswerTransaction())
    ->setAuthorPHID('herp')
    ->setObjectPHID('derp')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setTransactionType('beep')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderQuestionTransactionComment())
    ->setCommentVersion(1)
    ->setAuthorPHID('bloop')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setContent('blip')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderAnswerTransactionComment())
    ->setCommentVersion(1)
    ->setAuthorPHID('bloop')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setContent('blip')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  unset($unguarded);

  echo "OK!\n";

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3373

Differential Revision: https://secure.phabricator.com/D6584
2013-07-28 15:08:36 -07:00
Gareth Evans
70a5ec600d Allowing Closing of questions
Summary:
Fixes T3579

As a basic overview, this enables the author of a question to open/close a question.

Other bits;

- Add "Open" filter to the builtin queries
- Add "Status" to search form
- Refactor ponder constants
- Add coloured bars for different question statuses

Test Plan:
- Open/Close questions
- Search for some bits
- Use filters

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3579

Differential Revision: https://secure.phabricator.com/D6590

Conflicts:
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-07-27 18:41:03 -07:00
Bob Trahan
f75f3a0c3b Pholio - add concept of replacing images and primitive history view
Summary:
Now you can actually replace an image! Ref T3572. This ended up needing a wee bit of infrastructure to work...

 - add replace image transaction to pholio
 - add replacesImagePHID to PholioImage
 - tweaks to editor to properly update images with respect to replacement
   - add edges to track replacement
 - expose getNodes on graph query infrastructure to query the entire graph of who replaced who
 - move pholio image to new phid infrastructure

Still TODO - the history view should get chopped out a bit from the current view - no more inline comments / generally less functionality plus maybe a tweak or two to make this more sensical.

Test Plan: replaced images and played with history controller a little. works okay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3572

Differential Revision: https://secure.phabricator.com/D6560
2013-07-25 16:59:25 -07:00
epriestley
6750a48951 Surface task queue temporary failure rate in Daemon console
Summary: Fixes T3557. One thing which made T3557 kind of a mess was the lack of information about progress through temporary failures. Add a column which records a task's last failure time, and surface it in the console.

Test Plan: {F51277}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3557

Differential Revision: https://secure.phabricator.com/D6550
2013-07-23 16:58:22 -07:00
epriestley
17ee71d896 Use Application PHIDs in Maniphest
Summary: Ref T2715. Switch Maniphest to the new stuff.

Test Plan: Used `phid.query`; `phid.lookup` to load objects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6516
2013-07-22 12:17:35 -07:00
epriestley
1fb39a20d3 Move DifferentialRevision to application PHIDs
Summary: Ref T2715.

Test Plan: Used `phid.lookup` and `phid.query` to load handles. Grepped for `PHID_TYPE_DREV`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6509
2013-07-22 12:17:29 -07:00
epriestley
7ed6996604 Provide basic infrastructure for moving PHIDs, Handles and Object Names to applications
Summary:
See discussion in T2715. Currently, PHIDs are all hard coded in the PHID application. In the long run, we need to move them out into actual applications.

A specific immediate issue is Releeph, which uses a very very old and very broken mechanism to inject PHIDs in a way that only sort of works.

Moving forward, every PHID type will be provided by a `PhabricatorPHIDType` subclass, which will manage loading it, etc.

This also moves toward cleaning up the "load objects by name" (where "name" means something like `D12`) code, which is an //enormous// mess and spread across at least 4-5 callsites.

Test Plan: Used `phid.lookup` and `phid.query` to load Slowvotes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6502
2013-07-21 06:34:21 -07:00
Bob Trahan
001f76cbaa Projects - tighten up a few things
Summary:
Fixes T2675, T2676.

 - when the last person leaves a project it is archived.
 - a script to archive all memberless projects
 - better feed stories for the various policy edits you can do
 - phriction pages are also moved as you rename projects

Test Plan: edited some projects and noted reasonable feed stories. ran script against test data and it worked! left a last man standing project and it archived. renamed a project to "a" then "b" then "a" (etc) and it worked including phrictiondocument moves

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2676, T2675

Differential Revision: https://secure.phabricator.com/D6478
2013-07-17 16:43:37 -07:00
Bob Trahan
df264d8548 Pholio - support editing images - fixes T3489
Summary:
Nice title. We add three new transactions - IMAGE_FILE, IMAGE_NAME, and IMAGE_DESCRIPTION. The first is a bit like subscribers as it is a list of file phids. The latter have values of the form ($file_phid => $data), where $data is $name or $description respectively. This is because we need to collate transactions based on $file_phid...

Overall, this uses the _underyling files_ and not the "PholioImage" to determine if things are unique or not. That said, simply mark PholioImages as obsolete so inline comments about no-longer applicable PholioImages don't break.

Does a reasonable job implementing the mock. Note you can't "update" an image at this time, though you can delete and add at will.

Test Plan: played with pholio a ton.

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T3489

Differential Revision: https://secure.phabricator.com/D6441
2013-07-16 13:31:20 -07:00
epriestley
d2f8c5b5e7 Make slowvotes subscribable
Summary: Subscribing doesn't do anything yet, but you can subscribe!

Test Plan: {F50169}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6456
2013-07-16 10:30:28 -07:00
epriestley
ea52bcbcd6 Migrate Slowvote comments to ApplicationTransactions
Summary:
Move comments from the old table to ApplicationTransactions. Patch dances around which objects it uses since I intend to delete the comment table.

NOTE: This temporarily disables comment writes. I'll restore them shortly.

Test Plan: {F50166}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6454
2013-07-16 10:30:21 -07:00
epriestley
ef1bedef02 Add ApplicaionTransactions and a mutable view policy to Slowvote
Summary: Schema changes to modernize this app.

Test Plan: Ran schema changes, created a new slowvote. No real effects yet.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6453
2013-07-16 10:30:14 -07:00
epriestley
237ca095a6 Trim trailing whitespace from user realnames
Summary: Fixes T3069. Ref T3516. For a long time, LDAP import added trailing whitespace to real names when importing them (for example, `"John Smith"` would be imported as `"John Smith "`). We no longer do this, but should clean up the database since it seems highly unlikely that any user wants trailing whitespace in their real name.

Test Plan:
  $ ./bin/storage upgrade -f --apply phabricator:20130711.trimrealnames.php
  Applying patch 'phabricator:20130711.trimrealnames.php'...
  Trimming trailing whitespace from user real names...
  Trimming user 1 from 'Evan Priestley   ' to 'Evan Priestley'.
  User 4 is already trim.
  User 5 is already trim.
  User 6 is already trim.
  User 7 is already trim.
  User 8 is already trim.
  User 9 is already trim.
  User 10 is already trim.
  User 11 is already trim.
  User 12 is already trim.
  User 13 is already trim.
  User 14 is already trim.
  User 15 is already trim.
  User 21 is already trim.
  User 22 is already trim.
  User 26 is already trim.
  User 28 is already trim.
  User 29 is already trim.
  User 32 is already trim.
  User 33 is already trim.
  User 35 is already trim.
  User 39 is already trim.
  User 51 is already trim.
  User 53 is already trim.
  User 54 is already trim.
  User 55 is already trim.
  Done.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3069, T3516

Differential Revision: https://secure.phabricator.com/D6426
2013-07-11 10:10:06 -07:00
Bob Trahan
4fb3b27f1d Legalpad - add signature page
Summary: Fixes T3481. Sort of - this thing be very ugly. Also it assumes that you'll "always" want to sign terms. I was thinking in a future diff that should be optional as well as configurable, though it was unclear to me if either was worth pursuing...  Generally very hideous as the three main elements (PHUIDocument, AphrontErrorView, and AphrontForm with an AphrontFormInset) have never really played together before.

Test Plan: agreed to some test terms. noted UI displayed nicely. reloaded and noted UI told me I had signed it already. Went to different terms and filled them out wrong and got sensical errors.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3481

Differential Revision: https://secure.phabricator.com/D6399
2013-07-10 11:46:39 -07:00
Lauri-Henrik Jalonen
93e37e9060 Phabricator event timeline removed
Summary: Removed related files and references

Test Plan: Crossed my fingers

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2003

Differential Revision: https://secure.phabricator.com/D5744

Conflicts:
	src/__phutil_library_map__.php
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-07-09 18:07:42 -07:00
Bob Trahan
2d87bb29ac Legalpad - denormalize a field or two from DocumentBody to Document
Summary: this let's the list controller save a query. Fixes T3488. Note didn't bother denormalizing document body at all since I don't think we want to show a snippet.

Test Plan: viewed a list of legalpad documents - yay. viewed a legalpad document - yay. created a legalpad document - yay. edited a legalpad document - yay. edited with N authors - yay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3488

Differential Revision: https://secure.phabricator.com/D6382
2013-07-08 17:06:49 -07:00