1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-24 15:52:41 +01:00
Commit graph

386 commits

Author SHA1 Message Date
epriestley
ac19c55822 Formalize "manual" buildables in Harbormaster
Summary:
Ref T1049. Generally, it's useful to separate test/trial/manual runs from production/automatic runs.

For example, you don't want to email a bunch of people that the build is broken just because you messed something up when writing a new build plan. You'd rather try it first, then promote it into production once you have some good runs.

Similarly, test runs generally should not affect the outside world, etc. Finally, some build steps (like "wait for other buildables") may want to behave differently when run in production/automation than when run in a testing environment (where they should probably continue immediately).

So, formalize the distinction between automatic buildables (those created passively by the system in response to events) and manual buildables (those created explicitly by users). Add filtering, and stop the automated parts of the system from interacting with the manual parts (for example, we won't show manual results on revisions).

This also moves the "Apply Build Plan" to a third, new home: instead of the sidebar or Buildables, it's now on plans. I think this generally makes more sense given how things have developed. Broadly, this improves isolation of test environments.

Test Plan: Created some builds, browsed around, used filters, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7824
2013-12-26 10:40:43 -08:00
epriestley
c179ce6279 (Later...) Drop legacy Project transaction table
Summary: Ref T4010. I'll hold this for a bit, but we should eventually drop this table once the dust has settled.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7372
2013-12-19 07:03:51 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
James Rhodes
86ec4d6021 Implement policies in Phragment
Summary: This implements support for enforcing and setting policies in Phragment.

Test Plan: Set policies and ensured they were enforced successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7751
2013-12-13 14:42:12 +11:00
James Rhodes
8bb6e807f0 Implement snapshots in Phragment
Summary:
Ref T4212.  This implements snapshots in Phragment, which allows you to take a snapshot of a fragment at a given point in time, and download a ZIP of the snapshot as it was in this state.

There's also functionality for deleting and promoting snapshots.  You can promote a snapshot to either the latest version or any other snapshot of the fragment.

Test Plan: Clicked around, took some snapshots, promoted them to different points and deleted snapshots.  Also downloaded ZIPs of the snapshots and saw the right versions coming through for all the files downloaded.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205, T4212

Differential Revision: https://secure.phabricator.com/D7741
2013-12-09 08:24:50 +11:00
James Rhodes
25e7b7d53c Implement support for creating and updating fragments from ZIPs
Summary:
This implements support for creating and updating fragments from ZIP files.  It allows you to upload a ZIP via the Files application, create a fragment from it, and have it recursively imported into Phragment.  Updating that folder with another ZIP will recursively create, update and delete files as appropriate.

The logic for creating and updating fragments from files has also been centralized into the PhragmentFragment class.  Directories are also now supported; a directory fragment is simply a fragment that has no patches; thus a directory fragment can be converted to a file fragment by uploading a first patch for it.

Test Plan: Uploaded ZIP files through the interface and saw all of the fragments get created and updated as expected.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7729
2013-12-07 13:37:18 +11:00
James Rhodes
4c143ad3b2 Phragment v0
Summary: Ref T4205.  This is an initial implementation of Phragment.  You can create and browse fragments in the system (but you can't yet view a fragment's patches / history).

Test Plan: Clicked around and created fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7726
2013-12-07 12:43:49 +11:00
epriestley
9d474452f9 Make nonscalar field saves raise a more useful exception from LiskDAO
Summary:
If you do something like this:

  // Missing $user->getPHID()!
  $object->setUserPHID($user)->save();

...you get a very unhelpful exception:

  Expected a scalar or null for %s conversion. Query: %s

This doesn't give you any hints about what's wrong. Instead, provide a more useful exception:

  Unable to insert or update object of class DifferentialRevision, field 'title' has a nonscalar value.

Test Plan: {F87614}

Reviewers: hach-que, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7725
2013-12-06 15:47:40 -08:00
James Rhodes
79d153b85d Implement explicit build step ordering in Harbormaster
Summary: This implements support for explicitly marking the sequence of build steps.  Users can now drag and re-order build steps in plans, and artifact dependencies are re-calculated so that if you move "Run Command" before "Lease Host", the "Run Command" step has it's artifact setting cleared and thus the step becomes invalid.

Test Plan: Re-ordered build steps and observed dependencies being correctly recalculated.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7715
2013-12-06 14:12:15 +11:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
James Rhodes
53250d84df Introduce HarbormasterBuildTarget to snapshot build steps through a build
Summary: This implements build targets as outlined in D7582.  Build targets represent an instance of a build step particular to the build.  Logs and artifacts have been adjusted to attach to build targets instead of build / build step pairs.

Test Plan: Ran builds and clicked around the interface.  Everything seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7703
2013-12-05 12:01:12 +11:00
James Rhodes
ba16df0fed Restructure Drydock so that blueprints are instances in the DB
Summary:
//(this diff used to be about applying policies to blueprints)//

This restructures Drydock so that blueprints are instances in the DB, with an associated implementation class.  Thus resources now have a `blueprintPHID` instead of `blueprintClass` and DrydockBlueprint becomes a DAO.  The old DrydockBlueprint is renamed to DrydockBlueprintImplementation, and the DrydockBlueprint DAO has a `blueprintClass` column on it.

This now just implements CAN_VIEW and CAN_EDIT policies for blueprints, although they are probably not enforced in all of the places they could be.

Test Plan: Used the `create-resource` and `lease` commands.  Closed resources and leases in the UI.  Clicked around the new and old lists to make sure everything is still working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T2015

Differential Revision: https://secure.phabricator.com/D7638
2013-12-03 11:09:07 +11:00
epriestley
4b91c4f7ae Add UI for defining repository mirrors
Summary:
Ref T4038. This adds everything except the actual pushing part for mirrors.

This isn't the most beautiful or sophisticated UI, but I want get the authoritative repositories self-hosted and get users beta-ing hosting as soon as possible. We can do transactions, etc., later on.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7632
2013-11-22 15:23:50 -08:00
epriestley
51fb1ca16d Migrate repositories to use Passphrase for credential management
Summary: Fixes T4122. Ref T2230. Instead of storing credentials on each repository, store them in Passphrase. This allows easy creation/management of many repositories which share credentials.

Test Plan:
  - Upgraded repositories.
  - Created and edited repositories.
  - Pulled HTTP and SSH repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230, T4122

Differential Revision: https://secure.phabricator.com/D7629
2013-11-22 15:23:33 -08:00
epriestley
819f899013 Add an edge between Passphrase credentials and objects which use them
Summary: Ref T4122. Add an edge to keep track of where a credential is used, and show it in the UI.

Test Plan:
See "Used By":

{F84099}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7628
2013-11-22 15:23:23 -08:00
Bob Trahan
7b718bb033 Nuance - federate out the design of NuanceSource via NuanceSourceDefinition
Summary: ...and get the basic edit flow "working" for a new NuanceSourceDefinition - the Phabricator Form. ...and fix a dumb bug in the query class so when you redirect to the view page / try to edit an existing NuanceSource you don't fatal.

Test Plan: played around with the edit form and it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7585
2013-11-20 13:41:19 -08:00
epriestley
91d084624b Passphrase v0
Summary:
Ref T4122. Implements a credential management application for the uses described in T4122.

@chad, this needs an icon, HA HA HAHA HA BWW HA HA HA

bwahaha

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7608
2013-11-20 09:13:35 -08:00
epriestley
d9db1d61e0 Restore population of ownerOrdering to ManiphestTasks
Summary:
Ref T4110. This denormalized field used to power "Group By: Assigned" got dropped in the T2217 migration at some point.

Restore its population, and fix all the data in the database.

Test Plan: Ran migration, verified database came out reasonable-looking. Reassigned a task, verified database. Ran a "Group By: assigned" query.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4110

Differential Revision: https://secure.phabricator.com/D7602
2013-11-19 14:10:54 -08:00
epriestley
7f11e8d740 Improve handling of email verification and "activated" accounts
Summary:
Small step forward which improves existing stuff or lays groudwork for future stuff:

  - Currently, to check for email verification, we have to single-query the email address on every page. Instead, denoramlize it into the user object.
    - Migrate all the existing users.
    - When the user verifies an email, mark them as `isEmailVerified` if the email is their primary email.
    - Just make the checks look at the `isEmailVerified` field.
  - Add a new check, `isUserActivated()`, to cover email-verified plus disabled. Currently, a non-verified-but-not-disabled user could theoretically use Conduit over SSH, if anyone deployed it. Tighten that up.
  - Add an `isApproved` flag, which is always true for now. In a future diff, I want to add a default-on admin approval queue for new accounts, to prevent configuration mistakes. The way it will work is:
    - When the queue is enabled, registering users are created with `isApproved = false`.
    - Admins are sent an email, "[Phabricator] New User Approval (alincoln)", telling them that a new user is waiting for approval.
    - They go to the web UI and approve the user.
    - Manually-created accounts are auto-approved.
    - The email will have instructions for disabling the queue.

I think this queue will be helpful for new installs and give them peace of mind, and when you go to disable it we have a better opportunity to warn you about exactly what that means.

Generally, I want to improve the default safety of registration, since if you just blindly coast through the path of least resistance right now your install ends up pretty open, and realistically few installs are on VPNs.

Test Plan:
  - Ran migration, verified `isEmailVerified` populated correctly.
  - Created a new user, checked DB for verified (not verified).
  - Verified, checked DB (now verified).
  - Used Conduit, People, Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Differential Revision: https://secure.phabricator.com/D7572
2013-11-12 14:37:04 -08:00
James Rhodes
0ac1be7094 Implemented support for build logs
Summary:
Depends on D7519.

This implements support for build logs in Harbormaster.  This includes support for appending to a log from the "Run Remote Command" build step.

It also adds the ability to cancel builds.

Currently the build view page doesn't update the logs live; I'm sure this can be achieved with Javelin, but I don't have enough experience with Javelin to actually make it poll from updates to content in the background.

{F79151}

{F79153}

{F79150}

{F79152}

Test Plan:
Tested this by setting up SSH on a Windows machine and using a Remote Command configured with:

```
C:\Windows\system32\cmd.exe /C cd C:\Build && mkdir Build_${timestamp} && cd Build_${timestamp} && git clone --recursive https://github.com/hach-que/Tychaia.git && cd Tychaia && Protobuild.exe && C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe Tychaia.Windows.sln
```

and observed the output of the build stream from the Windows machine into Phabricator.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7521
2013-11-08 18:15:07 -08:00
Bob Trahan
66ae64f7bc Nuance - get some scaffolding up there
Summary:
I updated the wiki too - https://secure.phabricator.com/w/projects/pebkac/ - with what I am thinking right now. Rough plan here is

 - next diff:
  - implement editors and transactions
  - implement "web type" for contact source
   - /pebkac/item/new/ will be the entry point for this
  - implement "actions" on a contact
  - probably some "polish" on the scaffolding laid out here; like "create" permissions maybs
 - diffs after that:
  - implement "twitter" type for source
  - implement email reply handler stuff for item and source

Probs a great time to blast huge holes in all this stuff. :D

Test Plan: these pages load and arc lint doesn't complain

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran, chad

Differential Revision: https://secure.phabricator.com/D7465
2013-11-06 17:00:09 -08:00
epriestley
4f0f95f7b5 Assign PHIDs to all diffs
Summary:
Ref T1049. Ref T2222. `DifferentialDiff` does not currently have a PHID, but we need it for Harbormaster and ApplicationTransactions. See some discussion in D7501.

(I split the SQL into two sections so we can't fail in the middle. At some point, I'd like to do a pass on the migration stuff and get this happening automatically, and also simplify the PatchList.)

Test Plan:
  - Ran `bin/storage upgrade`.
  - Checked for valid PHIDs in the database.
  - Used `phid.query` to look up a diff by PHID.
  - Created a new diff and verified it got a PHID.

Reviewers: btrahan, hach-que

Reviewed By: btrahan

CC: aran, vrana

Maniphest Tasks: T2222, T1049

Differential Revision: https://secure.phabricator.com/D7513
2013-11-06 13:59:06 -08:00
James Rhodes
c514d34b94 Add build step implementation infrastructure and sleep build step.
Summary:
Depends on D7498.

This implements support for a "build step implementation".  Build steps have an associated class name (which makes the class in PHP) and a details field, which is serialized JSON (same as PhabricatorRepository).

This also implements a SleepBuildStepImplementation which just pauses the build for a specified period of seconds.

Test Plan:
Inserted a build step with `insert into harbormaster_buildstep (phid, buildPlanPHID, className, details, dateCreated, dateModified) values ('', 'PHID-HMCP-zkh5w6czfbfpk2gxwdeo', 'SleepBuildStepImplementation', '{"seconds":5}', NOW(), NOW());` (adjusting the build plan PHID as appropriate).

Started the daemon and applied the build plan to a buildable, and saw the daemon take a 5 second delay after creating `SleepBuildStepImplementation`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, chad

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7499
2013-11-05 13:34:44 -08:00
epriestley
0278b15ceb Implementation of VCS passwords against user.
Summary: This allows users to set their HTTP access passwords via Diffusion interface.

Test Plan: Clicked the "Set HTTP Access Password" link, set a password and saw it appear in the DB.

Reviewers: #blessed_reviewers, hach-que, btrahan

Reviewed By: hach-que

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7462
2013-11-01 08:34:11 -07:00
epriestley
3a39b01233 Add "RepositoryStatusMessage" and detailed information about initilization
Summary:
`RepositoryStatusMessage` is basically a key/value table associated with a repository that I'm using to let the daemons store the most recent event of a given type, so we can easily show it on the status dashboard. I think this will be a lot easier for users to figure out than digging through logfiles.

I'm also going to write the "this needs a pull" status here eventually, for reducing the time lapse between pushes and discovery.

  - Add storage for these messages.
  - Have the pull engine populate the INIT phase. I'll do the FETCH phase next.
  - Update the status readout to show all the various states.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7461
2013-10-30 16:04:19 -07:00
epriestley
9125095587 Distinguish between empty and unparsed commits in Diffusion
Summary:
Fixes T3416. Fixes T1733.

  - Adds a flag to the commit table showing whether or not we have parsed it.
  - The flag is set to `0` initially when the commit is discovered.
  - The flag is set to `1` when the changes are parsed.
  - The UI can now use the flag to distinguish between "empty commit" and "commit which we haven't imported changes for yet".
  - Simplify rendering code a little bit.
  - Fix an issue with the Message parser for empty commits.
  - There's a key on the flag so we can do `SELECT * FROM repository_commit WHERE repositoryID = %d AND importStatus = 0 LIMIT 1` soon, to determine if a repository is fully imported or not. This will let us improve the UI (Ref T776, Ref T3217).

Test Plan:
  - Ran `bin/storage upgrade -f`.
  - Created an empty commit.
  - Without the daemons running, ran `bin/repository pull GTEST` and `bin/repository discover GTEST`.
  - Viewed web UI to get the first screenshot ("Still Importing...").
  - Ran the message and change steps with `scripts/repository/reparse.php`.
  - Viewed web UI to get the second screenshot ("Empty Commit").

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776, T1733, T3416, T3217

Differential Revision: https://secure.phabricator.com/D7428
2013-10-29 15:32:41 -07:00
epriestley
86fe020a97 Add global "push" policy to Repositories
Summary: No editing or view yet, just adds the schema and a policy default. Part of D7391.

Test Plan: `bin/storage upgrade`

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7415
2013-10-29 15:32:40 -07:00
epriestley
b5a009337f Harbormaster v(-2)
Summary:
Ref T1049. I don't really want to sink too much time into this right now, but a seemingly reasonable architecture came to me in a dream. Here's a high-level overview of how things fit together:

  - **"Build"**: In Harbormaster, "build" means any process we want to run against a working copy. It might actually be building an executable, but it might also be running lint, running unit tests, generating documentation, generating symbols, running a deploy, setting up a sandcastle, etc.
  - `HarbormasterBuildable`: A "buildable" is some piece of code which build operations can run on. Generally, this is either a Differential diff or a Diffusion commit. The Buildable class just wraps those objects and provides a layer of abstraction. Currently, you can manually create a buildable from a commit. In the future, this will be done automatically.
  - `HarbormasterBuildStep`: A "build step" is an individual build operation, like "run lint", "run unit", "build docs", etc. The step defines how to perform the operation (for example, "run unit tests by executing 'arc unit'"). In this diff, this barely exists.
  - `HarbormasterBuildPlan`: This glues together build steps into groups or sequences. For example, you might want to "run unit", and then "deploy" if the tests pass. You can create a build plan which says "run step "unit tests", then run step "deploy" on success" or whatever. In the future, these will also contain triggers/conditions ("Automatically run this build plan against every commit") and probably be able to define failure actions ("If this plan fails, send someone an email"). Because build plans will run commands, only administrators can manage them.
  - `HarbormasterBuild`: This is the concrete result of running a `BuildPlan` against a `Buildable`. It tracks the build status and collects results, so you can see if the build is running/successful/failed. A `Buildable` may have several `Build`s, because you can execute more than one `BuildPlan` against it. For example, you might have a "documentation" build plan which you run continuously against HEAD, but a "unit" build plan which you want to run against every commit.
  - `HarbormasterBuildTarget`: This is the concrete result of running a `BuildStep` against a `Buildable`. These are children of `Build`. A step might be able to produce multiple targets, but generally this is something like "Unit Tests" or "Lint" and has an overall status, so you can see at a glance that unit tests were fine but lint had some issues.
  - `HarbormasterBuildItem`: An optional subitem for a target. For lint, this might be an individual file. For unit tests, an individual test. For normal builds, an executable. For deploys, a server. For documentation generation, there might just not be subitems.
  - `HarbormasterBuildLog`: Provides extra information, like command/execution transcripts. This is where stdout/stderr will get dumped, and general details and other messages.
  - `HarbormasterBuildArtifact`: Stores side effects or results from build steps. For example, something which builds a binary might put the binary in "Files" and then put its PHID here. Unit tests might put coverage information here. Generally, any build step which produces some high-level output object can use this table to record its existence.

This diff implements almost nothing and does nothing useful, but puts most of these object relationships in place. The two major things you can't easily do with these objects are:

  1) Run arbitrary cron jobs. Jenkins does this, but it feels tacked on and I don't know of anyone using it for that. We could create fake Buildables to get a similar effect, but if we need to do this I'd rather do it elsewhere in general. Build and cron/service/monitoring feel like pretty different problems to me.
  2) Run parameterized/matrix steps (maybe?). Bamboo has this plan/stage/task/job breakdown where a build step can generate a zillion actual jobs, like "build client on x86", "build server on x86", "build client on ARM", "build server on ARM", etc. We can sort of do this by having a Step map to multiple Targets, but I haven't really thought about it too much and it may end up being not-great. I'd guess we have like an 80% chance of getting a clean implementation if/when we get there. I suspect no one actually needs this, or when they do they'll just implement a custom Step and it can be parameterized at that level. I'm not too worried about this overall.

The major difference between this and Jenkins/Bamboo/TravisCI is that all three of those are **plan-centric**: the primary object in the system is a build plan, and the dashboard shows you all your build plans and the current status. I don't think this is the right model. One disadvantage is that you basically end up with top-level messaging that says "Trunk is broken", not "Trunk was broken by commit af32f392f". Harbormaster is **buildable-centric**: the primary object in the system is stuff you can run build operations against (commits/branches/revisions), and actual build plans are secondary. The main view will be "recent commits on this branch, and whether they're good or not" -- which I think is what's most important in a larger/more complex product -- not the pass/fail status of all jobs. This also makes it easier and more natural to integrate with Differential and Diffusion, which both care about the overall status of the commit/revision, not the current status of jobs.

Test Plan: Poked around, but this doesn't really do anything yet.

Reviewers: btrahan

Reviewed By: btrahan

CC: zeeg, chad, aran, seporaitis

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7368
2013-10-22 15:01:06 -07:00
epriestley
9b89e137cf Move Project transaction storage to modern tables
Summary:
Ref T4010. Projects have a weird proto-version of ApplicationTransactions which is very similar but not quite the same.

Move the storage to a modern format, but keep all the other code for now.

Test Plan: Migrated project transactions; edited projects.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7370
2013-10-22 13:49:28 -07:00
epriestley
d7a276346f Add a secret board view to Projects
Summary:
Ref T1344. This is //very// rough. Some UI issues:

  - Empty states for the board and columns are junky.
  - Column widths are crazy. I think we need to set them to fixed-width, since we may have an arbitrarily large number of columns?
  - I don't think we have the header UI elements in M10 yet and that mock is pretty old, so I sort of very roughly approximated it.
  - What should we do when you click a task title? Popping the whole task in a dialog is possible but needs a bunch of work to actually work. Might need to build "sheets" or something.
  - Icons are slightly clipped for some reason.
  - All the backend stuff is totally faked.

Generally, my plan is just to use these to implement all of T390. Specifically:

  - "Kanban" projects will have "Backlog" on the left. You'll drag them toward the right as you make progress.
  - "Milestone" projects will have "No Milestone" on the left, then "Milestone 9", "Milestone 8", etc.
  - "Sprint" projects will have "Backlog" on the left, then "Sprint 31", "Sprint 30", etc.

So all of these things end up being pretty much exactly the same, with some minor text changes and new columns showing up on the left vs the right or whatever.

Test Plan: See screenshot.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: chad, aran, sascha-egerer

Maniphest Tasks: T1344

Differential Revision: https://secure.phabricator.com/D7374
2013-10-21 21:11:36 -07:00
epriestley
cf1c06e157 Add custom field storage to Projects
Summary: Ref T4010. Adds storage and indexes for custom fields. These tables are the same as people/maniphest/differential.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7369
2013-10-21 17:01:13 -07:00
epriestley
f010730e49 Migrate all Differential inline comments to ApplicationTransactions
Summary:
Ref T2222. This implements step (1) described there, which is moving over all the inline comments.

The old and new tables are simliar. The only real trick here is that `transactionPHID` and `legacyCommentID` mean roughly the same thing (`null` if the inline is a draft, non-null if it has been submitted) but we don't have real `transactionPHID`s yet. We just make some up -- we'll backfill them later.

Two risks here:

  - I need to take a second look at the keys on this table. I think we need to tweak them a bit, and it will be less disruptive to do that before this migration than after.
  - This will take a while for Facebook, and other large installs with tens of thousands of revisions. I'll communicate this.

I'm otherwise pretty satisfied with this, seems to work well and is pretty low risk / non-disruptive.

Test Plan:
  - Before migrating, then after migrating:
    - Made a bunch of inlines (drafts, submitted).
    - Edited and deleted inlines.
    - Verified inlines showed up in preview.
    - Verified that inlines aren't indexed when they're drafts (`bin/search index D935`).
    - Verified that inlines ARE indexed when they're not drafts.
    - Verified that drafts inlines make revisions appear as "with draft" in the revision list.
  - Made left, right, and draft inlines.
  - Migrated (`bin/storage upgrade`).
  - Verified that my inlines from before the migration still showed up.
  - (Repeated all the stuff above.)
  - Manually inspected the inline comment table.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7139
2013-10-19 05:03:25 -07:00
epriestley
460051c1d8 Drop maniphest_savedquery table
Summary: This data was migrated by D6977 and is now obsolete. I'll hold this patch for a week or two in case we get reports of migration errors.

Test Plan: Ran storage upgrade, saw the table vanish. Grepped for references to the table.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6997
2013-10-19 04:56:17 -07:00
epriestley
3410cbd53e Add application and object level policy controls to Countdown
Summary: Ref T603. Give countdowns proper UI-level policy controls, and an application-level default policy. Put policy information in the header.

Test Plan:
  - Adjusted default policy.
  - Created new countdowns.
  - Edited countdowns.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7322
2013-10-16 10:36:08 -07:00
epriestley
073cb0e78c Make PhabricatorPolicyInterface require a getPHID() method
Summary:
Ref T603. This cleans up an existing callsite in the policy filter, and opens up some stuff in the future.

Some policy objects don't have real PHIDs:

  PhabricatorTokenGiven
  PhabricatorSavedQuery
  PhabricatorNamedQuery
  PhrequentUserTime
  PhabricatorFlag
  PhabricatorDaemonLog
  PhabricatorConduitMethodCallLog
  ConduitAPIMethod
  PhabricatorChatLogEvent
  PhabricatorChatLogChannel

Although it would be reasonable to add real PHIDs to some of these (like `ChatLogChannel`), it probably doesn't make much sense for others (`DaemonLog`, `MethodCallLog`). Just let them return `null`.

Also remove some duplicate `$id` and `$phid` properties. These are declared on `PhabricatorLiskDAO` and do not need to be redeclared.

Test Plan: Ran the `testEverythingImplemented` unit test, which verifies that all classes conform to the interface.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7306
2013-10-14 14:35:47 -07:00
epriestley
5899ae08b3 Add storage for custom policies
Summary: Ref T603. Allows custom policies to be saved. No integration with policy controls yet.

Test Plan:
  mysql> select * from policy where id = 3\G
  *************************** 1. row ***************************
             id: 3
           phid: PHID-PLCY-e4v2fnbyuibi4supl5tn
          rules: [{"action":"allow","rule":"PhabricatorPolicyRuleAdministrators","value":null},{"action":"allow","rule":"PhabricatorPolicyRuleProjects","value":["PHID-PROJ-cwovm5gn2ilubjehcdgd"]},{"action":"allow","rule":"PhabricatorPolicyRuleLunarPhase","value":"new"}]
  defaultAction: deny
    dateCreated: 1381437466
   dateModified: 1381437466
  1 row in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7282
2013-10-10 16:09:51 -07:00
epriestley
953ff197bf Allow Herald rules to be disabled, instead of deleted
Summary:
Ref T603. Ref T1279. Further improves transaction and policy support for Herald.

  - Instead of deleting rules (which wipes out history and can't be undone) allow them to be disabled.
  - Track disables with transactions.
  - Gate disables with policy controls.
  - Show policy and status information in the headers.
  - Show transaction history on rule detail screens.
  - Remove the delete controller.
  - Support disabled queries in the ApplicationSearch.

Test Plan:
  - Enabled and disabled rules.
  - Searched for enabled/disabled rules.
  - Verified disabled rules don't activate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279, T603

Differential Revision: https://secure.phabricator.com/D7247
2013-10-06 17:10:29 -07:00
epriestley
bd8071bb33 Add a (dst, type, src) key to Differential's edge table
Summary:
Ref T1279. This came to me in a dream.

The existing `differential_relationship` table has an `(objectPHID, type)` column, which theoretically is useful for queries like "revisions with X as a reviewer". In practice, I'm not sure it gets used much, but I can get it to show up in at least some query plans.

Add a similar index to the `edge` table. This sequences //before// D7227, which actually migrates the data.

Test Plan:
  - Ran `bin/storage upgrade`.
  - EXPLAIN'd a bunch of queries against different versions of the schema, this seemed helpful overall.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7232

Conflicts:
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-10-05 13:53:43 -07:00
epriestley
6bd2251fa2 Possibly fix an issue with storage upgrade, see <https://github.com/facebook/phabricator/issues/399> 2013-10-02 07:44:19 -07:00
epriestley
98bf001a58 Add viewPolicy and attachedToObjectPHID to PhabricatorFile
Summary:
Ref T603. Principally, I want to implement the rule "when you upload a file to an object, users must be able to see the object in order to see the file", since I think this is strongly in line with user expectation. For example, if you attach a file to a Conpherence, it should only be visible to members of that thread.

This adds storage for policies, but doesn't do anything interesting with it yet.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7175
2013-10-01 08:45:18 -07:00
epriestley
7091dad606 Allow macros to have associated audio and audio behaviors
Summary: Ref T3887. Implements storage and editors, but not the actual audio part.

Test Plan: Edited audio, audio behaviors of macros. Transactions and email looked good. Hit error cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3887

Differential Revision: https://secure.phabricator.com/D7159
2013-09-27 16:01:37 -07:00
epriestley
3cb67480e0 Adjust keys for new Differential inline comment table
Summary:
Ref T2222. This sequences //before// D7139 and sorts out keys on the table. In particular:

  - There was a fairly silly `draft` key modeled after Pholio; drop it.
  - Add a `revisionPHID` key. This is queried mostly-transitionally on the revision view screen.
  - Add a `changesetID` key. This is queried by a bunch of interfaces that want more surgical results than `revisionPHID` provides.
  - Add an `authorPHID, transactionPHID` key. This is queried on the list interface to find pending drafts.
  - Add a `legacy` key. This is queried by the feed publisher.

Test Plan: Used the query analyzer to hit all (I think?) of the pages, saw keyed queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7140
2013-09-26 13:48:36 -07:00
epriestley
5677cd23bd Add storage and classes for CustomField in Differential
Summary: Ref T3886. Adds the storage, indexes, and storage classes for modernizing Differential custom fields.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886

Differential Revision: https://secure.phabricator.com/D7138
2013-09-26 12:37:28 -07:00
epriestley
c458517cb4 Add viewPolicy, editPolicy, repositoryPHID columns to DifferentialRevision
Summary: Ref T603. Paves the way for policy controls.

Test Plan: Ran storage upgrade, bumbled around in Differential.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7133
2013-09-26 12:36:30 -07:00
epriestley
225a38c7d3 Add viewPolicy, editPolicy storage to tasks
Summary: Ref T603. Adds storage for custom policies.

Test Plan: Ran storage upgrade. Created and edited tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7118
2013-09-25 11:18:47 -07:00
epriestley
6e29e809b4 Fix "edit" for Maniphest comments
Summary:
Ref T2217. Fixes T3876. We incorrectly have a unique key on `(authorPHID, transactionPHID)`, which prevents saving multiple versions of a comment.

I'm not entirely sure why this exists. I think it came from Pholio (where it works for inlines, because it has an additional component, but maybe should be adjusted) but we might need to wipe it out of more apps too.

Test Plan: Edited a comment in Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217, T3876

Differential Revision: https://secure.phabricator.com/D7102
2013-09-24 11:10:31 -07:00
epriestley
1d7cbe202f Rename "transactionpro" table to "transaction"
Summary: Ref T2217. Cleans up the table names. Moves old data to `maniphest_transaction_legacy`. We'll drop that eventually once it's more clear that I didn't break the world.

Test Plan: Did reads/writes to/from these tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7094
2013-09-24 10:49:16 -07:00
epriestley
7abe9dc4c0 Migrate all Maniphest transaction data to new storage
Summary:
Ref T2217. This is the risky, hard part; everything after this should be smooth sailing. This is //mostly// clean, except:

  - The old format would opportunistically combine a comment with some other transaction type if it could. We no longer do that, so:
    - When migrating, "edit" + "comment" transactions need to be split in two.
    - When editing now, we should no longer combine these transaction types.
    - These changes are pretty straightforward and low-impact.
  - This migration promotes "auxiliary field" data to the new CustomField/StandardField format, so that's not a straight migration either. The formats are very similar, though.

Broadly, this takes the same attack that the auth migration did: proxy all the code through to the new storage. `ManiphestTransaction` is now just an API on top of `ManiphestTransactionPro`, which is the new storage format. The two formats are very similar, so this was mostly a straight copy from one table to the other.

Test Plan:
  - Without performing the migration, made a bunch of edits and comments on tasks and verified the new code works correctly.
  - Droped the test data and performed the migration.
  - Looked at the resulting data for obvious discrepancies.
  - Looked at a bunch of tasks and their transaction history.
  - Used Conduit to pull transaction data.
  - Edited task description and clicked "View Details" on transaction.
  - Used batch editor.
  - Made a bunch more edits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7068
2013-09-23 14:25:28 -07:00
epriestley
4c2bd2ca8e Add storage for new Maniphest transactions
Summary: Ref T2217. Add the tables and comment class for the new stuff. Not used yet.

Test Plan: Ran storage upgrade, browsed Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7066
2013-09-23 14:24:58 -07:00
epriestley
b8cb6ddaa5 Add some keys and policy fields to repositories
Summary:
  - Add some TODO'd keys.
  - Add policy fields.

Test Plan: Viewed repositories; created a new repository and verified it got the right default policy settings.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7056
2013-09-21 16:23:01 -07:00
epriestley
0558d53273 Convert maniphest to use standard fields
Summary: Ref T3794. Drop auxiliary field, use standard field.

Test Plan: Performed migration, field seemed to survive it intact. Edited and viewed tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3794

Differential Revision: https://secure.phabricator.com/D7036
2013-09-19 11:56:15 -07:00
epriestley
bd40e74400 Migrate auxiliary field storage to common field storage
Summary: Ref T418. Moves data from the Maniphest-specific table to the general one. This patch is a bit gross, but mostly about getting the reads and writes aimed correctly. Future patches will clean things up.

Test Plan: Migrated data across formats. Verified it survied the migration. Viewed and edited tasks' custom fields.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6999
2013-09-16 16:02:06 -07:00
epriestley
b0e145f2fe Provide generalized custom field storage and custom field indexes in Maniphest
Summary: Ref T418. Depends on D6992. This adds index and value storage for Maniphest custom fields.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6995
2013-09-16 14:06:41 -07:00
epriestley
c8574cf6fd Integrate ApplicationSearch with CustomField
Summary:
Ref T2625. Ref T3794. Ref T418. Ref T1703.

This is a more general version of D5278. It expands CustomField support to include real integration with ApplicationSearch.

Broadly, custom fields may elect to:

  - build indicies when objects are updated;
  - populate ApplicationSearch forms with new controls;
  - read inputs entered into those controls out of the request; and
  - apply constraints to search queries.

Some utility/helper stuff is provided to make this easier. This part could be cleaner, but seems reasonable for a first cut. In particular, the Query and SearchEngine must manually call all the hooks right now instead of everything happening magically. I think that's fine for the moment; they're pretty easy to get right.

Test Plan:
I added a new searchable "Company" field to People:

{F58229}

This also cleaned up the disable/reorder view a little bit:

{F58230}

As it did before, this field appears on the edit screen:

{F58231}

However, because it has `search`, it also appears on the search screen:

{F58232}

When queried, it returns the expected results:

{F58233}

And the actually good bit of all this is that the query can take advantage of indexes:

  mysql> explain SELECT * FROM `user` user JOIN `user_customfieldstringindex` `appsearch_0` ON `appsearch_0`.objectPHID = user.phid AND `appsearch_0`.indexKey = 'mk3Ndy476ge6' AND `appsearch_0`.indexValue IN ('phacility') ORDER BY user.id DESC LIMIT 101;
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  | id | select_type | table       | type   | possible_keys     | key      | key_len | ref                                      | rows | Extra                                        |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  |  1 | SIMPLE      | appsearch_0 | ref    | key_join,key_find | key_find | 232     | const,const                              |    1 | Using where; Using temporary; Using filesort |
  |  1 | SIMPLE      | user        | eq_ref | phid              | phid     | 194     | phabricator2_user.appsearch_0.objectPHID |    1 |                                              |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  2 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418, T1703, T2625, T3794

Differential Revision: https://secure.phabricator.com/D6992
2013-09-16 13:44:34 -07:00
epriestley
9864d562dd Migrate old Maniphest custom queries to new query infrastructure
Summary: Ref T2625. EVERYONE LOVES MIGRATIONS!!!

Test Plan:
  - Created and migrated a query with every field, verified results were preserved.
  - Created and migrated a query using "noproject" and "upforgrabs" magic, verified results were preserved.

Here's the pre-migration "everything" query:

{F58110}

Here it is after migration:

{F58111}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6977
2013-09-13 11:49:45 -07:00
Bob Trahan
ea0dc5625d Purge loadRelativeEdges
Summary:
Fixes T3821. Maybe. The existing code seemed to have a bug and actually return the //commit phid//. Judging by the function name this is not intended.

Also, sorry to step on toes here -- I thought no one was assigned and was curious about loadRelativeEdges and here we are...

Test Plan: lots of logic here as I have no idea how to use Releeph.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3821

Differential Revision: https://secure.phabricator.com/D6967
2013-09-13 11:40:52 -07:00
epriestley
e50eccf109 Provide and populate an object name index for Maniphest
Summary: See discussion in D6955. This provides a table we can JOIN against to (effectively) "ORDER BY project name", populates it intially, and keeps it up to date as projects are edited.

Test Plan:
  - Ran storage upgrade, verified projects populated into the table.
  - Edited a project, verified its entry updated.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6957
2013-09-12 13:06:44 -07:00
epriestley
df86f87289 Add a "dateCreated" key to Maniphest
Summary: Depends on D6952. Unpunts there since I'm rolling into a swamp full of schema changes.

Test Plan: Issued date-constrained query and saw key as a candidate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6954
2013-09-12 13:04:31 -07:00
epriestley
8f8c61be31 Remove legacy "touched" table and indexing
Summary: Noticed this in the schema. "Touches" were an idea that never really got off the ground, as we built out more/better notification channels instead. Essentially, they recorded any object you'd ever interacted with. Maybe this will be useful some day, but for now it does nothing and can't be interacted with. Nuke it.

Test Plan: `grep`, loaded Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6953
2013-09-12 13:04:09 -07:00
epriestley
da6cdc6bd0 Fix issue with loadRelativeEdges()
Summary: I'm going to just delete all this code at some point, but fixing it now means piles of single gets, so unbreak it first. I'll file something.

Test Plan: Releeph is less fataley.

Reviewers: andrewjcg, btrahan

Reviewed By: andrewjcg

CC: aran

Differential Revision: https://secure.phabricator.com/D6945
2013-09-11 17:24:02 -07:00
Bob Trahan
228496cdbe File - add transactions and editor
Summary: this ends up being a little weird since you can't actually edit files. Also, since we create files all sorts of ways, sometimes without even having a user, we don't  bother logging transactions for those events. Fixes T3651. Turns out this work is important for T3612, which is a priority of mine to help get Pholio out the door.

Test Plan: left a comment on a file. it worked! use bin/mail to verify mail content looked correct.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran, wez

Maniphest Tasks: T3651, T3612

Differential Revision: https://secure.phabricator.com/D6789
2013-09-05 13:11:02 -07:00
Gareth Evans
fcba0c74d9 Replace all "attach first..." exceptions with assertAttached()
Summary:
Ref T3599
Go through everything, grep a bit, replace some bits.

Test Plan: Navigate around a bit

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3599

Differential Revision: https://secure.phabricator.com/D6871
2013-09-03 06:02:14 -07:00
epriestley
41ac06959e Generate some amount of PHP class documentation
Summary:
Ref T988. This brings the class/interface atomizer over. A lot of parts of this are still varying degrees of very-rough, but most of the data ends up in approximatley the right place.

ALSO: PROGRESS BARS

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6817
2013-08-28 09:54:39 -07:00
epriestley
caf4a086d4 Add missing $port property
Summary: See rPd27e7c52b21865563bd56f37d5d422515aa40bb7.

Test Plan: eyeballed it

Reviewers: wez

Reviewed By: wez

CC: aran

Differential Revision: https://secure.phabricator.com/D6818
2013-08-26 17:14:57 -07:00
Wez Furlong
ec8162bcfc fix broken DatabaseConfigurationProvider interface
Summary: broken by rPd27e7c52b21865563bd56f37d5d422515aa40bb7

Test Plan: sigh

Reviewers: epriestley, edward, levijackson

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6816
2013-08-26 17:04:56 -07:00
epriestley
49e0aa203c Add storage for Releeph project and branch transactions
Summary: Ref T3663. Does what it says on the tin.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3663

Differential Revision: https://secure.phabricator.com/D6778
2013-08-21 12:31:51 -07:00
epriestley
ca0115b361 Support configuration-driven custom fields
Summary:
Ref T1702. Ref T3718. There are a couple of things going on here:

**PhabricatorCustomFieldList**: I added `PhabricatorCustomFieldList`, which is just a convenience class for dealing with lists of fields. Often, current field code does something like this inline in a Controller:

  foreach ($fields as $field) {
    // do some junk
  }

Often, that junk has some slightly subtle implications. Move all of it to `$list->doSomeJunk()` methods (like `appendFieldsToForm()`, `loadFieldsFromStorage()`) to reduce code duplication and prevent errors. This additionally moves an existing list-convenience method there, out of `PhabricatorPropertyListView`.

**PhabricatorUserConfiguredCustomFieldStorage**: Adds `PhabricatorUserConfiguredCustomFieldStorage` for storing custom field data (like "ICQ Handle", "Phone Number", "Desk", "Favorite Flower", etc).

**Configuration-Driven Custom Fields**: Previously, I was thinking about doing these with interfaces, but as I thought about it more I started to dislike that approach. Instead, I built proxies into `PhabricatorCustomField`. Basically, this means that fields (like a custom, configuration-driven "Favorite Flower" field) can just use some other Field to actually provide their implementation (like a "standard" field which knows how to render text areas). The previous approach would have involed subclasssing the "standard" field and implementing an interface, but that would mean that every application would have at least two "base" fields and generally just seemed bleh as I worked through it.

The cost of this approach is that we need a bunch of `proxy` junk in the base class, but that's a one-time cost and I think it simplifies all the implementations and makes them a lot less magical (e.g., all of the custom fields now extend the right base field classes).

**Fixed Some Bugs**: Some of this code hadn't really been run yet and had minor bugs.

Test Plan:
{F54240}
{F54241}
{F54242}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1702, T1703, T3718

Differential Revision: https://secure.phabricator.com/D6749
2013-08-14 12:33:53 -07:00
epriestley
9f41032693 Remove "cut point commit identifier" from Releeph
Summary: Ref T3656. Releeph denormalizes branch cut point identifiers into Branch objects, but this information isn't useful or used for sorting, filtering, or enforcing unique constraints. Instead, derive it via noramlized pathways from the `cutPointCommitPHID`.

Test Plan: Ran storage upgrade. Ran `releephwork.getbranch` and `releeph.getbranches`. Grepped for `cutPointCommitIdentifier`.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3656

Differential Revision: https://secure.phabricator.com/D6636
2013-08-14 09:01:38 -07:00
epriestley
a84cc777c8 Remove "Project ID" from Releeph Projects
Summary:
Fixes T3660. Releeph Projects currently have an unused one-to-one mapping to Phabricator projects. This isn't consistent with other applications and has no integrations or uses. Get rid of it.

NOTE: Waiting for signoff from @legneato on T3660 before pulling the trigger here.

Test Plan: Created and edited Releeph projects. Grepped for references to project ID; there are a dozen or so but they're all either Releeph projects or Arcanist projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3660

Differential Revision: https://secure.phabricator.com/D6635
2013-08-14 09:00:56 -07:00
epriestley
296818e129 Remove ReleephProject->repositoryID writes
Summary: Ref T3655. Depends on D6633. This removes the writes and the column.

Test Plan: Created a project, edited a project. Verified the table doesn't have any keys including this column.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3655

Differential Revision: https://secure.phabricator.com/D6634
2013-08-14 09:00:25 -07:00
epriestley
b1c4a258c9 Add ApplicationTransactions to Herald
Summary: Ref T2769. I'm planning to keep this pretty simple, but we have this ad-hoc edit log for rules already and some other mess that we can clean up.

Test Plan: No effect yet; see future changes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6654
2013-08-07 18:03:49 -07:00
epriestley
2820fdc89b Add PHIDs to Herald Rules
Summary: Ref T2769. Precursor to various Herald-related modernizations.

Test Plan: Ran migration; loaded Herald via web.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6648
2013-08-07 18:03:37 -07:00
Bob Trahan
0e6b5073cd Paste - add support for email replies and subscribers
Summary: Email replies and subscribers seem to go hand in hand so deploy both at once.

Test Plan: played around with bin/mail. Verified replies posted comments on the paste.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3650

Differential Revision: https://secure.phabricator.com/D6682
2013-08-05 17:11:46 -07:00
Bob Trahan
37a5c4b11a Paste - add transactions
Summary: Ref T3650. This adds a create transaction, transactions for metadata (title, langauge, view policy), and comments. Editor is used on all create /edit paths.

Test Plan: made some pastes via web and email - yay. edited pastes - yay. verified txns showed up on pastes and in feed correctly.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3516, T3650

Differential Revision: https://secure.phabricator.com/D6645
2013-08-02 12:56:58 -07:00
epriestley
71841e262a Migrate Ponder comments to ApplicationTransactions
Summary: Ref T3373. Bring the storage across.

Test Plan: {F52067}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3373

Differential Revision: https://secure.phabricator.com/D6606
2013-07-29 12:04:05 -07:00
epriestley
6172f2bbd1 Require answers' authors to be unique in Ponder
Summary: Ref T3578. I forget if this was an explicit decision or not, but we currently let the same user answer questions multiple times. I think this probably causes more confusion than it provides freedom. In conjunction with other UI issues (commenting being weird, notably), we're seeing some use of answers to comment, which is undesirable. Require each answer's author to be unique. Merge existing nonunique authors' answers.

Test Plan: {F52062}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3578

Differential Revision: https://secure.phabricator.com/D6605
2013-07-29 12:04:04 -07:00
epriestley
8f0c836063 Add question and answer transactions to Ponder
Summary: Ref T3373.

Test Plan:
Ran this script, saw valid inserts:

  <?php

  $unguarded = AphrontWriteGuard::beginScopedUnguardedWrites();

  id(new PonderQuestionTransaction())
    ->setAuthorPHID('herp')
    ->setObjectPHID('derp')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setTransactionType('beep')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderAnswerTransaction())
    ->setAuthorPHID('herp')
    ->setObjectPHID('derp')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setTransactionType('beep')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderQuestionTransactionComment())
    ->setCommentVersion(1)
    ->setAuthorPHID('bloop')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setContent('blip')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderAnswerTransactionComment())
    ->setCommentVersion(1)
    ->setAuthorPHID('bloop')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setContent('blip')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  unset($unguarded);

  echo "OK!\n";

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3373

Differential Revision: https://secure.phabricator.com/D6584
2013-07-28 15:08:36 -07:00
Gareth Evans
70a5ec600d Allowing Closing of questions
Summary:
Fixes T3579

As a basic overview, this enables the author of a question to open/close a question.

Other bits;

- Add "Open" filter to the builtin queries
- Add "Status" to search form
- Refactor ponder constants
- Add coloured bars for different question statuses

Test Plan:
- Open/Close questions
- Search for some bits
- Use filters

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3579

Differential Revision: https://secure.phabricator.com/D6590

Conflicts:
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-07-27 18:41:03 -07:00
Bob Trahan
f75f3a0c3b Pholio - add concept of replacing images and primitive history view
Summary:
Now you can actually replace an image! Ref T3572. This ended up needing a wee bit of infrastructure to work...

 - add replace image transaction to pholio
 - add replacesImagePHID to PholioImage
 - tweaks to editor to properly update images with respect to replacement
   - add edges to track replacement
 - expose getNodes on graph query infrastructure to query the entire graph of who replaced who
 - move pholio image to new phid infrastructure

Still TODO - the history view should get chopped out a bit from the current view - no more inline comments / generally less functionality plus maybe a tweak or two to make this more sensical.

Test Plan: replaced images and played with history controller a little. works okay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3572

Differential Revision: https://secure.phabricator.com/D6560
2013-07-25 16:59:25 -07:00
epriestley
6750a48951 Surface task queue temporary failure rate in Daemon console
Summary: Fixes T3557. One thing which made T3557 kind of a mess was the lack of information about progress through temporary failures. Add a column which records a task's last failure time, and surface it in the console.

Test Plan: {F51277}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3557

Differential Revision: https://secure.phabricator.com/D6550
2013-07-23 16:58:22 -07:00
epriestley
d2e5afb095 Use application PHIDs in Releeph, plus more
Summary:
Ref T2715. Ref T3551. Ref T603. This does a few things, but they're all sort of small:

  - We commonly use a `getX()` / `attachX()` pattern, but have very similar code in the `getX()` method every time. Provide a convenience method to make this pattern easier to write.
  - We use `willFilterPage()` in many queries, but it currently is called with zero or more results. This means we have a lot of "if no results, return nothing" boilerplate. Make it call only for one or more results.
  - Implement `PhabricatorPolicyInterface` on `ReleephBranch`. A branch has the same policy as its project.
  - Implement `ReleephBranchQuery`.
  - Move the branch PHID type to application PHID infrastructure.

Test Plan: Browsed Releeph. Used `phid.query` to query branch PHIDs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603, T2715, T3551

Differential Revision: https://secure.phabricator.com/D6512
2013-07-22 12:17:32 -07:00
epriestley
83c29da594 Mark dead databases as "dead" and don't dump, probe, or list them
Summary:
When we delete a database, we still need to create it in the patch sequence so that installs can upgrade correctly. However, we shouldn't try to dump, probe, or list it. Mark deleted databases (of which there is only one) as "dead" and don't dump them.

A specific problem this fixes is `bin/storage dump` failing when trying to dump `phabricator_timeline`, which no longer exists.

Test Plan: Ran `bin/storage dump`, `list`, `probe`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6496
2013-07-19 11:07:29 -07:00
Bob Trahan
001f76cbaa Projects - tighten up a few things
Summary:
Fixes T2675, T2676.

 - when the last person leaves a project it is archived.
 - a script to archive all memberless projects
 - better feed stories for the various policy edits you can do
 - phriction pages are also moved as you rename projects

Test Plan: edited some projects and noted reasonable feed stories. ran script against test data and it worked! left a last man standing project and it archived. renamed a project to "a" then "b" then "a" (etc) and it worked including phrictiondocument moves

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2676, T2675

Differential Revision: https://secure.phabricator.com/D6478
2013-07-17 16:43:37 -07:00
Bob Trahan
df264d8548 Pholio - support editing images - fixes T3489
Summary:
Nice title. We add three new transactions - IMAGE_FILE, IMAGE_NAME, and IMAGE_DESCRIPTION. The first is a bit like subscribers as it is a list of file phids. The latter have values of the form ($file_phid => $data), where $data is $name or $description respectively. This is because we need to collate transactions based on $file_phid...

Overall, this uses the _underyling files_ and not the "PholioImage" to determine if things are unique or not. That said, simply mark PholioImages as obsolete so inline comments about no-longer applicable PholioImages don't break.

Does a reasonable job implementing the mock. Note you can't "update" an image at this time, though you can delete and add at will.

Test Plan: played with pholio a ton.

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T3489

Differential Revision: https://secure.phabricator.com/D6441
2013-07-16 13:31:20 -07:00
epriestley
d2f8c5b5e7 Make slowvotes subscribable
Summary: Subscribing doesn't do anything yet, but you can subscribe!

Test Plan: {F50169}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6456
2013-07-16 10:30:28 -07:00
epriestley
ea52bcbcd6 Migrate Slowvote comments to ApplicationTransactions
Summary:
Move comments from the old table to ApplicationTransactions. Patch dances around which objects it uses since I intend to delete the comment table.

NOTE: This temporarily disables comment writes. I'll restore them shortly.

Test Plan: {F50166}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6454
2013-07-16 10:30:21 -07:00
epriestley
ef1bedef02 Add ApplicaionTransactions and a mutable view policy to Slowvote
Summary: Schema changes to modernize this app.

Test Plan: Ran schema changes, created a new slowvote. No real effects yet.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6453
2013-07-16 10:30:14 -07:00
Levi Jackson
d27e7c52b2 Add explicit mysql.port configuration
See: https://github.com/facebook/phabricator/pull/356

Reviewed by: epriestley
2013-07-14 16:06:23 -07:00
epriestley
237ca095a6 Trim trailing whitespace from user realnames
Summary: Fixes T3069. Ref T3516. For a long time, LDAP import added trailing whitespace to real names when importing them (for example, `"John Smith"` would be imported as `"John Smith "`). We no longer do this, but should clean up the database since it seems highly unlikely that any user wants trailing whitespace in their real name.

Test Plan:
  $ ./bin/storage upgrade -f --apply phabricator:20130711.trimrealnames.php
  Applying patch 'phabricator:20130711.trimrealnames.php'...
  Trimming trailing whitespace from user real names...
  Trimming user 1 from 'Evan Priestley   ' to 'Evan Priestley'.
  User 4 is already trim.
  User 5 is already trim.
  User 6 is already trim.
  User 7 is already trim.
  User 8 is already trim.
  User 9 is already trim.
  User 10 is already trim.
  User 11 is already trim.
  User 12 is already trim.
  User 13 is already trim.
  User 14 is already trim.
  User 15 is already trim.
  User 21 is already trim.
  User 22 is already trim.
  User 26 is already trim.
  User 28 is already trim.
  User 29 is already trim.
  User 32 is already trim.
  User 33 is already trim.
  User 35 is already trim.
  User 39 is already trim.
  User 51 is already trim.
  User 53 is already trim.
  User 54 is already trim.
  User 55 is already trim.
  Done.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3069, T3516

Differential Revision: https://secure.phabricator.com/D6426
2013-07-11 10:10:06 -07:00
Bob Trahan
4fb3b27f1d Legalpad - add signature page
Summary: Fixes T3481. Sort of - this thing be very ugly. Also it assumes that you'll "always" want to sign terms. I was thinking in a future diff that should be optional as well as configurable, though it was unclear to me if either was worth pursuing...  Generally very hideous as the three main elements (PHUIDocument, AphrontErrorView, and AphrontForm with an AphrontFormInset) have never really played together before.

Test Plan: agreed to some test terms. noted UI displayed nicely. reloaded and noted UI told me I had signed it already. Went to different terms and filled them out wrong and got sensical errors.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3481

Differential Revision: https://secure.phabricator.com/D6399
2013-07-10 11:46:39 -07:00
Lauri-Henrik Jalonen
93e37e9060 Phabricator event timeline removed
Summary: Removed related files and references

Test Plan: Crossed my fingers

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2003

Differential Revision: https://secure.phabricator.com/D5744

Conflicts:
	src/__phutil_library_map__.php
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-07-09 18:07:42 -07:00
Bob Trahan
2d87bb29ac Legalpad - denormalize a field or two from DocumentBody to Document
Summary: this let's the list controller save a query. Fixes T3488. Note didn't bother denormalizing document body at all since I don't think we want to show a snippet.

Test Plan: viewed a list of legalpad documents - yay. viewed a legalpad document - yay. created a legalpad document - yay. edited a legalpad document - yay. edited with N authors - yay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3488

Differential Revision: https://secure.phabricator.com/D6382
2013-07-08 17:06:49 -07:00
Bob Trahan
2c03cd931b Legalpad V0.2 - add mail integration
Summary:
Supports !unsubscribe and commenting on replies. Subscribers get mailed something reasonable. Fixes T3480.

Sneaks in /LX/ support. In the near future I want to have that /LX/ be a clean "signature" page sans all the edit actions and other fluff... Will resolve this as part of T3481.

Test Plan: used the metamta console to add comments and unsubscribe. added a phlog() inside mail code to verify mail bodies looked okay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3480

Differential Revision: https://secure.phabricator.com/D6369
2013-07-03 16:37:05 -07:00
epriestley
a0c5a6cdb6 Use %P for all sensitive command construction in Phabricator
Summary: Depends on D6366. Applies %P everywhere.

Test Plan: Ran various daemon commands via scripts, e.g. `bin/repository pull`, `bin/storage dump`.

Reviewers: btrahan, mbishopim3

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6367
2013-07-03 15:13:45 -07:00
epriestley
fcb56c6371 Move "scripts/sql/probe.php" to "bin/storage probe"
Summary: This makes it namespace/database/connection aware and a little easier to find. Also use pht() / PhutilConsole.

Test Plan: Ran `bin/storage probe`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6341
2013-07-02 16:34:17 -07:00
epriestley
c3b2184977 Mostly modernize Conduit logs
Summary:
  - Add GC support to conduit logs.
  - Add Query support to conduit logs.
  - Record the actual user PHID.
  - Show client name.
  - Support querying by specific method, so I can link to this from a setup issue.

@wez, this migration may not be fast. It took about 8 seconds for me to migrate 800,000 rows in the `conduit_methodcalllog` table. This adds a GC which should keep the table at a more manageable size in the future.

You can safely delete all data older than 30 days from this table, although you should do it by `id` instead of `dateCreated` since there's no key on `dateCreated` until this patch.

Test Plan:
  - Ran GC.
  - Looked at log UI.
  - Ran Conduit methods.

Reviewers: btrahan

Reviewed By: btrahan

CC: wez, aran

Differential Revision: https://secure.phabricator.com/D6332
2013-07-01 12:37:34 -07:00
Bob Trahan
f15ed36d3c legalpad schema
Summary:
Ref T3116. This is a large amount of schema for V0 but it seems relatively complete to the desired features in T3116.

The only thing of note that is missing is documentSignatures should have some sort of "signedStatus". "Un-signing" seemed weird to me, though I could imagination "pending signature". "Pending signature" could be done via edges pretty easily.

Plan is to have "Document" be at the top level and own policy. "DocumentBody" will store a version of title and text for each and every "edit" on a larger Document. "Edges" are to be used to tie Authors => Document for V0ish. Transactions are going to be used to store all the various edits possible here. Oh and DocumentSignatures will do what you expect, but include documentVersion as part of the key.

Test Plan: just some schema. `storage update` worked though!

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T3116

Differential Revision: https://secure.phabricator.com/D6323
2013-06-28 13:56:16 -07:00
epriestley
f54a5d8087 Add DoorkeeperExternalObject
Summary:
Ref T2852. This table holds data about external objects and allows us to write edges to them.

Objects are identified with an `<applicationType, applicationDomain, objectType, objectID>` tuple. For example, Asana tasks will be, e.g., `<asana, asana.com, asana:task, 93829279873>` or similar.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6271
2013-06-24 15:54:36 -07:00
epriestley
3124838d65 Undo D6266 (DifferentialComment PHID migration)
Summary:
Ref T2222. My path forward here wasn't very good -- I was thinking I could set `transactionPHID` for the inline comments as I migrated, but it must be unique and an individual DifferentialComment may have more than one inline comment. Dropping the unique requirement just creates more issues for us, not fewer.

So the migration in D6266 isn't actually useful. Undo it -- this can't be a straight revert because some installs may already have upgraded.

Test Plan: Ran new migrations, verified the world ended up back in the same place as before (made comments, viewed reivsions).

Reviewers: btrahan

Reviewed By: btrahan

CC: wez, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D6269
2013-06-24 11:00:35 -07:00
epriestley
75fa580f3f Add PHIDs to DifferentialComments
Summary:
Ref T2222. This adds PHIDs to all Differential comments so I can migrate the inlinecommment table to transaction_comment in the next diff.

@wez, this will issue a few million queries for Facebook (roughly, one for each Differential comment ever made). It's safe to skip the `.php` half of the patch, bring Phabricator up normally, and then apply this patch with Phabricator running if that eases the migration, although the next few diffs will probably be downtime-required migrations so maybe it's easier to just schedule some downtime.

Test Plan: Ran migration locally. Verified existing comments and new comments received PHIDs.

Reviewers: btrahan

Reviewed By: btrahan

CC: wez, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D6266
2013-06-21 18:41:14 -07:00
epriestley
8e8057c6fe Add LiskRawMigrationIterator
Summary: Ref T2222. I need this to migrate the Differential comment tables, because they are large. We have the similar `LiskMigrationIterator` already, but it won't work here because I intend to destroy the original objects after migrating them.

Test Plan:
Wrote a script to iterate over the `differential_comment` table, got reasonable output:

{P869}

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D6264
2013-06-21 12:55:07 -07:00
epriestley
44302d2f07 Add storage for new Differential transactions and transaction comments
Summary:
Ref T2222. Ref T1460. Depends on D6260.

This creates the new tables, but doesn't start using them. I added three new fields for {T1460}, to represent fixed/done/replied states.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T1460, T2222

Differential Revision: https://secure.phabricator.com/D6261
2013-06-21 12:54:29 -07:00
epriestley
3b9ccf11f2 Drive auth config with the database
Summary: Ref T1536. This is the last major migration. Moves us over to the DB and drops all the config stuff.

Test Plan:
  - Ran the migration.
  - Saw all my old config brought forward and respected, with accurate settings.
  - Ran LDAP import.
  - Grepped for all removed config options.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran, wez

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6243
2013-06-20 11:18:11 -07:00
epriestley
5f29ccaaca Add storage for Auth configuration in preparation for moving it into a web interface
Summary:
Ref T1536. Currently, we have about 40 auth-related configuration options. This is already roughly 20% of our config, and we want to add more providers. Additionally, we want to turn some of these auth options into multi-auth options (e.g., allow multiple Phabricator OAuth installs, or, theoretically multiple LDAP servers).

I'm going to move this into a separate "Auth" tool with a minimal CLI (`bin/auth`) interface and a more full web interface. Roughly:

  - Administrators will use the app to manage authentication providers.
  - The `bin/auth` CLI will provide a safety hatch if you lock yourself out by disabling all usable providers somehow.
  - We'll migrate existing configuration into the app and remove it.

General goals:

  - Make it much easier to configure authentication by providing an interface for it.
  - Make it easier to configure everything else by reducing the total number of available options.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6196
2013-06-17 10:48:41 -07:00
epriestley
8744cdb699 Migrate PhabricatorUserLDAPInfo to PhabricatorExternalAccount
Summary: Ref T1536. This is similar to D6172 but much simpler: we don't need to retain external interfaces here and can do a straight migration.

Test Plan: TBA

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6173
2013-06-16 09:55:55 -07:00
epriestley
8111dc74bf Migrate the OAuthInfo table to the ExternalAccount table
Summary: Ref T1536. Migrates the OAuthInfo table to ExternalAccount, and makes `PhabricatorUserOAuthInfo` a wrapper for an ExternalAccount.

Test Plan: Logged in with OAuth, registered with OAuth, linked/unlinked OAuth accounts, checked OAuth status screen, deleted an account with related OAuth.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6172
2013-06-14 07:04:41 -07:00
epriestley
8886416e30 Expand the "PhabricatorExternalAccount" table for new registration
Summary:
Ref T1536. This is the schema code for `PhabricatorExternalAccount` which was previously in D4647. I'm splitting it out so I can put it earlier in the sequence and because it's simple and standalone.

Expands `PhabricatorExternalAccount` to have everything we need for the rest of registration.

Test Plan: Implemented the remainder of new registration on top of this.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6169
2013-06-14 06:55:18 -07:00
epriestley
6ffbee115b Add ApplicationTransactions/CustomField based user profile editor
Summary:
Adds a profile edit controller (with just one field and on links to it) that uses ApplicationTransactions and CustomField.

{F45617}

My plan is to move the other profile fields to this interface and get rid of Settings -> Profile. Basically, these will be "settings":

  - Sex
  - Language
  - Timezone

These will be "profile":

  - Real Name
  - Title
  - Blurb
  - Profile Image (but I'm going to put this on a separate UI)
  - Other custom fields

Test Plan: Edited my realname using the new interface.

Reviewers: chad, seporaitis

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6152
2013-06-07 09:55:55 -07:00
epriestley
758586abda Allow builtin named queries to be disabled
Summary:
Applications come with builtin queries, but users might want to get rid of them. Allow users to disable named queries if they prefer.

This has one funky behavior, which is that the first time you disable a named query it goes to the top of your list. That will be fixed in the next diff, which will make them reorderable.

Test Plan: Added/edited/removed named queries, disabled/enabled builtin named queries.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6128
2013-06-05 05:28:25 -07:00
epriestley
d9848d3c46 Add a book controller and various amenities to Diviner's live view
Summary: Ref T988. Mostly backend changes, with a very rough frontend on top of them. See Conpherence discussion.

Test Plan: {F45010}

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6113
2013-06-04 11:15:34 -07:00
epriestley
3aae972406 Implement ApplicationSearch in Files
Summary: Ref T2625. Ref T1163. A couple of small generalization nudges, but this is almost entirely straightforward.

Test Plan: Executed various File queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1163, T2625

Differential Revision: https://secure.phabricator.com/D6091
2013-05-31 10:51:05 -07:00
epriestley
fb765b8c93 Add language and date ranges to Paste queries
Summary:
Ref T2625. Ref T3273. This is mostly a UI foil for T3273. Right now, to find tasks without owners or without projects you search for the magic strings "upforgrabs" and "noproject". Unsurprisingly, no users have ever figured this out. I want to get rid of it. Instead, these interfaces will look like:

      Assigned: [ Type a user name... ]
                [ X ] Find unassigned tasks.
      Projects: [ Type a project name... ]
                [ X ] Find tasks with no projects.

Seems reasonable, I think?

Test Plan: Searched for "rainbow, js", "rainbow + no language", "no language", date ranges, etc.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T2625, T3273

Differential Revision: https://secure.phabricator.com/D6085
2013-05-30 18:55:04 -07:00
epriestley
bccc1e1244 Add a date range query to Macros
Summary:
Ref T1163. Ref T2625. This could probably use some tweaks, but I kept things mostly-generic.

I added a new control for freeform dates so we can have it render help or whatever later on.

Test Plan: See screenshots.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625, T1163

Differential Revision: https://secure.phabricator.com/D6084
2013-05-30 17:32:12 -07:00
Jakub Vrana
32f91557f8 Store hash of session key
Summary:
This prevents security by obscurity.
If I have read-only access to the database then I can pretend to be any logged-in user.

I've used `PhabricatorHash::digest()` (even though we don't need salt as the hashed string is random) to be compatible with user log.

Test Plan:
Applied patch.
Verified I'm still logged in.
Logged out.
Logged in.

  $ arc tasks

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6080
2013-05-30 17:30:06 -07:00
epriestley
d63811d319 Store authors on image macros
Summary:
Currently, the author of an image macro is read from the attached file. This is messy and necessitates a join, and is not always correct. Instead, store the data when the macro is created.

This lays the groundwork for generalizing ApplicationSearch here. Ref T2625.

Test Plan: Migrated existing macros, created a new macro, checked web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6071
2013-05-29 15:05:44 -07:00
epriestley
2fd018ad92 Begin transacitonalizing repository edits and provide a more sensible edit interface
Summary:
Ref T2231, T603. Plan of attack here is pretty much:

  - Built out a new (currently not linked in the UI) edit interface in Diffusion which is transaction-based and has a sensible layout.
  - Build out a new create interface based on PagedForm which dumps into the new edit interface.
  - Throw the old stuff away.
  - Everyone lives happily ever after.

Test Plan:
{F44163}
{F44164}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D6029
2013-05-24 12:37:42 -07:00
Gareth Evans
183bb8c80f Stop writing empty strings to the ownerPHID column
Summary: If we're unassigning an owner from a task it should set the column to `NULL` rather than an empty string. Fixes T3239

Test Plan: Assigned and Unassigned a task. Make sure the db is doing as excpected. Ran the patch, checked the db.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3239

Differential Revision: https://secure.phabricator.com/D6017
2013-05-23 09:22:18 -07:00
Bob Trahan
a56ca7a043 Conpherence - kill the image-based header UI
Summary:
removes the whole custom image thing, instead using a more standard application crumbs. Gives this glorious space back to the compose area which is now tens of pixels taller. Also defaults it to the people widget. Basically, fixes T3160.

For now, you **CAN NOT** edit the title of a conpherence. I didn't want to jam in too much here. Next diff will be to change the widget icons into the dropdown switcher, which will also bring back the editing of titles.

Test Plan: looked at conpherence and it was pretty. Resized it vigorously and it wasn't too bad.

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T3160

Differential Revision: https://secure.phabricator.com/D5998
2013-05-22 16:05:47 -07:00
epriestley
b0a5f42244 Add "live" publisher and storage to Diviner
Summary:
Ref T988. This adds basics for the non-static publishing target:

  - Storage (called "Live", e.g. `DivinerLiveAtom` to distinguish it from shared classes like `DivinerAtom`).
  - Mostly populate the storage.
  - Some minor fixes and improvements.

Test Plan: Generated docs, looked at DB, saw mostly-sensible output.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D5973
2013-05-20 10:18:26 -07:00
epriestley
eabe3a4d33 Begin improving the soundness of received mail
Summary:
We/I broke a couple of things here recently (see D5911) and are doing some work here in general (see D5912, etc.).

Generally, this code is pretty oldschool and not especially well architected for modern application-oriented Phabricator. It hardcodes a lot of stuff which should be applications' responsibilites.

Take the first steps toward making it more solid to reduce the risk here. In particular:

  - Factor out the "self mail" and "duplicate mail" checks and add unit tests.
  - Make Message-ID hash handling automatic.

Test Plan: Ran unit tests.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5915
2013-05-13 16:32:19 -07:00
Edward Speyer
fcb7286533 ReleephRequest xactions
Summary:
Migrate to `PhabricatorApplicationTransactions` (`ReleephRequestTransactions` applied by `ReleephRequestTransactionalEditor`, instead of `ReleephRequestEvents` created by `ReleephRequestEditor`) and migrate all the old events into transactions.  Email is supported in the standard way (no more `ReleephRequestMail`) as well.

This also collapses the Releeph request create and edit controllers into one class, as well as breaking everyone's subject-based mail rules by standardising them (but which should be more easily filtered by looking at headers.)

Test Plan:
* Make requests, then pick them.
* Pick and revert the same request so that discovery happens way after `arc` has told Releeph about what's been happening.
* Try to pick something that fails to pick in a project with pick instructions (and see the instructions are in the email.)
* Load all of FB's Releeph data into my DB and run the `storage upgrade` script.
* Request a commit via the "action" in a Differential revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: epriestley, aran, Korvin, wez

Maniphest Tasks: T3092, T2720

Differential Revision: https://secure.phabricator.com/D5868
2013-05-11 15:20:09 +01:00
Bryan Cuccioli
3c1f402da3 Add ability to name saved queries.
Summary: Can name saved queries.

Test Plan: Try naming some saved queries using the form.

Reviewers: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D5878

Conflicts:

	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-05-10 16:45:45 -07:00
Edward Speyer
58f0f37873 Add mail keys to ReleephRequests
Summary: Adding mail-keys; required for `PhabricatorApplicationTransaction` support.

Test Plan: Upgrade an old database with this patch, observe the matrix: {F42620}

Reviewers: wez, epriestley

Reviewed By: epriestley

CC: epriestley, aran

Maniphest Tasks: T2720

Differential Revision: https://secure.phabricator.com/D5852
2013-05-08 10:38:07 +01:00
Edward Speyer
5b2fc6a184 Simplify ReleephRequest schema
Summary:
Removing a bunch of cache-style columns from `ReleephRequest`, where it's actually much easier to just load the information at runtime.

This makes sense for migrating to `PhabricatorApplicationTransactions`, where each xaction changes one aspect of a `ReleephRequest` at a time (rather than multiple columns at once.)

Test Plan: Request something, run `arc releeph` and watch the picks, pass on some RQs, run `arc releeph` and watch the reverts.

Reviewers: wez, epriestley

Reviewed By: epriestley

CC: epriestley, aran

Maniphest Tasks: T2720

Differential Revision: https://secure.phabricator.com/D5851
2013-05-08 10:27:20 +01:00
Lauri-Henrik Jalonen
271d6605a9 Countdown revamp
Summary:
countdown_timer table named to countdown.
datepoint and related stuff renamed to epoch.
Countdowns now have phids.
Various UI items changed from timer to countdown.

Test Plan: Did run storage upgrade and added some countdowns.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2624

Differential Revision: https://secure.phabricator.com/D5812
2013-05-03 15:53:36 -07:00
Bryan Cuccioli
7ad2eae47f Implement saving queries.
Summary: Enable saved query objects to actually be saved to the database.

Test Plan: Insert a call to save() and check that the query is written correctly.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D5775

Conflicts:

	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-04-30 10:48:16 -07:00
Bob Trahan
11cb2f4f6c Conpherence - paginate thread list
Summary: this is D5750 but just the conpherence part. fixes a few random conpherence bugs / quirks as well. Also messes with ApplicationTransactionEditor to expose the xactions so Conpherence doesn't over-update participation rows. Fixes T2429.

Test Plan: set LIMIT to 3. verified I could scroll down all conpherences. next, picked a conpherence "in the middle" to load. verified I could page up and down. next, picked a conpherence in the middle then had another user update that conpherence. verified as I paged up the conpherence re-loaded properly selected

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin, vrana

Maniphest Tasks: T2429

Differential Revision: https://secure.phabricator.com/D5783
2013-04-26 10:30:41 -07:00
epriestley
7a5f622820 General cleanup for adding payment methods in Phortune
Summary:
This has no real behavioral changes (except better error handling), it just factors things out to be a bit cleaner. In particular:

  - Move more shared form behaviors into the common JS form component.
  - Move more error handling into shared pathways.
  - Make the specialized Stripe / Balanced methods do less work.

This needs some more polish for nontrival errors (especially on the Balanced side) but none of the error behavior is worse than it was and a lot of it is much better.

Ref T2787.

Test Plan: Hit all invalid form errors, added valid payment methods with Stripe and Balacned.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5771
2013-04-25 09:49:32 -07:00
Afaque Hussain
3f7ae27b58 Sql Patch to update the user_externalaccount table.
Summary: Sql Patch to rename the externalaccount table to user_externalaccount and to add dateCreated, dateModified fields to the updated table.

Test Plan: {F41442}

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5770
2013-04-24 15:14:26 -07:00
Afaque Hussain
dc6cfe6e01 Created PhabricatorExternalAccount Class and Sql patch to create an external_account table.
Summary: Created PhabricatorExternalAccount class with only data members. Will discuss with you regarding the necessary functions to be implemented in this class. Sql Patch to create a new table for external_accounts. Will I have to write unit tests the new storage object? Sending you this diff so that you can comment on this to further improve :).

Test Plan: {F40977}

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T1536, T1205

Differential Revision: https://secure.phabricator.com/D5724
2013-04-19 11:40:24 -07:00
Jakub Vrana
e31e998f3b Convert differential.revisionPHID commit detail to edge
Summary: Migration doesn't delete differential.revisionPHID but maybe it should?

Test Plan: Reparsed commit, ran the migration, deleted differential.revisionPHID, looked at task with attached commit with attached revision.

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5634
2013-04-12 22:48:16 -07:00
Jakub Vrana
9da6c9ff15 Delete LiskDAO::IDS_PHID
Test Plan: Searched for `IDS_PHID` and loaded homepage.

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5619
2013-04-08 20:46:54 -07:00
Bob Trahan
754705df4e Conpherence - get back-end prepped for loading less transactions all the time
Summary: this just does the back-end migration. I realized that we don't need to keep track of cacheTitle and cachePhoto since those are based off recent participation handles and dynamic relative to who is viewing it. Also kept the "last seen phid" as I think that will be useful to have auto-scroll to where you last read.  Ref T2867.

Test Plan: did the migration. observed sensical values in the database. created a new conpherence - again sensical values. updated a conpherence - more sensical values.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2867

Differential Revision: https://secure.phabricator.com/D5567
2013-04-04 16:57:38 -07:00
Bryan Cuccioli
2334f63c2c Paginate token leader board.
Summary: Add pagination to leader board. Add key on token count in db.

Test Plan: Set page size to 1 and give tokens to two tasks.

Reviewers: epriestley, AnhNhan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5513
2013-04-01 08:16:01 -07:00
James Rhodes
dd3b3bdd5e Added initial storage structure for Phrequent.
Summary:
Added the initial storage structure (DB tables and DAO classes)
for Phrequent.

Test Plan:
Apply the patch and run `bin/storage upgrade`.  It should
complete successfully.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2857

Differential Revision: https://secure.phabricator.com/D5476
2013-03-29 21:30:15 -07:00
epriestley
abdaf90239 Phortune v0.1: products
Summary:
Ref T2787. A product is the abstract representation of something you can buy or rent/subscribe to. Although the interface isn't locked down yet, this would ultimately be internal/administrative.

Products likely have some user-facing skin on top of them: plans would have a purchasing/comparison flow, physical goods would have a storefront, etc., so products don't have any information like descriptions or images, just the data that Phortune needs to correctly bill accounts.

Generally, this is very basic for the moment.

Test Plan:
{F37594}
{F37595}
{F37596}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5441
2013-03-28 09:13:07 -07:00
epriestley
4f3b5f0ea9 Phortune v0.1: add payment methods
Summary:
Hook @btrahan's Stripe form to the rest of Phortune.

  - Users can add payment methods.
  - They are saved to Stripe and associated with PhortunePaymentMethods on our side.
  - Payment methods appear on account overview.

Test Plan:
{F37548}
{F37549}
{F37550}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5438
2013-03-28 09:11:42 -07:00
epriestley
960ac3b2a6 Phortune v0
Summary:
Ref T2787. This does very little so far, but makes inroads on accounts and billing. This is mostly just modeled on what Stripe looks like. The objects are:

  - **Account**: Has one or more authorized users, who can make manage the account. An example might be "Phacility", and the three of us would be able to manage it. A user may be associated with more than one account (e.g., a corporate account and a personal account) but the UI tries to simplify the common case of a single account.
  - **Payment Method**: Something we can get sweet sweet money from; for now, a credit card registered with Stripe. Payment methods are associated with an account.
  - **Product**: A good (one time charge) or service (recurring charge). This might be "t-shirt" or "enterprise plan" or "hourly support" or whatever else.
  - **Purchase**: Represents a user purchasing a Product for an Account, using a Payment Method. e.g., you bought a shirt, or started a plan, or purchased support.
  - **Charge**: Actual charges against payment methods. A Purchase can create more than one charge if it's a plan, or if the first charge fails and we re-bill.

This doesn't fully account for stuff like coupons/discounts yet but they should fit into the model without any issues.

This only implements `Account`, and that only partially.

Test Plan: {F37531}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5435
2013-03-28 09:10:34 -07:00
epriestley
696498934c Support edge transactions in ApplicationTransactions
Summary: Fixes T2655. Adds generic support for edge edits (e.g., membership or attached objects).

Test Plan: See next diff.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2655

Differential Revision: https://secure.phabricator.com/D5434
2013-03-28 08:34:34 -07:00
Bob Trahan
23dc686045 Conpherence - add per thread notification setting
Summary: Introduces a new settings panel for Conpherence specific settings.

Test Plan:
started a thread with a test user, thus two participants total. Replied to conpherence, toggling notification settings in between. Verified 1 or 2 emails were sent as appropos to the current toggle.

Toggled global setting and verified setting was updated in conpherences where nothing was specified. Verified setting conpherence setting overrides global setting.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2521

Differential Revision: https://secure.phabricator.com/D5391
2013-03-26 13:30:35 -07:00
Anh Nhan Nguyen
489f9e7dfe Added subscriptions to Phriction documents
Summary:
Fixes T2694

added edge infrastructure for Phriction

added mail subject prefix option for Phriction

added messy mail support for subscribers

adds edges to the phriction db, along with the subscriber interface
which gives us subscriptions for free.

simple display of subscribers, adequate to the current design and
sufficient fallbacks for exceptional cases. @chad may
be mailed about that one more UI element may be added to his redesign

mail support is messy. not generic at all. only sends to subscribed non-authors.

Test Plan:
tried out all kinds of stuff. applied patch, subscribed, unsubscribed with multiple
accs. verified proper

edited documents, verified that mail was sent in MetaMTA. Verified
contents, tos and stuff by looking into the db, comparing PHIDs etc.

functional testing per serious MTA (that is, AWS SES) worked wonderfully.

Here's how the subscription list looks like:
{F36320, layout=link}

Reviewers: epriestley, chad, btrahan

Reviewed By: epriestley

CC: hfcorriez, aran, Korvin

Maniphest Tasks: T2686, T2694

Differential Revision: https://secure.phabricator.com/D5372

Conflicts:

	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-03-21 10:28:22 -07:00
epriestley
9a515171f4 Phlux v0
Summary:
Yup, it's sitevars.

No conduit or caching or fancy stuff yet.

Ref T2793.

Test Plan:
{F36525}
{F36526}
{F36527}
{F36528}
{F36529}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2793

Differential Revision: https://secure.phabricator.com/D5397
2013-03-20 18:01:52 -07:00
Afaque Hussain
77a8765824 Schema patch to add a column to Phabricator_File.file table
Summary: Added a column called explicit_upload to Phabricator_File.file table

Test Plan: By chekcing locally if the the column has been added in table using mysql commands.

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5384
2013-03-19 15:00:44 -07:00
Edward Speyer
2497e5b5ed Releeph (Phabricator part)
Summary: A copy of the Releeph release tool.

Test Plan: Generally, click everything at least once.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2094

Differential Revision: https://secure.phabricator.com/D4932
2013-03-15 11:28:43 +00:00
vrana
1091dc7aa1 Save blame info to lint messages
Test Plan:
Applied the patch.
Looked at blame and plain blame of SVN and Git file.
Ran the lint saver.
Looked at lint messages list.
/diffusion/lint/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5218
2013-03-06 16:19:01 -08:00
James Rhodes
cd99850cb8 Increase Maniphest auxiliary storage size.
Test Plan:
Store large amounts of text into a string auxiliary field.  It should
be stored successfully rather than truncated.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2575

Differential Revision: https://secure.phabricator.com/D5246
2013-03-06 10:46:56 -08:00
epriestley
fe78944c9d Prepare Diffusion for hovercards
Summary:
Move Diffusion to be hovercard-ready, and expand our ability to resolve commit references.

  - Link unqualified hashes of 7 characters or more which match a commit.
  - Link qualified hashes of 5 characters or more which match a commit.
  - Support `{...}` syntax.

Test Plan: {F33896}

Reviewers: chad, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5121
2013-02-27 08:04:54 -08:00
Afaque Hussain
dd076a813f SQL patch to drop channel field.
Summary: Deleted the channel field and added a sql patch to drop the channel field.

Test Plan: I have messed up my local mysql:P, hence by storage upgrade is failing. Anyways, The chatlog_event table shouldn't contain the channel column now.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5011
2013-02-22 07:29:40 -08:00
epriestley
b32bfb6541 Render commit summaries when rendering handles
Summary:
Fixes T2563. Instead of rendering "rPnnnnnn", render "rPnnnnnn: add feature X". Tweak Audit tables to accommodate.

@vrana / @nh, this migration might take a while. You could safely skip it when deploying and then run it after deployment.

I think I fixed all the other places where these render, but might have missed something.

Test Plan:
  - Ran first schema migration, clicked around to make sure nothing broke.
  - Ran `scripts/repository/reparse.php --message rXyyyyy`, verified summary populated.
  - Ran second migration.
  - Checked task/diffusion/audit/differential for weird rendering.

Reviewers: vrana

Reviewed By: vrana

CC: nh, aran, chrisbolt, allixsenos

Maniphest Tasks: T2563

Differential Revision: https://secure.phabricator.com/D5012
2013-02-21 15:09:35 -08:00
kwadwo
762ace810d Allow files to TTL and and be garbage collected
Summary: Added ttl field to files. Gabage collect files with expired ttl

Test Plan: created file with a ttl. Let garbage collector run

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4987
2013-02-20 13:38:36 -08:00
epriestley
8d79c7282d Accommodate long daemon command lines
Summary: Fixes T2559 with an incredibly original patch which I came up with myself.

Test Plan:
  $ ./bin/storage upgrade -f
  Applying patch 'phabricator:20130218.longdaemon.sql'...
  Storage is up to date. Use 'storage status' for details.

Reviewers: edward

Reviewed By: edward

CC: aran

Maniphest Tasks: T2559

Differential Revision: https://secure.phabricator.com/D5006
2013-02-18 11:51:42 -08:00
Afaque Hussain
61c26463bc Php schema patch to update channel id's of past events.
Summary: Php schema patch to update channel id's of past events.

Test Plan: Having some proxy issues here due to which connection is timing out and bot is not able to log into IRC. Bot connects to IRC in my home though ! So I wasn't able to quite to test this by running storage upgrade.

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5000
2013-02-18 10:53:53 -08:00
epriestley
26aac16346 Garbage collect TTL'd cache entries from the general cache
Summary: We currently garbage collect general cache entries after a set period of time (30 days by default), but the recent changes to DarkConsole have left us writing a lot of large, short-TTL data to the cache. In addition to a maximum age, GC cache entires after they TTL out.

Test Plan: Ran GC daemon, saw TTL'd entries get collected. Inserted a TTL'd entry, saw it get collected by GC. Saw non-ttl'd entries not get collected.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4990
2013-02-17 09:13:49 -08:00
epriestley
49c40d209d Tokens v1
Summary:
Features!

  - Giving tokens.
  - Taking tokens back.
  - Not giving tokens.

Test Plan: See screenshots.

Reviewers: chad, vrana

Reviewed By: chad

CC: aran, btrahan

Maniphest Tasks: T2541

Differential Revision: https://secure.phabricator.com/D4964
2013-02-15 07:47:14 -08:00
epriestley
176ee9a889 Fix {Fnnn} rule in Remarkup
Summary:
Remarkup rule callbacks now get SafeHTML matches instead of string matches. If they call:

  $some_lisk_dao->load($matches[1]);

..as is the case with the `{F123}` rule, we reject the SafeHTML as an invalid ID and return null.

Allow load() to string convert any object (which will either succeed or fatal in an obviously-broken way).

(Long ago we threw instead of returning null here, but it meant we had to do a lot of redundant checks.)

Test Plan: `{F123}` shows an image again. `{C1}` embeds a countdown.

Reviewers: vrana, chad

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4961
2013-02-14 13:09:19 -08:00
Afaque Hussain
73991bb262 Added channel ID to events
Summary: Added a column channelID column to phabricator_chatlog.chatlog_event

Test Plan: Checked through mysql to see if table is updated

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4956
2013-02-14 12:27:18 -08:00
Afaque Hussain
9baada1571 Schema Patch to Add a New Table
Summary: Added 20130214.chatlogchannel.sql in resources/sql/patches to add a new table

Test Plan: Hmmmmm .......

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4952
2013-02-14 11:37:05 -08:00
Bob Trahan
68814d4eca add conpherence schema patch to the list so it runs
Summary: ...i tend to forget to do this for some reason. my bad.

Test Plan: NA

Reviewers: epriestley, chad

Reviewed By: chad

CC: aran, Korvin

Maniphest Tasks: T2503

Differential Revision: https://secure.phabricator.com/D4840
2013-02-06 15:28:24 -08:00
vrana
8c99938aad Convert revision unsubscribers to edges
Test Plan: Ran the migration on a single revision, verified DB, called `loadUnsubscribedPHIDs()`.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4786
2013-02-04 11:36:55 -08:00
Lauri-Henrik Jalonen
5cb8787d91 Removed psth column from herald transcript
Summary: Added patch file to remove psth column in herald transcript tabe

Test Plan: Verified that column was removed with ./bin/storage upgrade

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2425

Differential Revision: https://secure.phabricator.com/D4672
2013-01-26 15:22:18 -08:00
Bob Trahan
b873f3f991 Conpherence V1
Summary: it's ugly. but it works. basically. See T2399 for a roughly prioritized list of what still needs to happen.

Test Plan:
- created a conpherence with myself from my profile
- created a conpherence with myself from "new conpherence"
- created a conphernece with another from "new conpherence"
- created a conpherence with several others
- created a conpherence with files in the initial post
 - verified files via comment text ("{F232} is awesome!") and via traditional attach
- edited a conpherence image
 - verified it showed up in the header and in the conpherence menu on the left
- edited a conpherence title
 - verified it showed up in the header and in the conpherence menu on the right
- verified each widget showed up when clicked and displayed the proper data
 - calendar being an exception since it sucks so hard right now.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, epriestley, chad, codeblock, Korvin

Maniphest Tasks: T2301

Differential Revision: https://secure.phabricator.com/D4620
2013-01-24 17:23:05 -08:00
Nick Pellegrino
3e6fa43658 getConfigEnv fails fast when key is not found and no default value is given.
Summary:
T2345
getConfig throws an Exception when the key does not exist.
Also removes dead code that throws an Exception.

Test Plan:
Reloaded the Phabricator home page.  In the process, found
2 Exceptions thrown due to nonexistent keys.  After addressing these problems,
the home page loads without Exceptions.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4541
2013-01-19 12:11:28 -08:00
vrana
aa3e95cbeb Cache connection in bin/storage
Summary: Connection takes .3s from dev server to master.

Test Plan:
  $ bin/storage --trace upgrade --namespace x
  $ bin/storage --trace destroy --namespace x

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4480
2013-01-17 11:00:36 -08:00
Mailson Menezes
712e22208c Store width and height metadata of image files
Summary: Also provide a way to update old files metadata.

Test Plan: Create a revision which includes a image file. Check whether the widht, height metadata exists. Run `scripts/files/manage_files.php metadata --all` to update previously uploaded files.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2101

Differential Revision: https://secure.phabricator.com/D4347
2013-01-07 09:46:43 -08:00
Bob Trahan
3448781c40 de-duplicate emails received by phabricator multiple times
Summary: this can happen if you have Phabricator and email lists co-mingling such that Phabricator receives an email multiple times. we can prevent this from then spamming everyone or otherwise taking the action multiple times by storing a message id hash and dropping the message if we have more than one message that matches.

Test Plan: simulated sending the same email multiple times on the command line. noted only the first one made it through.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1726

Differential Revision: https://secure.phabricator.com/D4328
2013-01-03 17:04:30 -08:00
epriestley
0902543fc8 Port MySQL settings to PHP
Summary:
  - Ports MySQL settings to PHP.
  - Removes "mysql.retries" -- this existed only because Magic Numbers Are Bad, but there is no concievable reason it should ever be set to anything other than 3.
  - Introduced "Hidden" config, which isn't visible from the web (for SaaS, we'll just mark anything with secret keys as "hidden").
  - Introduced "Masked" config, which will be masked in darkconsole once that gets updated.
  - "Hidden" implies "Masked" and "Locked".
  - Moved "storage.default-namespace" here -- it probably makes more sense than core; this was my bad in T2255.
  - Put cancel button back for hidden/locked config.
  - Introduce 'class' config type.

Test Plan: Viewed MySQL options. None are editable.

Reviewers: codeblock, btrahan

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4326
2013-01-03 06:01:14 -08:00
Bob Trahan
2f6104bf8b add code to drop old databases
Summary: these existed once, are no more, and don't get cleaned up in the current code path

Test Plan: storage destroy --dryrun -- noted the correct database names

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2237

Differential Revision: https://secure.phabricator.com/D4329
2013-01-02 19:02:31 -08:00
epriestley
32e4a7a37f Use transactions to show edit history for Configuration
Summary: Use ApplicationTransactions in Config to create an edit history. Resolves T2256.

Test Plan: {F28477}

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2256

Differential Revision: https://secure.phabricator.com/D4314
2013-01-01 18:14:41 -08:00
Ricky Elrod
a774620042 Start of a config web interface.
Summary:
This is somewhat clowny, particularly in how it handles JSON encode/decode, but
I've commented why I did things the way I did. The goal is to store minified JSON
but show pretty-printed JSON where possible, to the user editing it.

Test Plan:
* Went to /config/ and saw a list of keys from the `default` config.
* Clicked on one of them, submitted the default value successfully.
* Changed the value to invalid JSON and got a decent error.
* Changed the value to valid JSON and checked the DB to confirm it saved.
* Confirmed the DB values were minified.
* Confirmed the user-facing values were pretty-printed where they could be.
* Confirmed that PHIDs were getting assigned properly and that isDeleted
  properly defaulted to false/0.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2246

Differential Revision: https://secure.phabricator.com/D4290
2012-12-27 15:21:21 -08:00
epriestley
aae5f9efd3 Implement a more compact, general database-backed key-value cache
Summary:
See discussion in D4204. Facebook currently has a 314MB remarkup cache with a 55MB index, which is slow to access. Under the theory that this is an index size/quality problem (the current index is on a potentially-384-byte field, with many keys sharing prefixes), provide a more general index with fancy new features:

  - It implements PhutilKeyValueCache, so it can be a component in cache stacks and supports TTL.
  - It has a 12-byte hash-based key.
  - It automatically compresses large blocks of data (most of what we store is highly-compressible HTML).

Test Plan:
  - Basics:
    - Loaded /paste/, saw caches generate and save.
    - Reloaded /paste/, saw the page hit cache.
  - GC:
    - Ran GC daemon, saw nothing.
    - Set maximum lifetime to 1 second, ran GC daemon, saw it collect the entire cache.
  - Deflate:
    - Selected row formats from the database, saw a mixture of 'raw' and 'deflate' storage.
    - Used profiler to verify that 'deflate' is fast (12 calls @ 220us on my paste list).
  - Ran unit tests

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4259
2012-12-21 14:17:56 -08:00
epriestley
ba7723d905 Modernize Macro application
Summary: Adds feed, email, notifications, comments, partial editing, subscriptions, enable/disable, flags and crumbs to Macro.

Test Plan:
{F26839}
{F26840}
{F26841}
{F26842}
{F26843}
{F26844}
{F26845}

Reviewers: vrana, btrahan, chad

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2157, T175, T2104

Differential Revision: https://secure.phabricator.com/D4141
2012-12-11 14:01:03 -08:00
epriestley
7b6fa0db12 Genericize transactions in Pholio
Summary:
Split Pholio's transaction implementation into generic and application-specific parts. Moves us toward generic transactions, with support for:

  - Editing and deleting comments.
  - Setting visibility of individual comments (I'm not a fan of this feature but we'll see).

I want to move everything to a more generic piece of infrastructure but there's very little they can share right now so adding transactions to, e.g., Paste or Macros (T2157) means massive amounts of similar code.

Tons of work left to do here, but I think it basically works. Here's a screenshot:

{F26820}

Test Plan: Made transactions in Pholio.

Reviewers: btrahan, vrana, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2104

Differential Revision: https://secure.phabricator.com/D4136
2012-12-11 13:59:20 -08:00
vrana
4f615ad2a9 Allow excluding paths from package
Summary: Resolves T2149.

Test Plan:
  $ bin/storage upgrade

# /owners/ - saw +
# /owners/package/1/ - saw +
# /owners/edit/1/ - added exclude paths, saw correct e-mail
# /rPabc123 - included paths are still highlighted and excluded not
# /owners/view/search/?path=/included/ - found
# /owners/view/search/?path=/excluded/ - not found
# owners.query - path: /included/
# owners.query - path: /excluded/
# new unit test

  PhabricatorOwnersPackage::loadAffectedPackages(
    $repository,
    array('/excluded/b.php'));

  PhabricatorOwnersPackage::loadAffectedPackages(
    $repository,
    array('/included/a.php'));

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2149

Differential Revision: https://secure.phabricator.com/D4102
2012-12-07 16:33:16 -08:00
epriestley
fc9ad37b26 Add very basic scaffolding for Pholio
Summary:
I'm not going to land this until it's a bit more fleshed out since it would just confuse users, but this is probably more reviewable as a few diffs adding a couple features than one ULTRA-diff adding everything. Implement application basics for Pholio. This does more or less nothing, but adds storage, subscribe, flag, markup, indexing, query basics, PHIDs, handle loads, a couple of realy really basic controllers, etc.
Basic hierarchy is:

  - **Moleskine**: Top-level object like a Differential Revision, like "Ponder Feed Ideas".
  - **Image**: Each Moleskine has one or more images, like the unexpanded / expanded / mobile / empty states of feed.
  - **Transaction**: Comment or edit, like Maniphest. I generally want to move most apps to a transaction model so we can log edits.
  - **PixelComment**: Equivalent of an inline comment.

Test Plan: Created a fake object and viewed it.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran, davidreuss

Maniphest Tasks: T2097

Differential Revision: https://secure.phabricator.com/D3817
2012-11-21 17:22:36 -08:00
epriestley
ee2e85a0bb Fix several migration issues with the Task/Counter patch
Summary:
People hit three issues with D3914:

  - As per T2059, we applied a schema change from a `.php` patch, which currently does not work if you use a different user to make schema changes than for normal use.
    - Since the change in question is idempotent, just move it to a `.sql` patch. We'll follow up in T2059 and fix it properly.
  - Rogue daemons at several installs used old code (expecting autoincrement) to insert into the new table (no autoincrement), thereby creating tasks with ID 0.
    - Rename the table so they'll fail.
    - This also makes the code a little more consistent.
  - Some installs now have tasks with ID 0.
    - Use checks against null rather than against 0 so we can process these tasks.

The major issues this fixes are the schema upgrade failure in T2059, and the infinite loops in T2072 and elsewhere.

This isn't really a fully statisfactory fix. I'll discuss some next steps in T2072.

Test Plan: Created new tasks via MetaMTA/Differential. Ran tasks with `phd debug taskmaster`. Inserted a task 0 and verified it ran and archived correctly.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2072, T2059

Differential Revision: https://secure.phabricator.com/D3973
2012-11-16 10:19:22 -08:00
vrana
23a046b3cd Allow saving lint errors to database
Summary: This saves lint errors to the path change of current commit. It requires pushed revision. It doesn't save difference from previous commit mentioned in T2038#comment-4 - I don't plan doing it after all, everything would be much more complicated and the amount of data saved with this approach isn't that bad.

Test Plan: Applied patch, ran script, verified DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2038

Differential Revision: https://secure.phabricator.com/D3899
2012-11-08 15:39:43 -08:00
Bob Trahan
9966af50dd Delete PhabricatorRemarkupRuleProxyImage
Summary: don't need it now that uploading files is so easy. Plus it made for some buggy jonx if / when there were bad image links coupled with caching. In theory this is a lot less pretty though if folks linked to a bunch of files served elsewhere using images.

Test Plan: http://does-not-exist.com/imaginary.jpg rendered as a link!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2000

Differential Revision: https://secure.phabricator.com/D3908
2012-11-07 14:31:43 -08:00
epriestley
7332599e03 Provide an IDS_COUNTER mechanism for ID assignment
Summary: See D3912 for discussion. InnoDB may reuse autoincrement IDs after restart; provide a way to avoid it.

Test Plan: Unit tests. Scheduled and executed tasks through `drydock lease --type host` and `phd debug taskmaster`.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3914
2012-11-07 13:33:07 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
89b37f0357 Make various Drydock improvements
Summary:
Tightens up a bunch of stuff:

  - In `drydock lease`, pull and print logs so the user can see what's happening.
  - Remove `DrydockAllocator`, which was a dumb class that did nothing. Move the tiny amount of logic it held directly to `DrydockLease`.
  - Move `resourceType` from worker task metadata directly to `DrydockLease`. Other things (like the web UI) can be more informative with this information available.
  - Pass leases to `allocateResource()`. We always allocate in response to a lease activation request, and the lease often has vital information. This also allows us to associate logs with leases correctly.

Test Plan: Ran `drydock lease --type host` and saw it perform a host allocation in EC2.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3870
2012-11-01 16:53:17 -07:00
epriestley
f0fdcf1a51 Undumb the Drydock resource allocator pipeline
Summary:
This was the major goal of D3859/D3855, and to a lesser degree D3854/D3852.

As Drydock is allocating a resource, it may need to allocate other resources first. For example, if it's allocating a working copy, it may need to allocate a host first.

Currently, we have the process basically queue up the allocation (insert a task into the queue) and sleep() until it finishes. This is problematic for a bunch of reasons, but the major one is that if allocation takes more resources (host, port, machine, DNS) than you have daemons, they could all end up sleeping and waiting for some other daemon to do their work. This is really stupid. Even if you only take up some of them, you're spending slots sleeping when you could be doing useful work.

To partially get around this and make the CLI experience less dumb, there's this goofy `synchronous` flag that gets passed around everywhere and pushes the workflow through a pile of special cases. Basically the `synchronous` flag causes us to do everything in-process. But this is dumb too because we'd rather do things in parallel if we can, and we have to have a lot of special case code to make it work at all.

Get rid of all of this. Instead of sleep()ing, try to work on the tasks that need to be worked on. If another daemon grabbed them already that's fine, but in the worst case we just gracefully degrade and do everything in process. So we get the best of both worlds: if we have parallelizable tasks and free daemons, things will execute in parallel. If we have nonparallelizable tasks or no free daemons, things will execute in process.

Test Plan: Ran `drydock_control.php --trace` and saw it perform cascading allocations without sleeping or special casing.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3861
2012-11-01 11:30:42 -07:00
Edward Speyer
fe8351c930 Delete stale fixtures with bin/storage destroy
Summary:
When a PhabricatorTestCase dies after creating storage fixtures, it leaves
those storage fixtures around.  This doesn't happen often, but when it does
happen it's a pain to cleanup.  The --unittest-fixtures option helps automate
that cleanup.

Test Plan:
Ran it with --dryrun, then for real.  Became overwhelmed with a Zen like peace
after regarding the tidiness and beauty of SHOW DATABASES.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3858
2012-10-31 18:36:38 -07:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
Jakub Vrana
d506ffbd30 Add regression test for LiskDAO::__call()
Summary: D3606

Test Plan: Deleted `call()`, ran the test.

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3814
2012-10-24 14:00:43 -07:00
Bob Trahan
60466d3bcc Create a status tool by giving /calendar/ some teeth
Summary: you can now add, edit, and delete status events. also added a "description" to status events and surface it in the big calendar view on mouse hover. some refactoring changes as well to make validation logic centralized within the storage class.

Test Plan: added, edited, deleted. yay.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T407

Differential Revision: https://secure.phabricator.com/D3810
2012-10-24 13:22:24 -07:00
vrana
fd87a88d71 Fix static variable usage in Lisk
Summary:
See comment.
This can reveal some pretty bad bugs but HPHP handles this correctly so we already know about them.

Test Plan:
Added `phlog()` to `__call()` and observed what is defined for each method (under PHP). Also:

  class C {
    function __call($name, $args) {
      static $class;
      if (!$class) {
        $class = get_class($this);
      }
      return $class;
    }
  }

  class D extends C {
  }

  class E extends C {
  }

  $d = new D;
  $e = new E;
  var_dump($d->x());
  var_dump($e->x()); // Prints D under PHP!

See also D3754.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1261

Differential Revision: https://secure.phabricator.com/D3753
2012-10-20 05:04:38 -07:00
epriestley
a50b8e39b1 Make posts 1:1 with blogs and implement policy controls
Summary:
This leaves the UI in a pretty rough state, but implements blog policy controls and queries, and 1:1 relationships between posts and blogs. Needs a bunch more cleanup but seemed like an okayish breaking point in terms of cohesiveness.

Posts have these rules:

  - Drafts are visible only to the author.
  - Published posts are visible to anyone who can see the blog they appear on.
  - Posts are only editable by the author.

...so we don't need any special policy UI or state to accommodate these rules.

Posts may have no blog if they're grandfathered in or you write a post to a blog and then lose the ability to see the blog. This is the messiest edge case -- specifically:

  - You write a post to blog A.
  - You publish the post.
  - I edit the "Visible To:" for blog A and set it to exclude you.

What we do in this case is let you see the post in "My Posts", but you can no longer see the blog and you'll see the post as not being part of a blog. We can maybe give you some UI to let you move it later or something.

Test Plan: Hit all (I think?) of the interfaces without issues. Definitely some UI problems still right now.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3694
2012-10-15 14:50:04 -07:00
epriestley
dbcf2e44e8 Make PhameBlogs respect policies
Summary:
Adds "can view" and "can edit" policies to blogs. Replaces "bloggers" with "can join".

This doesn't fully remove "bloggers" because I didn't want this to get too crazy/huge.

Test Plan: Created, edited, deleted blogs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3693
2012-10-15 14:49:52 -07:00
vrana
b605878792 Properly name database in patch list
Summary:
Blames to D3659.
It's not that old (< 1 day) to fix it properly (add patch for dropping xhpast DB).

Test Plan:
  $ arc unit src/applications/calendar/storage/__tests__/PhabricatorCalendarHolidayTestCase.php

Reviewers: btrahan, epriestley

Reviewed By: btrahan

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3670
2012-10-09 15:15:36 -07:00
epriestley
2532cb9613 Add mail keys to Ponder questions
Summary:
We need to go slightly farther to stub reply handler functionality for Ponder in at least some configurations, where we rely on the presence of a unique random key to generate per-object or per-object+user reply addresses.

This should probably be formalized in an interface since it's currently pretty ad-hoc.

Test Plan:
  - Made comments in Ponder under a per-user email configuration.
  - Ran migration, verified mail keys were generated.
  - Ran migration again (with --apply), verified existing questions were skipped.
  - Created a new question, verified mail key generation.

Reviewers: pieter

Reviewed By: pieter

CC: aran

Maniphest Tasks: T1873

Differential Revision: https://secure.phabricator.com/D3665
2012-10-08 20:14:58 -07:00
vrana
cbde56cdce Properly create xhpast database
Summary:
It isn't deleted by `storage destroy`.

This should be a no-op on current storage because we execute `CREATE DATABASE IF NOT EXISTS`.

Test Plan:
  $ bin/storage destroy --dryrun

Reviewers: nh, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3659
2012-10-08 16:09:20 -07:00
epriestley
cd9d78c107 Allow Fact analysis of commits
Summary: For immutable objects, just use the ID as a cursor.

Test Plan:
  - Analyzed commits from an empty cursor.
  - Checked that cursor was good.
  - Pulled some more commits.
  - Analyzed commits again, verified it only hit the new ones.
  - Verified the graph of "Count of CMIT" looked reasonable.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1866

Differential Revision: https://secure.phabricator.com/D3656
2012-10-08 13:27:06 -07:00
epriestley
ea32c541e4 Remove all optimistic lock code from Lisk
Summary: We never use this and almost certainly never will. It's been in Lisk for ~7 years but is a solution in search of a problem. It causes a conflict with any DAO that has a `version` column.

Test Plan: Browsed around, performed inserts and updates. Edited a Phriction document.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran, leslie.chong

Differential Revision: https://secure.phabricator.com/D3625
2012-10-04 13:55:43 -07:00
epriestley
fe6022cd97 Load only valid properties in LiskDAO::loadFromArray()
Summary:
Fixes a TODO, and silences a warning introduced by D3601.

There are several cases where we load data like:

  SELECT *, ... AS extraData FROM ...

...and then pass it to `loadAllFromArray()`. Currently, this causes us to set an `extraData` property on the object.

This idiom seems fairly useful and non-dangerous, so I made `loadFromArray()` just drop extra keys.

Since we hit this loop a potentially huge number of times (10,000+ for full Maniphest pages) I did some microoptimization. Lisk is hot enough that it's one of the few places where it's worthwhile (see D1291).

Test Plan: Loaded homepage, no longer got warnings about `viewerIsMember` from Project queries. Browsed ~10 apps, didn't see any issues.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3606
2012-10-03 15:42:44 -07:00
vrana
4682e0c104 Warn against writing to undeclared properties
Summary:
I make this error quite often: I forget to declare a property I am writing to or I make a typo in it.
PHP implicitly creates a public property which I don't like.

I would much rather see a linter warning me against this than this runtime check but writing it is very difficult:

- We need to explore all parents of the class we are checking.
- It is even possible that children will declare that property but it's OK to treat this as error anyway.
- We can extend also builtin or external classes.
- It's somewhat doable for `$this` but even more complex for any `$obj` because we don't know the class of it.

This should catch significant part of these errors and I'm fine with that.

I don't plan escalating to exception because this error is not fatal and should not stop the application from working.

Test Plan: Loaded homepage, checked log.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3601
2012-10-03 11:54:03 -07:00
vrana
7c39c4ca7d Declare common Lisk properties
Summary:
Calling `->setPHID()` or other common Lisk setters creates an implicit public property `$phid`.
I don't like implicit properties and I see them as errors.
Its public visibility also makes me nervous and is vulnerable to bypassing any setters we may create.

Test Plan: Loaded homepage, checked log.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3600
2012-10-03 11:53:37 -07:00
Bob Trahan
9e1b643896 Phame - allow blogs to specify custom URIs
Summary: this then enables people to create blog.theircompany.com. And for us, blog.phacility.com...!

Test Plan:
 - created custom URIs of various goodness and verified the error messages were sensical.
 - verified if "false" in configuration then custom uri stuff disappears

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3542
2012-09-30 17:10:27 -07:00
vrana
d119ac672f Remember action in Differential comment draft
Summary:
It happens to me quite often that I leave the window with revision (by closing it or by visiting a link from it).
When I return then the comment draft is there so I clowncopterize it but forget that I wanted to take some other action than Comment.

Test Plan: Selected "Add Reviewers", added some reviewers, closed the window, opened it - the action and reviewers were still there.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3530
2012-09-21 13:05:09 -07:00
vrana
8e7ae7b33a Optimize reindex_everything.php
Summary:
We have two troubles with this script:

# Our revisions and commits don't fit in the memory. (Our tasks do :-).)
# Reindexing revisions is slow.

Test Plan: Ran it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3483
2012-09-17 13:34:41 -07:00
vrana
3c2cb13153 Fix bad rebase
Auditors: nh
2012-09-17 13:29:09 -07:00