1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-28 16:30:59 +01:00
Commit graph

359 commits

Author SHA1 Message Date
James Rhodes
c514d34b94 Add build step implementation infrastructure and sleep build step.
Summary:
Depends on D7498.

This implements support for a "build step implementation".  Build steps have an associated class name (which makes the class in PHP) and a details field, which is serialized JSON (same as PhabricatorRepository).

This also implements a SleepBuildStepImplementation which just pauses the build for a specified period of seconds.

Test Plan:
Inserted a build step with `insert into harbormaster_buildstep (phid, buildPlanPHID, className, details, dateCreated, dateModified) values ('', 'PHID-HMCP-zkh5w6czfbfpk2gxwdeo', 'SleepBuildStepImplementation', '{"seconds":5}', NOW(), NOW());` (adjusting the build plan PHID as appropriate).

Started the daemon and applied the build plan to a buildable, and saw the daemon take a 5 second delay after creating `SleepBuildStepImplementation`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, chad

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7499
2013-11-05 13:34:44 -08:00
epriestley
0278b15ceb Implementation of VCS passwords against user.
Summary: This allows users to set their HTTP access passwords via Diffusion interface.

Test Plan: Clicked the "Set HTTP Access Password" link, set a password and saw it appear in the DB.

Reviewers: #blessed_reviewers, hach-que, btrahan

Reviewed By: hach-que

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7462
2013-11-01 08:34:11 -07:00
epriestley
3a39b01233 Add "RepositoryStatusMessage" and detailed information about initilization
Summary:
`RepositoryStatusMessage` is basically a key/value table associated with a repository that I'm using to let the daemons store the most recent event of a given type, so we can easily show it on the status dashboard. I think this will be a lot easier for users to figure out than digging through logfiles.

I'm also going to write the "this needs a pull" status here eventually, for reducing the time lapse between pushes and discovery.

  - Add storage for these messages.
  - Have the pull engine populate the INIT phase. I'll do the FETCH phase next.
  - Update the status readout to show all the various states.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7461
2013-10-30 16:04:19 -07:00
epriestley
9125095587 Distinguish between empty and unparsed commits in Diffusion
Summary:
Fixes T3416. Fixes T1733.

  - Adds a flag to the commit table showing whether or not we have parsed it.
  - The flag is set to `0` initially when the commit is discovered.
  - The flag is set to `1` when the changes are parsed.
  - The UI can now use the flag to distinguish between "empty commit" and "commit which we haven't imported changes for yet".
  - Simplify rendering code a little bit.
  - Fix an issue with the Message parser for empty commits.
  - There's a key on the flag so we can do `SELECT * FROM repository_commit WHERE repositoryID = %d AND importStatus = 0 LIMIT 1` soon, to determine if a repository is fully imported or not. This will let us improve the UI (Ref T776, Ref T3217).

Test Plan:
  - Ran `bin/storage upgrade -f`.
  - Created an empty commit.
  - Without the daemons running, ran `bin/repository pull GTEST` and `bin/repository discover GTEST`.
  - Viewed web UI to get the first screenshot ("Still Importing...").
  - Ran the message and change steps with `scripts/repository/reparse.php`.
  - Viewed web UI to get the second screenshot ("Empty Commit").

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776, T1733, T3416, T3217

Differential Revision: https://secure.phabricator.com/D7428
2013-10-29 15:32:41 -07:00
epriestley
86fe020a97 Add global "push" policy to Repositories
Summary: No editing or view yet, just adds the schema and a policy default. Part of D7391.

Test Plan: `bin/storage upgrade`

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7415
2013-10-29 15:32:40 -07:00
epriestley
b5a009337f Harbormaster v(-2)
Summary:
Ref T1049. I don't really want to sink too much time into this right now, but a seemingly reasonable architecture came to me in a dream. Here's a high-level overview of how things fit together:

  - **"Build"**: In Harbormaster, "build" means any process we want to run against a working copy. It might actually be building an executable, but it might also be running lint, running unit tests, generating documentation, generating symbols, running a deploy, setting up a sandcastle, etc.
  - `HarbormasterBuildable`: A "buildable" is some piece of code which build operations can run on. Generally, this is either a Differential diff or a Diffusion commit. The Buildable class just wraps those objects and provides a layer of abstraction. Currently, you can manually create a buildable from a commit. In the future, this will be done automatically.
  - `HarbormasterBuildStep`: A "build step" is an individual build operation, like "run lint", "run unit", "build docs", etc. The step defines how to perform the operation (for example, "run unit tests by executing 'arc unit'"). In this diff, this barely exists.
  - `HarbormasterBuildPlan`: This glues together build steps into groups or sequences. For example, you might want to "run unit", and then "deploy" if the tests pass. You can create a build plan which says "run step "unit tests", then run step "deploy" on success" or whatever. In the future, these will also contain triggers/conditions ("Automatically run this build plan against every commit") and probably be able to define failure actions ("If this plan fails, send someone an email"). Because build plans will run commands, only administrators can manage them.
  - `HarbormasterBuild`: This is the concrete result of running a `BuildPlan` against a `Buildable`. It tracks the build status and collects results, so you can see if the build is running/successful/failed. A `Buildable` may have several `Build`s, because you can execute more than one `BuildPlan` against it. For example, you might have a "documentation" build plan which you run continuously against HEAD, but a "unit" build plan which you want to run against every commit.
  - `HarbormasterBuildTarget`: This is the concrete result of running a `BuildStep` against a `Buildable`. These are children of `Build`. A step might be able to produce multiple targets, but generally this is something like "Unit Tests" or "Lint" and has an overall status, so you can see at a glance that unit tests were fine but lint had some issues.
  - `HarbormasterBuildItem`: An optional subitem for a target. For lint, this might be an individual file. For unit tests, an individual test. For normal builds, an executable. For deploys, a server. For documentation generation, there might just not be subitems.
  - `HarbormasterBuildLog`: Provides extra information, like command/execution transcripts. This is where stdout/stderr will get dumped, and general details and other messages.
  - `HarbormasterBuildArtifact`: Stores side effects or results from build steps. For example, something which builds a binary might put the binary in "Files" and then put its PHID here. Unit tests might put coverage information here. Generally, any build step which produces some high-level output object can use this table to record its existence.

This diff implements almost nothing and does nothing useful, but puts most of these object relationships in place. The two major things you can't easily do with these objects are:

  1) Run arbitrary cron jobs. Jenkins does this, but it feels tacked on and I don't know of anyone using it for that. We could create fake Buildables to get a similar effect, but if we need to do this I'd rather do it elsewhere in general. Build and cron/service/monitoring feel like pretty different problems to me.
  2) Run parameterized/matrix steps (maybe?). Bamboo has this plan/stage/task/job breakdown where a build step can generate a zillion actual jobs, like "build client on x86", "build server on x86", "build client on ARM", "build server on ARM", etc. We can sort of do this by having a Step map to multiple Targets, but I haven't really thought about it too much and it may end up being not-great. I'd guess we have like an 80% chance of getting a clean implementation if/when we get there. I suspect no one actually needs this, or when they do they'll just implement a custom Step and it can be parameterized at that level. I'm not too worried about this overall.

The major difference between this and Jenkins/Bamboo/TravisCI is that all three of those are **plan-centric**: the primary object in the system is a build plan, and the dashboard shows you all your build plans and the current status. I don't think this is the right model. One disadvantage is that you basically end up with top-level messaging that says "Trunk is broken", not "Trunk was broken by commit af32f392f". Harbormaster is **buildable-centric**: the primary object in the system is stuff you can run build operations against (commits/branches/revisions), and actual build plans are secondary. The main view will be "recent commits on this branch, and whether they're good or not" -- which I think is what's most important in a larger/more complex product -- not the pass/fail status of all jobs. This also makes it easier and more natural to integrate with Differential and Diffusion, which both care about the overall status of the commit/revision, not the current status of jobs.

Test Plan: Poked around, but this doesn't really do anything yet.

Reviewers: btrahan

Reviewed By: btrahan

CC: zeeg, chad, aran, seporaitis

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7368
2013-10-22 15:01:06 -07:00
epriestley
9b89e137cf Move Project transaction storage to modern tables
Summary:
Ref T4010. Projects have a weird proto-version of ApplicationTransactions which is very similar but not quite the same.

Move the storage to a modern format, but keep all the other code for now.

Test Plan: Migrated project transactions; edited projects.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7370
2013-10-22 13:49:28 -07:00
epriestley
d7a276346f Add a secret board view to Projects
Summary:
Ref T1344. This is //very// rough. Some UI issues:

  - Empty states for the board and columns are junky.
  - Column widths are crazy. I think we need to set them to fixed-width, since we may have an arbitrarily large number of columns?
  - I don't think we have the header UI elements in M10 yet and that mock is pretty old, so I sort of very roughly approximated it.
  - What should we do when you click a task title? Popping the whole task in a dialog is possible but needs a bunch of work to actually work. Might need to build "sheets" or something.
  - Icons are slightly clipped for some reason.
  - All the backend stuff is totally faked.

Generally, my plan is just to use these to implement all of T390. Specifically:

  - "Kanban" projects will have "Backlog" on the left. You'll drag them toward the right as you make progress.
  - "Milestone" projects will have "No Milestone" on the left, then "Milestone 9", "Milestone 8", etc.
  - "Sprint" projects will have "Backlog" on the left, then "Sprint 31", "Sprint 30", etc.

So all of these things end up being pretty much exactly the same, with some minor text changes and new columns showing up on the left vs the right or whatever.

Test Plan: See screenshot.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: chad, aran, sascha-egerer

Maniphest Tasks: T1344

Differential Revision: https://secure.phabricator.com/D7374
2013-10-21 21:11:36 -07:00
epriestley
cf1c06e157 Add custom field storage to Projects
Summary: Ref T4010. Adds storage and indexes for custom fields. These tables are the same as people/maniphest/differential.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7369
2013-10-21 17:01:13 -07:00
epriestley
f010730e49 Migrate all Differential inline comments to ApplicationTransactions
Summary:
Ref T2222. This implements step (1) described there, which is moving over all the inline comments.

The old and new tables are simliar. The only real trick here is that `transactionPHID` and `legacyCommentID` mean roughly the same thing (`null` if the inline is a draft, non-null if it has been submitted) but we don't have real `transactionPHID`s yet. We just make some up -- we'll backfill them later.

Two risks here:

  - I need to take a second look at the keys on this table. I think we need to tweak them a bit, and it will be less disruptive to do that before this migration than after.
  - This will take a while for Facebook, and other large installs with tens of thousands of revisions. I'll communicate this.

I'm otherwise pretty satisfied with this, seems to work well and is pretty low risk / non-disruptive.

Test Plan:
  - Before migrating, then after migrating:
    - Made a bunch of inlines (drafts, submitted).
    - Edited and deleted inlines.
    - Verified inlines showed up in preview.
    - Verified that inlines aren't indexed when they're drafts (`bin/search index D935`).
    - Verified that inlines ARE indexed when they're not drafts.
    - Verified that drafts inlines make revisions appear as "with draft" in the revision list.
  - Made left, right, and draft inlines.
  - Migrated (`bin/storage upgrade`).
  - Verified that my inlines from before the migration still showed up.
  - (Repeated all the stuff above.)
  - Manually inspected the inline comment table.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7139
2013-10-19 05:03:25 -07:00
epriestley
460051c1d8 Drop maniphest_savedquery table
Summary: This data was migrated by D6977 and is now obsolete. I'll hold this patch for a week or two in case we get reports of migration errors.

Test Plan: Ran storage upgrade, saw the table vanish. Grepped for references to the table.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6997
2013-10-19 04:56:17 -07:00
epriestley
3410cbd53e Add application and object level policy controls to Countdown
Summary: Ref T603. Give countdowns proper UI-level policy controls, and an application-level default policy. Put policy information in the header.

Test Plan:
  - Adjusted default policy.
  - Created new countdowns.
  - Edited countdowns.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7322
2013-10-16 10:36:08 -07:00
epriestley
5899ae08b3 Add storage for custom policies
Summary: Ref T603. Allows custom policies to be saved. No integration with policy controls yet.

Test Plan:
  mysql> select * from policy where id = 3\G
  *************************** 1. row ***************************
             id: 3
           phid: PHID-PLCY-e4v2fnbyuibi4supl5tn
          rules: [{"action":"allow","rule":"PhabricatorPolicyRuleAdministrators","value":null},{"action":"allow","rule":"PhabricatorPolicyRuleProjects","value":["PHID-PROJ-cwovm5gn2ilubjehcdgd"]},{"action":"allow","rule":"PhabricatorPolicyRuleLunarPhase","value":"new"}]
  defaultAction: deny
    dateCreated: 1381437466
   dateModified: 1381437466
  1 row in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7282
2013-10-10 16:09:51 -07:00
epriestley
953ff197bf Allow Herald rules to be disabled, instead of deleted
Summary:
Ref T603. Ref T1279. Further improves transaction and policy support for Herald.

  - Instead of deleting rules (which wipes out history and can't be undone) allow them to be disabled.
  - Track disables with transactions.
  - Gate disables with policy controls.
  - Show policy and status information in the headers.
  - Show transaction history on rule detail screens.
  - Remove the delete controller.
  - Support disabled queries in the ApplicationSearch.

Test Plan:
  - Enabled and disabled rules.
  - Searched for enabled/disabled rules.
  - Verified disabled rules don't activate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279, T603

Differential Revision: https://secure.phabricator.com/D7247
2013-10-06 17:10:29 -07:00
epriestley
65ddefad8b Migrate all Differential reviewer data to edges
Summary:
Ref T1279. @champo did a lot of this work already; we've been doing double writes for a long time.

Add "double reads" (reading the edge table as both the "relationship" table and as the "reviewer status" table), and migrate all the data.

I'm not bothering to try to recover old reviewer status (e.g., we could infer from transactions who accepted old revisions) because it wold be very complicated and doesn't seem too valuable.

Test Plan:
  - Without doing the migration, used Differential. Verified that reads and writes worked. Most of the data was there anyway since we've been double-writing.
  - Performed the migration. Verified that everything was still unchanged.
  - Dropped the edge table, verified all reviweer data vanished.
  - Migrated again, verified the reviewer stuff was restored.
  - Did various cc/reviewer/subscriber queries, got consistent results.

Reviewers: btrahan

Reviewed By: btrahan

CC: champo, aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7227
2013-10-05 13:54:02 -07:00
epriestley
bd8071bb33 Add a (dst, type, src) key to Differential's edge table
Summary:
Ref T1279. This came to me in a dream.

The existing `differential_relationship` table has an `(objectPHID, type)` column, which theoretically is useful for queries like "revisions with X as a reviewer". In practice, I'm not sure it gets used much, but I can get it to show up in at least some query plans.

Add a similar index to the `edge` table. This sequences //before// D7227, which actually migrates the data.

Test Plan:
  - Ran `bin/storage upgrade`.
  - EXPLAIN'd a bunch of queries against different versions of the schema, this seemed helpful overall.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7232

Conflicts:
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-10-05 13:53:43 -07:00
epriestley
7dde01df76 Fix issues with first-time account registration
Summary: This worked originally, but the migration broke slightly after the
config was deprecated, and there was another minor issue during setup.
2013-10-05 08:02:41 -07:00
epriestley
98bf001a58 Add viewPolicy and attachedToObjectPHID to PhabricatorFile
Summary:
Ref T603. Principally, I want to implement the rule "when you upload a file to an object, users must be able to see the object in order to see the file", since I think this is strongly in line with user expectation. For example, if you attach a file to a Conpherence, it should only be visible to members of that thread.

This adds storage for policies, but doesn't do anything interesting with it yet.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7175
2013-10-01 08:45:18 -07:00
epriestley
7091dad606 Allow macros to have associated audio and audio behaviors
Summary: Ref T3887. Implements storage and editors, but not the actual audio part.

Test Plan: Edited audio, audio behaviors of macros. Transactions and email looked good. Hit error cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3887

Differential Revision: https://secure.phabricator.com/D7159
2013-09-27 16:01:37 -07:00
epriestley
3cb67480e0 Adjust keys for new Differential inline comment table
Summary:
Ref T2222. This sequences //before// D7139 and sorts out keys on the table. In particular:

  - There was a fairly silly `draft` key modeled after Pholio; drop it.
  - Add a `revisionPHID` key. This is queried mostly-transitionally on the revision view screen.
  - Add a `changesetID` key. This is queried by a bunch of interfaces that want more surgical results than `revisionPHID` provides.
  - Add an `authorPHID, transactionPHID` key. This is queried on the list interface to find pending drafts.
  - Add a `legacy` key. This is queried by the feed publisher.

Test Plan: Used the query analyzer to hit all (I think?) of the pages, saw keyed queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7140
2013-09-26 13:48:36 -07:00
epriestley
5677cd23bd Add storage and classes for CustomField in Differential
Summary: Ref T3886. Adds the storage, indexes, and storage classes for modernizing Differential custom fields.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886

Differential Revision: https://secure.phabricator.com/D7138
2013-09-26 12:37:28 -07:00
epriestley
c458517cb4 Add viewPolicy, editPolicy, repositoryPHID columns to DifferentialRevision
Summary: Ref T603. Paves the way for policy controls.

Test Plan: Ran storage upgrade, bumbled around in Differential.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7133
2013-09-26 12:36:30 -07:00
epriestley
225a38c7d3 Add viewPolicy, editPolicy storage to tasks
Summary: Ref T603. Adds storage for custom policies.

Test Plan: Ran storage upgrade. Created and edited tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7118
2013-09-25 11:18:47 -07:00
epriestley
960da82f33 Fix two migration issues with Maniphest
Summary: The new editor saves tasks before applying transactions, so fields need default values.
Also, I goofed the table name.

Auditors: btrahan
2013-09-24 12:03:47 -07:00
epriestley
6e29e809b4 Fix "edit" for Maniphest comments
Summary:
Ref T2217. Fixes T3876. We incorrectly have a unique key on `(authorPHID, transactionPHID)`, which prevents saving multiple versions of a comment.

I'm not entirely sure why this exists. I think it came from Pholio (where it works for inlines, because it has an additional component, but maybe should be adjusted) but we might need to wipe it out of more apps too.

Test Plan: Edited a comment in Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217, T3876

Differential Revision: https://secure.phabricator.com/D7102
2013-09-24 11:10:31 -07:00
epriestley
1d7cbe202f Rename "transactionpro" table to "transaction"
Summary: Ref T2217. Cleans up the table names. Moves old data to `maniphest_transaction_legacy`. We'll drop that eventually once it's more clear that I didn't break the world.

Test Plan: Did reads/writes to/from these tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7094
2013-09-24 10:49:16 -07:00
epriestley
01ecf1a236 Rename ManiphestTransactionPro -> ManiphestTransaction
Summary: Ref T2217. Pro is the new standard.

Test Plan: Lots of `grep`, made a pile of Maniphest views/edits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7093
2013-09-24 10:49:06 -07:00
epriestley
7abe9dc4c0 Migrate all Maniphest transaction data to new storage
Summary:
Ref T2217. This is the risky, hard part; everything after this should be smooth sailing. This is //mostly// clean, except:

  - The old format would opportunistically combine a comment with some other transaction type if it could. We no longer do that, so:
    - When migrating, "edit" + "comment" transactions need to be split in two.
    - When editing now, we should no longer combine these transaction types.
    - These changes are pretty straightforward and low-impact.
  - This migration promotes "auxiliary field" data to the new CustomField/StandardField format, so that's not a straight migration either. The formats are very similar, though.

Broadly, this takes the same attack that the auth migration did: proxy all the code through to the new storage. `ManiphestTransaction` is now just an API on top of `ManiphestTransactionPro`, which is the new storage format. The two formats are very similar, so this was mostly a straight copy from one table to the other.

Test Plan:
  - Without performing the migration, made a bunch of edits and comments on tasks and verified the new code works correctly.
  - Droped the test data and performed the migration.
  - Looked at the resulting data for obvious discrepancies.
  - Looked at a bunch of tasks and their transaction history.
  - Used Conduit to pull transaction data.
  - Edited task description and clicked "View Details" on transaction.
  - Used batch editor.
  - Made a bunch more edits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7068
2013-09-23 14:25:28 -07:00
epriestley
4c2bd2ca8e Add storage for new Maniphest transactions
Summary: Ref T2217. Add the tables and comment class for the new stuff. Not used yet.

Test Plan: Ran storage upgrade, browsed Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7066
2013-09-23 14:24:58 -07:00
epriestley
b8cb6ddaa5 Add some keys and policy fields to repositories
Summary:
  - Add some TODO'd keys.
  - Add policy fields.

Test Plan: Viewed repositories; created a new repository and verified it got the right default policy settings.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7056
2013-09-21 16:23:01 -07:00
epriestley
0558d53273 Convert maniphest to use standard fields
Summary: Ref T3794. Drop auxiliary field, use standard field.

Test Plan: Performed migration, field seemed to survive it intact. Edited and viewed tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3794

Differential Revision: https://secure.phabricator.com/D7036
2013-09-19 11:56:15 -07:00
epriestley
bd40e74400 Migrate auxiliary field storage to common field storage
Summary: Ref T418. Moves data from the Maniphest-specific table to the general one. This patch is a bit gross, but mostly about getting the reads and writes aimed correctly. Future patches will clean things up.

Test Plan: Migrated data across formats. Verified it survied the migration. Viewed and edited tasks' custom fields.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6999
2013-09-16 16:02:06 -07:00
epriestley
b0e145f2fe Provide generalized custom field storage and custom field indexes in Maniphest
Summary: Ref T418. Depends on D6992. This adds index and value storage for Maniphest custom fields.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6995
2013-09-16 14:06:41 -07:00
epriestley
c8574cf6fd Integrate ApplicationSearch with CustomField
Summary:
Ref T2625. Ref T3794. Ref T418. Ref T1703.

This is a more general version of D5278. It expands CustomField support to include real integration with ApplicationSearch.

Broadly, custom fields may elect to:

  - build indicies when objects are updated;
  - populate ApplicationSearch forms with new controls;
  - read inputs entered into those controls out of the request; and
  - apply constraints to search queries.

Some utility/helper stuff is provided to make this easier. This part could be cleaner, but seems reasonable for a first cut. In particular, the Query and SearchEngine must manually call all the hooks right now instead of everything happening magically. I think that's fine for the moment; they're pretty easy to get right.

Test Plan:
I added a new searchable "Company" field to People:

{F58229}

This also cleaned up the disable/reorder view a little bit:

{F58230}

As it did before, this field appears on the edit screen:

{F58231}

However, because it has `search`, it also appears on the search screen:

{F58232}

When queried, it returns the expected results:

{F58233}

And the actually good bit of all this is that the query can take advantage of indexes:

  mysql> explain SELECT * FROM `user` user JOIN `user_customfieldstringindex` `appsearch_0` ON `appsearch_0`.objectPHID = user.phid AND `appsearch_0`.indexKey = 'mk3Ndy476ge6' AND `appsearch_0`.indexValue IN ('phacility') ORDER BY user.id DESC LIMIT 101;
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  | id | select_type | table       | type   | possible_keys     | key      | key_len | ref                                      | rows | Extra                                        |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  |  1 | SIMPLE      | appsearch_0 | ref    | key_join,key_find | key_find | 232     | const,const                              |    1 | Using where; Using temporary; Using filesort |
  |  1 | SIMPLE      | user        | eq_ref | phid              | phid     | 194     | phabricator2_user.appsearch_0.objectPHID |    1 |                                              |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  2 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418, T1703, T2625, T3794

Differential Revision: https://secure.phabricator.com/D6992
2013-09-16 13:44:34 -07:00
epriestley
5e274778d1 Restore one missing "order" value to the saved query migration script
Summary: Caught this when migrating my queries in production.

Auditors: btrahan
2013-09-13 12:00:50 -07:00
epriestley
9864d562dd Migrate old Maniphest custom queries to new query infrastructure
Summary: Ref T2625. EVERYONE LOVES MIGRATIONS!!!

Test Plan:
  - Created and migrated a query with every field, verified results were preserved.
  - Created and migrated a query using "noproject" and "upforgrabs" magic, verified results were preserved.

Here's the pre-migration "everything" query:

{F58110}

Here it is after migration:

{F58111}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6977
2013-09-13 11:49:45 -07:00
epriestley
b38a688cf7 Ignore failures when rebuilding project search index in Maniphest migration
Summary: See <https://github.com/facebook/phabricator/issues/386>. Just ignore any issues here, they aren't worth complaining about.

Auditors: btrahan
2013-09-13 09:39:46 -07:00
epriestley
e50eccf109 Provide and populate an object name index for Maniphest
Summary: See discussion in D6955. This provides a table we can JOIN against to (effectively) "ORDER BY project name", populates it intially, and keeps it up to date as projects are edited.

Test Plan:
  - Ran storage upgrade, verified projects populated into the table.
  - Edited a project, verified its entry updated.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6957
2013-09-12 13:06:44 -07:00
epriestley
df86f87289 Add a "dateCreated" key to Maniphest
Summary: Depends on D6952. Unpunts there since I'm rolling into a swamp full of schema changes.

Test Plan: Issued date-constrained query and saw key as a candidate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6954
2013-09-12 13:04:31 -07:00
epriestley
8f8c61be31 Remove legacy "touched" table and indexing
Summary: Noticed this in the schema. "Touches" were an idea that never really got off the ground, as we built out more/better notification channels instead. Essentially, they recorded any object you'd ever interacted with. Maybe this will be useful some day, but for now it does nothing and can't be interacted with. Nuke it.

Test Plan: `grep`, loaded Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6953
2013-09-12 13:04:09 -07:00
epriestley
2d240142c8 Fix 20130820.file-mailkey-populate.php for databases in strict mode
Summary:
Fixes T3803. Turns out my advice was sort of terrible. :/

In strict mode, the `INSERT INTO x (y, z)` raises an error unless `(y, z, ...)` includes //all// columns without default values.

Test Plan: Ran `storage upgrade` on a strict-mode install. Verified no inserts were performed.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3803

Differential Revision: https://secure.phabricator.com/D6894
2013-09-05 15:58:28 -07:00
Bob Trahan
228496cdbe File - add transactions and editor
Summary: this ends up being a little weird since you can't actually edit files. Also, since we create files all sorts of ways, sometimes without even having a user, we don't  bother logging transactions for those events. Fixes T3651. Turns out this work is important for T3612, which is a priority of mine to help get Pholio out the door.

Test Plan: left a comment on a file. it worked! use bin/mail to verify mail content looked correct.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran, wez

Maniphest Tasks: T3651, T3612

Differential Revision: https://secure.phabricator.com/D6789
2013-09-05 13:11:02 -07:00
epriestley
41ac06959e Generate some amount of PHP class documentation
Summary:
Ref T988. This brings the class/interface atomizer over. A lot of parts of this are still varying degrees of very-rough, but most of the data ends up in approximatley the right place.

ALSO: PROGRESS BARS

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6817
2013-08-28 09:54:39 -07:00
epriestley
49e0aa203c Add storage for Releeph project and branch transactions
Summary: Ref T3663. Does what it says on the tin.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3663

Differential Revision: https://secure.phabricator.com/D6778
2013-08-21 12:31:51 -07:00
epriestley
973bb0c76e Remove ReleephRequestEvent
Summary: Ref T3663. This is obsolete code which is used only in this migration, which Facebook has already performed and which isn't relevant for any other installs.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3663

Differential Revision: https://secure.phabricator.com/D6777
2013-08-21 12:31:50 -07:00
epriestley
ca0115b361 Support configuration-driven custom fields
Summary:
Ref T1702. Ref T3718. There are a couple of things going on here:

**PhabricatorCustomFieldList**: I added `PhabricatorCustomFieldList`, which is just a convenience class for dealing with lists of fields. Often, current field code does something like this inline in a Controller:

  foreach ($fields as $field) {
    // do some junk
  }

Often, that junk has some slightly subtle implications. Move all of it to `$list->doSomeJunk()` methods (like `appendFieldsToForm()`, `loadFieldsFromStorage()`) to reduce code duplication and prevent errors. This additionally moves an existing list-convenience method there, out of `PhabricatorPropertyListView`.

**PhabricatorUserConfiguredCustomFieldStorage**: Adds `PhabricatorUserConfiguredCustomFieldStorage` for storing custom field data (like "ICQ Handle", "Phone Number", "Desk", "Favorite Flower", etc).

**Configuration-Driven Custom Fields**: Previously, I was thinking about doing these with interfaces, but as I thought about it more I started to dislike that approach. Instead, I built proxies into `PhabricatorCustomField`. Basically, this means that fields (like a custom, configuration-driven "Favorite Flower" field) can just use some other Field to actually provide their implementation (like a "standard" field which knows how to render text areas). The previous approach would have involed subclasssing the "standard" field and implementing an interface, but that would mean that every application would have at least two "base" fields and generally just seemed bleh as I worked through it.

The cost of this approach is that we need a bunch of `proxy` junk in the base class, but that's a one-time cost and I think it simplifies all the implementations and makes them a lot less magical (e.g., all of the custom fields now extend the right base field classes).

**Fixed Some Bugs**: Some of this code hadn't really been run yet and had minor bugs.

Test Plan:
{F54240}
{F54241}
{F54242}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1702, T1703, T3718

Differential Revision: https://secure.phabricator.com/D6749
2013-08-14 12:33:53 -07:00
epriestley
9f41032693 Remove "cut point commit identifier" from Releeph
Summary: Ref T3656. Releeph denormalizes branch cut point identifiers into Branch objects, but this information isn't useful or used for sorting, filtering, or enforcing unique constraints. Instead, derive it via noramlized pathways from the `cutPointCommitPHID`.

Test Plan: Ran storage upgrade. Ran `releephwork.getbranch` and `releeph.getbranches`. Grepped for `cutPointCommitIdentifier`.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3656

Differential Revision: https://secure.phabricator.com/D6636
2013-08-14 09:01:38 -07:00
epriestley
a84cc777c8 Remove "Project ID" from Releeph Projects
Summary:
Fixes T3660. Releeph Projects currently have an unused one-to-one mapping to Phabricator projects. This isn't consistent with other applications and has no integrations or uses. Get rid of it.

NOTE: Waiting for signoff from @legneato on T3660 before pulling the trigger here.

Test Plan: Created and edited Releeph projects. Grepped for references to project ID; there are a dozen or so but they're all either Releeph projects or Arcanist projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3660

Differential Revision: https://secure.phabricator.com/D6635
2013-08-14 09:00:56 -07:00
epriestley
296818e129 Remove ReleephProject->repositoryID writes
Summary: Ref T3655. Depends on D6633. This removes the writes and the column.

Test Plan: Created a project, edited a project. Verified the table doesn't have any keys including this column.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3655

Differential Revision: https://secure.phabricator.com/D6634
2013-08-14 09:00:25 -07:00
epriestley
b1c4a258c9 Add ApplicationTransactions to Herald
Summary: Ref T2769. I'm planning to keep this pretty simple, but we have this ad-hoc edit log for rules already and some other mess that we can clean up.

Test Plan: No effect yet; see future changes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6654
2013-08-07 18:03:49 -07:00