1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-22 13:30:55 +01:00
Commit graph

411 commits

Author SHA1 Message Date
epriestley
8f9b7f4196 Move Differential to proper subscriptions
Summary:
Ref T2222. Ref T4415. We're still writing Differential subscription stuff into this weird legacy `differential_relationship` table, which is like an edge table but extremely ancient.

Move it into a proper table.

I've removed `withSubscriptions()` from `DifferentialRevisionQuery`. It was weird, doesn't work consistently with other similar filters, and was only used by the API. Now it means "ccs", which is consistent with the ApplicationSearch UI and with Maniphest.

Test Plan:
Without migrating, added and removed subscribers via various workflows. Queried for subscribers. Everything worked as expected.

Ran the migration, verified data survived.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Maniphest Tasks: T2222, T4415

Differential Revision: https://secure.phabricator.com/D8202
2014-02-12 08:53:40 -08:00
epriestley
305fb3fbd9 Migrate all Differential comment text into new storage
Summary:
Ref T2222. Currently, `DifferentialComment` stores both (a) the text of comments and (b) various other transaction details. This data needs to map to both Transactions and TransactionComments in the long run. This diff separates out all the data which is bound for the TransactionComment table, so that when we migrate `DifferentialComment` itself it will //only// need to migrate into the Transactions table. This is a much simpler migration than the inline comment one was, partly because it set up infrastructure and partly because the data is less complex.

Basically, I'm just proxying the read/write for the comment text into the other table. All readers already go through the Query class, and there are only three writers (preview, comment, implicit comment on diff update) which are all highly regular and straightforward to test.

We can also back out of this diff very easily: doing double writes cost only one line of code (`$this->content = $content;`) so we have proper double writes and a trivial revert path.

Test Plan:
  - Without migrating, added comments and saw them show up.
  - Migrated.
  - Saw all the old comments, and no damage to the new ones.
  - Added new comments.
  - Used comment preview.
  - Updated a revision to implicitly create an update comment and verified it looked OK.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8196
2014-02-11 11:34:15 -08:00
epriestley
b1243e549c Allow unsubscription from projects
Summary:
Fixes T4379. Several changes:

  - Migrate all project members into subscribers.
  - When members are added or removed, subscribe or unsubscribe them.
  - Show sub/unsub in the UI.
  - Determine mailable membership of projects by querying subscribers.

Test Plan:
  - As `duck`, joined a project.
  - Added the project as a reviewer to a revision.
  - Commented on the revision.
  - Observed `duck` receive mail.
  - Unsubscribed as `duck`.
  - Observed no mail.
  - Resubscribed as `duck`.
  - Mail again.
  - Joined/left project, checked sub/unsub status.
  - Ran migration, looked at database.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, asherkin

Maniphest Tasks: T4379

Differential Revision: https://secure.phabricator.com/D8189
2014-02-11 07:45:56 -08:00
Bob Trahan
1830868007 Herald - make herald condition of herald rule display better
Summary: ...by the surprising step of changing how this data is stored from id to phid. Also a small fix to not allow "disabled" rules to be used as herald rule conditions, i.e. can't make a rule that depends on a disabled rule.

Test Plan: viewed existing herald rule that had a rule condition and noted nice new display using handle. made a new rule that had a rule condition and verified it worked correctly.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8186
2014-02-10 14:40:09 -08:00
epriestley
8c84ed61b1 Migrate project profiles onto projects, and remove ProjectProfile object
Summary:
Ref T4379. Long ago, the "Project" vs "ProjectProfile" split was intended to allow a bunch of special fields on projects without burdening the simple use cases, but CustomField handles that far better and far more generally, and doing this makes using ApplicationTransactions a pain to get right, so get rid of it.

The only remaining field is `profileImagePHID`, which we can just move to the main Project object. This is custom enough that I think it's reasonable not to express it as a custom field.

Test Plan: Created a project, set profile, edited project, viewed in typeahead, ran migration, verified database results.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4379

Differential Revision: https://secure.phabricator.com/D8183
2014-02-10 14:32:14 -08:00
epriestley
1520065ec0 Migrate project blurb/description to standard custom field storage
Summary: Ref T4379. Major goal here is to remove `ProjectProfile` so all edits use ApplicationTransactions. This also makes things more flexible, allowing users to disable this field if they don't like it.

Test Plan: Ran migration, verified data survived, edited/created projects, reordered fields.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4379

Differential Revision: https://secure.phabricator.com/D8182
2014-02-10 14:31:57 -08:00
epriestley
b13a51adeb Fix inadvertent forward dependency in task owner migration
Summary:
See <https://github.com/facebook/phabricator/issues/505>. When the status/event table moved, it broke this migration, which implicitly loads statuses by loading events.

Instead, access just the row we care about.

Test Plan: Used `--apply` to apply the new version of the patch.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D8162
2014-02-07 09:17:11 -08:00
epriestley
eef2727ea6 Fix accidental forward dependency in old commit summary migration
Summary: See IRC. This migration inadvertently depends on the columns in the commit table, because it calls `save()`, and thus broke for installs with data after we added the `importStatus` column. Since that was ~9 months after this patch, probably not many installs are affected.

Test Plan: Ran patch locally with `--apply` on data. Had user verify fix.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8152
2014-02-06 10:33:54 -08:00
epriestley
4c7acbafac Assign PHIDs to calendar events
Summary: Ref T4375. We're going to need these for a bunch of infrastructure to work.

Test Plan: Ran migrations, checked DB, used `phid.query`.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4375

Differential Revision: https://secure.phabricator.com/D8151
2014-02-06 10:10:43 -08:00
epriestley
ab636f36bf Rename PhabricatorUserStatus to PhabricatorCalendarEvent
Summary: Ref T4375. We never join this table, so this is a pretty straight find/replace.

Test Plan: Browsed around Calendar, verified that nothing seemed broken. Saw my red dot in other apps.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4375

Differential Revision: https://secure.phabricator.com/D8145
2014-02-06 10:07:29 -08:00
epriestley
cb605ad5ee Add edit/view plumbing for dashboards and panels
Summary: Ref T3583. This doesn't add any dashboard/panel-specific code beyond headers/titles/buttons/etc., but allows you to create and view dashboards and panel skeletons.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8131
2014-02-03 10:52:15 -08:00
epriestley
88436349b8 Add a GC for sent and received mail
Summary: Ref T4368. We don't currently GC these tables, and the sent mail table is one of the largest on `secure.phabricator.com`. There's no value in retaining this information indefinitely. Instead, retain it for 90 days, which should be plenty of time to debug/diagnose any issues.

Test Plan: Ran `phd debug garbage`, saw it clean up a reasonable amount of data from these tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4368

Differential Revision: https://secure.phabricator.com/D8127
2014-02-03 10:51:31 -08:00
epriestley
4b0ef353e4 Remove retry/failure mechanisms from MetaMTA
Summary:
Fixes T4202. We have old code in MetaMTA which implements gradual backoff and maximum retries.

However, we have more general code in the task queue which does this, too. We can just use the more general stuff in the task queue; it obsoletes the specific stuff in MetaMTA, which is more complex and ran into some kind of issue in T4202.

Remove `retryCount`, `nextRetry` (obsoleted by task queue retry mechanisms) and "simulated failures" (no longer in use).

Generally, modern infrastructure has replaced these mechanisms with more general ones.

Test Plan:
  - Sent mail.
  - Observed unsendable mail failing in reasonable ways in the queue.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4202

Differential Revision: https://secure.phabricator.com/D8115
2014-02-01 14:35:42 -08:00
epriestley
73924dfa18 Add initial skeleton for Dashboard application
Summary:
Ref T3583. General idea here is:

  - Users will be able to create `DashboardPanel`s, which are things like the jump nav, or a minifeed, or recent assigned tasks, or recent tokens given, or whatever else.
  - The `DashboardPanel`s can be combined into `Dashboard`s, which select specific panels and arrange them in some layout (and maybe have a few other options eventually).
  - Then, you'll be able to set a specific `Dashboard` for your home page, and maybe for project home pages. But you can also use `Dashboard`s on their own if you just like dashboards.

My plan is pretty much:

  - Put in basic infrastructure for dashboards (this diff).
  - Add basic create/edit (next few diffs).
  - Once dashboards sort of work, do the homepage integration.

This diff does very little: you can't create dashboards or panels yet, and thus there are no dashboards to look at. This is all skeleton code, pretty much.

IMPORTANT: We need an icon bwahahahahaha

Test Plan:
omg si purrfect

{F106367}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8109
2014-01-30 11:43:24 -08:00
epriestley
014a873773 Update DifferentialDiff: add repositoryPHID, drop parentRevisionID
Summary:
Moves away from ArcanistProjects:

  - Adds storage for diffs to be directly associated with a repository (instead of indirectly, through arcanist projects). Not really populated yet.
  - Drops `parentRevisionID`, which is obsoleted by the "Depends On" edge. This is not exposed in the UI anywhere and doesn't do anything. Resolves TODO.

Test Plan: Ran storage upgrades, browsed around, lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8072
2014-01-26 15:29:22 -08:00
epriestley
a9e11ed8c1 Update quickstart SQL
Summary: Ref T4327. The change parser unit tests need database fixtures, which are getting a bit slow to build. Speed them up by updating the quickstart.

Test Plan: Initialized new storage via quickstart, clicked around, everything seemed to work properly. Ran all unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8004
2014-01-17 16:11:16 -08:00
epriestley
f4b9efe256 Introduce ref cursors for repository parsing
Summary:
Ref T4327. I want to make change parsing testable; one thing which is blocking this is that the Git discovery process is still part of `PullLocal` daemon instead of being part of `DiscoveryEngine`. The unit test stuff which I want to use for change parsing relies on `DiscoveryEngine` to discover repositories during unit tests.

The major reason git discovery isn't part of `DiscoveryEngine` is that it relies on the messy "autoclose" logic, which we never implemented for Mercurial. Generally, I don't like how autoclose was implemented: it's complicated and gross and too hard to figure out and extend.

Instead, I want to do something more similar to what we do for pushes, which is cleaner overall. Basically this means remembering the old branch heads from the last time we parsed a repository, and figuring out what's new by comparing the old and new branch heads. This should give us several advantages:

  - It should be simpler to understand than the autoclose stuff, which is pretty mind-numbing, at least for me.
  - It will let us satisfy branch and tag queries cheaply (from the database) instead of having to go to the repository. We could also satisfy some ref-resolve queries from the database.
  - It should be easier to extend to Mercurial.

This implements the basics -- pretty much a table to store the cursors, which we update only for Git for now.

Test Plan:
  - Ran migration.
  - Ran `bin/repository discover X --trace --verbose` on various repositories with branches and tags, before and after modifying pushes.
  - Pushed commits to a git repo.
  - Looked at database tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7982
2014-01-17 11:48:53 -08:00
epriestley
2ec45d42a6 Remove session limits and sequencing
Summary:
Ref T4310. Fixes T3720. This change:

  - Removes concurrent session limits. Instead, unused sessions are GC'd after a while.
  - Collapses all existing "web-1", "web-2", etc., sessions into "web" sessions.
  - Dramatically simplifies the code for establishing a session (like omg).

Test Plan: Ran migration, checked Sessions panel and database for sanity. Used existing session. Logged out, logged in. Ran Conduit commands.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4310, T3720

Differential Revision: https://secure.phabricator.com/D7978
2014-01-15 17:27:59 -08:00
Bob Trahan
d740374cca Legalpad - add policy rule for legalpad document signatures
Summary:
Ref T3116. This creates a policy rule where you can require a signature on a given legalpad document.

NOTE: signatures must be for the *latest* document version.

Test Plan: made a task have a custom policy requiring a legalpad signature. verified non-signers were locked out.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3116

Differential Revision: https://secure.phabricator.com/D7977
2014-01-15 16:48:44 -08:00
epriestley
acb141cf52 Expire and garbage collect unused sessions
Summary:
Ref T3720. Ref T4310. Currently, we limit the maximum number of concurrent sessions of each type. This is primarily because sessions predate garbage collection and we had no way to prevent the session table from growing fairly quickly and without bound unless we did this.

Now that we have GC (and it's modular!) we can just expire unused sessions after a while and throw them away:

  - Add a `sessionExpires` column to the table, with a key.
  - Add a GC for old sessions.
  - When we establish a session, set `sessionExpires` to the current time plus the session TTL.
  - When a user uses a session and has used up more than 20% of the time on it, extend the session.

In addition to this, we could also rotate sessions, but I think that provides very little value. If we do want to implement it, we should hold it until after T3720 / T4310.

Test Plan:
  - Ran schema changes.
  - Looked at database.
  - Tested GC:
    - Started GC.
    - Set expires on one row to the past.
    - Restarted GC.
    - Verified GC nuked the session.
  - Logged in.
  - Logged out.
  - Ran Conduit method.
  - Tested refresh:
    - Set threshold to 0.0001% instead of 20%.
    - Loaded page.
    - Saw a session extension ever few page loads.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4310, T3720

Differential Revision: https://secure.phabricator.com/D7976
2014-01-15 13:56:16 -08:00
epriestley
a64228b03f Give the session table a normal id column as a primary key
Summary:
Ref T4310. Ref T3720. Two major things are going on here:

  - I'm making this table work more like a standard table, which, e.g., makes `delete()` simpler to implement.
  - Currently, the primary key is `(userPHID, type)`. I want to get rid of this, issue unlimited sessions, and GC old sessions. This means we can't have a unique key on `(userPHID, type)` anymore. This removes it as the primary key and adds it as a normal key instead. There's no functional change -- the code to generate sessions guarantees that it will never write duplicate rows or write additional rows -- but allows us to drop the `-1`, `-2` qualifiers in the future.
  - Also of note, our task is made far simpler here because MySQL will automatically assign values to new `AUTO_INCREMENT` columns, so we don't have to migrate to get real IDs.

Test Plan: Ran migrations, verified table looked sane. Logged out, logged in.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3720, T4310

Differential Revision: https://secure.phabricator.com/D7975
2014-01-15 13:55:18 -08:00
Bob Trahan
41d2a09536 Legalpad - make it work for not logged in users
Summary: Adds "verified" and "secretKey" to Legalpad document signatures. For logged in users using an email address they own, things are verified right away. Otherwise, the email is sent a verification letter. When the user clicks the link the signature is marked verified.

Test Plan: signed the document with a bogus email address not logged in. verified the email that would be sent looked good from command line. followed link and successfully verified bogus email address

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran, asherkin

Maniphest Tasks: T4283

Differential Revision: https://secure.phabricator.com/D7930
2014-01-14 17:17:18 -08:00
Mikael Knutsson
5417f91b77 Adding the create flow for Project Board (Workphlow) columns.
Summary: This adds in the create flow for the Project board columns on the super secret board page which totally doesn't do anything right now.

Test Plan:
1. Apply diff.
2. Go to super secret page.
3. Click link close to top with a way too long name.
4. Enter a name for the column.
5. Enjoy a new column briefly before realising you cannot remove it.
6. Stay happy!

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: tmaroschik, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7925
2014-01-09 16:12:11 -08:00
epriestley
c020837397 Add transactions to Drydock blueprint editing
Summary: Ref T2015. Fixes TODO.

Test Plan: {F100338}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7922
2014-01-09 12:19:54 -08:00
epriestley
962aca664f Add names to Drydock blueprints
Summary:
Ref T2015. Adds human-readable names to Drydock blueprints.

Also the new patches stuff is so much nicer.

Test Plan: Edited, created, and reviewed migrated blueprints.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7918
2014-01-09 10:56:34 -08:00
epriestley
1786093c6e Replace "Cancel Build" with "Stop", "Resume" and "Restart"
Summary:
Ref T1049. Currently you can cancel a build, but now that we're tracking a lot more state we can stop, resume, and restart builds.

When the user issues a command against a build, I'm writing it into an auxiliary queue (`HarbormasterBuildCommand`) and then reading them out in the worker. This is mostly to avoid race messes where we try to `save()` the object in multiple places: basically, the BuildEngine is the //only// thing that writes to Build objects, and it holds a lock while it does it.

Test Plan:
  - Created a plan which runs "sleep 2" a bunch of times in a row.
  - Stopped, resumed, and restarted it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7892
2014-01-06 12:32:20 -08:00
epriestley
6abe65bfdc Add mailKey to macros
Summary:
If you have private replies on and a Macro reply handler set, we try to access `getMailKey()` and fail. See P1039 for a trace.

(Thanks to @Korvin for picking this up.)

Test Plan: Set configuration, repro'd the exception, applied the patch, then disabled/enabled a macro.

Reviewers: btrahan

Reviewed By: btrahan

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7896
2014-01-06 12:17:23 -08:00
epriestley
09341be10f Remove repository shortcuts
Summary:
Repositories currently have a no-UI "shortcut" feature which is only used by Facebook (and I'm not sure it's even used). As implemented, this feature is policy-oblivious and kind of nonsensical. Throw it away.

I'm open to reimplementing this, but I want to see some level of interest in it before I do. The new implementation would add shortcuts to each repository, similar to how mirrors work. My original plan was to follow this up with such an implementation (it's half-implemented in my sandbox), but as I worked through it I'm not sure it's really valuable.

Test Plan: Browsed repository list, grep.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Differential Revision: https://secure.phabricator.com/D7862
2014-01-02 11:59:27 -08:00
epriestley
f5fb3f05dc Lay most groundwork for Herald object rules
Summary:
Ref T4264. This gets most of the plumbing in for "object" rules, which will bind to a specific object, like a repository or project.

It does not yet let you actually create these rules.

Test Plan: Ran `storage upgrade`, created/edited rules, browsed Herald.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4264

Differential Revision: https://secure.phabricator.com/D7847
2013-12-27 13:17:10 -08:00
epriestley
ac19c55822 Formalize "manual" buildables in Harbormaster
Summary:
Ref T1049. Generally, it's useful to separate test/trial/manual runs from production/automatic runs.

For example, you don't want to email a bunch of people that the build is broken just because you messed something up when writing a new build plan. You'd rather try it first, then promote it into production once you have some good runs.

Similarly, test runs generally should not affect the outside world, etc. Finally, some build steps (like "wait for other buildables") may want to behave differently when run in production/automation than when run in a testing environment (where they should probably continue immediately).

So, formalize the distinction between automatic buildables (those created passively by the system in response to events) and manual buildables (those created explicitly by users). Add filtering, and stop the automated parts of the system from interacting with the manual parts (for example, we won't show manual results on revisions).

This also moves the "Apply Build Plan" to a third, new home: instead of the sidebar or Buildables, it's now on plans. I think this generally makes more sense given how things have developed. Broadly, this improves isolation of test environments.

Test Plan: Created some builds, browsed around, used filters, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7824
2013-12-26 10:40:43 -08:00
epriestley
c179ce6279 (Later...) Drop legacy Project transaction table
Summary: Ref T4010. I'll hold this for a bit, but we should eventually drop this table once the dust has settled.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7372
2013-12-19 07:03:51 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
James Rhodes
86ec4d6021 Implement policies in Phragment
Summary: This implements support for enforcing and setting policies in Phragment.

Test Plan: Set policies and ensured they were enforced successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7751
2013-12-13 14:42:12 +11:00
James Rhodes
8bb6e807f0 Implement snapshots in Phragment
Summary:
Ref T4212.  This implements snapshots in Phragment, which allows you to take a snapshot of a fragment at a given point in time, and download a ZIP of the snapshot as it was in this state.

There's also functionality for deleting and promoting snapshots.  You can promote a snapshot to either the latest version or any other snapshot of the fragment.

Test Plan: Clicked around, took some snapshots, promoted them to different points and deleted snapshots.  Also downloaded ZIPs of the snapshots and saw the right versions coming through for all the files downloaded.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205, T4212

Differential Revision: https://secure.phabricator.com/D7741
2013-12-09 08:24:50 +11:00
James Rhodes
25e7b7d53c Implement support for creating and updating fragments from ZIPs
Summary:
This implements support for creating and updating fragments from ZIP files.  It allows you to upload a ZIP via the Files application, create a fragment from it, and have it recursively imported into Phragment.  Updating that folder with another ZIP will recursively create, update and delete files as appropriate.

The logic for creating and updating fragments from files has also been centralized into the PhragmentFragment class.  Directories are also now supported; a directory fragment is simply a fragment that has no patches; thus a directory fragment can be converted to a file fragment by uploading a first patch for it.

Test Plan: Uploaded ZIP files through the interface and saw all of the fragments get created and updated as expected.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7729
2013-12-07 13:37:18 +11:00
James Rhodes
4c143ad3b2 Phragment v0
Summary: Ref T4205.  This is an initial implementation of Phragment.  You can create and browse fragments in the system (but you can't yet view a fragment's patches / history).

Test Plan: Clicked around and created fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7726
2013-12-07 12:43:49 +11:00
James Rhodes
79d153b85d Implement explicit build step ordering in Harbormaster
Summary: This implements support for explicitly marking the sequence of build steps.  Users can now drag and re-order build steps in plans, and artifact dependencies are re-calculated so that if you move "Run Command" before "Lease Host", the "Run Command" step has it's artifact setting cleared and thus the step becomes invalid.

Test Plan: Re-ordered build steps and observed dependencies being correctly recalculated.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7715
2013-12-06 14:12:15 +11:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
James Rhodes
53250d84df Introduce HarbormasterBuildTarget to snapshot build steps through a build
Summary: This implements build targets as outlined in D7582.  Build targets represent an instance of a build step particular to the build.  Logs and artifacts have been adjusted to attach to build targets instead of build / build step pairs.

Test Plan: Ran builds and clicked around the interface.  Everything seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7703
2013-12-05 12:01:12 +11:00
James Rhodes
ba16df0fed Restructure Drydock so that blueprints are instances in the DB
Summary:
//(this diff used to be about applying policies to blueprints)//

This restructures Drydock so that blueprints are instances in the DB, with an associated implementation class.  Thus resources now have a `blueprintPHID` instead of `blueprintClass` and DrydockBlueprint becomes a DAO.  The old DrydockBlueprint is renamed to DrydockBlueprintImplementation, and the DrydockBlueprint DAO has a `blueprintClass` column on it.

This now just implements CAN_VIEW and CAN_EDIT policies for blueprints, although they are probably not enforced in all of the places they could be.

Test Plan: Used the `create-resource` and `lease` commands.  Closed resources and leases in the UI.  Clicked around the new and old lists to make sure everything is still working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T2015

Differential Revision: https://secure.phabricator.com/D7638
2013-12-03 11:09:07 +11:00
epriestley
95c2b50974 Shorten extremely long credential names when migrating them
Summary: Fixes T4183. If you have too many repositories sharing the same credential and MySQL is in strict mode, we'll fail a query when trying to write a credential with a name longer than 255 characters. Instead, shorten the variable-length part to 128 characters.

Test Plan: Wiped credentials column and successfully re-ran migration with `storage upgrade --force --apply phabricator:20131121.repocredentials.2.mig.php`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4183

Differential Revision: https://secure.phabricator.com/D7677
2013-12-02 11:26:12 -08:00
epriestley
04b099d9ba Fix an issue with repository credential migration for older repositories
Summary: See IRC. It's possible to have a functional repository with the SSH username only in the URL. Look there if the username property isn't set. These should all be older repostiories.

Test Plan: Did a `--force --apply` upgrade.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D7665
2013-11-27 11:09:07 -08:00
epriestley
4b91c4f7ae Add UI for defining repository mirrors
Summary:
Ref T4038. This adds everything except the actual pushing part for mirrors.

This isn't the most beautiful or sophisticated UI, but I want get the authoritative repositories self-hosted and get users beta-ing hosting as soon as possible. We can do transactions, etc., later on.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7632
2013-11-22 15:23:50 -08:00
epriestley
51fb1ca16d Migrate repositories to use Passphrase for credential management
Summary: Fixes T4122. Ref T2230. Instead of storing credentials on each repository, store them in Passphrase. This allows easy creation/management of many repositories which share credentials.

Test Plan:
  - Upgraded repositories.
  - Created and edited repositories.
  - Pulled HTTP and SSH repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230, T4122

Differential Revision: https://secure.phabricator.com/D7629
2013-11-22 15:23:33 -08:00
epriestley
819f899013 Add an edge between Passphrase credentials and objects which use them
Summary: Ref T4122. Add an edge to keep track of where a credential is used, and show it in the UI.

Test Plan:
See "Used By":

{F84099}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7628
2013-11-22 15:23:23 -08:00
Bob Trahan
7b718bb033 Nuance - federate out the design of NuanceSource via NuanceSourceDefinition
Summary: ...and get the basic edit flow "working" for a new NuanceSourceDefinition - the Phabricator Form. ...and fix a dumb bug in the query class so when you redirect to the view page / try to edit an existing NuanceSource you don't fatal.

Test Plan: played around with the edit form and it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7585
2013-11-20 13:41:19 -08:00
epriestley
91d084624b Passphrase v0
Summary:
Ref T4122. Implements a credential management application for the uses described in T4122.

@chad, this needs an icon, HA HA HAHA HA BWW HA HA HA

bwahaha

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7608
2013-11-20 09:13:35 -08:00
epriestley
d9db1d61e0 Restore population of ownerOrdering to ManiphestTasks
Summary:
Ref T4110. This denormalized field used to power "Group By: Assigned" got dropped in the T2217 migration at some point.

Restore its population, and fix all the data in the database.

Test Plan: Ran migration, verified database came out reasonable-looking. Reassigned a task, verified database. Ran a "Group By: assigned" query.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4110

Differential Revision: https://secure.phabricator.com/D7602
2013-11-19 14:10:54 -08:00
epriestley
7f11e8d740 Improve handling of email verification and "activated" accounts
Summary:
Small step forward which improves existing stuff or lays groudwork for future stuff:

  - Currently, to check for email verification, we have to single-query the email address on every page. Instead, denoramlize it into the user object.
    - Migrate all the existing users.
    - When the user verifies an email, mark them as `isEmailVerified` if the email is their primary email.
    - Just make the checks look at the `isEmailVerified` field.
  - Add a new check, `isUserActivated()`, to cover email-verified plus disabled. Currently, a non-verified-but-not-disabled user could theoretically use Conduit over SSH, if anyone deployed it. Tighten that up.
  - Add an `isApproved` flag, which is always true for now. In a future diff, I want to add a default-on admin approval queue for new accounts, to prevent configuration mistakes. The way it will work is:
    - When the queue is enabled, registering users are created with `isApproved = false`.
    - Admins are sent an email, "[Phabricator] New User Approval (alincoln)", telling them that a new user is waiting for approval.
    - They go to the web UI and approve the user.
    - Manually-created accounts are auto-approved.
    - The email will have instructions for disabling the queue.

I think this queue will be helpful for new installs and give them peace of mind, and when you go to disable it we have a better opportunity to warn you about exactly what that means.

Generally, I want to improve the default safety of registration, since if you just blindly coast through the path of least resistance right now your install ends up pretty open, and realistically few installs are on VPNs.

Test Plan:
  - Ran migration, verified `isEmailVerified` populated correctly.
  - Created a new user, checked DB for verified (not verified).
  - Verified, checked DB (now verified).
  - Used Conduit, People, Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Differential Revision: https://secure.phabricator.com/D7572
2013-11-12 14:37:04 -08:00
James Rhodes
0ac1be7094 Implemented support for build logs
Summary:
Depends on D7519.

This implements support for build logs in Harbormaster.  This includes support for appending to a log from the "Run Remote Command" build step.

It also adds the ability to cancel builds.

Currently the build view page doesn't update the logs live; I'm sure this can be achieved with Javelin, but I don't have enough experience with Javelin to actually make it poll from updates to content in the background.

{F79151}

{F79153}

{F79150}

{F79152}

Test Plan:
Tested this by setting up SSH on a Windows machine and using a Remote Command configured with:

```
C:\Windows\system32\cmd.exe /C cd C:\Build && mkdir Build_${timestamp} && cd Build_${timestamp} && git clone --recursive https://github.com/hach-que/Tychaia.git && cd Tychaia && Protobuild.exe && C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe Tychaia.Windows.sln
```

and observed the output of the build stream from the Windows machine into Phabricator.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7521
2013-11-08 18:15:07 -08:00