1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-20 20:40:56 +01:00
Commit graph

561 commits

Author SHA1 Message Date
Mukunda Modell
25ae4c458d Protect file data with a one-time-token
Test Plan: currently untested work in progress

Reviewers: #blessed_reviewers, epriestley

Subscribers: rush898, aklapper, Korvin, epriestley

Projects: #wikimedia

Maniphest Tasks: T5685

Differential Revision: https://secure.phabricator.com/D10054
2014-08-11 07:32:17 -07:00
epriestley
24a6eeb8d8 Allow the workboard backlog column to be reordered
Summary:
Fixes T5677.

  - Instead of using `sequence == 0` to mean "this is the backlog column", flag the column explicitly.
  - Migrate existing sequence 0 columns to have the flag.
  - Add the flag when initializing or copying a board.
  - Remove special backlog logic when reordering columns.

Test Plan:
  - Migrated columns, viewed some boards, they looked identical.
  - Reordered the backlog column a bunch of times (first, last, middle, dragged other stuff around).
  - Added tasks to a project, saw them show up in the reordered backlog.
  - Initialized a new board and saw a backlog column show up.
  - Copied an existing board and saw the backlog column come over.
  - Tried to hide a backlog column.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5677

Differential Revision: https://secure.phabricator.com/D10189
2014-08-08 15:50:36 -07:00
epriestley
09868271bd Move board relationships to dedicated storage
Summary:
Fixes T5476. Using edges to store which objects are on which board columns ends up being pretty awkward. In particular, it makes T4807 very difficult to implement.

Introduce a dedicated `BoardColumnPosition` storage.

This doesn't affect ordering rules (T4807) yet: boards are still arranged by priority. We just read which tasks are on which columns out of a new table.

Test Plan:
  - Migrated data, then viewed some boards. Saw exactly the same data.
  - Dragged tasks from column to column.
  - Created a task directly into a column.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5476

Differential Revision: https://secure.phabricator.com/D10160
2014-08-06 15:09:09 -07:00
Joshua Spence
8fd098329b Rename AphrontQueryException subclasses
Summary: Ref T5655. Depends on D10149.

Test Plan: Ran `arc unit`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin, hach-que

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D10150
2014-08-06 07:51:21 +10:00
epriestley
89b942c183 Move Audit to proper Subscriptions
Summary:
Ref T4896. Currently, subscriptions to commits are stored as auditors with a special "CC" type.

Instead, use normal subscriptions storage, reads and writes.

Test Plan:
  - Ran migration and verified data still looked good.
  - Viewed commits in UI and saw "subscribers".
  - Saw "Automatically Subscribed", clicked Subscribe/Unsubscribe on a non-authored commit, saw subscriptions update.
  - Pushed a commit through Herald rules and saw them trigger subscriptions and auditors.
  - Used "Add CCs".
  - Added CCs with mentions.

Reviewers: btrahan, joshuaspence

Reviewed By: btrahan, joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10103
2014-08-02 00:06:13 -07:00
James Rhodes
dfa9b27a94 Use tabs on build targets and allow build steps to have a description
Summary:
Ref T1049. This uses tabs on build targets to hide the configuration details and variables by default, instead promoting the target name, it's status and a description of the build step.  The description is a new field on each build step.

The primary advantage of having a description on build steps is that DevOps can configure appropriate description information (including any troubleshooting information for build failures) on build steps, and developers who have builds fail against their code review can then look at this information.

Test Plan: Viewed a build plan and saw the appropriate information.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D10093
2014-08-01 08:09:15 +10:00
epriestley
fa2fcc7852 Fix missed migration constant from PHIDType rename
Summary: See D9986, we missed this one.

Test Plan: `grep`

Reviewers: joshuaspence, btrahan, bizrad6

Reviewed By: bizrad6

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D10098
2014-07-31 11:30:40 -07:00
James Rhodes
cad41ea294 Implement build simulation; convert Harbormaster to be purely dependency based
Summary:
Depends on D9806.  This implements the build simulator, which is used to calculate the order of build steps in the plan editor.  This includes a migration script to convert existing plans from sequential based to dependency based, and then drops the sequence column.

Because build plans are now dependency based, the grippable and re-order behaviour has been removed.

Test Plan: Tested the migration, saw the dependencies appear correctly.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9847
2014-07-31 11:39:49 +10:00
epriestley
f965126dc4 Migrate audit comments to transactions
Summary:
Ref T4896. Depends on D10052. This is the major/scary migration, but not really so bad. It is substantially similar to D8210, but less complex because there are fewer actions here.

This moves `PhabricatorAuditComment` storage to `PhabricatorAuditTransaction`, then reads `PhabricatorAuditComment`s as a proxy around the new objects.

Test Plan:
  - Before migrating, browsed around. Nothing appeared broken.
  - Migrated cleanly.
  - Viewed old transactions (inlines, comments, accept/reject/etc, add auditors, add ccs, implicit CCs).
  - Added all of those comment types.
  - Edited a draft.
  - Deleted a draft.
  - Spot checked the database for sanity.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10055
2014-07-28 15:00:46 -07:00
epriestley
3d78c0eff7 Migrate Audit comment text into new storage
Summary:
Ref T4896. This is substantially similar to D8196.

Migrate the comment text out of the `audit_comment` table and into the `audit_transaction_comment` table. Do double reads on `PhabricatorAuditComment` so the APIs aren't disturbed. The old table is still updated.

Test Plan:
  - Before applying migration, cleared cache and browsed around. Things looked fine, except no comment text.
  - Applied migration.
  - Cleared cache, browsed around, saw all my old comments.
  - Added some new comments.
  - Spot checked migrated and new rows in database.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10020
2014-07-24 18:00:30 -07:00
epriestley
c01aa794c1 Migrate Audit inline comments to new storage
Summary:
Ref T4896. This is substantially identical to the process which Differential followed, and mostly copied from the original Differential migration and the Differential proxy object.

Basically, we move all the data over but the application can't tell, and the same APIs do reads and writes to the new table.

Test Plan:
 - Browsed UI before migrating, everything looked fine (but no inlines).
 - Ran migration.
 - Verified draft and published comments survived migration.
 - Added a draft.
 - Previewed draft.
 - Submitted draft.
 - Viewed standalone with drafts and published comments.
 - Sanity checked data in database, didn't see anything unusual.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10018
2014-07-24 17:59:54 -07:00
epriestley
416f3d9ede Add storage for new audit transactions and comments
Summary:
Ref T4896. This adds the new storage, without any code changes.

This storage is substantially identical to the Differential storage, except that `changesetID` has been replaced by `pathID`.

I've retained the properties intended to be used to implement T1460. They might not be quite right, but at least we'll be able to make any fixes consistently to both applications. For now, these fields are empty and ignored.

Test Plan: Ran `./bin/storage upgrade`. Nothing calls this code yet.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10017
2014-07-24 17:59:43 -07:00
Joshua Spence
97a8700e45 Rename PHIDType classes
Summary: Ref T5655. Rename `PhabricatorPHIDType` subclasses for clarity (see discussion in D9839). I'm not too keen on some of the resulting class names, so feel free to suggest alternatives.

Test Plan: Ran unit tests.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin, hach-que

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9986
2014-07-24 08:05:46 +10:00
epriestley
d6eb1c67e7 Phortune Cart Status, some one-time support
Summary:
Ref T2787. Carts need a status so we can tell if they've been purchased. Also kind of get WePay working as a one-time provider, and let charges not have a methodPHID (they won't for one-time providers).

All the status stuff is still super crazy rough and you can do things like start a checkout, add a bunch of stuff to your cart, complete the checkout, and have Phabricator think you paid for all the stuff you added. But this is fine for now since you can't actually edit carts, and also none of this is at all usable anyway. I'll refine some of the workflows in future diffs, for now I'm just getting things hooked up and technically working.

Test Plan:
  - Purcahsed a cart and got a sort of status/done screen instead of a "your money is gone" exception.
  - Went through the WePay flow and got a successful test checkout.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D10003
2014-07-23 10:36:25 -07:00
epriestley
4c0f15b94b Phortune Charges
Summary: Ref T2787. Makes charges a real object, allows providers to apply them. We are now (just barely) capable of stealing users' money.

Test Plan: {F179584}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D10002
2014-07-23 10:36:12 -07:00
epriestley
6ec1f35870 Phortune Carts and Purchases
Summary:
Ref T2787. Make carts and purchases real objects, with storage, that kind-of work.

Roughly, the idea here is that applications create "purchases" (like "1 large t-shirt") and add them to "carts" (a user can have a lot of different carts at the same time), then hand things off to Phortune to deal with actualy charging a card. Roughly this works like Paypal or other similar systems do, except Phortune is the thing the user gets handed off to.

This doesn't do anything interesting/useful yet.

Also fix some bugs and update some UI.

Test Plan: Added a product to a cart, saw it in cart screen.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D10001
2014-07-23 10:34:08 -07:00
epriestley
e561a5fe73 Fix incorrect old name for Asana auth provider in provider migration
Summary: The old class name isn't quite correct.

I'm just updating the migration rather than adding a new one to fix it since
this was very recently introduced and affects only installs using Asana auth,
so it's realistic that the number of affected installs might be 0.

Affected installs can use `--apply` to safely rerun the migration.

Auditors: joshuaspence
2014-07-23 09:46:06 -07:00
Joshua Spence
e0a7d47e0d Add a missing migration for transaction actors
Summary: Add a missing migration which should have been included in D9982. Harbormaster and Herald PHIDs are used as actors in some transactions.

Test Plan: Ran `./bin/storage upgrade`. Saw a transaction render correctly as "Herald assigned this task to alincoln" instead of "Unknown Object (Application) assigned this task to alincoln".

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10028
2014-07-23 23:49:22 +10:00
Joshua Spence
86c399b657 Rename PhabricatorApplication subclasses
Summary: Ref T5655. Some discussion in D9839. Generally speaking, `Phabricator{$name}Application` is clearer than `PhabricatorApplication{$name}`.

Test Plan:
# Pinned and uninstalled some applications.
# Applied patch and performed migrations.
# Verified that the pinned applications were still pinned and that the uninstalled applications were still uninstalled.
# Performed a sanity check on the database contents.

Reviewers: btrahan, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: hach-que, epriestley, Korvin

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9982
2014-07-23 10:03:09 +10:00
Joshua Spence
458cf8c898 Add a missing migration
Summary: This migration script is required for D9999, which has already landed.

Test Plan: Ran `./bin/storage upgrade` and can log in again.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10010
2014-07-22 06:26:41 -07:00
Joshua Spence
f4b05312cd Fix broken references to auth adapters
This was broken in D9999 but somehow didn't fail linting or unit tests.

Auditors: epriestley
2014-07-22 21:20:45 +10:00
epriestley
a115810912 Give projects a proper on-demand datasource
Summary:
Fixes T5614. Ref T4420. Other than the "users" datasource and a couple of others, many datasources ignore what the user typed and just return all results, then rely on the client to filter them.

This works fine for rarely used ("legalpad documents") or always small ("task priorities", "applications") datasets, but is something we should graudally move away from as datasets get larger.

Add a token table to projects, populate it, and use it to drive the datasource query. Additionally, expose it on the applicationsearch UI.

Test Plan:
  - Ran migration.
  - Manually checked the table.
  - Searched for projects by name from ApplicationSearch.
  - Searched for projects by name from typeahead.
  - Manually checked the typeahead response.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5614, T4420

Differential Revision: https://secure.phabricator.com/D9896
2014-07-17 16:35:54 -07:00
epriestley
aa79539789 Move task/project storage to edges
Summary: Ref T5245. This moves the actual storage over and stops reads and writes to the old table.

Test Plan:
  - Verified tasks retained projects across the migration.
  - Added and removed projects from tasks.
  - Searched for: all, any, users' projects, not-in-projects, no-projects.

Reviewers: chad, btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9850
2014-07-17 15:42:30 -07:00
epriestley
8cbfb49b4e Remove all edge events
Summary:
Ref T5245. These were a bad idea.

We no longer need actors for edge edits either, so remove those. Generally, edges have fit into the policy model as pure/low-level infrastructure, and they do not have any policy or capability information in and of themselves.

Test Plan: `grep`

Reviewers: chad, btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9840
2014-07-17 15:41:42 -07:00
epriestley
533e799c5f Modernize task/revision edges and write inverse transactions
Summary:
Ref T5245. See some discussion in D9838.

When we attach object A to object B, we'd like to write transactions on both sides but only write the actual edges once.

To do this, allow edge types to `shouldWriteInverseTransactions()`. When an edge type opts into this, have editors apply the inverse transactions before writing the edge. These inverse transactions don't actually apply effects, they just show up in the transaction log.

Test Plan: Attached and detached revisions from tasks, saw transactions appear on both sides of the operation.

Reviewers: chad, btrahan, joshuaspence

Reviewed By: btrahan, joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9839
2014-07-17 15:41:08 -07:00
epriestley
ca6b3a632d Remove uniqueness constraint from "sequence" key in project column table
Summary: Fixes T5611. We don't need sequences to be unique, and it makes it a pain to update them.

Test Plan: Dragged some columns around.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5611

Differential Revision: https://secure.phabricator.com/D9914
2014-07-12 11:24:35 -07:00
Joshua Spence
9a679bf374 Allow worker tasks to have priorities
Summary: Fixes T5336. Currently, `PhabricatorWorkerLeaseQuery` is basically FIFO. It makes more sense for the queue to be a priority-queue, and to assign higher priorities to alerts (email and SMS).

Test Plan: Created dummy tasks in the queue (with different priorities). Verified that the priority field was set correctly in the DB and that the priority was shown on the `/daemon/` page. Started a `PhabricatorTaskmasterDaemon` and verified that the higher priority tasks were executed before lower priority tasks.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5336

Differential Revision: https://secure.phabricator.com/D9871
2014-07-12 03:02:06 +10:00
epriestley
eb28a7caef Add an optional preamble to Legalpad documents
Summary:
Fixes T5532. Allow documents to have a preamble in the header which can be used to explain who should sign a document and why.

Particularly, I plan to use this to navigate the corporate vs individual stuff more sensibly.

Test Plan: {F174228}

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5532

Differential Revision: https://secure.phabricator.com/D9819
2014-07-04 09:41:27 -07:00
James Rhodes
a3d50118e1 Allow users to specify names of build steps
Summary: Ref T1049.  This provides a user-configurable name field on build steps, which allows users to uniquely identify their steps.  The intention is that this field will be used in D9806 to better identify the dependencies (rather than showing an unhelpful PHID).

Test Plan: Set the name of some build steps, saw it appear in the correct places.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D9816
2014-07-05 01:56:02 +10:00
epriestley
e8d217b8bd Allow Legalpad documents to designate corporate signers
Summary:
Ref T5532. This adds:

  - Documents can designate that they should be signed by "Corporations" or "Individuals".
  - Corporate documents get different fields and a different exemption process.
  - Basically everything works the same but this is like a zillion lines of form code.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5532

Differential Revision: https://secure.phabricator.com/D9812
2014-07-04 08:04:28 -07:00
epriestley
b6ea2735d7 Allow Legalpad document managers to add signature exemptions
Summary:
Ref T5532. Allow document managers to add exemptions, which act like signatures but are tracked a little differently.

The primary use case for us is users who sign a corporate CLA and need a user-level exemption if they don't want to sign an individual CLA.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5532

Differential Revision: https://secure.phabricator.com/D9795
2014-07-02 04:59:35 -07:00
epriestley
c9366acbec Allow dashboard panels to be archived
Summary: Ref T5471. Adds an archived state for panels. Archived panels don't show up in the default query view or in the "Add Existing Panel" workflow.

Test Plan:
  - Archived a panel.
  - Activated a panel.
  - Viewed / searched for archived/active panels.
  - Popped "Add Existing Panel" dropdown and saw it omit archived panels.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5471

Differential Revision: https://secure.phabricator.com/D9779
2014-07-01 17:50:28 -07:00
epriestley
ffc1b5c26a Allow users to search for signatures by name and email substrings
Summary:
Ref T3116. In the case of anonymous signers, there's no way to do a quick way to check if someone has signed a doc since you can't query by their (nonexistent) external account ID.

Move "name" and "email" to first-class columns and let the engine search for them.

Test Plan: Searched for signatures with name and email fragments.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T3116

Differential Revision: https://secure.phabricator.com/D9776
2014-06-29 07:51:03 -07:00
epriestley
be47f2141a Allow projects icon color to be selected from several fabulous shades
Summary: This further helps differentiate types/roles for projects.

Test Plan: {F169758}

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9710
2014-06-25 22:01:58 -07:00
epriestley
9c423647d1 Drop DifferentialComment and DifferentialInlineComment
Summary: Ref T2222. No remaining readers or writers. Holding for a while.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8242
2014-06-24 05:29:54 -07:00
epriestley
1d90577d8c Drop old Differential custom field storage
Summary: Ref T2222. See D8355. I'll hold this for a while.

Test Plan: Ran migration.

Reviewers: chad, btrahan

Reviewed By: chad

Subscribers: epriestley, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8356
2014-06-22 08:15:14 -07:00
epriestley
490ae05eb8 Drop differential_relationship table
Summary: Ref T2222. I'll hold this, but there are no more reads or writes from/to this table in the application.

Test Plan: Grepped for usage, ran migration, browsed around.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8203
2014-06-22 08:12:11 -07:00
Joshua Spence
f52fbf6117 Unify the local and global view for ./bin/phd status.
Summary:
Ref T4209. Unifies the local (`./bin/phd status`) and global (`./bin/phd status --all`) view into a single table. This generally makes it easy to administer daemons running across multiple hosts.

Depends on D9606.

Test Plan:
```
> sudo ./bin/phd status
ID Host      PID  Started                 Daemon                               Arguments
38 localhost 2282 Jun 18 2014, 7:52:56 AM PhabricatorRepositoryPullLocalDaemon
39 localhost 2289 Jun 18 2014, 7:52:57 AM PhabricatorGarbageCollectorDaemon
40 localhost 2294 Jun 18 2014, 7:52:57 AM PhabricatorTaskmasterDaemon
41 localhost 2314 Jun 18 2014, 7:52:58 AM PhabricatorTaskmasterDaemon
42 localhost 2319 Jun 18 2014, 7:52:59 AM PhabricatorTaskmasterDaemon
43 localhost 2328 Jun 18 2014, 7:53:00 AM PhabricatorTaskmasterDaemon
44 localhost 2354 Jun 18 2014, 7:53:08 AM PhabricatorRepositoryPullLocalDaemon X --not Y
```

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4209

Differential Revision: https://secure.phabricator.com/D9607
2014-06-18 11:44:52 +10:00
epriestley
df3736e81e Garbage collect daemon logs
Summary:
We already have GC for daemon log events, but not for daemon logs themselves.

Collect old daemon logs which aren't still running.

Test Plan: Ran `phd debug garbage`, observed old logs get cleaned up. Started some daemons, re-ran garbage, made sure they stuck around.

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9610
2014-06-17 15:33:08 -07:00
epriestley
868ff166b1 Give Pholio mocks a configurable edit policy
Summary:
Ref T4566. Currently, mocks have a conservative (author only), immutable default edit policy.

Instead:

  - Let the edit policy be changed.
  - Default the edit policy to "all users", similar to other applications.
  - Add an application-level setting for it.
  - Migrate existing edit policies to be consistent with the old policy (just the author).

This stops short of adding a separate "owner" and letting that be changed, since Pholio doesn't really have any review/approve type features (at least, so far). We can look at doing this if we get more feedback about it, or if we make owners more meaningful (e.g., add more "review-like" process to mocks).

Test Plan:
  - Ran migration scripts.
  - Confirmed existing mocks retained their effective policies (author only).
  - Created a new mock, saw edit policy.
  - Changed edit policy.
  - Changed global edit policy default.
  - Tried to edit a mock I couldn't edit.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T4566

Differential Revision: https://secure.phabricator.com/D9550
2014-06-15 10:28:16 -07:00
Joshua Spence
d0128afa29 Applied various linter fixes.
Summary: Applied some more linter fixes that I previously missed because my global `arc` install was out-of-date.

Test Plan: Will run `arc unit` on another host.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9443
2014-06-09 16:04:12 -07:00
Joshua Spence
0a62f13464 Change double quotes to single quotes.
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.

Test Plan: Eyeballed it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9431
2014-06-09 11:36:50 -07:00
epriestley
0aa913805d Add an alternate "modern" hunk datastore
Summary:
Ref T4045. Ref T5179. Hunk storage has two major issues:

  - It's utf8, but actual diffs are binary.
  - It's huge and can't be compressed or archived.

This introduces a second datastore which solves these problems: by recording hunk encoding, supporting compression, and supporting alternate storage. There's no actual compression or storage support yet, but there's space in the table for them.

Since nothing actually uses hunk IDs, it's fine to have these tables exist at the same time and use the same IDs. We can migrate data between the tables gradually without requiring downtime or disrupting installs.

Test Plan:
  - There are no writes to the new table yet.
  - The only effect this has is making us issue one extra query when looking for hunks.
  - Observed the query issue, but everything else continue working fine.
  - Created a new diff.
  - Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4045, T5179

Differential Revision: https://secure.phabricator.com/D9290
2014-06-03 18:01:22 -07:00
epriestley
99c72a32d0 Allow installs to require multi-factor authentication for all users
Summary: Ref T5089. Adds a `security.require-multi-factor-auth` which forces all users to enroll in MFA before they can use their accounts.

Test Plan:
Config:

{F159750}

Roadblock:

{F159748}

After configuration:

{F159749}

  - Required MFA, got roadblocked, added MFA, got unblocked.
  - Removed MFA, got blocked again.
  - Used `bin/auth strip` to strip MFA, got blocked.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5089

Differential Revision: https://secure.phabricator.com/D9285
2014-06-03 16:50:27 -07:00
epriestley
63ed126b2a Point github.com/facebook URIs at github.com/phacility insead
Summary: Point everything at the new canonical URI.

Test Plan: `grep`

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9328
2014-05-29 08:33:25 -07:00
Bob Trahan
102befdede Project - add ability to select an icon for typeaheads and such
Summary: Fixes T5090. Introduced getIcon into Handle stack which allows you to specify a per handle icon. getIcon falls back ot getTypeIcon.

Test Plan: changed the icon on a project a bunch. verified transactions showed up. verified icon showed up in typeahead. verified icon showed up in tokens that were pre-generated (not typed in). units test passed.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5090

Differential Revision: https://secure.phabricator.com/D9264
2014-05-23 10:41:24 -07:00
Bob Trahan
922e5c0849 Projects - add "Additional Hashtags" to projects
Summary:
Fixes T4021. Chooses to keep a "primary" slug based off the name - including all that lovely logic - and allow the user to specify "additional" slugs. Expose these as "hashtags" to the user.

Sets us up for a fun diff where we can delete all the Project => Phriction automagicalness. In terms of this diff, see the TODOs i added.

Test Plan:
added a primary slug as an additional slug - got an error. added a slug in use on another project - got an error. added multiple good slugs and they worked. removed slugs and it worked. made some remark using multiple new slugs and they all linked to the correct project

ran epriestley's case

 - Create project "A".
 - Give it additional slug "B".
 - Try to create project "B".

and i got a nice error about hashtag collision

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4021

Differential Revision: https://secure.phabricator.com/D9250
2014-05-22 11:19:03 -07:00
epriestley
cac61980f9 Add "temporary tokens" to auth, for SMS codes, TOTP codes, reset codes, etc
Summary:
Ref T4398. We have several auth-related systems which require (or are improved by) the ability to hand out one-time codes which expire after a short period of time.

In particular, these are:

  - SMS multi-factor: we need to be able to hand out one-time codes for this in order to prove the user has the phone.
  - Password reset emails: we use a time-based rotating token right now, but we could improve this with a one-time token, so once you reset your password the link is dead.
  - TOTP auth: we don't need to verify/invalidate keys, but can improve security by doing so.

This adds a generic one-time code storage table, and strengthens the TOTP enrollment process by using it. Specifically, you can no longer edit the enrollment form (the one with a QR code) to force your own key as the TOTP key: only keys Phabricator generated are accepted. This has no practical security impact, but generally helps raise the barrier potential attackers face.

Followup changes will use this for reset emails, then implement SMS multi-factor.

Test Plan:
  - Enrolled in TOTP multi-factor auth.
  - Submitted an error in the form, saw the same key presented.
  - Edited the form with web tools to provide a different key, saw it reject and the server generate an alternate.
  - Change the expiration to 5 seconds instead of 1 hour, submitted the form over and over again, saw it cycle the key after 5 seconds.
  - Looked at the database and saw the tokens I expected.
  - Ran the GC and saw all the 5-second expiry tokens get cleaned up.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D9217
2014-05-20 11:43:45 -07:00
Bob Trahan
5f33aa5b4f Dashboards - add ability to install dashboard as home
Summary:
See title. Adds PhabricatorDashboardInstall data object which scopes installs to objectPHID + applicationClass. This is because we already have a collision for user home pages and user profiles. Assume only one dashboard per objectPHID + applicationClass though at the database level.

Fixes T5076.

Test Plan: From dashboard view, installed a dashboard - success! Went back to dashboard view and uninstalled it!

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5076

Differential Revision: https://secure.phabricator.com/D9206
2014-05-19 16:09:31 -07:00
lkassianik
3d457a53be Close pholio mocks
Summary: Fixes T4299, Add status dropdown to mock edit view

Test Plan: Edit mock, close mock, thumbnail title should read (Disabled). Default mocks list should show only open mocks.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: chad, epriestley, Korvin

Maniphest Tasks: T4299

Differential Revision: https://secure.phabricator.com/D9145
2014-05-19 11:34:23 -07:00
epriestley
15c408be1d Change LONGTEXT cache column to BINARY
See T4898. As it turns out, `latin1` LONGTEXT is not approximately equivalent
to LONGBLOB.
2014-05-17 22:38:56 -07:00
epriestley
d744d5d859 Fix binary/utf8 issues with Differential changeset parse cache
Summary:
Fixes T4898. After we increased the strictness of the `%s` conversion, most `serialize()` output is rejected from the cache.

Drop the cache, change the column type to latin1_bin, and then use `%B` to mark the data as binary during query construction.

Test Plan: Viewed Differential, saw cache fills.

Reviewers: btrahan, spicyj

Reviewed By: spicyj

Subscribers: epriestley

Maniphest Tasks: T4898

Differential Revision: https://secure.phabricator.com/D9171
2014-05-17 16:34:13 -07:00
epriestley
a74545c9da Provide a rough, unstable API for reporting coverage into Diffusion
Summary:
Ref T4994. This stuff works:

  - You can dump a blob of coverage information into `diffusion.updatecoverage`. This wipes existing coverage information and replaces it.
  - It shows up when viewing files.
  - It shows up when viewing commits.

This stuff does not work:

  - When viewing files, the Javascript hover interaction isn't tied in yet.
  - We always show this information, even if you're behind the commit where it was generated.
  - You can't do incremental updates.
  - There's no aggregation at the file (this file has 90% coverage), diff (the changes in this commit are 90% covered), or directory (the code in this directory has 90% coverage) levels yet.
  - This is probably not the final form of the UI, storage, or API, so you should expect occasional changes over time. I've marked the method as "Unstable" for now.

Test Plan:
  - Ran `save_lint.php` to check for collateral damage; it worked fine.
  - Ran `save_lint.php` on a new branch to check creation.
  - Published some fake coverage information.
  - Viewed an affected commit.
  - Viewed an affected file.

{F151915}

{F151916}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: jhurwitz, epriestley, zeeg

Maniphest Tasks: T5044, T4994

Differential Revision: https://secure.phabricator.com/D9022
2014-05-17 16:10:54 -07:00
Tal Shiri
43d45c4956 can now tell phabricator you trust an auth provider's emails (useful for Google OAuth), which will mark emails as "verified" and will skip email verification.
Summary: This is useful when you're trying to onboard an entire office and you end up using the Google OAuth anyway.

Test Plan: tested locally. Maybe I should write some tests?

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9150
2014-05-16 14:14:06 -07:00
Bob Trahan
6300955661 Dashboards - add layout mode to dashboards
Summary:
This gets us the ability to specify a "layout mode" and which column a panel should appear in at panel add time. Changing the layout mode from a multi column view to a single column view or vice versa will reset all panels to the left most column.

You can also drag and drop where columns appear via the "arrange" mode.

We also have a new dashboard create flow. Create dashboard -> arrange mode. (As opposed to view mode.) This could all possibly use massaging.

Fixes T4996.

Test Plan:
made a dashboard with panels in multiple columns. verified correct widths for various layout modes

re-arranged collumns like whoa.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4996

Differential Revision: https://secure.phabricator.com/D9031
2014-05-15 19:12:40 -07:00
Aviv Eyal
f2c0e94ea8 Show command transactions in Harbormaster builds
Summary:
Create transaction, editor, etc, and move command generation over to editor.
Show in a timeline in the buildable page.

Also prevent Engine from creating an empty transaction when build starts (Fixes T4885).

Fixes T4886.

Test Plan: Restart builds and buildables, look at timeline.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4885, T4886

Differential Revision: https://secure.phabricator.com/D9110
2014-05-15 07:04:34 -07:00
epriestley
4a7499f230 Fix credential upgrade issue after introduction of isLocked column
Summary: Fixes T5035. This migration isn't forward compatible after schema mutation.

Test Plan: Ran locally, will get reporting user to confirm.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: gera, epriestley

Maniphest Tasks: T5035

Differential Revision: https://secure.phabricator.com/D9101
2014-05-13 12:14:27 -07:00
epriestley
95eab2f3b0 Record parent relationships when discovering commits
Summary:
Ref T4455. This adds a `repository_parents` table which stores `<childCommitID, parentCommitID>` relationships.

For new commits, it is populated when commits are discovered.

For older commits, there's a `bin/repository parents` script to rebuild the data.

Right now, there's no UI suggestion that you should run the script. I haven't come up with a super clean way to do this, and this table will only improve performance for now, so it's not important that we get everyone to run the script right away. I'm just leaving it for the moment, and we can figure out how to tell admins to run it later.

The ultimate goal is to solve T2683, but solving T4455 gets us some stuff anyway (for example, we can serve `diffusion.commitparentsquery` faster out of this cache).

Test Plan:
  - Used `bin/repository discover` to discover new commits in Git, SVN and Mercurial repositories.
  - Used `bin/repository parents` to rebuild Git and Mercurial repositories (SVN repos just exit with a message).
  - Verified that the table appears to be sensible.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: jhurwitz, epriestley

Maniphest Tasks: T4455

Differential Revision: https://secure.phabricator.com/D9044
2014-05-12 11:47:22 -07:00
Bob Trahan
e96c363eef Add SMS support
Summary:
Provides a working SMS implementation with support for Twilio.

This version doesn't really retry if we get any gruff at all. Future versions should retry.

Test Plan: used bin/sms to send messages and look at them.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: aurelijus, epriestley, Korvin

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D8930
2014-05-09 12:47:21 -07:00
epriestley
bd7420c4bb Allow pastes to be edited
Summary: Fixes T4814.

Test Plan: Edited pastes from the web UI.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4814

Differential Revision: https://secure.phabricator.com/D8970
2014-05-04 11:11:46 -07:00
epriestley
7fb89686d4 Upstream a patch to cover a migration issue
Summary: Fixes T4928. I'm not sure how this column was missing, but this patch can't hurt.

Test Plan: Reasoned about behavior.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4928

Differential Revision: https://secure.phabricator.com/D8967
2014-05-04 10:48:25 -07:00
lkassianik
d7b7b19337 Add a "Lock Permanently" action to Passphrase
Summary: Fixes T4931. Each new credential should come with the ability to lock the credential permanently, so that no one can ever edit again. Each existing credential must allow user to lock existing credential.

Test Plan: Create new credential, verify that you can lock it before saving it. Open existing unlocked credential, verify that option to lock it exists. Once credential is locked, the option to reveal it should be disabled, and editing the credential won't allow username/password updates.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4931

Differential Revision: https://secure.phabricator.com/D8947
2014-05-02 18:21:51 -07:00
epriestley
2022a70e16 Implement bin/remove, for structured destruction of objects
Summary:
Ref T4749. Ref T3265. Ref T4909. Several goals here:

  - Move user destruction to the CLI to limit the power of rogue admins.
  - Start consolidating all "destroy named object" scripts into a single UI, to make it easier to know how to destroy things.
  - Structure object destruction so we can do a better and more automatic job of cleaning up transactions, edges, search indexes, etc.
  - Log when we destroy objects so there's a record if data goes missing.

Test Plan: Used `bin/remove destroy` to destroy several users.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3265, T4749, T4909

Differential Revision: https://secure.phabricator.com/D8940
2014-05-01 18:23:31 -07:00
epriestley
50376aad04 Require multiple auth factors to establish web sessions
Summary:
Ref T4398. This prompts users for multi-factor auth on login.

Roughly, this introduces the idea of "partial" sessions, which we haven't finished constructing yet. In practice, this means the session has made it through primary auth but not through multi-factor auth. Add a workflow for bringing a partial session up to a full one.

Test Plan:
  - Used Conduit.
  - Logged in as multi-factor user.
  - Logged in as no-factor user.
  - Tried to do non-login-things with a partial session.
  - Reviewed account activity logs.

{F149295}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D8922
2014-05-01 10:23:02 -07:00
epriestley
941f0ba7ae Allow panels to appear on dashboards
Summary:
Ref T3583. Adds edges, query relationships, etc. Lots of debugging/temporary UI.

My general intent here is to use edges to track where panels appear, and then put additional data on the dashboard itself to control layout, positioning, etc.

Dashboards don't actually render yet so this is still pretty boring.

Test Plan:
{F149175}

{F149176}

{F149177}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8916
2014-04-30 14:28:55 -07:00
epriestley
ea954c37e4 Add dashboard panel types
Summary: Ref T3583. These will be the primary class carrying panel implementations.

Test Plan:
{F149125}

{F149126}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8912
2014-04-30 14:28:20 -07:00
epriestley
17709bc167 Add multi-factor auth and TOTP support
Summary:
Ref T4398. This is still pretty rough and isn't exposed in the UI yet, but basically works. Some missing features / areas for improvement:

  - Rate limiting attempts (see TODO).
  - Marking tokens used after they're used once (see TODO), maybe. I can't think of ways an attacker could capture a token without also capturing a session, offhand.
  - Actually turning this on (see TODO).
  - This workflow is pretty wordy. It would be nice to calm it down a bit.
  - But also add more help/context to help users figure out what's going on here, I think it's not very obvious if you don't already know what "TOTP" is.
  - Add admin tool to strip auth factors off an account ("Help, I lost my phone and can't log in!").
  - Add admin tool to show users who don't have multi-factor auth? (so you can pester them)
  - Generate QR codes to make the transfer process easier (they're fairly complicated).
  - Make the "entering hi-sec" workflow actually check for auth factors and use them correctly.
  - Turn this on so users can use it.
  - Adding SMS as an option would be nice eventually.
  - Adding "password" as an option, maybe? TOTP feels fairly good to me.

I'll post a couple of screens...

Test Plan:
  - Added TOTP token with Google Authenticator.
  - Added TOTP token with Authy.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D8875
2014-04-28 09:27:11 -07:00
epriestley
f42ec84d0c Add "High Security" mode to support multi-factor auth
Summary:
Ref T4398. This is roughly a "sudo" mode, like GitHub has for accessing SSH keys, or Facebook has for managing credit cards. GitHub actually calls theirs "sudo" mode, but I think that's too technical for big parts of our audience. I've gone with "high security mode".

This doesn't actually get exposed in the UI yet (and we don't have any meaningful auth factors to prompt the user for) but the workflow works overall. I'll go through it in a comment, since I need to arrange some screenshots.

Test Plan: See guided walkthrough.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D8851
2014-04-27 17:31:11 -07:00
lkassianik
9a827096a7 Ability to close poll
Summary: Fixes T3566 List of poll actions should include ability to close an open poll or reopen a closed poll.

Test Plan: Poll author should be able to close/reopen poll. Non-author should get policy screen when attempting to close/reopen poll.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T3566

Differential Revision: https://secure.phabricator.com/D8846
2014-04-24 12:02:56 -07:00
epriestley
2ac8457cb9 [Later] Drop the project profile table
Summary: Going to sit on this for a bit so we can fall back to it if needbe, but this table no longer has any reads or writes in the application.

Test Plan: Applied patch locally and poked around.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: aran

Differential Revision: https://secure.phabricator.com/D8190
2014-04-24 08:15:24 -07:00
epriestley
4bbd2d5203 Make an old migration a little more robust
Summary: See https://github.com/facebook/phabricator/pull/507

Test Plan: This is hard to test since the migration no longer runs against HEAD, but pull 507 strongly implies this is the correct fix.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8837
2014-04-23 14:22:02 -07:00
epriestley
1a3ac09975 Add "requestedObjectPHID" to ReleephRequest
Summary:
Ref T3551. Currently, ReleephRequests don't have a direct concept of the //object// being requested. You can request `D123`, but that is just a convenient way to write `rXyyyy`.

When the UI wants to display information about a revision, it deduces it by examining the commit.

This is primarily an attack on T3551, so we don't need to load <commit -> edge -> revision> (in an ad-hoc way) to get revisions. Instead, when you request a revision we keep track of it and can load it directly later.

Later, this will let us do more things: for example, if you request a branch, we can automatically update the commits (as GitHub does), etc. (Repository branches will need PHIDs first, of course.)

This adds and populates the column but doesn't use it yet. The second part of the migration could safely be run while Phabricator is up, although even for Facebook this table is probably quite small.

Test Plan:
  - Ran migration.
  - Verified existing requests associated sensibly.
  - Created a new commit request.
  - Created a new revision request.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3551

Differential Revision: https://secure.phabricator.com/D8822
2014-04-20 11:55:18 -07:00
epriestley
4918773afe Drop nonsense buildStatus field from Buildable
Summary:
Ref T4809. Buildables currently have buildStatus and buildableStatus. Neither are used, and no one knows why we have two.

I'm going to use buildableStatus shortly, but buildStatus is meaningless; burn it.

Test Plan: `grep`, examined similar get/set calls, created a new buildable, ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4809

Differential Revision: https://secure.phabricator.com/D8796
2014-04-17 16:01:06 -07:00
epriestley
ab7d89edc8 Use better secrets in generating account tokens
Summary:
When we generate account tokens for CSRF keys and email verification, one of the inputs we use is the user's password hash. Users won't always have a password hash, so this is a weak input to key generation. This also couples CSRF weirdly with auth concerns.

Instead, give users a dedicated secret for use in token generation which is used only for this purpose.

Test Plan:
  - Ran upgrade scripts.
  - Verified all users got new secrets.
  - Created a new user.
  - Verified they got a secret.
  - Submitted CSRF'd forms, they worked.
  - Adjusted the CSRF token and submitted CSRF'd forms, verified they don't work.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8748
2014-04-10 11:45:10 -07:00
epriestley
582ec54465 Add a checkbox to the LDAP auth configuration UI to "Always Search"
Summary: Fixes T3208. This forces us to bind+search even if there are no anonymous credentials.

Test Plan: Checked the box, saved the form. Unchecked the box, saved the form. LDAP??

Reviewers: Firehed

Reviewed By: Firehed

Subscribers: epriestley

Maniphest Tasks: T3208

Differential Revision: https://secure.phabricator.com/D8723
2014-04-08 11:36:23 -07:00
epriestley
847b7977c1 Add semi-generic rate limiting infrastructure
Summary:
This adds a system which basically keeps a record of recent actions, who took them, and how many "points" they were worth, like:

  epriestley email.add 1 1233989813
  epriestley email.add 1 1234298239
  epriestley email.add 1 1238293981

We can use this to rate-limit actions by examining how many actions the user has taken in the past hour (i.e., their total score) and comparing that to an allowed limit.

One major thing I want to use this for is to limit the amount of error email we'll send to an email address. A big concern I have with sending more error email is that we'll end up in loops. We have some protections against this in headers already, but hard-limiting the system so it won't send more than a few errors to a particular address per hour should provide a reasonable secondary layer of protection.

This use case (where the "actor" needs to be an email address) is why the table uses strings + hashes instead of PHIDs. For external users, it might be appropriate to rate limit by cookies or IPs, too.

To prove it works, I rate limited adding email addresses. This is a very, very low-risk security thing where a user with an account can enumerate addresses (by checking if they get an error) and sort of spam/annoy people (by adding their address over and over again). Limiting them to 6 actions / hour should satisfy all real users while preventing these behaviors.

Test Plan:
This dialog is uggos but I'll fix that in a sec:

{F137406}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8683
2014-04-03 11:22:38 -07:00
epriestley
f239d81c1c Allow very long notes on flags
Summary: Fixes T4703. This is a VARCHAR(255) for no particular reason.

Test Plan: {F136160}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T4703

Differential Revision: https://secure.phabricator.com/D8652
2014-03-30 19:51:46 -07:00
epriestley
d3dbbec88d Rename Releeph "Project" transactions to "Product"
Summary: Ref T3549. This table isn't written to yet; rename it and the DAOs and modernize the history controller.

Test Plan: Viewed history page for a product.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3549

Differential Revision: https://secure.phabricator.com/D8633
2014-03-29 09:15:09 -07:00
Bob Trahan
655ac9927f Workboards - add column detail page
Summary: followup to D8544. This ends up creating an editor + transactions to get the job done.

Test Plan: made a column - saw a nice created transaction. edited the name - saw a nice name edit. deleted the column - saw a deleted transaction, updated "deleted" ui, and hte action change to activate. "Activated" the column and saw a transaction and updated UI. Tried to delete a column with tasks in it and got an error.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D8620
2014-03-26 14:40:47 -07:00
epriestley
543a313387 Skip a very old project reindex migration
Summary:
During migration of very old installs, this script no longer runs properly since at HEAD it can't index against older schemas.

Since it's pretty fluff, just toss it. Installs can run `bin/search index --type PROJ` after finishing migrations to achieve the same effect, if necessary.

Test Plan: eyeballed it

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8619
2014-03-26 13:51:49 -07:00
epriestley
a5f55d506f Provide a real object ("PhabricatorRepositoryPushEvent") to represent an entire push transaction
Summary:
Ref T4677. Currently, we record individual actions in a push as PhabricatorRepositoryPushLogs, but tie them together only loosely with a `transactionKey`.

Provide a real PushEvent object, and move some of the denormalized fields to it. This primarily just gives us more robust infrastructure for building, e.g., email about pushes, for T4677, since we can act on real PHIDs rather than passing awkward identifiers around.

Test Plan:
  - Performed migration.
  - Looked at database for consistency.
  - Browsed/queried push logs.
  - Pushed a bunch of stuff.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4677

Differential Revision: https://secure.phabricator.com/D8615
2014-03-26 13:51:06 -07:00
epriestley
d6b937ca27 Allow external systems to send messages to build targets
Summary:
Ref T1049. Allows external systems to send a message to a build target. The primary intended use case is:

  - You make an HTTP request to Jenkins.
  - The build goes into a "waiting" state.
  - Later, Jenkins calls `harbormaster.sendmessage` to report that the target passed or failed.
  - The build continues as appropriate.

This is deceptively complicated because:

  - There are a lot of race concerns. We might get a message back from an external system before it even responds to the request we made. We want to make sure we process these messages no matter when we receive them.
  - These messages need to be sent to a build target (vs a build or buildable) because we'll get into trouble with parallelization later on otherwise (Jenkins is told to do 3 builds; we can't tell which ones failed or what overall state is unless the message are sent to targets).
  - I initially thought about implementing this as a separate "Wait for a response from an external system" build step. This gets a lot more complicated for users once we do parallelization, though. Particularly, in the case where you've told Jenkins to do 3 builds, the three "wait" steps need to know which target they're waiting for (and jenkins needs to know some unique identifier for each target). So this pretty much boils down to a more complicated, more error-prone version of using target PHIDs.

This makes the already-muddy Build UI a bit worse, but it needs a general clarity pass anyway (it's showing way too much uninteresting data, and should show a better summary of results instead).

Test Plan:
  - This doesn't really do anything interesting yet.
  - Used Conduit to send messages to build plans.
  - Viewed the messages on the build screen.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D8604
2014-03-25 16:11:28 -07:00
epriestley
cec8d10731 Rename concrete Harbormaster step implementations
Summary: Ref T1049. For consistency, rename these to "Harbormaster...".

Test Plan: Ran migration, ran builds, everything still works fine.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D8602
2014-03-25 16:09:51 -07:00
epriestley
a246c85c6b Use ApplicationTransactions and CustomField to implement build steps
Summary:
Ref T1049. Fixes T4602. Moves all the funky field stuff to CustomField. Uses ApplicationTransactions to apply and record edits.

This makes "artifact" fields a little less nice (but still perfectly usable). With D8599, I think they're reasonable overall. We can improve this in the future.

All other field types are better (e.g., fixes weird bugs with "bool", fixes lots of weird behavior around required fields), and this gives us access to many new field types.

Test Plan:
Made a bunch of step edits. Here's an example:

{F133694}

Note that:

  - "Required" fields work correctly.
  - the transaction record is shown at the bottom of the page.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4602, T1049

Differential Revision: https://secure.phabricator.com/D8600
2014-03-25 16:08:40 -07:00
epriestley
0a76d82a7c Use string constants, not integer constants, to represent task status internally
Summary:
Ref T1812. I think integer constants are going to be confusing and error prone for users to interact with. For example, because we use 0-5, adding a second "open" status like "needs verification" without disrupting the existing statuses would require users to define a status with, e.g., constant `6`, but order it between constants `0` and `1`. And if they later remove statuses, they need to avoid reusing existing constants.

Instead, use more manageable string constants like "open", "resolved", etc.

We must migrate three tables:

  - The task table itself, to update task status.
  - The transaction table, to update historic status changes.
  - The saved query table, to update saved queries which specify status sets.

Test Plan:
  - Saved a query with complicated status filters.
  - Ran migrations.
  - Looked at the query, at existing tasks, and at task transactions.
  - Forced migrations to run again to verify idempotentcy/safety.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1812

Differential Revision: https://secure.phabricator.com/D8583
2014-03-25 13:58:14 -07:00
Bob Trahan
809e5a0389 Workboards - let users delete columns
Summary: Fixes T4408. I had to add a "status" to colum. I think we'll need this once we get fancier anyway but for now we have "active" and deleted.

Test Plan: deleted a column. noted reloaded workboard with all those tasks back in the default colun. loaded a task and saw the initial transaction had a "Disabled" icon next to the deleted workboard. also saw the new transaction back to the default column worked.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4408

Differential Revision: https://secure.phabricator.com/D8544
2014-03-18 10:40:31 -07:00
epriestley
5b2887b69b Add "Date Updated" query fields for Maniphest
Summary:
Fixes T4637.

  - We already allow you to order by this column but don't have a key on it. Add one.
  - Expose UI for querying on ranges.

Test Plan:
  - Ran some queries, got reasonable-looking results and no table scans.

Reviewers: btrahan, bigo

Reviewed By: bigo

Subscribers: bigo, epriestley

Maniphest Tasks: T4637

Differential Revision: https://secure.phabricator.com/D8557
2014-03-17 15:53:07 -07:00
epriestley
be92bf182f Drop Maniphest legacy transaction table
Summary: Ref T2217. I'll hold this for a month or so, but once we're confident the migration didn't ruin anything we should nuke this old data -- it's just an insurance policy against discovering migration issues.

Test Plan: Will run in a month or so.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7104
2014-03-12 06:04:45 -07:00
epriestley
2dbfb1d5fb Remove DifferentialComment
Summary: Ref T2222. Remove this; no more callsites.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8477
2014-03-11 13:02:33 -07:00
epriestley
03abde0b25 Fix Diviner links to articles by title
Summary:
Ref T988. This fixes the biggest current problem with Diviner, which is dead links to articles.

In the new Diviner, articles can have both a "name" (derived from the file name, and used in the URI) and a "title" (optional, specified explicitly). For example, we have one document with the name "feedback" and the title "Give Feedback! Get Support!".

On disk, we want to use the name for the actual file where the text lives ("feedback.diviner"). We also want to use the name in the URI, to generate a clean URI and to allow us to retitle the document slightly without breaking links to it (for example, we renamed the "Backup" document to "Backups and Migrations").

However, when displaying the article we want to use the title.

Currently, you can //only// link to the name, not the title. This is inconvenient:

  - We have a bunch of existing docs which link to titles.
  - It's natural/intuitive to link to titles.
  - Linking to titles makes it easier/cheaper to generate documentation, because we don't need to be able to resolve things at render time.

To remedy this, allow links to target either names or titles. If we miss on a name query, we'll do a title query. This is implemented with a slug hash to allow approximately correct titles (wrong case/spacing/punctuation, e.g.) and sidestep all the UTF8/column length issues.

(In the long run, atom resolution should theoretically be more sophistiated than it is now, and we should do render-time lookups on at least some documents to catch bad links. However, this is fairly complicated and a relatively advanced feature, and I think allowing links to titles is desirable no matter what.)

Test Plan: The user documentation book now has valid links to articles when the titles and names differ.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D8407
2014-03-05 12:07:26 -08:00
epriestley
bca0ad8fdd Make "EditPro" controller work with diff updates
Summary:
Ref T2222. Make the "EditPro" controller accommodate diff updates, and support the transaction type. This one is pretty straightforward.

Also make `revisionPHID` in the comments table nullable to fix the "Edit" action.

Test Plan:
  - Created new revision.
  - Updated revision.
  - Tried to do some invalid stuff.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8376
2014-02-28 16:49:22 -08:00
epriestley
cd7171ec6e Migrate old AuxiliaryField storage to modern CustomField storage
Summary: Ref T2222. Ref T3886. Differential has a legacy storage table for auxiliary fields; move the data to modern storage.

Test Plan:
  - Ran migration.
  - Verified fields still worked properly afterward (view, edit, etc).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886, T2222

Differential Revision: https://secure.phabricator.com/D8355
2014-02-26 16:52:30 -08:00
Joshua Spence
6270114767 Various linter fixes.
Summary:
- Removed trailing newlines.
- Added newline at EOF.
- Removed leading newlines.
- Trimmed trailing whitespace.
- Spelling fix.
- Added newline at EOF

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: hach-que, chad, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8344
2014-02-26 12:44:58 -08:00
epriestley
0f0673b9e5 Remove "dateCommitted" field from DifferentialRevision
Summary: Ref T2222. This is obsolete and no longer used. We could deduce it from transactions or commits in modern Phabricator if we wanted it. We may implement a more general mechanism for T4434.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8330
2014-02-25 12:36:14 -08:00
epriestley
70b008d18d Add test coverage that our definition of BMP agrees with MySQL
Summary:
Ref T1191. Test that MySQL's rules match those of `phutil_is_utf8_with_only_bmp_characters()`:

  - Build a string with //every// character that we consider to be a BMP character.
  - Write it into MySQL.
  - Read it back out.
  - Make sure MySQL didn't truncate it.

Test Plan: Ran unit test. This test runs pretty quickly (50ms), the string with every character isn't all that enormous.

Reviewers: btrahan, arice

Reviewed By: arice

CC: chad, arice, aran

Maniphest Tasks: T1191

Differential Revision: https://secure.phabricator.com/D8314
2014-02-23 16:20:38 -08:00
Bob Trahan
dcd7a316d2 Differential - add DifferentialDraft to track whether revisions have draft feedback or not
Summary: ...do it somewhat generically, so we could fairly easily add this to other applications. Fixes T3496. I got a wee bit lazy and decided not to migrate existing drafts. My excuses aside from laziness are doing it this way will let us see if anyone complains, we can always do a migration later if people do complain, and there's likely to be a lot of garbage data for older / bigger installs, and the migration didn't seem worth itgiven it would also likely be expensive in these cases.

Test Plan: made a draft inline comment on DX and observed DX had a note icon on Differential home page. made a draft comment on DX and observed DX had a note icon on Differential home page. deleted a draft inline comment and noted icon disappeared from Differential homepage. Submitted a draft comment + inline comment and noted icon disappeared.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3496

Differential Revision: https://secure.phabricator.com/D8275
2014-02-18 16:25:16 -08:00
epriestley
b96ab5aadf Modernize VCS password storage to use shared hash infrastructure
Summary: Fixes T4443. Plug VCS passwords into the shared key stretching. They don't use any real stretching now (I anticipated doing something like T4443 eventually) so we can just migrate them into stretching all at once.

Test Plan:
  - Viewed VCS settings.
  - Used VCS password after migration.
  - Set VCS password.
  - Upgraded VCS password by using it.
  - Used VCS password some more.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4443

Differential Revision: https://secure.phabricator.com/D8272
2014-02-18 14:09:36 -08:00
epriestley
5778627e41 Provide more storage space for password hashes and migrate existing hashes to "md5:"
Summary: Ref T4443. Provide more space; remove the hack-glue.

Test Plan: Logged out, logged in, inspected database.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4443

Differential Revision: https://secure.phabricator.com/D8269
2014-02-18 14:09:36 -08:00
epriestley
f3cbc0e006 Move many task status hardcodes into ManiphestTaskStatus
Summary:
Ref T1812. This cleans up most of the easy hard-coded references to specific statuses:

  - The "fixes" language moves into ManiphestTaskStatus.
  - Add a method to list open statuses.
  - Add a method to test if a status is open.
  - Add a method to get default status for new tasks.

Test Plan: Browsed around, lint, grep, created, filtered and updated tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1812

Differential Revision: https://secure.phabricator.com/D8264
2014-02-17 15:59:31 -08:00
epriestley
18938b5310 Migrate Differential comments to ApplicationTransactions
Summary:
Ref T2222. This is the big one.

This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions.

The migration is pretty straightforward:

  - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.".
  - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook).
  - If a comment added or removed reviewers, it gets a "changed reviewers" transaction.
  - If a comment added CCs, it gets a "subscribers" transaction.
  - If a comment added comment text, it gets a "comment" transaction.
  - For each inline attached to a comment, we generate an "inline" transaction.

Most comments generate a small number of transactions, but a few generate a significant number.

At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines).

Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table.

NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code.

Specifically, they look like this:

{F112270}

Test Plan:
I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place.
IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master.

I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made.

Reviewers: btrahan

CC: chad, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 14:34:48 -08:00