1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-23 05:50:55 +01:00
Commit graph

925 commits

Author SHA1 Message Date
epriestley
09fb5667cc Allow users to back initiatives in Fund
Summary:
Ref T5835. This is still completely made up (no payment integration), but you can "back" an initiative, type a number in the box, and generate a database row. You can then seach for backers and things you've backed and such.

Notable changes:

  - Renamed "FundBacking" to "FundBacker". The former name was sort of because you can back things multiple times, but stuff like `$backings` was just too weird.
  - I think that's it?

Test Plan:
  - Backed an initiative.
  - Viewed that I became a backer.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5835

Differential Revision: https://secure.phabricator.com/D10486
2014-09-12 06:31:11 -07:00
epriestley
e4f399b9fa Scaffolding for Fund
Summary:
Ref T5835. This is all pretty boilerplate, and does not interact with Phortune at all yet.

You can create "Initiatives", which have a title and description, and support most of the expected infrastructure (policies, transactions, mentions, edges, appsearch, remakrup, etc).

Only notable decisions:

  - Initiatives have an explicit owner. I think it's good to have a single clearly-responsible user behind an initiative.
  - I think that's it?

Test Plan:
  - Created an initiative.
  - Edited an initiative.
  - Changed application policy defaults.
  - Searched for initiatives.
  - Subscribed to an initiative.
  - Opened/closed an initiative.
  - Used `I123` and `{I123}` in remarkup.
  - Destroyed an initiative.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5835

Differential Revision: https://secure.phabricator.com/D10481
2014-09-11 13:38:58 -07:00
Joshua Spence
d6671cbbe1 Don't use parentheses for echo unnecessarily
Summary: As established in D10122.

Test Plan: I basically ran `arc lint --everything --apply-patches`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10437
2014-09-08 10:03:19 +10:00
epriestley
8038af4bd5 Fix old image macros and memes for logged out users
Summary:
Fixes T6013. Old image macros/memes never had the file edge written.

We also never wrote file edges for audio.

Finally, the meme controller didn't allow public access.

Write edges for images and audio, perform a migration to populate the historic ones, and make the Editor keep them up to date going forward.

Test Plan:
  - Updated image, saw new image attach and old image detach.
  - Updated audio, saw new audio attach and old audio detach.
  - Ran migration.
  - Viewed memes as a logged-out user.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6013

Differential Revision: https://secure.phabricator.com/D10411
2014-09-04 12:50:51 -07:00
epriestley
6be8d65763 Convert two missed phutil_utf8_shorten() callsites
Summary: Fixes T6006. These didn't get caught by D10392.

Test Plan: Forced migration to re-run; ran SSH commands against Phabricator.

Auditors: btrahan
2014-08-30 07:20:35 -07:00
Bob Trahan
6f246bd351 Daemons - add a config check for out of date daemon environment
Summary: Fixes T4881.

Test Plan: made a config change, saw the issue, restarted daemons and it went away

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4881

Differential Revision: https://secure.phabricator.com/D10339
2014-08-22 14:52:36 -07:00
James Rhodes
efadfbbc97 Implement build generations in Harbormaster
Summary:
Ref T5932.  Ref T5936.  This implements build generations in Harbormaster, which provides the infrastructure required to both show users the previous states of restarted builds and to allow users to forcefully abort builds (and their targets).

You can view previous generations of a build by adding `?g=<n>` to the URI, but this isn't exposed in the UI anywhere yet.

Test Plan: Ran a build plan with a Sleep step in it.  Reconfigured it for various sleep times and viewed previous generations of the build after restarting it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5932, T5936

Differential Revision: https://secure.phabricator.com/D10321
2014-08-21 22:55:24 +10:00
James Rhodes
6f85c22faf Create an index on harbormaster_buildlog to reduce page load times
Summary: Resolves T5895.  This reduces page load times significantly when looking at builds.

Test Plan: Viewed a build, saw the page load a lot faster.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5895

Differential Revision: https://secure.phabricator.com/D10286
2014-08-20 09:19:57 +10:00
James Rhodes
26f283fe21 Implement passphrase.query for querying credentials
Summary: Resolves T5868.  This implements `passphrase.query` and a mechanism for allowing Conduit access to credentials.

Test Plan: Tested locally.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: talshiri, epriestley, Korvin

Maniphest Tasks: T5868

Differential Revision: https://secure.phabricator.com/D10262
2014-08-16 22:41:03 +10:00
epriestley
607e99490b Migrate "cancdn" to "canCDN" in the database
Summary: Ref T5884. We migrated with "canCDN" and then had live writes with "cancdn". Move everything to "canCDN" for consistency.

Test Plan: Ran migration, verified DB only has "canCDN" afterward.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5884

Differential Revision: https://secure.phabricator.com/D10273
2014-08-15 11:07:40 -07:00
epriestley
e616f166ae Provide a setup warning about using the default MySQL stopword file
Summary:
Fixes T2605.

  - Add a setup warning about the stopword file.
  - Provide a simpler stopword file.

Test Plan:
  - Hit setup warning.
  - Resolved it according to instructions.
  - Added "various" to a task, then searched for it, found the task.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2605

Differential Revision: https://secure.phabricator.com/D10258
2014-08-13 15:34:09 -07:00
epriestley
94389fcd9f Allow projects to be filtered by icon and color
Summary: Ref T5819. Implements basic icon and color filtering for projects.

Test Plan: {F189350}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5819

Differential Revision: https://secure.phabricator.com/D10230
2014-08-12 08:04:38 -07:00
James Rhodes
efc82c727b Measure how long build targets take in Harbormaster
Summary:
Ref T1049.  This keeps track of how long a build target takes to execute in Harbormaster and displays it in the build view page.  I'm not sure whether "Started" is really that useful once the target has completed?

Also, I change the name of the time taken depending on whether or not the target has completed; if it's still in progress it's called "Elapsed" and if it's completed then it's "Duration".  The primary reason for this is that "Duration" sounds like post tense, whereas "Elapsed" is current tense.  I'm not sure whether this is okay or not?

Test Plan: Ran a Sleep build step and saw the target dates / times appear correctly.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: talshiri, epriestley, Korvin

Maniphest Tasks: T5824, T1049

Differential Revision: https://secure.phabricator.com/D10174
2014-08-12 08:34:43 +10:00
epriestley
31e1653a4e Convert pushlog and refcursor to BLOB storage
Summary:
Fixes T5840. Some time ago I incorrectly believed that `latin1_bin` collation was synonymous with "binary". It is not, and does not permit UTF8 characters outside of BMP, among other sequences.

These two tables currently have `LONGTEXT` columns which should be `LONGBLOB`. The table design is explicilty intended to accommodate invalid/unreasonably long ref names, but the collation prevents this from working properly.

After T1191, we'll have a general system for resolving this, but a user hit an issue yesterday (T5840) with a brnach name containing Chinese characters.

Test Plan:
  - Tried emoji inserts into both tables, was rebuffed.
  - Ran migration.
  - Performed emoji inserts into both tables.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5840

Differential Revision: https://secure.phabricator.com/D10217
2014-08-11 12:29:46 -07:00
Mukunda Modell
25ae4c458d Protect file data with a one-time-token
Test Plan: currently untested work in progress

Reviewers: #blessed_reviewers, epriestley

Subscribers: rush898, aklapper, Korvin, epriestley

Projects: #wikimedia

Maniphest Tasks: T5685

Differential Revision: https://secure.phabricator.com/D10054
2014-08-11 07:32:17 -07:00
epriestley
24a6eeb8d8 Allow the workboard backlog column to be reordered
Summary:
Fixes T5677.

  - Instead of using `sequence == 0` to mean "this is the backlog column", flag the column explicitly.
  - Migrate existing sequence 0 columns to have the flag.
  - Add the flag when initializing or copying a board.
  - Remove special backlog logic when reordering columns.

Test Plan:
  - Migrated columns, viewed some boards, they looked identical.
  - Reordered the backlog column a bunch of times (first, last, middle, dragged other stuff around).
  - Added tasks to a project, saw them show up in the reordered backlog.
  - Initialized a new board and saw a backlog column show up.
  - Copied an existing board and saw the backlog column come over.
  - Tried to hide a backlog column.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5677

Differential Revision: https://secure.phabricator.com/D10189
2014-08-08 15:50:36 -07:00
epriestley
09868271bd Move board relationships to dedicated storage
Summary:
Fixes T5476. Using edges to store which objects are on which board columns ends up being pretty awkward. In particular, it makes T4807 very difficult to implement.

Introduce a dedicated `BoardColumnPosition` storage.

This doesn't affect ordering rules (T4807) yet: boards are still arranged by priority. We just read which tasks are on which columns out of a new table.

Test Plan:
  - Migrated data, then viewed some boards. Saw exactly the same data.
  - Dragged tasks from column to column.
  - Created a task directly into a column.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5476

Differential Revision: https://secure.phabricator.com/D10160
2014-08-06 15:09:09 -07:00
Joshua Spence
8fd098329b Rename AphrontQueryException subclasses
Summary: Ref T5655. Depends on D10149.

Test Plan: Ran `arc unit`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin, hach-que

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D10150
2014-08-06 07:51:21 +10:00
epriestley
89b942c183 Move Audit to proper Subscriptions
Summary:
Ref T4896. Currently, subscriptions to commits are stored as auditors with a special "CC" type.

Instead, use normal subscriptions storage, reads and writes.

Test Plan:
  - Ran migration and verified data still looked good.
  - Viewed commits in UI and saw "subscribers".
  - Saw "Automatically Subscribed", clicked Subscribe/Unsubscribe on a non-authored commit, saw subscriptions update.
  - Pushed a commit through Herald rules and saw them trigger subscriptions and auditors.
  - Used "Add CCs".
  - Added CCs with mentions.

Reviewers: btrahan, joshuaspence

Reviewed By: btrahan, joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10103
2014-08-02 00:06:13 -07:00
James Rhodes
dfa9b27a94 Use tabs on build targets and allow build steps to have a description
Summary:
Ref T1049. This uses tabs on build targets to hide the configuration details and variables by default, instead promoting the target name, it's status and a description of the build step.  The description is a new field on each build step.

The primary advantage of having a description on build steps is that DevOps can configure appropriate description information (including any troubleshooting information for build failures) on build steps, and developers who have builds fail against their code review can then look at this information.

Test Plan: Viewed a build plan and saw the appropriate information.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D10093
2014-08-01 08:09:15 +10:00
epriestley
fa2fcc7852 Fix missed migration constant from PHIDType rename
Summary: See D9986, we missed this one.

Test Plan: `grep`

Reviewers: joshuaspence, btrahan, bizrad6

Reviewed By: bizrad6

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D10098
2014-07-31 11:30:40 -07:00
James Rhodes
cad41ea294 Implement build simulation; convert Harbormaster to be purely dependency based
Summary:
Depends on D9806.  This implements the build simulator, which is used to calculate the order of build steps in the plan editor.  This includes a migration script to convert existing plans from sequential based to dependency based, and then drops the sequence column.

Because build plans are now dependency based, the grippable and re-order behaviour has been removed.

Test Plan: Tested the migration, saw the dependencies appear correctly.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9847
2014-07-31 11:39:49 +10:00
epriestley
f965126dc4 Migrate audit comments to transactions
Summary:
Ref T4896. Depends on D10052. This is the major/scary migration, but not really so bad. It is substantially similar to D8210, but less complex because there are fewer actions here.

This moves `PhabricatorAuditComment` storage to `PhabricatorAuditTransaction`, then reads `PhabricatorAuditComment`s as a proxy around the new objects.

Test Plan:
  - Before migrating, browsed around. Nothing appeared broken.
  - Migrated cleanly.
  - Viewed old transactions (inlines, comments, accept/reject/etc, add auditors, add ccs, implicit CCs).
  - Added all of those comment types.
  - Edited a draft.
  - Deleted a draft.
  - Spot checked the database for sanity.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10055
2014-07-28 15:00:46 -07:00
epriestley
3d78c0eff7 Migrate Audit comment text into new storage
Summary:
Ref T4896. This is substantially similar to D8196.

Migrate the comment text out of the `audit_comment` table and into the `audit_transaction_comment` table. Do double reads on `PhabricatorAuditComment` so the APIs aren't disturbed. The old table is still updated.

Test Plan:
  - Before applying migration, cleared cache and browsed around. Things looked fine, except no comment text.
  - Applied migration.
  - Cleared cache, browsed around, saw all my old comments.
  - Added some new comments.
  - Spot checked migrated and new rows in database.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10020
2014-07-24 18:00:30 -07:00
epriestley
c01aa794c1 Migrate Audit inline comments to new storage
Summary:
Ref T4896. This is substantially identical to the process which Differential followed, and mostly copied from the original Differential migration and the Differential proxy object.

Basically, we move all the data over but the application can't tell, and the same APIs do reads and writes to the new table.

Test Plan:
 - Browsed UI before migrating, everything looked fine (but no inlines).
 - Ran migration.
 - Verified draft and published comments survived migration.
 - Added a draft.
 - Previewed draft.
 - Submitted draft.
 - Viewed standalone with drafts and published comments.
 - Sanity checked data in database, didn't see anything unusual.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10018
2014-07-24 17:59:54 -07:00
epriestley
416f3d9ede Add storage for new audit transactions and comments
Summary:
Ref T4896. This adds the new storage, without any code changes.

This storage is substantially identical to the Differential storage, except that `changesetID` has been replaced by `pathID`.

I've retained the properties intended to be used to implement T1460. They might not be quite right, but at least we'll be able to make any fixes consistently to both applications. For now, these fields are empty and ignored.

Test Plan: Ran `./bin/storage upgrade`. Nothing calls this code yet.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10017
2014-07-24 17:59:43 -07:00
Joshua Spence
97a8700e45 Rename PHIDType classes
Summary: Ref T5655. Rename `PhabricatorPHIDType` subclasses for clarity (see discussion in D9839). I'm not too keen on some of the resulting class names, so feel free to suggest alternatives.

Test Plan: Ran unit tests.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin, hach-que

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9986
2014-07-24 08:05:46 +10:00
epriestley
d6eb1c67e7 Phortune Cart Status, some one-time support
Summary:
Ref T2787. Carts need a status so we can tell if they've been purchased. Also kind of get WePay working as a one-time provider, and let charges not have a methodPHID (they won't for one-time providers).

All the status stuff is still super crazy rough and you can do things like start a checkout, add a bunch of stuff to your cart, complete the checkout, and have Phabricator think you paid for all the stuff you added. But this is fine for now since you can't actually edit carts, and also none of this is at all usable anyway. I'll refine some of the workflows in future diffs, for now I'm just getting things hooked up and technically working.

Test Plan:
  - Purcahsed a cart and got a sort of status/done screen instead of a "your money is gone" exception.
  - Went through the WePay flow and got a successful test checkout.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D10003
2014-07-23 10:36:25 -07:00
epriestley
4c0f15b94b Phortune Charges
Summary: Ref T2787. Makes charges a real object, allows providers to apply them. We are now (just barely) capable of stealing users' money.

Test Plan: {F179584}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D10002
2014-07-23 10:36:12 -07:00
epriestley
6ec1f35870 Phortune Carts and Purchases
Summary:
Ref T2787. Make carts and purchases real objects, with storage, that kind-of work.

Roughly, the idea here is that applications create "purchases" (like "1 large t-shirt") and add them to "carts" (a user can have a lot of different carts at the same time), then hand things off to Phortune to deal with actualy charging a card. Roughly this works like Paypal or other similar systems do, except Phortune is the thing the user gets handed off to.

This doesn't do anything interesting/useful yet.

Also fix some bugs and update some UI.

Test Plan: Added a product to a cart, saw it in cart screen.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D10001
2014-07-23 10:34:08 -07:00
epriestley
e561a5fe73 Fix incorrect old name for Asana auth provider in provider migration
Summary: The old class name isn't quite correct.

I'm just updating the migration rather than adding a new one to fix it since
this was very recently introduced and affects only installs using Asana auth,
so it's realistic that the number of affected installs might be 0.

Affected installs can use `--apply` to safely rerun the migration.

Auditors: joshuaspence
2014-07-23 09:46:06 -07:00
Joshua Spence
e0a7d47e0d Add a missing migration for transaction actors
Summary: Add a missing migration which should have been included in D9982. Harbormaster and Herald PHIDs are used as actors in some transactions.

Test Plan: Ran `./bin/storage upgrade`. Saw a transaction render correctly as "Herald assigned this task to alincoln" instead of "Unknown Object (Application) assigned this task to alincoln".

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10028
2014-07-23 23:49:22 +10:00
Joshua Spence
86c399b657 Rename PhabricatorApplication subclasses
Summary: Ref T5655. Some discussion in D9839. Generally speaking, `Phabricator{$name}Application` is clearer than `PhabricatorApplication{$name}`.

Test Plan:
# Pinned and uninstalled some applications.
# Applied patch and performed migrations.
# Verified that the pinned applications were still pinned and that the uninstalled applications were still uninstalled.
# Performed a sanity check on the database contents.

Reviewers: btrahan, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: hach-que, epriestley, Korvin

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9982
2014-07-23 10:03:09 +10:00
Joshua Spence
458cf8c898 Add a missing migration
Summary: This migration script is required for D9999, which has already landed.

Test Plan: Ran `./bin/storage upgrade` and can log in again.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10010
2014-07-22 06:26:41 -07:00
Joshua Spence
f4b05312cd Fix broken references to auth adapters
This was broken in D9999 but somehow didn't fail linting or unit tests.

Auditors: epriestley
2014-07-22 21:20:45 +10:00
epriestley
a115810912 Give projects a proper on-demand datasource
Summary:
Fixes T5614. Ref T4420. Other than the "users" datasource and a couple of others, many datasources ignore what the user typed and just return all results, then rely on the client to filter them.

This works fine for rarely used ("legalpad documents") or always small ("task priorities", "applications") datasets, but is something we should graudally move away from as datasets get larger.

Add a token table to projects, populate it, and use it to drive the datasource query. Additionally, expose it on the applicationsearch UI.

Test Plan:
  - Ran migration.
  - Manually checked the table.
  - Searched for projects by name from ApplicationSearch.
  - Searched for projects by name from typeahead.
  - Manually checked the typeahead response.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5614, T4420

Differential Revision: https://secure.phabricator.com/D9896
2014-07-17 16:35:54 -07:00
epriestley
aa79539789 Move task/project storage to edges
Summary: Ref T5245. This moves the actual storage over and stops reads and writes to the old table.

Test Plan:
  - Verified tasks retained projects across the migration.
  - Added and removed projects from tasks.
  - Searched for: all, any, users' projects, not-in-projects, no-projects.

Reviewers: chad, btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9850
2014-07-17 15:42:30 -07:00
epriestley
8cbfb49b4e Remove all edge events
Summary:
Ref T5245. These were a bad idea.

We no longer need actors for edge edits either, so remove those. Generally, edges have fit into the policy model as pure/low-level infrastructure, and they do not have any policy or capability information in and of themselves.

Test Plan: `grep`

Reviewers: chad, btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9840
2014-07-17 15:41:42 -07:00
epriestley
533e799c5f Modernize task/revision edges and write inverse transactions
Summary:
Ref T5245. See some discussion in D9838.

When we attach object A to object B, we'd like to write transactions on both sides but only write the actual edges once.

To do this, allow edge types to `shouldWriteInverseTransactions()`. When an edge type opts into this, have editors apply the inverse transactions before writing the edge. These inverse transactions don't actually apply effects, they just show up in the transaction log.

Test Plan: Attached and detached revisions from tasks, saw transactions appear on both sides of the operation.

Reviewers: chad, btrahan, joshuaspence

Reviewed By: btrahan, joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9839
2014-07-17 15:41:08 -07:00
epriestley
ca6b3a632d Remove uniqueness constraint from "sequence" key in project column table
Summary: Fixes T5611. We don't need sequences to be unique, and it makes it a pain to update them.

Test Plan: Dragged some columns around.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5611

Differential Revision: https://secure.phabricator.com/D9914
2014-07-12 11:24:35 -07:00
Joshua Spence
9a679bf374 Allow worker tasks to have priorities
Summary: Fixes T5336. Currently, `PhabricatorWorkerLeaseQuery` is basically FIFO. It makes more sense for the queue to be a priority-queue, and to assign higher priorities to alerts (email and SMS).

Test Plan: Created dummy tasks in the queue (with different priorities). Verified that the priority field was set correctly in the DB and that the priority was shown on the `/daemon/` page. Started a `PhabricatorTaskmasterDaemon` and verified that the higher priority tasks were executed before lower priority tasks.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5336

Differential Revision: https://secure.phabricator.com/D9871
2014-07-12 03:02:06 +10:00
epriestley
eb28a7caef Add an optional preamble to Legalpad documents
Summary:
Fixes T5532. Allow documents to have a preamble in the header which can be used to explain who should sign a document and why.

Particularly, I plan to use this to navigate the corporate vs individual stuff more sensibly.

Test Plan: {F174228}

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5532

Differential Revision: https://secure.phabricator.com/D9819
2014-07-04 09:41:27 -07:00
James Rhodes
a3d50118e1 Allow users to specify names of build steps
Summary: Ref T1049.  This provides a user-configurable name field on build steps, which allows users to uniquely identify their steps.  The intention is that this field will be used in D9806 to better identify the dependencies (rather than showing an unhelpful PHID).

Test Plan: Set the name of some build steps, saw it appear in the correct places.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D9816
2014-07-05 01:56:02 +10:00
epriestley
e8d217b8bd Allow Legalpad documents to designate corporate signers
Summary:
Ref T5532. This adds:

  - Documents can designate that they should be signed by "Corporations" or "Individuals".
  - Corporate documents get different fields and a different exemption process.
  - Basically everything works the same but this is like a zillion lines of form code.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5532

Differential Revision: https://secure.phabricator.com/D9812
2014-07-04 08:04:28 -07:00
epriestley
b6ea2735d7 Allow Legalpad document managers to add signature exemptions
Summary:
Ref T5532. Allow document managers to add exemptions, which act like signatures but are tracked a little differently.

The primary use case for us is users who sign a corporate CLA and need a user-level exemption if they don't want to sign an individual CLA.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5532

Differential Revision: https://secure.phabricator.com/D9795
2014-07-02 04:59:35 -07:00
epriestley
c9366acbec Allow dashboard panels to be archived
Summary: Ref T5471. Adds an archived state for panels. Archived panels don't show up in the default query view or in the "Add Existing Panel" workflow.

Test Plan:
  - Archived a panel.
  - Activated a panel.
  - Viewed / searched for archived/active panels.
  - Popped "Add Existing Panel" dropdown and saw it omit archived panels.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5471

Differential Revision: https://secure.phabricator.com/D9779
2014-07-01 17:50:28 -07:00
epriestley
ffc1b5c26a Allow users to search for signatures by name and email substrings
Summary:
Ref T3116. In the case of anonymous signers, there's no way to do a quick way to check if someone has signed a doc since you can't query by their (nonexistent) external account ID.

Move "name" and "email" to first-class columns and let the engine search for them.

Test Plan: Searched for signatures with name and email fragments.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T3116

Differential Revision: https://secure.phabricator.com/D9776
2014-06-29 07:51:03 -07:00
epriestley
be47f2141a Allow projects icon color to be selected from several fabulous shades
Summary: This further helps differentiate types/roles for projects.

Test Plan: {F169758}

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9710
2014-06-25 22:01:58 -07:00
epriestley
9c423647d1 Drop DifferentialComment and DifferentialInlineComment
Summary: Ref T2222. No remaining readers or writers. Holding for a while.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8242
2014-06-24 05:29:54 -07:00
epriestley
1d90577d8c Drop old Differential custom field storage
Summary: Ref T2222. See D8355. I'll hold this for a while.

Test Plan: Ran migration.

Reviewers: chad, btrahan

Reviewed By: chad

Subscribers: epriestley, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8356
2014-06-22 08:15:14 -07:00
epriestley
490ae05eb8 Drop differential_relationship table
Summary: Ref T2222. I'll hold this, but there are no more reads or writes from/to this table in the application.

Test Plan: Grepped for usage, ran migration, browsed around.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8203
2014-06-22 08:12:11 -07:00
Joshua Spence
f52fbf6117 Unify the local and global view for ./bin/phd status.
Summary:
Ref T4209. Unifies the local (`./bin/phd status`) and global (`./bin/phd status --all`) view into a single table. This generally makes it easy to administer daemons running across multiple hosts.

Depends on D9606.

Test Plan:
```
> sudo ./bin/phd status
ID Host      PID  Started                 Daemon                               Arguments
38 localhost 2282 Jun 18 2014, 7:52:56 AM PhabricatorRepositoryPullLocalDaemon
39 localhost 2289 Jun 18 2014, 7:52:57 AM PhabricatorGarbageCollectorDaemon
40 localhost 2294 Jun 18 2014, 7:52:57 AM PhabricatorTaskmasterDaemon
41 localhost 2314 Jun 18 2014, 7:52:58 AM PhabricatorTaskmasterDaemon
42 localhost 2319 Jun 18 2014, 7:52:59 AM PhabricatorTaskmasterDaemon
43 localhost 2328 Jun 18 2014, 7:53:00 AM PhabricatorTaskmasterDaemon
44 localhost 2354 Jun 18 2014, 7:53:08 AM PhabricatorRepositoryPullLocalDaemon X --not Y
```

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4209

Differential Revision: https://secure.phabricator.com/D9607
2014-06-18 11:44:52 +10:00
epriestley
df3736e81e Garbage collect daemon logs
Summary:
We already have GC for daemon log events, but not for daemon logs themselves.

Collect old daemon logs which aren't still running.

Test Plan: Ran `phd debug garbage`, observed old logs get cleaned up. Started some daemons, re-ran garbage, made sure they stuck around.

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9610
2014-06-17 15:33:08 -07:00
epriestley
868ff166b1 Give Pholio mocks a configurable edit policy
Summary:
Ref T4566. Currently, mocks have a conservative (author only), immutable default edit policy.

Instead:

  - Let the edit policy be changed.
  - Default the edit policy to "all users", similar to other applications.
  - Add an application-level setting for it.
  - Migrate existing edit policies to be consistent with the old policy (just the author).

This stops short of adding a separate "owner" and letting that be changed, since Pholio doesn't really have any review/approve type features (at least, so far). We can look at doing this if we get more feedback about it, or if we make owners more meaningful (e.g., add more "review-like" process to mocks).

Test Plan:
  - Ran migration scripts.
  - Confirmed existing mocks retained their effective policies (author only).
  - Created a new mock, saw edit policy.
  - Changed edit policy.
  - Changed global edit policy default.
  - Tried to edit a mock I couldn't edit.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T4566

Differential Revision: https://secure.phabricator.com/D9550
2014-06-15 10:28:16 -07:00
Joshua Spence
d0128afa29 Applied various linter fixes.
Summary: Applied some more linter fixes that I previously missed because my global `arc` install was out-of-date.

Test Plan: Will run `arc unit` on another host.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9443
2014-06-09 16:04:12 -07:00
Joshua Spence
0a62f13464 Change double quotes to single quotes.
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.

Test Plan: Eyeballed it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9431
2014-06-09 11:36:50 -07:00
epriestley
0aa913805d Add an alternate "modern" hunk datastore
Summary:
Ref T4045. Ref T5179. Hunk storage has two major issues:

  - It's utf8, but actual diffs are binary.
  - It's huge and can't be compressed or archived.

This introduces a second datastore which solves these problems: by recording hunk encoding, supporting compression, and supporting alternate storage. There's no actual compression or storage support yet, but there's space in the table for them.

Since nothing actually uses hunk IDs, it's fine to have these tables exist at the same time and use the same IDs. We can migrate data between the tables gradually without requiring downtime or disrupting installs.

Test Plan:
  - There are no writes to the new table yet.
  - The only effect this has is making us issue one extra query when looking for hunks.
  - Observed the query issue, but everything else continue working fine.
  - Created a new diff.
  - Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4045, T5179

Differential Revision: https://secure.phabricator.com/D9290
2014-06-03 18:01:22 -07:00
epriestley
99c72a32d0 Allow installs to require multi-factor authentication for all users
Summary: Ref T5089. Adds a `security.require-multi-factor-auth` which forces all users to enroll in MFA before they can use their accounts.

Test Plan:
Config:

{F159750}

Roadblock:

{F159748}

After configuration:

{F159749}

  - Required MFA, got roadblocked, added MFA, got unblocked.
  - Removed MFA, got blocked again.
  - Used `bin/auth strip` to strip MFA, got blocked.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5089

Differential Revision: https://secure.phabricator.com/D9285
2014-06-03 16:50:27 -07:00
epriestley
63ed126b2a Point github.com/facebook URIs at github.com/phacility insead
Summary: Point everything at the new canonical URI.

Test Plan: `grep`

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9328
2014-05-29 08:33:25 -07:00
Bob Trahan
102befdede Project - add ability to select an icon for typeaheads and such
Summary: Fixes T5090. Introduced getIcon into Handle stack which allows you to specify a per handle icon. getIcon falls back ot getTypeIcon.

Test Plan: changed the icon on a project a bunch. verified transactions showed up. verified icon showed up in typeahead. verified icon showed up in tokens that were pre-generated (not typed in). units test passed.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5090

Differential Revision: https://secure.phabricator.com/D9264
2014-05-23 10:41:24 -07:00
Bob Trahan
922e5c0849 Projects - add "Additional Hashtags" to projects
Summary:
Fixes T4021. Chooses to keep a "primary" slug based off the name - including all that lovely logic - and allow the user to specify "additional" slugs. Expose these as "hashtags" to the user.

Sets us up for a fun diff where we can delete all the Project => Phriction automagicalness. In terms of this diff, see the TODOs i added.

Test Plan:
added a primary slug as an additional slug - got an error. added a slug in use on another project - got an error. added multiple good slugs and they worked. removed slugs and it worked. made some remark using multiple new slugs and they all linked to the correct project

ran epriestley's case

 - Create project "A".
 - Give it additional slug "B".
 - Try to create project "B".

and i got a nice error about hashtag collision

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4021

Differential Revision: https://secure.phabricator.com/D9250
2014-05-22 11:19:03 -07:00
epriestley
cac61980f9 Add "temporary tokens" to auth, for SMS codes, TOTP codes, reset codes, etc
Summary:
Ref T4398. We have several auth-related systems which require (or are improved by) the ability to hand out one-time codes which expire after a short period of time.

In particular, these are:

  - SMS multi-factor: we need to be able to hand out one-time codes for this in order to prove the user has the phone.
  - Password reset emails: we use a time-based rotating token right now, but we could improve this with a one-time token, so once you reset your password the link is dead.
  - TOTP auth: we don't need to verify/invalidate keys, but can improve security by doing so.

This adds a generic one-time code storage table, and strengthens the TOTP enrollment process by using it. Specifically, you can no longer edit the enrollment form (the one with a QR code) to force your own key as the TOTP key: only keys Phabricator generated are accepted. This has no practical security impact, but generally helps raise the barrier potential attackers face.

Followup changes will use this for reset emails, then implement SMS multi-factor.

Test Plan:
  - Enrolled in TOTP multi-factor auth.
  - Submitted an error in the form, saw the same key presented.
  - Edited the form with web tools to provide a different key, saw it reject and the server generate an alternate.
  - Change the expiration to 5 seconds instead of 1 hour, submitted the form over and over again, saw it cycle the key after 5 seconds.
  - Looked at the database and saw the tokens I expected.
  - Ran the GC and saw all the 5-second expiry tokens get cleaned up.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D9217
2014-05-20 11:43:45 -07:00
Bob Trahan
5f33aa5b4f Dashboards - add ability to install dashboard as home
Summary:
See title. Adds PhabricatorDashboardInstall data object which scopes installs to objectPHID + applicationClass. This is because we already have a collision for user home pages and user profiles. Assume only one dashboard per objectPHID + applicationClass though at the database level.

Fixes T5076.

Test Plan: From dashboard view, installed a dashboard - success! Went back to dashboard view and uninstalled it!

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5076

Differential Revision: https://secure.phabricator.com/D9206
2014-05-19 16:09:31 -07:00
lkassianik
3d457a53be Close pholio mocks
Summary: Fixes T4299, Add status dropdown to mock edit view

Test Plan: Edit mock, close mock, thumbnail title should read (Disabled). Default mocks list should show only open mocks.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: chad, epriestley, Korvin

Maniphest Tasks: T4299

Differential Revision: https://secure.phabricator.com/D9145
2014-05-19 11:34:23 -07:00
epriestley
15c408be1d Change LONGTEXT cache column to BINARY
See T4898. As it turns out, `latin1` LONGTEXT is not approximately equivalent
to LONGBLOB.
2014-05-17 22:38:56 -07:00
epriestley
d744d5d859 Fix binary/utf8 issues with Differential changeset parse cache
Summary:
Fixes T4898. After we increased the strictness of the `%s` conversion, most `serialize()` output is rejected from the cache.

Drop the cache, change the column type to latin1_bin, and then use `%B` to mark the data as binary during query construction.

Test Plan: Viewed Differential, saw cache fills.

Reviewers: btrahan, spicyj

Reviewed By: spicyj

Subscribers: epriestley

Maniphest Tasks: T4898

Differential Revision: https://secure.phabricator.com/D9171
2014-05-17 16:34:13 -07:00
epriestley
a74545c9da Provide a rough, unstable API for reporting coverage into Diffusion
Summary:
Ref T4994. This stuff works:

  - You can dump a blob of coverage information into `diffusion.updatecoverage`. This wipes existing coverage information and replaces it.
  - It shows up when viewing files.
  - It shows up when viewing commits.

This stuff does not work:

  - When viewing files, the Javascript hover interaction isn't tied in yet.
  - We always show this information, even if you're behind the commit where it was generated.
  - You can't do incremental updates.
  - There's no aggregation at the file (this file has 90% coverage), diff (the changes in this commit are 90% covered), or directory (the code in this directory has 90% coverage) levels yet.
  - This is probably not the final form of the UI, storage, or API, so you should expect occasional changes over time. I've marked the method as "Unstable" for now.

Test Plan:
  - Ran `save_lint.php` to check for collateral damage; it worked fine.
  - Ran `save_lint.php` on a new branch to check creation.
  - Published some fake coverage information.
  - Viewed an affected commit.
  - Viewed an affected file.

{F151915}

{F151916}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: jhurwitz, epriestley, zeeg

Maniphest Tasks: T5044, T4994

Differential Revision: https://secure.phabricator.com/D9022
2014-05-17 16:10:54 -07:00
Tal Shiri
43d45c4956 can now tell phabricator you trust an auth provider's emails (useful for Google OAuth), which will mark emails as "verified" and will skip email verification.
Summary: This is useful when you're trying to onboard an entire office and you end up using the Google OAuth anyway.

Test Plan: tested locally. Maybe I should write some tests?

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9150
2014-05-16 14:14:06 -07:00
Bob Trahan
6300955661 Dashboards - add layout mode to dashboards
Summary:
This gets us the ability to specify a "layout mode" and which column a panel should appear in at panel add time. Changing the layout mode from a multi column view to a single column view or vice versa will reset all panels to the left most column.

You can also drag and drop where columns appear via the "arrange" mode.

We also have a new dashboard create flow. Create dashboard -> arrange mode. (As opposed to view mode.) This could all possibly use massaging.

Fixes T4996.

Test Plan:
made a dashboard with panels in multiple columns. verified correct widths for various layout modes

re-arranged collumns like whoa.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4996

Differential Revision: https://secure.phabricator.com/D9031
2014-05-15 19:12:40 -07:00
Aviv Eyal
f2c0e94ea8 Show command transactions in Harbormaster builds
Summary:
Create transaction, editor, etc, and move command generation over to editor.
Show in a timeline in the buildable page.

Also prevent Engine from creating an empty transaction when build starts (Fixes T4885).

Fixes T4886.

Test Plan: Restart builds and buildables, look at timeline.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4885, T4886

Differential Revision: https://secure.phabricator.com/D9110
2014-05-15 07:04:34 -07:00
epriestley
4a7499f230 Fix credential upgrade issue after introduction of isLocked column
Summary: Fixes T5035. This migration isn't forward compatible after schema mutation.

Test Plan: Ran locally, will get reporting user to confirm.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: gera, epriestley

Maniphest Tasks: T5035

Differential Revision: https://secure.phabricator.com/D9101
2014-05-13 12:14:27 -07:00
epriestley
95eab2f3b0 Record parent relationships when discovering commits
Summary:
Ref T4455. This adds a `repository_parents` table which stores `<childCommitID, parentCommitID>` relationships.

For new commits, it is populated when commits are discovered.

For older commits, there's a `bin/repository parents` script to rebuild the data.

Right now, there's no UI suggestion that you should run the script. I haven't come up with a super clean way to do this, and this table will only improve performance for now, so it's not important that we get everyone to run the script right away. I'm just leaving it for the moment, and we can figure out how to tell admins to run it later.

The ultimate goal is to solve T2683, but solving T4455 gets us some stuff anyway (for example, we can serve `diffusion.commitparentsquery` faster out of this cache).

Test Plan:
  - Used `bin/repository discover` to discover new commits in Git, SVN and Mercurial repositories.
  - Used `bin/repository parents` to rebuild Git and Mercurial repositories (SVN repos just exit with a message).
  - Verified that the table appears to be sensible.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: jhurwitz, epriestley

Maniphest Tasks: T4455

Differential Revision: https://secure.phabricator.com/D9044
2014-05-12 11:47:22 -07:00
Bob Trahan
e96c363eef Add SMS support
Summary:
Provides a working SMS implementation with support for Twilio.

This version doesn't really retry if we get any gruff at all. Future versions should retry.

Test Plan: used bin/sms to send messages and look at them.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: aurelijus, epriestley, Korvin

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D8930
2014-05-09 12:47:21 -07:00
epriestley
bd7420c4bb Allow pastes to be edited
Summary: Fixes T4814.

Test Plan: Edited pastes from the web UI.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4814

Differential Revision: https://secure.phabricator.com/D8970
2014-05-04 11:11:46 -07:00
epriestley
7fb89686d4 Upstream a patch to cover a migration issue
Summary: Fixes T4928. I'm not sure how this column was missing, but this patch can't hurt.

Test Plan: Reasoned about behavior.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4928

Differential Revision: https://secure.phabricator.com/D8967
2014-05-04 10:48:25 -07:00
lkassianik
d7b7b19337 Add a "Lock Permanently" action to Passphrase
Summary: Fixes T4931. Each new credential should come with the ability to lock the credential permanently, so that no one can ever edit again. Each existing credential must allow user to lock existing credential.

Test Plan: Create new credential, verify that you can lock it before saving it. Open existing unlocked credential, verify that option to lock it exists. Once credential is locked, the option to reveal it should be disabled, and editing the credential won't allow username/password updates.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4931

Differential Revision: https://secure.phabricator.com/D8947
2014-05-02 18:21:51 -07:00
epriestley
2022a70e16 Implement bin/remove, for structured destruction of objects
Summary:
Ref T4749. Ref T3265. Ref T4909. Several goals here:

  - Move user destruction to the CLI to limit the power of rogue admins.
  - Start consolidating all "destroy named object" scripts into a single UI, to make it easier to know how to destroy things.
  - Structure object destruction so we can do a better and more automatic job of cleaning up transactions, edges, search indexes, etc.
  - Log when we destroy objects so there's a record if data goes missing.

Test Plan: Used `bin/remove destroy` to destroy several users.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3265, T4749, T4909

Differential Revision: https://secure.phabricator.com/D8940
2014-05-01 18:23:31 -07:00
epriestley
50376aad04 Require multiple auth factors to establish web sessions
Summary:
Ref T4398. This prompts users for multi-factor auth on login.

Roughly, this introduces the idea of "partial" sessions, which we haven't finished constructing yet. In practice, this means the session has made it through primary auth but not through multi-factor auth. Add a workflow for bringing a partial session up to a full one.

Test Plan:
  - Used Conduit.
  - Logged in as multi-factor user.
  - Logged in as no-factor user.
  - Tried to do non-login-things with a partial session.
  - Reviewed account activity logs.

{F149295}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D8922
2014-05-01 10:23:02 -07:00
epriestley
941f0ba7ae Allow panels to appear on dashboards
Summary:
Ref T3583. Adds edges, query relationships, etc. Lots of debugging/temporary UI.

My general intent here is to use edges to track where panels appear, and then put additional data on the dashboard itself to control layout, positioning, etc.

Dashboards don't actually render yet so this is still pretty boring.

Test Plan:
{F149175}

{F149176}

{F149177}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8916
2014-04-30 14:28:55 -07:00
epriestley
ea954c37e4 Add dashboard panel types
Summary: Ref T3583. These will be the primary class carrying panel implementations.

Test Plan:
{F149125}

{F149126}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8912
2014-04-30 14:28:20 -07:00
epriestley
17709bc167 Add multi-factor auth and TOTP support
Summary:
Ref T4398. This is still pretty rough and isn't exposed in the UI yet, but basically works. Some missing features / areas for improvement:

  - Rate limiting attempts (see TODO).
  - Marking tokens used after they're used once (see TODO), maybe. I can't think of ways an attacker could capture a token without also capturing a session, offhand.
  - Actually turning this on (see TODO).
  - This workflow is pretty wordy. It would be nice to calm it down a bit.
  - But also add more help/context to help users figure out what's going on here, I think it's not very obvious if you don't already know what "TOTP" is.
  - Add admin tool to strip auth factors off an account ("Help, I lost my phone and can't log in!").
  - Add admin tool to show users who don't have multi-factor auth? (so you can pester them)
  - Generate QR codes to make the transfer process easier (they're fairly complicated).
  - Make the "entering hi-sec" workflow actually check for auth factors and use them correctly.
  - Turn this on so users can use it.
  - Adding SMS as an option would be nice eventually.
  - Adding "password" as an option, maybe? TOTP feels fairly good to me.

I'll post a couple of screens...

Test Plan:
  - Added TOTP token with Google Authenticator.
  - Added TOTP token with Authy.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D8875
2014-04-28 09:27:11 -07:00
epriestley
f42ec84d0c Add "High Security" mode to support multi-factor auth
Summary:
Ref T4398. This is roughly a "sudo" mode, like GitHub has for accessing SSH keys, or Facebook has for managing credit cards. GitHub actually calls theirs "sudo" mode, but I think that's too technical for big parts of our audience. I've gone with "high security mode".

This doesn't actually get exposed in the UI yet (and we don't have any meaningful auth factors to prompt the user for) but the workflow works overall. I'll go through it in a comment, since I need to arrange some screenshots.

Test Plan: See guided walkthrough.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4398

Differential Revision: https://secure.phabricator.com/D8851
2014-04-27 17:31:11 -07:00
lkassianik
9a827096a7 Ability to close poll
Summary: Fixes T3566 List of poll actions should include ability to close an open poll or reopen a closed poll.

Test Plan: Poll author should be able to close/reopen poll. Non-author should get policy screen when attempting to close/reopen poll.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T3566

Differential Revision: https://secure.phabricator.com/D8846
2014-04-24 12:02:56 -07:00
epriestley
2ac8457cb9 [Later] Drop the project profile table
Summary: Going to sit on this for a bit so we can fall back to it if needbe, but this table no longer has any reads or writes in the application.

Test Plan: Applied patch locally and poked around.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: aran

Differential Revision: https://secure.phabricator.com/D8190
2014-04-24 08:15:24 -07:00
epriestley
4bbd2d5203 Make an old migration a little more robust
Summary: See https://github.com/facebook/phabricator/pull/507

Test Plan: This is hard to test since the migration no longer runs against HEAD, but pull 507 strongly implies this is the correct fix.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8837
2014-04-23 14:22:02 -07:00
epriestley
1a3ac09975 Add "requestedObjectPHID" to ReleephRequest
Summary:
Ref T3551. Currently, ReleephRequests don't have a direct concept of the //object// being requested. You can request `D123`, but that is just a convenient way to write `rXyyyy`.

When the UI wants to display information about a revision, it deduces it by examining the commit.

This is primarily an attack on T3551, so we don't need to load <commit -> edge -> revision> (in an ad-hoc way) to get revisions. Instead, when you request a revision we keep track of it and can load it directly later.

Later, this will let us do more things: for example, if you request a branch, we can automatically update the commits (as GitHub does), etc. (Repository branches will need PHIDs first, of course.)

This adds and populates the column but doesn't use it yet. The second part of the migration could safely be run while Phabricator is up, although even for Facebook this table is probably quite small.

Test Plan:
  - Ran migration.
  - Verified existing requests associated sensibly.
  - Created a new commit request.
  - Created a new revision request.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3551

Differential Revision: https://secure.phabricator.com/D8822
2014-04-20 11:55:18 -07:00
epriestley
4918773afe Drop nonsense buildStatus field from Buildable
Summary:
Ref T4809. Buildables currently have buildStatus and buildableStatus. Neither are used, and no one knows why we have two.

I'm going to use buildableStatus shortly, but buildStatus is meaningless; burn it.

Test Plan: `grep`, examined similar get/set calls, created a new buildable, ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4809

Differential Revision: https://secure.phabricator.com/D8796
2014-04-17 16:01:06 -07:00
epriestley
ab7d89edc8 Use better secrets in generating account tokens
Summary:
When we generate account tokens for CSRF keys and email verification, one of the inputs we use is the user's password hash. Users won't always have a password hash, so this is a weak input to key generation. This also couples CSRF weirdly with auth concerns.

Instead, give users a dedicated secret for use in token generation which is used only for this purpose.

Test Plan:
  - Ran upgrade scripts.
  - Verified all users got new secrets.
  - Created a new user.
  - Verified they got a secret.
  - Submitted CSRF'd forms, they worked.
  - Adjusted the CSRF token and submitted CSRF'd forms, verified they don't work.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8748
2014-04-10 11:45:10 -07:00
epriestley
582ec54465 Add a checkbox to the LDAP auth configuration UI to "Always Search"
Summary: Fixes T3208. This forces us to bind+search even if there are no anonymous credentials.

Test Plan: Checked the box, saved the form. Unchecked the box, saved the form. LDAP??

Reviewers: Firehed

Reviewed By: Firehed

Subscribers: epriestley

Maniphest Tasks: T3208

Differential Revision: https://secure.phabricator.com/D8723
2014-04-08 11:36:23 -07:00
epriestley
847b7977c1 Add semi-generic rate limiting infrastructure
Summary:
This adds a system which basically keeps a record of recent actions, who took them, and how many "points" they were worth, like:

  epriestley email.add 1 1233989813
  epriestley email.add 1 1234298239
  epriestley email.add 1 1238293981

We can use this to rate-limit actions by examining how many actions the user has taken in the past hour (i.e., their total score) and comparing that to an allowed limit.

One major thing I want to use this for is to limit the amount of error email we'll send to an email address. A big concern I have with sending more error email is that we'll end up in loops. We have some protections against this in headers already, but hard-limiting the system so it won't send more than a few errors to a particular address per hour should provide a reasonable secondary layer of protection.

This use case (where the "actor" needs to be an email address) is why the table uses strings + hashes instead of PHIDs. For external users, it might be appropriate to rate limit by cookies or IPs, too.

To prove it works, I rate limited adding email addresses. This is a very, very low-risk security thing where a user with an account can enumerate addresses (by checking if they get an error) and sort of spam/annoy people (by adding their address over and over again). Limiting them to 6 actions / hour should satisfy all real users while preventing these behaviors.

Test Plan:
This dialog is uggos but I'll fix that in a sec:

{F137406}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8683
2014-04-03 11:22:38 -07:00
epriestley
f239d81c1c Allow very long notes on flags
Summary: Fixes T4703. This is a VARCHAR(255) for no particular reason.

Test Plan: {F136160}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T4703

Differential Revision: https://secure.phabricator.com/D8652
2014-03-30 19:51:46 -07:00
epriestley
d3dbbec88d Rename Releeph "Project" transactions to "Product"
Summary: Ref T3549. This table isn't written to yet; rename it and the DAOs and modernize the history controller.

Test Plan: Viewed history page for a product.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3549

Differential Revision: https://secure.phabricator.com/D8633
2014-03-29 09:15:09 -07:00
Bob Trahan
655ac9927f Workboards - add column detail page
Summary: followup to D8544. This ends up creating an editor + transactions to get the job done.

Test Plan: made a column - saw a nice created transaction. edited the name - saw a nice name edit. deleted the column - saw a deleted transaction, updated "deleted" ui, and hte action change to activate. "Activated" the column and saw a transaction and updated UI. Tried to delete a column with tasks in it and got an error.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D8620
2014-03-26 14:40:47 -07:00
epriestley
543a313387 Skip a very old project reindex migration
Summary:
During migration of very old installs, this script no longer runs properly since at HEAD it can't index against older schemas.

Since it's pretty fluff, just toss it. Installs can run `bin/search index --type PROJ` after finishing migrations to achieve the same effect, if necessary.

Test Plan: eyeballed it

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8619
2014-03-26 13:51:49 -07:00
epriestley
a5f55d506f Provide a real object ("PhabricatorRepositoryPushEvent") to represent an entire push transaction
Summary:
Ref T4677. Currently, we record individual actions in a push as PhabricatorRepositoryPushLogs, but tie them together only loosely with a `transactionKey`.

Provide a real PushEvent object, and move some of the denormalized fields to it. This primarily just gives us more robust infrastructure for building, e.g., email about pushes, for T4677, since we can act on real PHIDs rather than passing awkward identifiers around.

Test Plan:
  - Performed migration.
  - Looked at database for consistency.
  - Browsed/queried push logs.
  - Pushed a bunch of stuff.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4677

Differential Revision: https://secure.phabricator.com/D8615
2014-03-26 13:51:06 -07:00
epriestley
d6b937ca27 Allow external systems to send messages to build targets
Summary:
Ref T1049. Allows external systems to send a message to a build target. The primary intended use case is:

  - You make an HTTP request to Jenkins.
  - The build goes into a "waiting" state.
  - Later, Jenkins calls `harbormaster.sendmessage` to report that the target passed or failed.
  - The build continues as appropriate.

This is deceptively complicated because:

  - There are a lot of race concerns. We might get a message back from an external system before it even responds to the request we made. We want to make sure we process these messages no matter when we receive them.
  - These messages need to be sent to a build target (vs a build or buildable) because we'll get into trouble with parallelization later on otherwise (Jenkins is told to do 3 builds; we can't tell which ones failed or what overall state is unless the message are sent to targets).
  - I initially thought about implementing this as a separate "Wait for a response from an external system" build step. This gets a lot more complicated for users once we do parallelization, though. Particularly, in the case where you've told Jenkins to do 3 builds, the three "wait" steps need to know which target they're waiting for (and jenkins needs to know some unique identifier for each target). So this pretty much boils down to a more complicated, more error-prone version of using target PHIDs.

This makes the already-muddy Build UI a bit worse, but it needs a general clarity pass anyway (it's showing way too much uninteresting data, and should show a better summary of results instead).

Test Plan:
  - This doesn't really do anything interesting yet.
  - Used Conduit to send messages to build plans.
  - Viewed the messages on the build screen.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D8604
2014-03-25 16:11:28 -07:00
epriestley
cec8d10731 Rename concrete Harbormaster step implementations
Summary: Ref T1049. For consistency, rename these to "Harbormaster...".

Test Plan: Ran migration, ran builds, everything still works fine.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D8602
2014-03-25 16:09:51 -07:00
epriestley
a246c85c6b Use ApplicationTransactions and CustomField to implement build steps
Summary:
Ref T1049. Fixes T4602. Moves all the funky field stuff to CustomField. Uses ApplicationTransactions to apply and record edits.

This makes "artifact" fields a little less nice (but still perfectly usable). With D8599, I think they're reasonable overall. We can improve this in the future.

All other field types are better (e.g., fixes weird bugs with "bool", fixes lots of weird behavior around required fields), and this gives us access to many new field types.

Test Plan:
Made a bunch of step edits. Here's an example:

{F133694}

Note that:

  - "Required" fields work correctly.
  - the transaction record is shown at the bottom of the page.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4602, T1049

Differential Revision: https://secure.phabricator.com/D8600
2014-03-25 16:08:40 -07:00
epriestley
0a76d82a7c Use string constants, not integer constants, to represent task status internally
Summary:
Ref T1812. I think integer constants are going to be confusing and error prone for users to interact with. For example, because we use 0-5, adding a second "open" status like "needs verification" without disrupting the existing statuses would require users to define a status with, e.g., constant `6`, but order it between constants `0` and `1`. And if they later remove statuses, they need to avoid reusing existing constants.

Instead, use more manageable string constants like "open", "resolved", etc.

We must migrate three tables:

  - The task table itself, to update task status.
  - The transaction table, to update historic status changes.
  - The saved query table, to update saved queries which specify status sets.

Test Plan:
  - Saved a query with complicated status filters.
  - Ran migrations.
  - Looked at the query, at existing tasks, and at task transactions.
  - Forced migrations to run again to verify idempotentcy/safety.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1812

Differential Revision: https://secure.phabricator.com/D8583
2014-03-25 13:58:14 -07:00
Bob Trahan
809e5a0389 Workboards - let users delete columns
Summary: Fixes T4408. I had to add a "status" to colum. I think we'll need this once we get fancier anyway but for now we have "active" and deleted.

Test Plan: deleted a column. noted reloaded workboard with all those tasks back in the default colun. loaded a task and saw the initial transaction had a "Disabled" icon next to the deleted workboard. also saw the new transaction back to the default column worked.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4408

Differential Revision: https://secure.phabricator.com/D8544
2014-03-18 10:40:31 -07:00
epriestley
5b2887b69b Add "Date Updated" query fields for Maniphest
Summary:
Fixes T4637.

  - We already allow you to order by this column but don't have a key on it. Add one.
  - Expose UI for querying on ranges.

Test Plan:
  - Ran some queries, got reasonable-looking results and no table scans.

Reviewers: btrahan, bigo

Reviewed By: bigo

Subscribers: bigo, epriestley

Maniphest Tasks: T4637

Differential Revision: https://secure.phabricator.com/D8557
2014-03-17 15:53:07 -07:00
epriestley
be92bf182f Drop Maniphest legacy transaction table
Summary: Ref T2217. I'll hold this for a month or so, but once we're confident the migration didn't ruin anything we should nuke this old data -- it's just an insurance policy against discovering migration issues.

Test Plan: Will run in a month or so.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7104
2014-03-12 06:04:45 -07:00
epriestley
2dbfb1d5fb Remove DifferentialComment
Summary: Ref T2222. Remove this; no more callsites.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8477
2014-03-11 13:02:33 -07:00
epriestley
03abde0b25 Fix Diviner links to articles by title
Summary:
Ref T988. This fixes the biggest current problem with Diviner, which is dead links to articles.

In the new Diviner, articles can have both a "name" (derived from the file name, and used in the URI) and a "title" (optional, specified explicitly). For example, we have one document with the name "feedback" and the title "Give Feedback! Get Support!".

On disk, we want to use the name for the actual file where the text lives ("feedback.diviner"). We also want to use the name in the URI, to generate a clean URI and to allow us to retitle the document slightly without breaking links to it (for example, we renamed the "Backup" document to "Backups and Migrations").

However, when displaying the article we want to use the title.

Currently, you can //only// link to the name, not the title. This is inconvenient:

  - We have a bunch of existing docs which link to titles.
  - It's natural/intuitive to link to titles.
  - Linking to titles makes it easier/cheaper to generate documentation, because we don't need to be able to resolve things at render time.

To remedy this, allow links to target either names or titles. If we miss on a name query, we'll do a title query. This is implemented with a slug hash to allow approximately correct titles (wrong case/spacing/punctuation, e.g.) and sidestep all the UTF8/column length issues.

(In the long run, atom resolution should theoretically be more sophistiated than it is now, and we should do render-time lookups on at least some documents to catch bad links. However, this is fairly complicated and a relatively advanced feature, and I think allowing links to titles is desirable no matter what.)

Test Plan: The user documentation book now has valid links to articles when the titles and names differ.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D8407
2014-03-05 12:07:26 -08:00
epriestley
bca0ad8fdd Make "EditPro" controller work with diff updates
Summary:
Ref T2222. Make the "EditPro" controller accommodate diff updates, and support the transaction type. This one is pretty straightforward.

Also make `revisionPHID` in the comments table nullable to fix the "Edit" action.

Test Plan:
  - Created new revision.
  - Updated revision.
  - Tried to do some invalid stuff.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8376
2014-02-28 16:49:22 -08:00
epriestley
cd7171ec6e Migrate old AuxiliaryField storage to modern CustomField storage
Summary: Ref T2222. Ref T3886. Differential has a legacy storage table for auxiliary fields; move the data to modern storage.

Test Plan:
  - Ran migration.
  - Verified fields still worked properly afterward (view, edit, etc).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886, T2222

Differential Revision: https://secure.phabricator.com/D8355
2014-02-26 16:52:30 -08:00
Joshua Spence
6270114767 Various linter fixes.
Summary:
- Removed trailing newlines.
- Added newline at EOF.
- Removed leading newlines.
- Trimmed trailing whitespace.
- Spelling fix.
- Added newline at EOF

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: hach-que, chad, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8344
2014-02-26 12:44:58 -08:00
epriestley
0f0673b9e5 Remove "dateCommitted" field from DifferentialRevision
Summary: Ref T2222. This is obsolete and no longer used. We could deduce it from transactions or commits in modern Phabricator if we wanted it. We may implement a more general mechanism for T4434.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8330
2014-02-25 12:36:14 -08:00
epriestley
70b008d18d Add test coverage that our definition of BMP agrees with MySQL
Summary:
Ref T1191. Test that MySQL's rules match those of `phutil_is_utf8_with_only_bmp_characters()`:

  - Build a string with //every// character that we consider to be a BMP character.
  - Write it into MySQL.
  - Read it back out.
  - Make sure MySQL didn't truncate it.

Test Plan: Ran unit test. This test runs pretty quickly (50ms), the string with every character isn't all that enormous.

Reviewers: btrahan, arice

Reviewed By: arice

CC: chad, arice, aran

Maniphest Tasks: T1191

Differential Revision: https://secure.phabricator.com/D8314
2014-02-23 16:20:38 -08:00
Bob Trahan
dcd7a316d2 Differential - add DifferentialDraft to track whether revisions have draft feedback or not
Summary: ...do it somewhat generically, so we could fairly easily add this to other applications. Fixes T3496. I got a wee bit lazy and decided not to migrate existing drafts. My excuses aside from laziness are doing it this way will let us see if anyone complains, we can always do a migration later if people do complain, and there's likely to be a lot of garbage data for older / bigger installs, and the migration didn't seem worth itgiven it would also likely be expensive in these cases.

Test Plan: made a draft inline comment on DX and observed DX had a note icon on Differential home page. made a draft comment on DX and observed DX had a note icon on Differential home page. deleted a draft inline comment and noted icon disappeared from Differential homepage. Submitted a draft comment + inline comment and noted icon disappeared.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3496

Differential Revision: https://secure.phabricator.com/D8275
2014-02-18 16:25:16 -08:00
epriestley
b96ab5aadf Modernize VCS password storage to use shared hash infrastructure
Summary: Fixes T4443. Plug VCS passwords into the shared key stretching. They don't use any real stretching now (I anticipated doing something like T4443 eventually) so we can just migrate them into stretching all at once.

Test Plan:
  - Viewed VCS settings.
  - Used VCS password after migration.
  - Set VCS password.
  - Upgraded VCS password by using it.
  - Used VCS password some more.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4443

Differential Revision: https://secure.phabricator.com/D8272
2014-02-18 14:09:36 -08:00
epriestley
5778627e41 Provide more storage space for password hashes and migrate existing hashes to "md5:"
Summary: Ref T4443. Provide more space; remove the hack-glue.

Test Plan: Logged out, logged in, inspected database.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4443

Differential Revision: https://secure.phabricator.com/D8269
2014-02-18 14:09:36 -08:00
epriestley
f3cbc0e006 Move many task status hardcodes into ManiphestTaskStatus
Summary:
Ref T1812. This cleans up most of the easy hard-coded references to specific statuses:

  - The "fixes" language moves into ManiphestTaskStatus.
  - Add a method to list open statuses.
  - Add a method to test if a status is open.
  - Add a method to get default status for new tasks.

Test Plan: Browsed around, lint, grep, created, filtered and updated tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1812

Differential Revision: https://secure.phabricator.com/D8264
2014-02-17 15:59:31 -08:00
epriestley
18938b5310 Migrate Differential comments to ApplicationTransactions
Summary:
Ref T2222. This is the big one.

This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions.

The migration is pretty straightforward:

  - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.".
  - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook).
  - If a comment added or removed reviewers, it gets a "changed reviewers" transaction.
  - If a comment added CCs, it gets a "subscribers" transaction.
  - If a comment added comment text, it gets a "comment" transaction.
  - For each inline attached to a comment, we generate an "inline" transaction.

Most comments generate a small number of transactions, but a few generate a significant number.

At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines).

Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table.

NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code.

Specifically, they look like this:

{F112270}

Test Plan:
I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place.
IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master.

I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made.

Reviewers: btrahan

CC: chad, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 14:34:48 -08:00
epriestley
8f9b7f4196 Move Differential to proper subscriptions
Summary:
Ref T2222. Ref T4415. We're still writing Differential subscription stuff into this weird legacy `differential_relationship` table, which is like an edge table but extremely ancient.

Move it into a proper table.

I've removed `withSubscriptions()` from `DifferentialRevisionQuery`. It was weird, doesn't work consistently with other similar filters, and was only used by the API. Now it means "ccs", which is consistent with the ApplicationSearch UI and with Maniphest.

Test Plan:
Without migrating, added and removed subscribers via various workflows. Queried for subscribers. Everything worked as expected.

Ran the migration, verified data survived.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Maniphest Tasks: T2222, T4415

Differential Revision: https://secure.phabricator.com/D8202
2014-02-12 08:53:40 -08:00
epriestley
305fb3fbd9 Migrate all Differential comment text into new storage
Summary:
Ref T2222. Currently, `DifferentialComment` stores both (a) the text of comments and (b) various other transaction details. This data needs to map to both Transactions and TransactionComments in the long run. This diff separates out all the data which is bound for the TransactionComment table, so that when we migrate `DifferentialComment` itself it will //only// need to migrate into the Transactions table. This is a much simpler migration than the inline comment one was, partly because it set up infrastructure and partly because the data is less complex.

Basically, I'm just proxying the read/write for the comment text into the other table. All readers already go through the Query class, and there are only three writers (preview, comment, implicit comment on diff update) which are all highly regular and straightforward to test.

We can also back out of this diff very easily: doing double writes cost only one line of code (`$this->content = $content;`) so we have proper double writes and a trivial revert path.

Test Plan:
  - Without migrating, added comments and saw them show up.
  - Migrated.
  - Saw all the old comments, and no damage to the new ones.
  - Added new comments.
  - Used comment preview.
  - Updated a revision to implicitly create an update comment and verified it looked OK.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8196
2014-02-11 11:34:15 -08:00
epriestley
b1243e549c Allow unsubscription from projects
Summary:
Fixes T4379. Several changes:

  - Migrate all project members into subscribers.
  - When members are added or removed, subscribe or unsubscribe them.
  - Show sub/unsub in the UI.
  - Determine mailable membership of projects by querying subscribers.

Test Plan:
  - As `duck`, joined a project.
  - Added the project as a reviewer to a revision.
  - Commented on the revision.
  - Observed `duck` receive mail.
  - Unsubscribed as `duck`.
  - Observed no mail.
  - Resubscribed as `duck`.
  - Mail again.
  - Joined/left project, checked sub/unsub status.
  - Ran migration, looked at database.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, asherkin

Maniphest Tasks: T4379

Differential Revision: https://secure.phabricator.com/D8189
2014-02-11 07:45:56 -08:00
Bob Trahan
1830868007 Herald - make herald condition of herald rule display better
Summary: ...by the surprising step of changing how this data is stored from id to phid. Also a small fix to not allow "disabled" rules to be used as herald rule conditions, i.e. can't make a rule that depends on a disabled rule.

Test Plan: viewed existing herald rule that had a rule condition and noted nice new display using handle. made a new rule that had a rule condition and verified it worked correctly.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8186
2014-02-10 14:40:09 -08:00
epriestley
8c84ed61b1 Migrate project profiles onto projects, and remove ProjectProfile object
Summary:
Ref T4379. Long ago, the "Project" vs "ProjectProfile" split was intended to allow a bunch of special fields on projects without burdening the simple use cases, but CustomField handles that far better and far more generally, and doing this makes using ApplicationTransactions a pain to get right, so get rid of it.

The only remaining field is `profileImagePHID`, which we can just move to the main Project object. This is custom enough that I think it's reasonable not to express it as a custom field.

Test Plan: Created a project, set profile, edited project, viewed in typeahead, ran migration, verified database results.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4379

Differential Revision: https://secure.phabricator.com/D8183
2014-02-10 14:32:14 -08:00
epriestley
1520065ec0 Migrate project blurb/description to standard custom field storage
Summary: Ref T4379. Major goal here is to remove `ProjectProfile` so all edits use ApplicationTransactions. This also makes things more flexible, allowing users to disable this field if they don't like it.

Test Plan: Ran migration, verified data survived, edited/created projects, reordered fields.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4379

Differential Revision: https://secure.phabricator.com/D8182
2014-02-10 14:31:57 -08:00
epriestley
b13a51adeb Fix inadvertent forward dependency in task owner migration
Summary:
See <https://github.com/facebook/phabricator/issues/505>. When the status/event table moved, it broke this migration, which implicitly loads statuses by loading events.

Instead, access just the row we care about.

Test Plan: Used `--apply` to apply the new version of the patch.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D8162
2014-02-07 09:17:11 -08:00
epriestley
eef2727ea6 Fix accidental forward dependency in old commit summary migration
Summary: See IRC. This migration inadvertently depends on the columns in the commit table, because it calls `save()`, and thus broke for installs with data after we added the `importStatus` column. Since that was ~9 months after this patch, probably not many installs are affected.

Test Plan: Ran patch locally with `--apply` on data. Had user verify fix.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8152
2014-02-06 10:33:54 -08:00
epriestley
4c7acbafac Assign PHIDs to calendar events
Summary: Ref T4375. We're going to need these for a bunch of infrastructure to work.

Test Plan: Ran migrations, checked DB, used `phid.query`.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4375

Differential Revision: https://secure.phabricator.com/D8151
2014-02-06 10:10:43 -08:00
epriestley
ab636f36bf Rename PhabricatorUserStatus to PhabricatorCalendarEvent
Summary: Ref T4375. We never join this table, so this is a pretty straight find/replace.

Test Plan: Browsed around Calendar, verified that nothing seemed broken. Saw my red dot in other apps.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4375

Differential Revision: https://secure.phabricator.com/D8145
2014-02-06 10:07:29 -08:00
epriestley
cb605ad5ee Add edit/view plumbing for dashboards and panels
Summary: Ref T3583. This doesn't add any dashboard/panel-specific code beyond headers/titles/buttons/etc., but allows you to create and view dashboards and panel skeletons.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8131
2014-02-03 10:52:15 -08:00
epriestley
88436349b8 Add a GC for sent and received mail
Summary: Ref T4368. We don't currently GC these tables, and the sent mail table is one of the largest on `secure.phabricator.com`. There's no value in retaining this information indefinitely. Instead, retain it for 90 days, which should be plenty of time to debug/diagnose any issues.

Test Plan: Ran `phd debug garbage`, saw it clean up a reasonable amount of data from these tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4368

Differential Revision: https://secure.phabricator.com/D8127
2014-02-03 10:51:31 -08:00
epriestley
4b0ef353e4 Remove retry/failure mechanisms from MetaMTA
Summary:
Fixes T4202. We have old code in MetaMTA which implements gradual backoff and maximum retries.

However, we have more general code in the task queue which does this, too. We can just use the more general stuff in the task queue; it obsoletes the specific stuff in MetaMTA, which is more complex and ran into some kind of issue in T4202.

Remove `retryCount`, `nextRetry` (obsoleted by task queue retry mechanisms) and "simulated failures" (no longer in use).

Generally, modern infrastructure has replaced these mechanisms with more general ones.

Test Plan:
  - Sent mail.
  - Observed unsendable mail failing in reasonable ways in the queue.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4202

Differential Revision: https://secure.phabricator.com/D8115
2014-02-01 14:35:42 -08:00
epriestley
73924dfa18 Add initial skeleton for Dashboard application
Summary:
Ref T3583. General idea here is:

  - Users will be able to create `DashboardPanel`s, which are things like the jump nav, or a minifeed, or recent assigned tasks, or recent tokens given, or whatever else.
  - The `DashboardPanel`s can be combined into `Dashboard`s, which select specific panels and arrange them in some layout (and maybe have a few other options eventually).
  - Then, you'll be able to set a specific `Dashboard` for your home page, and maybe for project home pages. But you can also use `Dashboard`s on their own if you just like dashboards.

My plan is pretty much:

  - Put in basic infrastructure for dashboards (this diff).
  - Add basic create/edit (next few diffs).
  - Once dashboards sort of work, do the homepage integration.

This diff does very little: you can't create dashboards or panels yet, and thus there are no dashboards to look at. This is all skeleton code, pretty much.

IMPORTANT: We need an icon bwahahahahaha

Test Plan:
omg si purrfect

{F106367}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3583

Differential Revision: https://secure.phabricator.com/D8109
2014-01-30 11:43:24 -08:00
epriestley
014a873773 Update DifferentialDiff: add repositoryPHID, drop parentRevisionID
Summary:
Moves away from ArcanistProjects:

  - Adds storage for diffs to be directly associated with a repository (instead of indirectly, through arcanist projects). Not really populated yet.
  - Drops `parentRevisionID`, which is obsoleted by the "Depends On" edge. This is not exposed in the UI anywhere and doesn't do anything. Resolves TODO.

Test Plan: Ran storage upgrades, browsed around, lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8072
2014-01-26 15:29:22 -08:00
epriestley
a9e11ed8c1 Update quickstart SQL
Summary: Ref T4327. The change parser unit tests need database fixtures, which are getting a bit slow to build. Speed them up by updating the quickstart.

Test Plan: Initialized new storage via quickstart, clicked around, everything seemed to work properly. Ran all unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8004
2014-01-17 16:11:16 -08:00
epriestley
f4b9efe256 Introduce ref cursors for repository parsing
Summary:
Ref T4327. I want to make change parsing testable; one thing which is blocking this is that the Git discovery process is still part of `PullLocal` daemon instead of being part of `DiscoveryEngine`. The unit test stuff which I want to use for change parsing relies on `DiscoveryEngine` to discover repositories during unit tests.

The major reason git discovery isn't part of `DiscoveryEngine` is that it relies on the messy "autoclose" logic, which we never implemented for Mercurial. Generally, I don't like how autoclose was implemented: it's complicated and gross and too hard to figure out and extend.

Instead, I want to do something more similar to what we do for pushes, which is cleaner overall. Basically this means remembering the old branch heads from the last time we parsed a repository, and figuring out what's new by comparing the old and new branch heads. This should give us several advantages:

  - It should be simpler to understand than the autoclose stuff, which is pretty mind-numbing, at least for me.
  - It will let us satisfy branch and tag queries cheaply (from the database) instead of having to go to the repository. We could also satisfy some ref-resolve queries from the database.
  - It should be easier to extend to Mercurial.

This implements the basics -- pretty much a table to store the cursors, which we update only for Git for now.

Test Plan:
  - Ran migration.
  - Ran `bin/repository discover X --trace --verbose` on various repositories with branches and tags, before and after modifying pushes.
  - Pushed commits to a git repo.
  - Looked at database tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7982
2014-01-17 11:48:53 -08:00
epriestley
2ec45d42a6 Remove session limits and sequencing
Summary:
Ref T4310. Fixes T3720. This change:

  - Removes concurrent session limits. Instead, unused sessions are GC'd after a while.
  - Collapses all existing "web-1", "web-2", etc., sessions into "web" sessions.
  - Dramatically simplifies the code for establishing a session (like omg).

Test Plan: Ran migration, checked Sessions panel and database for sanity. Used existing session. Logged out, logged in. Ran Conduit commands.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4310, T3720

Differential Revision: https://secure.phabricator.com/D7978
2014-01-15 17:27:59 -08:00
Bob Trahan
d740374cca Legalpad - add policy rule for legalpad document signatures
Summary:
Ref T3116. This creates a policy rule where you can require a signature on a given legalpad document.

NOTE: signatures must be for the *latest* document version.

Test Plan: made a task have a custom policy requiring a legalpad signature. verified non-signers were locked out.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3116

Differential Revision: https://secure.phabricator.com/D7977
2014-01-15 16:48:44 -08:00
epriestley
acb141cf52 Expire and garbage collect unused sessions
Summary:
Ref T3720. Ref T4310. Currently, we limit the maximum number of concurrent sessions of each type. This is primarily because sessions predate garbage collection and we had no way to prevent the session table from growing fairly quickly and without bound unless we did this.

Now that we have GC (and it's modular!) we can just expire unused sessions after a while and throw them away:

  - Add a `sessionExpires` column to the table, with a key.
  - Add a GC for old sessions.
  - When we establish a session, set `sessionExpires` to the current time plus the session TTL.
  - When a user uses a session and has used up more than 20% of the time on it, extend the session.

In addition to this, we could also rotate sessions, but I think that provides very little value. If we do want to implement it, we should hold it until after T3720 / T4310.

Test Plan:
  - Ran schema changes.
  - Looked at database.
  - Tested GC:
    - Started GC.
    - Set expires on one row to the past.
    - Restarted GC.
    - Verified GC nuked the session.
  - Logged in.
  - Logged out.
  - Ran Conduit method.
  - Tested refresh:
    - Set threshold to 0.0001% instead of 20%.
    - Loaded page.
    - Saw a session extension ever few page loads.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4310, T3720

Differential Revision: https://secure.phabricator.com/D7976
2014-01-15 13:56:16 -08:00
epriestley
a64228b03f Give the session table a normal id column as a primary key
Summary:
Ref T4310. Ref T3720. Two major things are going on here:

  - I'm making this table work more like a standard table, which, e.g., makes `delete()` simpler to implement.
  - Currently, the primary key is `(userPHID, type)`. I want to get rid of this, issue unlimited sessions, and GC old sessions. This means we can't have a unique key on `(userPHID, type)` anymore. This removes it as the primary key and adds it as a normal key instead. There's no functional change -- the code to generate sessions guarantees that it will never write duplicate rows or write additional rows -- but allows us to drop the `-1`, `-2` qualifiers in the future.
  - Also of note, our task is made far simpler here because MySQL will automatically assign values to new `AUTO_INCREMENT` columns, so we don't have to migrate to get real IDs.

Test Plan: Ran migrations, verified table looked sane. Logged out, logged in.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3720, T4310

Differential Revision: https://secure.phabricator.com/D7975
2014-01-15 13:55:18 -08:00
Bob Trahan
41d2a09536 Legalpad - make it work for not logged in users
Summary: Adds "verified" and "secretKey" to Legalpad document signatures. For logged in users using an email address they own, things are verified right away. Otherwise, the email is sent a verification letter. When the user clicks the link the signature is marked verified.

Test Plan: signed the document with a bogus email address not logged in. verified the email that would be sent looked good from command line. followed link and successfully verified bogus email address

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran, asherkin

Maniphest Tasks: T4283

Differential Revision: https://secure.phabricator.com/D7930
2014-01-14 17:17:18 -08:00
Mikael Knutsson
5417f91b77 Adding the create flow for Project Board (Workphlow) columns.
Summary: This adds in the create flow for the Project board columns on the super secret board page which totally doesn't do anything right now.

Test Plan:
1. Apply diff.
2. Go to super secret page.
3. Click link close to top with a way too long name.
4. Enter a name for the column.
5. Enjoy a new column briefly before realising you cannot remove it.
6. Stay happy!

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: tmaroschik, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7925
2014-01-09 16:12:11 -08:00
epriestley
c020837397 Add transactions to Drydock blueprint editing
Summary: Ref T2015. Fixes TODO.

Test Plan: {F100338}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7922
2014-01-09 12:19:54 -08:00
epriestley
962aca664f Add names to Drydock blueprints
Summary:
Ref T2015. Adds human-readable names to Drydock blueprints.

Also the new patches stuff is so much nicer.

Test Plan: Edited, created, and reviewed migrated blueprints.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7918
2014-01-09 10:56:34 -08:00
epriestley
1786093c6e Replace "Cancel Build" with "Stop", "Resume" and "Restart"
Summary:
Ref T1049. Currently you can cancel a build, but now that we're tracking a lot more state we can stop, resume, and restart builds.

When the user issues a command against a build, I'm writing it into an auxiliary queue (`HarbormasterBuildCommand`) and then reading them out in the worker. This is mostly to avoid race messes where we try to `save()` the object in multiple places: basically, the BuildEngine is the //only// thing that writes to Build objects, and it holds a lock while it does it.

Test Plan:
  - Created a plan which runs "sleep 2" a bunch of times in a row.
  - Stopped, resumed, and restarted it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7892
2014-01-06 12:32:20 -08:00
epriestley
6abe65bfdc Add mailKey to macros
Summary:
If you have private replies on and a Macro reply handler set, we try to access `getMailKey()` and fail. See P1039 for a trace.

(Thanks to @Korvin for picking this up.)

Test Plan: Set configuration, repro'd the exception, applied the patch, then disabled/enabled a macro.

Reviewers: btrahan

Reviewed By: btrahan

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7896
2014-01-06 12:17:23 -08:00
epriestley
09341be10f Remove repository shortcuts
Summary:
Repositories currently have a no-UI "shortcut" feature which is only used by Facebook (and I'm not sure it's even used). As implemented, this feature is policy-oblivious and kind of nonsensical. Throw it away.

I'm open to reimplementing this, but I want to see some level of interest in it before I do. The new implementation would add shortcuts to each repository, similar to how mirrors work. My original plan was to follow this up with such an implementation (it's half-implemented in my sandbox), but as I worked through it I'm not sure it's really valuable.

Test Plan: Browsed repository list, grep.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Differential Revision: https://secure.phabricator.com/D7862
2014-01-02 11:59:27 -08:00
epriestley
f5fb3f05dc Lay most groundwork for Herald object rules
Summary:
Ref T4264. This gets most of the plumbing in for "object" rules, which will bind to a specific object, like a repository or project.

It does not yet let you actually create these rules.

Test Plan: Ran `storage upgrade`, created/edited rules, browsed Herald.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4264

Differential Revision: https://secure.phabricator.com/D7847
2013-12-27 13:17:10 -08:00
epriestley
ac19c55822 Formalize "manual" buildables in Harbormaster
Summary:
Ref T1049. Generally, it's useful to separate test/trial/manual runs from production/automatic runs.

For example, you don't want to email a bunch of people that the build is broken just because you messed something up when writing a new build plan. You'd rather try it first, then promote it into production once you have some good runs.

Similarly, test runs generally should not affect the outside world, etc. Finally, some build steps (like "wait for other buildables") may want to behave differently when run in production/automation than when run in a testing environment (where they should probably continue immediately).

So, formalize the distinction between automatic buildables (those created passively by the system in response to events) and manual buildables (those created explicitly by users). Add filtering, and stop the automated parts of the system from interacting with the manual parts (for example, we won't show manual results on revisions).

This also moves the "Apply Build Plan" to a third, new home: instead of the sidebar or Buildables, it's now on plans. I think this generally makes more sense given how things have developed. Broadly, this improves isolation of test environments.

Test Plan: Created some builds, browsed around, used filters, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7824
2013-12-26 10:40:43 -08:00
epriestley
c179ce6279 (Later...) Drop legacy Project transaction table
Summary: Ref T4010. I'll hold this for a bit, but we should eventually drop this table once the dust has settled.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7372
2013-12-19 07:03:51 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
James Rhodes
86ec4d6021 Implement policies in Phragment
Summary: This implements support for enforcing and setting policies in Phragment.

Test Plan: Set policies and ensured they were enforced successfully.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7751
2013-12-13 14:42:12 +11:00
James Rhodes
8bb6e807f0 Implement snapshots in Phragment
Summary:
Ref T4212.  This implements snapshots in Phragment, which allows you to take a snapshot of a fragment at a given point in time, and download a ZIP of the snapshot as it was in this state.

There's also functionality for deleting and promoting snapshots.  You can promote a snapshot to either the latest version or any other snapshot of the fragment.

Test Plan: Clicked around, took some snapshots, promoted them to different points and deleted snapshots.  Also downloaded ZIPs of the snapshots and saw the right versions coming through for all the files downloaded.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205, T4212

Differential Revision: https://secure.phabricator.com/D7741
2013-12-09 08:24:50 +11:00
James Rhodes
25e7b7d53c Implement support for creating and updating fragments from ZIPs
Summary:
This implements support for creating and updating fragments from ZIP files.  It allows you to upload a ZIP via the Files application, create a fragment from it, and have it recursively imported into Phragment.  Updating that folder with another ZIP will recursively create, update and delete files as appropriate.

The logic for creating and updating fragments from files has also been centralized into the PhragmentFragment class.  Directories are also now supported; a directory fragment is simply a fragment that has no patches; thus a directory fragment can be converted to a file fragment by uploading a first patch for it.

Test Plan: Uploaded ZIP files through the interface and saw all of the fragments get created and updated as expected.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7729
2013-12-07 13:37:18 +11:00
James Rhodes
4c143ad3b2 Phragment v0
Summary: Ref T4205.  This is an initial implementation of Phragment.  You can create and browse fragments in the system (but you can't yet view a fragment's patches / history).

Test Plan: Clicked around and created fragments.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4205

Differential Revision: https://secure.phabricator.com/D7726
2013-12-07 12:43:49 +11:00
James Rhodes
79d153b85d Implement explicit build step ordering in Harbormaster
Summary: This implements support for explicitly marking the sequence of build steps.  Users can now drag and re-order build steps in plans, and artifact dependencies are re-calculated so that if you move "Run Command" before "Lease Host", the "Run Command" step has it's artifact setting cleared and thus the step becomes invalid.

Test Plan: Re-ordered build steps and observed dependencies being correctly recalculated.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7715
2013-12-06 14:12:15 +11:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
James Rhodes
53250d84df Introduce HarbormasterBuildTarget to snapshot build steps through a build
Summary: This implements build targets as outlined in D7582.  Build targets represent an instance of a build step particular to the build.  Logs and artifacts have been adjusted to attach to build targets instead of build / build step pairs.

Test Plan: Ran builds and clicked around the interface.  Everything seemed to work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T1049

Differential Revision: https://secure.phabricator.com/D7703
2013-12-05 12:01:12 +11:00
James Rhodes
ba16df0fed Restructure Drydock so that blueprints are instances in the DB
Summary:
//(this diff used to be about applying policies to blueprints)//

This restructures Drydock so that blueprints are instances in the DB, with an associated implementation class.  Thus resources now have a `blueprintPHID` instead of `blueprintClass` and DrydockBlueprint becomes a DAO.  The old DrydockBlueprint is renamed to DrydockBlueprintImplementation, and the DrydockBlueprint DAO has a `blueprintClass` column on it.

This now just implements CAN_VIEW and CAN_EDIT policies for blueprints, although they are probably not enforced in all of the places they could be.

Test Plan: Used the `create-resource` and `lease` commands.  Closed resources and leases in the UI.  Clicked around the new and old lists to make sure everything is still working.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4111, T2015

Differential Revision: https://secure.phabricator.com/D7638
2013-12-03 11:09:07 +11:00
epriestley
95c2b50974 Shorten extremely long credential names when migrating them
Summary: Fixes T4183. If you have too many repositories sharing the same credential and MySQL is in strict mode, we'll fail a query when trying to write a credential with a name longer than 255 characters. Instead, shorten the variable-length part to 128 characters.

Test Plan: Wiped credentials column and successfully re-ran migration with `storage upgrade --force --apply phabricator:20131121.repocredentials.2.mig.php`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4183

Differential Revision: https://secure.phabricator.com/D7677
2013-12-02 11:26:12 -08:00
epriestley
04b099d9ba Fix an issue with repository credential migration for older repositories
Summary: See IRC. It's possible to have a functional repository with the SSH username only in the URL. Look there if the username property isn't set. These should all be older repostiories.

Test Plan: Did a `--force --apply` upgrade.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D7665
2013-11-27 11:09:07 -08:00
epriestley
4b91c4f7ae Add UI for defining repository mirrors
Summary:
Ref T4038. This adds everything except the actual pushing part for mirrors.

This isn't the most beautiful or sophisticated UI, but I want get the authoritative repositories self-hosted and get users beta-ing hosting as soon as possible. We can do transactions, etc., later on.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7632
2013-11-22 15:23:50 -08:00
epriestley
51fb1ca16d Migrate repositories to use Passphrase for credential management
Summary: Fixes T4122. Ref T2230. Instead of storing credentials on each repository, store them in Passphrase. This allows easy creation/management of many repositories which share credentials.

Test Plan:
  - Upgraded repositories.
  - Created and edited repositories.
  - Pulled HTTP and SSH repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230, T4122

Differential Revision: https://secure.phabricator.com/D7629
2013-11-22 15:23:33 -08:00
epriestley
819f899013 Add an edge between Passphrase credentials and objects which use them
Summary: Ref T4122. Add an edge to keep track of where a credential is used, and show it in the UI.

Test Plan:
See "Used By":

{F84099}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7628
2013-11-22 15:23:23 -08:00
Bob Trahan
7b718bb033 Nuance - federate out the design of NuanceSource via NuanceSourceDefinition
Summary: ...and get the basic edit flow "working" for a new NuanceSourceDefinition - the Phabricator Form. ...and fix a dumb bug in the query class so when you redirect to the view page / try to edit an existing NuanceSource you don't fatal.

Test Plan: played around with the edit form and it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7585
2013-11-20 13:41:19 -08:00
epriestley
91d084624b Passphrase v0
Summary:
Ref T4122. Implements a credential management application for the uses described in T4122.

@chad, this needs an icon, HA HA HAHA HA BWW HA HA HA

bwahaha

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T4122

Differential Revision: https://secure.phabricator.com/D7608
2013-11-20 09:13:35 -08:00
epriestley
d9db1d61e0 Restore population of ownerOrdering to ManiphestTasks
Summary:
Ref T4110. This denormalized field used to power "Group By: Assigned" got dropped in the T2217 migration at some point.

Restore its population, and fix all the data in the database.

Test Plan: Ran migration, verified database came out reasonable-looking. Reassigned a task, verified database. Ran a "Group By: assigned" query.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4110

Differential Revision: https://secure.phabricator.com/D7602
2013-11-19 14:10:54 -08:00
epriestley
7f11e8d740 Improve handling of email verification and "activated" accounts
Summary:
Small step forward which improves existing stuff or lays groudwork for future stuff:

  - Currently, to check for email verification, we have to single-query the email address on every page. Instead, denoramlize it into the user object.
    - Migrate all the existing users.
    - When the user verifies an email, mark them as `isEmailVerified` if the email is their primary email.
    - Just make the checks look at the `isEmailVerified` field.
  - Add a new check, `isUserActivated()`, to cover email-verified plus disabled. Currently, a non-verified-but-not-disabled user could theoretically use Conduit over SSH, if anyone deployed it. Tighten that up.
  - Add an `isApproved` flag, which is always true for now. In a future diff, I want to add a default-on admin approval queue for new accounts, to prevent configuration mistakes. The way it will work is:
    - When the queue is enabled, registering users are created with `isApproved = false`.
    - Admins are sent an email, "[Phabricator] New User Approval (alincoln)", telling them that a new user is waiting for approval.
    - They go to the web UI and approve the user.
    - Manually-created accounts are auto-approved.
    - The email will have instructions for disabling the queue.

I think this queue will be helpful for new installs and give them peace of mind, and when you go to disable it we have a better opportunity to warn you about exactly what that means.

Generally, I want to improve the default safety of registration, since if you just blindly coast through the path of least resistance right now your install ends up pretty open, and realistically few installs are on VPNs.

Test Plan:
  - Ran migration, verified `isEmailVerified` populated correctly.
  - Created a new user, checked DB for verified (not verified).
  - Verified, checked DB (now verified).
  - Used Conduit, People, Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Differential Revision: https://secure.phabricator.com/D7572
2013-11-12 14:37:04 -08:00
James Rhodes
0ac1be7094 Implemented support for build logs
Summary:
Depends on D7519.

This implements support for build logs in Harbormaster.  This includes support for appending to a log from the "Run Remote Command" build step.

It also adds the ability to cancel builds.

Currently the build view page doesn't update the logs live; I'm sure this can be achieved with Javelin, but I don't have enough experience with Javelin to actually make it poll from updates to content in the background.

{F79151}

{F79153}

{F79150}

{F79152}

Test Plan:
Tested this by setting up SSH on a Windows machine and using a Remote Command configured with:

```
C:\Windows\system32\cmd.exe /C cd C:\Build && mkdir Build_${timestamp} && cd Build_${timestamp} && git clone --recursive https://github.com/hach-que/Tychaia.git && cd Tychaia && Protobuild.exe && C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe Tychaia.Windows.sln
```

and observed the output of the build stream from the Windows machine into Phabricator.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7521
2013-11-08 18:15:07 -08:00
Bob Trahan
66ae64f7bc Nuance - get some scaffolding up there
Summary:
I updated the wiki too - https://secure.phabricator.com/w/projects/pebkac/ - with what I am thinking right now. Rough plan here is

 - next diff:
  - implement editors and transactions
  - implement "web type" for contact source
   - /pebkac/item/new/ will be the entry point for this
  - implement "actions" on a contact
  - probably some "polish" on the scaffolding laid out here; like "create" permissions maybs
 - diffs after that:
  - implement "twitter" type for source
  - implement email reply handler stuff for item and source

Probs a great time to blast huge holes in all this stuff. :D

Test Plan: these pages load and arc lint doesn't complain

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran, chad

Differential Revision: https://secure.phabricator.com/D7465
2013-11-06 17:00:09 -08:00
epriestley
4f0f95f7b5 Assign PHIDs to all diffs
Summary:
Ref T1049. Ref T2222. `DifferentialDiff` does not currently have a PHID, but we need it for Harbormaster and ApplicationTransactions. See some discussion in D7501.

(I split the SQL into two sections so we can't fail in the middle. At some point, I'd like to do a pass on the migration stuff and get this happening automatically, and also simplify the PatchList.)

Test Plan:
  - Ran `bin/storage upgrade`.
  - Checked for valid PHIDs in the database.
  - Used `phid.query` to look up a diff by PHID.
  - Created a new diff and verified it got a PHID.

Reviewers: btrahan, hach-que

Reviewed By: btrahan

CC: aran, vrana

Maniphest Tasks: T2222, T1049

Differential Revision: https://secure.phabricator.com/D7513
2013-11-06 13:59:06 -08:00
James Rhodes
c514d34b94 Add build step implementation infrastructure and sleep build step.
Summary:
Depends on D7498.

This implements support for a "build step implementation".  Build steps have an associated class name (which makes the class in PHP) and a details field, which is serialized JSON (same as PhabricatorRepository).

This also implements a SleepBuildStepImplementation which just pauses the build for a specified period of seconds.

Test Plan:
Inserted a build step with `insert into harbormaster_buildstep (phid, buildPlanPHID, className, details, dateCreated, dateModified) values ('', 'PHID-HMCP-zkh5w6czfbfpk2gxwdeo', 'SleepBuildStepImplementation', '{"seconds":5}', NOW(), NOW());` (adjusting the build plan PHID as appropriate).

Started the daemon and applied the build plan to a buildable, and saw the daemon take a 5 second delay after creating `SleepBuildStepImplementation`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, chad

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7499
2013-11-05 13:34:44 -08:00
epriestley
0278b15ceb Implementation of VCS passwords against user.
Summary: This allows users to set their HTTP access passwords via Diffusion interface.

Test Plan: Clicked the "Set HTTP Access Password" link, set a password and saw it appear in the DB.

Reviewers: #blessed_reviewers, hach-que, btrahan

Reviewed By: hach-que

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7462
2013-11-01 08:34:11 -07:00
epriestley
3a39b01233 Add "RepositoryStatusMessage" and detailed information about initilization
Summary:
`RepositoryStatusMessage` is basically a key/value table associated with a repository that I'm using to let the daemons store the most recent event of a given type, so we can easily show it on the status dashboard. I think this will be a lot easier for users to figure out than digging through logfiles.

I'm also going to write the "this needs a pull" status here eventually, for reducing the time lapse between pushes and discovery.

  - Add storage for these messages.
  - Have the pull engine populate the INIT phase. I'll do the FETCH phase next.
  - Update the status readout to show all the various states.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7461
2013-10-30 16:04:19 -07:00
epriestley
9125095587 Distinguish between empty and unparsed commits in Diffusion
Summary:
Fixes T3416. Fixes T1733.

  - Adds a flag to the commit table showing whether or not we have parsed it.
  - The flag is set to `0` initially when the commit is discovered.
  - The flag is set to `1` when the changes are parsed.
  - The UI can now use the flag to distinguish between "empty commit" and "commit which we haven't imported changes for yet".
  - Simplify rendering code a little bit.
  - Fix an issue with the Message parser for empty commits.
  - There's a key on the flag so we can do `SELECT * FROM repository_commit WHERE repositoryID = %d AND importStatus = 0 LIMIT 1` soon, to determine if a repository is fully imported or not. This will let us improve the UI (Ref T776, Ref T3217).

Test Plan:
  - Ran `bin/storage upgrade -f`.
  - Created an empty commit.
  - Without the daemons running, ran `bin/repository pull GTEST` and `bin/repository discover GTEST`.
  - Viewed web UI to get the first screenshot ("Still Importing...").
  - Ran the message and change steps with `scripts/repository/reparse.php`.
  - Viewed web UI to get the second screenshot ("Empty Commit").

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776, T1733, T3416, T3217

Differential Revision: https://secure.phabricator.com/D7428
2013-10-29 15:32:41 -07:00
epriestley
86fe020a97 Add global "push" policy to Repositories
Summary: No editing or view yet, just adds the schema and a policy default. Part of D7391.

Test Plan: `bin/storage upgrade`

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7415
2013-10-29 15:32:40 -07:00
epriestley
b5a009337f Harbormaster v(-2)
Summary:
Ref T1049. I don't really want to sink too much time into this right now, but a seemingly reasonable architecture came to me in a dream. Here's a high-level overview of how things fit together:

  - **"Build"**: In Harbormaster, "build" means any process we want to run against a working copy. It might actually be building an executable, but it might also be running lint, running unit tests, generating documentation, generating symbols, running a deploy, setting up a sandcastle, etc.
  - `HarbormasterBuildable`: A "buildable" is some piece of code which build operations can run on. Generally, this is either a Differential diff or a Diffusion commit. The Buildable class just wraps those objects and provides a layer of abstraction. Currently, you can manually create a buildable from a commit. In the future, this will be done automatically.
  - `HarbormasterBuildStep`: A "build step" is an individual build operation, like "run lint", "run unit", "build docs", etc. The step defines how to perform the operation (for example, "run unit tests by executing 'arc unit'"). In this diff, this barely exists.
  - `HarbormasterBuildPlan`: This glues together build steps into groups or sequences. For example, you might want to "run unit", and then "deploy" if the tests pass. You can create a build plan which says "run step "unit tests", then run step "deploy" on success" or whatever. In the future, these will also contain triggers/conditions ("Automatically run this build plan against every commit") and probably be able to define failure actions ("If this plan fails, send someone an email"). Because build plans will run commands, only administrators can manage them.
  - `HarbormasterBuild`: This is the concrete result of running a `BuildPlan` against a `Buildable`. It tracks the build status and collects results, so you can see if the build is running/successful/failed. A `Buildable` may have several `Build`s, because you can execute more than one `BuildPlan` against it. For example, you might have a "documentation" build plan which you run continuously against HEAD, but a "unit" build plan which you want to run against every commit.
  - `HarbormasterBuildTarget`: This is the concrete result of running a `BuildStep` against a `Buildable`. These are children of `Build`. A step might be able to produce multiple targets, but generally this is something like "Unit Tests" or "Lint" and has an overall status, so you can see at a glance that unit tests were fine but lint had some issues.
  - `HarbormasterBuildItem`: An optional subitem for a target. For lint, this might be an individual file. For unit tests, an individual test. For normal builds, an executable. For deploys, a server. For documentation generation, there might just not be subitems.
  - `HarbormasterBuildLog`: Provides extra information, like command/execution transcripts. This is where stdout/stderr will get dumped, and general details and other messages.
  - `HarbormasterBuildArtifact`: Stores side effects or results from build steps. For example, something which builds a binary might put the binary in "Files" and then put its PHID here. Unit tests might put coverage information here. Generally, any build step which produces some high-level output object can use this table to record its existence.

This diff implements almost nothing and does nothing useful, but puts most of these object relationships in place. The two major things you can't easily do with these objects are:

  1) Run arbitrary cron jobs. Jenkins does this, but it feels tacked on and I don't know of anyone using it for that. We could create fake Buildables to get a similar effect, but if we need to do this I'd rather do it elsewhere in general. Build and cron/service/monitoring feel like pretty different problems to me.
  2) Run parameterized/matrix steps (maybe?). Bamboo has this plan/stage/task/job breakdown where a build step can generate a zillion actual jobs, like "build client on x86", "build server on x86", "build client on ARM", "build server on ARM", etc. We can sort of do this by having a Step map to multiple Targets, but I haven't really thought about it too much and it may end up being not-great. I'd guess we have like an 80% chance of getting a clean implementation if/when we get there. I suspect no one actually needs this, or when they do they'll just implement a custom Step and it can be parameterized at that level. I'm not too worried about this overall.

The major difference between this and Jenkins/Bamboo/TravisCI is that all three of those are **plan-centric**: the primary object in the system is a build plan, and the dashboard shows you all your build plans and the current status. I don't think this is the right model. One disadvantage is that you basically end up with top-level messaging that says "Trunk is broken", not "Trunk was broken by commit af32f392f". Harbormaster is **buildable-centric**: the primary object in the system is stuff you can run build operations against (commits/branches/revisions), and actual build plans are secondary. The main view will be "recent commits on this branch, and whether they're good or not" -- which I think is what's most important in a larger/more complex product -- not the pass/fail status of all jobs. This also makes it easier and more natural to integrate with Differential and Diffusion, which both care about the overall status of the commit/revision, not the current status of jobs.

Test Plan: Poked around, but this doesn't really do anything yet.

Reviewers: btrahan

Reviewed By: btrahan

CC: zeeg, chad, aran, seporaitis

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7368
2013-10-22 15:01:06 -07:00
epriestley
9b89e137cf Move Project transaction storage to modern tables
Summary:
Ref T4010. Projects have a weird proto-version of ApplicationTransactions which is very similar but not quite the same.

Move the storage to a modern format, but keep all the other code for now.

Test Plan: Migrated project transactions; edited projects.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7370
2013-10-22 13:49:28 -07:00
epriestley
d7a276346f Add a secret board view to Projects
Summary:
Ref T1344. This is //very// rough. Some UI issues:

  - Empty states for the board and columns are junky.
  - Column widths are crazy. I think we need to set them to fixed-width, since we may have an arbitrarily large number of columns?
  - I don't think we have the header UI elements in M10 yet and that mock is pretty old, so I sort of very roughly approximated it.
  - What should we do when you click a task title? Popping the whole task in a dialog is possible but needs a bunch of work to actually work. Might need to build "sheets" or something.
  - Icons are slightly clipped for some reason.
  - All the backend stuff is totally faked.

Generally, my plan is just to use these to implement all of T390. Specifically:

  - "Kanban" projects will have "Backlog" on the left. You'll drag them toward the right as you make progress.
  - "Milestone" projects will have "No Milestone" on the left, then "Milestone 9", "Milestone 8", etc.
  - "Sprint" projects will have "Backlog" on the left, then "Sprint 31", "Sprint 30", etc.

So all of these things end up being pretty much exactly the same, with some minor text changes and new columns showing up on the left vs the right or whatever.

Test Plan: See screenshot.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: chad, aran, sascha-egerer

Maniphest Tasks: T1344

Differential Revision: https://secure.phabricator.com/D7374
2013-10-21 21:11:36 -07:00
epriestley
cf1c06e157 Add custom field storage to Projects
Summary: Ref T4010. Adds storage and indexes for custom fields. These tables are the same as people/maniphest/differential.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T4010

Differential Revision: https://secure.phabricator.com/D7369
2013-10-21 17:01:13 -07:00
epriestley
f010730e49 Migrate all Differential inline comments to ApplicationTransactions
Summary:
Ref T2222. This implements step (1) described there, which is moving over all the inline comments.

The old and new tables are simliar. The only real trick here is that `transactionPHID` and `legacyCommentID` mean roughly the same thing (`null` if the inline is a draft, non-null if it has been submitted) but we don't have real `transactionPHID`s yet. We just make some up -- we'll backfill them later.

Two risks here:

  - I need to take a second look at the keys on this table. I think we need to tweak them a bit, and it will be less disruptive to do that before this migration than after.
  - This will take a while for Facebook, and other large installs with tens of thousands of revisions. I'll communicate this.

I'm otherwise pretty satisfied with this, seems to work well and is pretty low risk / non-disruptive.

Test Plan:
  - Before migrating, then after migrating:
    - Made a bunch of inlines (drafts, submitted).
    - Edited and deleted inlines.
    - Verified inlines showed up in preview.
    - Verified that inlines aren't indexed when they're drafts (`bin/search index D935`).
    - Verified that inlines ARE indexed when they're not drafts.
    - Verified that drafts inlines make revisions appear as "with draft" in the revision list.
  - Made left, right, and draft inlines.
  - Migrated (`bin/storage upgrade`).
  - Verified that my inlines from before the migration still showed up.
  - (Repeated all the stuff above.)
  - Manually inspected the inline comment table.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7139
2013-10-19 05:03:25 -07:00
epriestley
460051c1d8 Drop maniphest_savedquery table
Summary: This data was migrated by D6977 and is now obsolete. I'll hold this patch for a week or two in case we get reports of migration errors.

Test Plan: Ran storage upgrade, saw the table vanish. Grepped for references to the table.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6997
2013-10-19 04:56:17 -07:00
epriestley
3410cbd53e Add application and object level policy controls to Countdown
Summary: Ref T603. Give countdowns proper UI-level policy controls, and an application-level default policy. Put policy information in the header.

Test Plan:
  - Adjusted default policy.
  - Created new countdowns.
  - Edited countdowns.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7322
2013-10-16 10:36:08 -07:00
epriestley
5899ae08b3 Add storage for custom policies
Summary: Ref T603. Allows custom policies to be saved. No integration with policy controls yet.

Test Plan:
  mysql> select * from policy where id = 3\G
  *************************** 1. row ***************************
             id: 3
           phid: PHID-PLCY-e4v2fnbyuibi4supl5tn
          rules: [{"action":"allow","rule":"PhabricatorPolicyRuleAdministrators","value":null},{"action":"allow","rule":"PhabricatorPolicyRuleProjects","value":["PHID-PROJ-cwovm5gn2ilubjehcdgd"]},{"action":"allow","rule":"PhabricatorPolicyRuleLunarPhase","value":"new"}]
  defaultAction: deny
    dateCreated: 1381437466
   dateModified: 1381437466
  1 row in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7282
2013-10-10 16:09:51 -07:00
epriestley
953ff197bf Allow Herald rules to be disabled, instead of deleted
Summary:
Ref T603. Ref T1279. Further improves transaction and policy support for Herald.

  - Instead of deleting rules (which wipes out history and can't be undone) allow them to be disabled.
  - Track disables with transactions.
  - Gate disables with policy controls.
  - Show policy and status information in the headers.
  - Show transaction history on rule detail screens.
  - Remove the delete controller.
  - Support disabled queries in the ApplicationSearch.

Test Plan:
  - Enabled and disabled rules.
  - Searched for enabled/disabled rules.
  - Verified disabled rules don't activate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279, T603

Differential Revision: https://secure.phabricator.com/D7247
2013-10-06 17:10:29 -07:00
epriestley
65ddefad8b Migrate all Differential reviewer data to edges
Summary:
Ref T1279. @champo did a lot of this work already; we've been doing double writes for a long time.

Add "double reads" (reading the edge table as both the "relationship" table and as the "reviewer status" table), and migrate all the data.

I'm not bothering to try to recover old reviewer status (e.g., we could infer from transactions who accepted old revisions) because it wold be very complicated and doesn't seem too valuable.

Test Plan:
  - Without doing the migration, used Differential. Verified that reads and writes worked. Most of the data was there anyway since we've been double-writing.
  - Performed the migration. Verified that everything was still unchanged.
  - Dropped the edge table, verified all reviweer data vanished.
  - Migrated again, verified the reviewer stuff was restored.
  - Did various cc/reviewer/subscriber queries, got consistent results.

Reviewers: btrahan

Reviewed By: btrahan

CC: champo, aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7227
2013-10-05 13:54:02 -07:00
epriestley
bd8071bb33 Add a (dst, type, src) key to Differential's edge table
Summary:
Ref T1279. This came to me in a dream.

The existing `differential_relationship` table has an `(objectPHID, type)` column, which theoretically is useful for queries like "revisions with X as a reviewer". In practice, I'm not sure it gets used much, but I can get it to show up in at least some query plans.

Add a similar index to the `edge` table. This sequences //before// D7227, which actually migrates the data.

Test Plan:
  - Ran `bin/storage upgrade`.
  - EXPLAIN'd a bunch of queries against different versions of the schema, this seemed helpful overall.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7232

Conflicts:
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-10-05 13:53:43 -07:00
epriestley
7dde01df76 Fix issues with first-time account registration
Summary: This worked originally, but the migration broke slightly after the
config was deprecated, and there was another minor issue during setup.
2013-10-05 08:02:41 -07:00
epriestley
98bf001a58 Add viewPolicy and attachedToObjectPHID to PhabricatorFile
Summary:
Ref T603. Principally, I want to implement the rule "when you upload a file to an object, users must be able to see the object in order to see the file", since I think this is strongly in line with user expectation. For example, if you attach a file to a Conpherence, it should only be visible to members of that thread.

This adds storage for policies, but doesn't do anything interesting with it yet.

Test Plan: Ran `bin/storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7175
2013-10-01 08:45:18 -07:00
epriestley
7091dad606 Allow macros to have associated audio and audio behaviors
Summary: Ref T3887. Implements storage and editors, but not the actual audio part.

Test Plan: Edited audio, audio behaviors of macros. Transactions and email looked good. Hit error cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3887

Differential Revision: https://secure.phabricator.com/D7159
2013-09-27 16:01:37 -07:00
epriestley
3cb67480e0 Adjust keys for new Differential inline comment table
Summary:
Ref T2222. This sequences //before// D7139 and sorts out keys on the table. In particular:

  - There was a fairly silly `draft` key modeled after Pholio; drop it.
  - Add a `revisionPHID` key. This is queried mostly-transitionally on the revision view screen.
  - Add a `changesetID` key. This is queried by a bunch of interfaces that want more surgical results than `revisionPHID` provides.
  - Add an `authorPHID, transactionPHID` key. This is queried on the list interface to find pending drafts.
  - Add a `legacy` key. This is queried by the feed publisher.

Test Plan: Used the query analyzer to hit all (I think?) of the pages, saw keyed queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D7140
2013-09-26 13:48:36 -07:00
epriestley
5677cd23bd Add storage and classes for CustomField in Differential
Summary: Ref T3886. Adds the storage, indexes, and storage classes for modernizing Differential custom fields.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886

Differential Revision: https://secure.phabricator.com/D7138
2013-09-26 12:37:28 -07:00
epriestley
c458517cb4 Add viewPolicy, editPolicy, repositoryPHID columns to DifferentialRevision
Summary: Ref T603. Paves the way for policy controls.

Test Plan: Ran storage upgrade, bumbled around in Differential.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7133
2013-09-26 12:36:30 -07:00
epriestley
225a38c7d3 Add viewPolicy, editPolicy storage to tasks
Summary: Ref T603. Adds storage for custom policies.

Test Plan: Ran storage upgrade. Created and edited tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7118
2013-09-25 11:18:47 -07:00
epriestley
960da82f33 Fix two migration issues with Maniphest
Summary: The new editor saves tasks before applying transactions, so fields need default values.
Also, I goofed the table name.

Auditors: btrahan
2013-09-24 12:03:47 -07:00
epriestley
6e29e809b4 Fix "edit" for Maniphest comments
Summary:
Ref T2217. Fixes T3876. We incorrectly have a unique key on `(authorPHID, transactionPHID)`, which prevents saving multiple versions of a comment.

I'm not entirely sure why this exists. I think it came from Pholio (where it works for inlines, because it has an additional component, but maybe should be adjusted) but we might need to wipe it out of more apps too.

Test Plan: Edited a comment in Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217, T3876

Differential Revision: https://secure.phabricator.com/D7102
2013-09-24 11:10:31 -07:00
epriestley
1d7cbe202f Rename "transactionpro" table to "transaction"
Summary: Ref T2217. Cleans up the table names. Moves old data to `maniphest_transaction_legacy`. We'll drop that eventually once it's more clear that I didn't break the world.

Test Plan: Did reads/writes to/from these tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7094
2013-09-24 10:49:16 -07:00
epriestley
01ecf1a236 Rename ManiphestTransactionPro -> ManiphestTransaction
Summary: Ref T2217. Pro is the new standard.

Test Plan: Lots of `grep`, made a pile of Maniphest views/edits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7093
2013-09-24 10:49:06 -07:00
epriestley
7abe9dc4c0 Migrate all Maniphest transaction data to new storage
Summary:
Ref T2217. This is the risky, hard part; everything after this should be smooth sailing. This is //mostly// clean, except:

  - The old format would opportunistically combine a comment with some other transaction type if it could. We no longer do that, so:
    - When migrating, "edit" + "comment" transactions need to be split in two.
    - When editing now, we should no longer combine these transaction types.
    - These changes are pretty straightforward and low-impact.
  - This migration promotes "auxiliary field" data to the new CustomField/StandardField format, so that's not a straight migration either. The formats are very similar, though.

Broadly, this takes the same attack that the auth migration did: proxy all the code through to the new storage. `ManiphestTransaction` is now just an API on top of `ManiphestTransactionPro`, which is the new storage format. The two formats are very similar, so this was mostly a straight copy from one table to the other.

Test Plan:
  - Without performing the migration, made a bunch of edits and comments on tasks and verified the new code works correctly.
  - Droped the test data and performed the migration.
  - Looked at the resulting data for obvious discrepancies.
  - Looked at a bunch of tasks and their transaction history.
  - Used Conduit to pull transaction data.
  - Edited task description and clicked "View Details" on transaction.
  - Used batch editor.
  - Made a bunch more edits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7068
2013-09-23 14:25:28 -07:00
epriestley
4c2bd2ca8e Add storage for new Maniphest transactions
Summary: Ref T2217. Add the tables and comment class for the new stuff. Not used yet.

Test Plan: Ran storage upgrade, browsed Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7066
2013-09-23 14:24:58 -07:00
epriestley
b8cb6ddaa5 Add some keys and policy fields to repositories
Summary:
  - Add some TODO'd keys.
  - Add policy fields.

Test Plan: Viewed repositories; created a new repository and verified it got the right default policy settings.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7056
2013-09-21 16:23:01 -07:00
epriestley
0558d53273 Convert maniphest to use standard fields
Summary: Ref T3794. Drop auxiliary field, use standard field.

Test Plan: Performed migration, field seemed to survive it intact. Edited and viewed tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3794

Differential Revision: https://secure.phabricator.com/D7036
2013-09-19 11:56:15 -07:00
epriestley
bd40e74400 Migrate auxiliary field storage to common field storage
Summary: Ref T418. Moves data from the Maniphest-specific table to the general one. This patch is a bit gross, but mostly about getting the reads and writes aimed correctly. Future patches will clean things up.

Test Plan: Migrated data across formats. Verified it survied the migration. Viewed and edited tasks' custom fields.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6999
2013-09-16 16:02:06 -07:00
epriestley
b0e145f2fe Provide generalized custom field storage and custom field indexes in Maniphest
Summary: Ref T418. Depends on D6992. This adds index and value storage for Maniphest custom fields.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418

Differential Revision: https://secure.phabricator.com/D6995
2013-09-16 14:06:41 -07:00
epriestley
c8574cf6fd Integrate ApplicationSearch with CustomField
Summary:
Ref T2625. Ref T3794. Ref T418. Ref T1703.

This is a more general version of D5278. It expands CustomField support to include real integration with ApplicationSearch.

Broadly, custom fields may elect to:

  - build indicies when objects are updated;
  - populate ApplicationSearch forms with new controls;
  - read inputs entered into those controls out of the request; and
  - apply constraints to search queries.

Some utility/helper stuff is provided to make this easier. This part could be cleaner, but seems reasonable for a first cut. In particular, the Query and SearchEngine must manually call all the hooks right now instead of everything happening magically. I think that's fine for the moment; they're pretty easy to get right.

Test Plan:
I added a new searchable "Company" field to People:

{F58229}

This also cleaned up the disable/reorder view a little bit:

{F58230}

As it did before, this field appears on the edit screen:

{F58231}

However, because it has `search`, it also appears on the search screen:

{F58232}

When queried, it returns the expected results:

{F58233}

And the actually good bit of all this is that the query can take advantage of indexes:

  mysql> explain SELECT * FROM `user` user JOIN `user_customfieldstringindex` `appsearch_0` ON `appsearch_0`.objectPHID = user.phid AND `appsearch_0`.indexKey = 'mk3Ndy476ge6' AND `appsearch_0`.indexValue IN ('phacility') ORDER BY user.id DESC LIMIT 101;
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  | id | select_type | table       | type   | possible_keys     | key      | key_len | ref                                      | rows | Extra                                        |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  |  1 | SIMPLE      | appsearch_0 | ref    | key_join,key_find | key_find | 232     | const,const                              |    1 | Using where; Using temporary; Using filesort |
  |  1 | SIMPLE      | user        | eq_ref | phid              | phid     | 194     | phabricator2_user.appsearch_0.objectPHID |    1 |                                              |
  +----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
  2 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T418, T1703, T2625, T3794

Differential Revision: https://secure.phabricator.com/D6992
2013-09-16 13:44:34 -07:00
epriestley
5e274778d1 Restore one missing "order" value to the saved query migration script
Summary: Caught this when migrating my queries in production.

Auditors: btrahan
2013-09-13 12:00:50 -07:00
epriestley
9864d562dd Migrate old Maniphest custom queries to new query infrastructure
Summary: Ref T2625. EVERYONE LOVES MIGRATIONS!!!

Test Plan:
  - Created and migrated a query with every field, verified results were preserved.
  - Created and migrated a query using "noproject" and "upforgrabs" magic, verified results were preserved.

Here's the pre-migration "everything" query:

{F58110}

Here it is after migration:

{F58111}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6977
2013-09-13 11:49:45 -07:00
epriestley
b38a688cf7 Ignore failures when rebuilding project search index in Maniphest migration
Summary: See <https://github.com/facebook/phabricator/issues/386>. Just ignore any issues here, they aren't worth complaining about.

Auditors: btrahan
2013-09-13 09:39:46 -07:00
epriestley
e50eccf109 Provide and populate an object name index for Maniphest
Summary: See discussion in D6955. This provides a table we can JOIN against to (effectively) "ORDER BY project name", populates it intially, and keeps it up to date as projects are edited.

Test Plan:
  - Ran storage upgrade, verified projects populated into the table.
  - Edited a project, verified its entry updated.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6957
2013-09-12 13:06:44 -07:00
epriestley
df86f87289 Add a "dateCreated" key to Maniphest
Summary: Depends on D6952. Unpunts there since I'm rolling into a swamp full of schema changes.

Test Plan: Issued date-constrained query and saw key as a candidate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6954
2013-09-12 13:04:31 -07:00
epriestley
8f8c61be31 Remove legacy "touched" table and indexing
Summary: Noticed this in the schema. "Touches" were an idea that never really got off the ground, as we built out more/better notification channels instead. Essentially, they recorded any object you'd ever interacted with. Maybe this will be useful some day, but for now it does nothing and can't be interacted with. Nuke it.

Test Plan: `grep`, loaded Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6953
2013-09-12 13:04:09 -07:00
epriestley
2d240142c8 Fix 20130820.file-mailkey-populate.php for databases in strict mode
Summary:
Fixes T3803. Turns out my advice was sort of terrible. :/

In strict mode, the `INSERT INTO x (y, z)` raises an error unless `(y, z, ...)` includes //all// columns without default values.

Test Plan: Ran `storage upgrade` on a strict-mode install. Verified no inserts were performed.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3803

Differential Revision: https://secure.phabricator.com/D6894
2013-09-05 15:58:28 -07:00
Bob Trahan
228496cdbe File - add transactions and editor
Summary: this ends up being a little weird since you can't actually edit files. Also, since we create files all sorts of ways, sometimes without even having a user, we don't  bother logging transactions for those events. Fixes T3651. Turns out this work is important for T3612, which is a priority of mine to help get Pholio out the door.

Test Plan: left a comment on a file. it worked! use bin/mail to verify mail content looked correct.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran, wez

Maniphest Tasks: T3651, T3612

Differential Revision: https://secure.phabricator.com/D6789
2013-09-05 13:11:02 -07:00
epriestley
41ac06959e Generate some amount of PHP class documentation
Summary:
Ref T988. This brings the class/interface atomizer over. A lot of parts of this are still varying degrees of very-rough, but most of the data ends up in approximatley the right place.

ALSO: PROGRESS BARS

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6817
2013-08-28 09:54:39 -07:00
epriestley
49e0aa203c Add storage for Releeph project and branch transactions
Summary: Ref T3663. Does what it says on the tin.

Test Plan: Ran `storage upgrade`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3663

Differential Revision: https://secure.phabricator.com/D6778
2013-08-21 12:31:51 -07:00
epriestley
973bb0c76e Remove ReleephRequestEvent
Summary: Ref T3663. This is obsolete code which is used only in this migration, which Facebook has already performed and which isn't relevant for any other installs.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3663

Differential Revision: https://secure.phabricator.com/D6777
2013-08-21 12:31:50 -07:00
epriestley
ca0115b361 Support configuration-driven custom fields
Summary:
Ref T1702. Ref T3718. There are a couple of things going on here:

**PhabricatorCustomFieldList**: I added `PhabricatorCustomFieldList`, which is just a convenience class for dealing with lists of fields. Often, current field code does something like this inline in a Controller:

  foreach ($fields as $field) {
    // do some junk
  }

Often, that junk has some slightly subtle implications. Move all of it to `$list->doSomeJunk()` methods (like `appendFieldsToForm()`, `loadFieldsFromStorage()`) to reduce code duplication and prevent errors. This additionally moves an existing list-convenience method there, out of `PhabricatorPropertyListView`.

**PhabricatorUserConfiguredCustomFieldStorage**: Adds `PhabricatorUserConfiguredCustomFieldStorage` for storing custom field data (like "ICQ Handle", "Phone Number", "Desk", "Favorite Flower", etc).

**Configuration-Driven Custom Fields**: Previously, I was thinking about doing these with interfaces, but as I thought about it more I started to dislike that approach. Instead, I built proxies into `PhabricatorCustomField`. Basically, this means that fields (like a custom, configuration-driven "Favorite Flower" field) can just use some other Field to actually provide their implementation (like a "standard" field which knows how to render text areas). The previous approach would have involed subclasssing the "standard" field and implementing an interface, but that would mean that every application would have at least two "base" fields and generally just seemed bleh as I worked through it.

The cost of this approach is that we need a bunch of `proxy` junk in the base class, but that's a one-time cost and I think it simplifies all the implementations and makes them a lot less magical (e.g., all of the custom fields now extend the right base field classes).

**Fixed Some Bugs**: Some of this code hadn't really been run yet and had minor bugs.

Test Plan:
{F54240}
{F54241}
{F54242}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1702, T1703, T3718

Differential Revision: https://secure.phabricator.com/D6749
2013-08-14 12:33:53 -07:00
epriestley
9f41032693 Remove "cut point commit identifier" from Releeph
Summary: Ref T3656. Releeph denormalizes branch cut point identifiers into Branch objects, but this information isn't useful or used for sorting, filtering, or enforcing unique constraints. Instead, derive it via noramlized pathways from the `cutPointCommitPHID`.

Test Plan: Ran storage upgrade. Ran `releephwork.getbranch` and `releeph.getbranches`. Grepped for `cutPointCommitIdentifier`.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3656

Differential Revision: https://secure.phabricator.com/D6636
2013-08-14 09:01:38 -07:00
epriestley
a84cc777c8 Remove "Project ID" from Releeph Projects
Summary:
Fixes T3660. Releeph Projects currently have an unused one-to-one mapping to Phabricator projects. This isn't consistent with other applications and has no integrations or uses. Get rid of it.

NOTE: Waiting for signoff from @legneato on T3660 before pulling the trigger here.

Test Plan: Created and edited Releeph projects. Grepped for references to project ID; there are a dozen or so but they're all either Releeph projects or Arcanist projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3660

Differential Revision: https://secure.phabricator.com/D6635
2013-08-14 09:00:56 -07:00
epriestley
296818e129 Remove ReleephProject->repositoryID writes
Summary: Ref T3655. Depends on D6633. This removes the writes and the column.

Test Plan: Created a project, edited a project. Verified the table doesn't have any keys including this column.

Reviewers: btrahan

Reviewed By: btrahan

CC: LegNeato, aran

Maniphest Tasks: T3655

Differential Revision: https://secure.phabricator.com/D6634
2013-08-14 09:00:25 -07:00
epriestley
b1c4a258c9 Add ApplicationTransactions to Herald
Summary: Ref T2769. I'm planning to keep this pretty simple, but we have this ad-hoc edit log for rules already and some other mess that we can clean up.

Test Plan: No effect yet; see future changes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6654
2013-08-07 18:03:49 -07:00
epriestley
2820fdc89b Add PHIDs to Herald Rules
Summary: Ref T2769. Precursor to various Herald-related modernizations.

Test Plan: Ran migration; loaded Herald via web.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6648
2013-08-07 18:03:37 -07:00
Bob Trahan
0e6b5073cd Paste - add support for email replies and subscribers
Summary: Email replies and subscribers seem to go hand in hand so deploy both at once.

Test Plan: played around with bin/mail. Verified replies posted comments on the paste.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3650

Differential Revision: https://secure.phabricator.com/D6682
2013-08-05 17:11:46 -07:00
epriestley
ed9edc5d3a Minor, fix Paste SQL patch for databases with all warnings turned on.
This column has no default value and throws if you have maximum warnings activated:

  EXCEPTION: (AphrontQueryException) #1364: Field 'commentVersion' doesn't have a default value...

Auditors: btrahan
2013-08-04 11:32:32 -07:00
Bob Trahan
37a5c4b11a Paste - add transactions
Summary: Ref T3650. This adds a create transaction, transactions for metadata (title, langauge, view policy), and comments. Editor is used on all create /edit paths.

Test Plan: made some pastes via web and email - yay. edited pastes - yay. verified txns showed up on pastes and in feed correctly.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3516, T3650

Differential Revision: https://secure.phabricator.com/D6645
2013-08-02 12:56:58 -07:00
epriestley
5cc3bbf721 Use application PHIDs for application transactions
Summary: Ref T2715. Ref T3578. Load application transactions through application PHID infrastructure.

Test Plan: Viewed feed, saw successful loads of application transaction objects and rendered feed stories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715, T3578

Differential Revision: https://secure.phabricator.com/D6617
2013-07-29 12:04:15 -07:00
epriestley
71841e262a Migrate Ponder comments to ApplicationTransactions
Summary: Ref T3373. Bring the storage across.

Test Plan: {F52067}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3373

Differential Revision: https://secure.phabricator.com/D6606
2013-07-29 12:04:05 -07:00
epriestley
6172f2bbd1 Require answers' authors to be unique in Ponder
Summary: Ref T3578. I forget if this was an explicit decision or not, but we currently let the same user answer questions multiple times. I think this probably causes more confusion than it provides freedom. In conjunction with other UI issues (commenting being weird, notably), we're seeing some use of answers to comment, which is undesirable. Require each answer's author to be unique. Merge existing nonunique authors' answers.

Test Plan: {F52062}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3578

Differential Revision: https://secure.phabricator.com/D6605
2013-07-29 12:04:04 -07:00
epriestley
8f0c836063 Add question and answer transactions to Ponder
Summary: Ref T3373.

Test Plan:
Ran this script, saw valid inserts:

  <?php

  $unguarded = AphrontWriteGuard::beginScopedUnguardedWrites();

  id(new PonderQuestionTransaction())
    ->setAuthorPHID('herp')
    ->setObjectPHID('derp')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setTransactionType('beep')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderAnswerTransaction())
    ->setAuthorPHID('herp')
    ->setObjectPHID('derp')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setTransactionType('beep')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderQuestionTransactionComment())
    ->setCommentVersion(1)
    ->setAuthorPHID('bloop')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setContent('blip')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  id(new PonderAnswerTransactionComment())
    ->setCommentVersion(1)
    ->setAuthorPHID('bloop')
    ->setViewPolicy('blarp')
    ->setEditPolicy('goop')
    ->setContent('blip')
    ->setContentSource(PhabricatorContentSource::newForSource('derp', array()))
    ->save();

  unset($unguarded);

  echo "OK!\n";

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3373

Differential Revision: https://secure.phabricator.com/D6584
2013-07-28 15:08:36 -07:00
Gareth Evans
70a5ec600d Allowing Closing of questions
Summary:
Fixes T3579

As a basic overview, this enables the author of a question to open/close a question.

Other bits;

- Add "Open" filter to the builtin queries
- Add "Status" to search form
- Refactor ponder constants
- Add coloured bars for different question statuses

Test Plan:
- Open/Close questions
- Search for some bits
- Use filters

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3579

Differential Revision: https://secure.phabricator.com/D6590

Conflicts:
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-07-27 18:41:03 -07:00
Bob Trahan
f75f3a0c3b Pholio - add concept of replacing images and primitive history view
Summary:
Now you can actually replace an image! Ref T3572. This ended up needing a wee bit of infrastructure to work...

 - add replace image transaction to pholio
 - add replacesImagePHID to PholioImage
 - tweaks to editor to properly update images with respect to replacement
   - add edges to track replacement
 - expose getNodes on graph query infrastructure to query the entire graph of who replaced who
 - move pholio image to new phid infrastructure

Still TODO - the history view should get chopped out a bit from the current view - no more inline comments / generally less functionality plus maybe a tweak or two to make this more sensical.

Test Plan: replaced images and played with history controller a little. works okay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3572

Differential Revision: https://secure.phabricator.com/D6560
2013-07-25 16:59:25 -07:00
epriestley
6750a48951 Surface task queue temporary failure rate in Daemon console
Summary: Fixes T3557. One thing which made T3557 kind of a mess was the lack of information about progress through temporary failures. Add a column which records a task's last failure time, and surface it in the console.

Test Plan: {F51277}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3557

Differential Revision: https://secure.phabricator.com/D6550
2013-07-23 16:58:22 -07:00
epriestley
17ee71d896 Use Application PHIDs in Maniphest
Summary: Ref T2715. Switch Maniphest to the new stuff.

Test Plan: Used `phid.query`; `phid.lookup` to load objects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6516
2013-07-22 12:17:35 -07:00
epriestley
1fb39a20d3 Move DifferentialRevision to application PHIDs
Summary: Ref T2715.

Test Plan: Used `phid.lookup` and `phid.query` to load handles. Grepped for `PHID_TYPE_DREV`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6509
2013-07-22 12:17:29 -07:00
epriestley
7ed6996604 Provide basic infrastructure for moving PHIDs, Handles and Object Names to applications
Summary:
See discussion in T2715. Currently, PHIDs are all hard coded in the PHID application. In the long run, we need to move them out into actual applications.

A specific immediate issue is Releeph, which uses a very very old and very broken mechanism to inject PHIDs in a way that only sort of works.

Moving forward, every PHID type will be provided by a `PhabricatorPHIDType` subclass, which will manage loading it, etc.

This also moves toward cleaning up the "load objects by name" (where "name" means something like `D12`) code, which is an //enormous// mess and spread across at least 4-5 callsites.

Test Plan: Used `phid.lookup` and `phid.query` to load Slowvotes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6502
2013-07-21 06:34:21 -07:00
Bob Trahan
001f76cbaa Projects - tighten up a few things
Summary:
Fixes T2675, T2676.

 - when the last person leaves a project it is archived.
 - a script to archive all memberless projects
 - better feed stories for the various policy edits you can do
 - phriction pages are also moved as you rename projects

Test Plan: edited some projects and noted reasonable feed stories. ran script against test data and it worked! left a last man standing project and it archived. renamed a project to "a" then "b" then "a" (etc) and it worked including phrictiondocument moves

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2676, T2675

Differential Revision: https://secure.phabricator.com/D6478
2013-07-17 16:43:37 -07:00
Bob Trahan
df264d8548 Pholio - support editing images - fixes T3489
Summary:
Nice title. We add three new transactions - IMAGE_FILE, IMAGE_NAME, and IMAGE_DESCRIPTION. The first is a bit like subscribers as it is a list of file phids. The latter have values of the form ($file_phid => $data), where $data is $name or $description respectively. This is because we need to collate transactions based on $file_phid...

Overall, this uses the _underyling files_ and not the "PholioImage" to determine if things are unique or not. That said, simply mark PholioImages as obsolete so inline comments about no-longer applicable PholioImages don't break.

Does a reasonable job implementing the mock. Note you can't "update" an image at this time, though you can delete and add at will.

Test Plan: played with pholio a ton.

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T3489

Differential Revision: https://secure.phabricator.com/D6441
2013-07-16 13:31:20 -07:00
epriestley
d2f8c5b5e7 Make slowvotes subscribable
Summary: Subscribing doesn't do anything yet, but you can subscribe!

Test Plan: {F50169}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6456
2013-07-16 10:30:28 -07:00
epriestley
ea52bcbcd6 Migrate Slowvote comments to ApplicationTransactions
Summary:
Move comments from the old table to ApplicationTransactions. Patch dances around which objects it uses since I intend to delete the comment table.

NOTE: This temporarily disables comment writes. I'll restore them shortly.

Test Plan: {F50166}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6454
2013-07-16 10:30:21 -07:00
epriestley
ef1bedef02 Add ApplicaionTransactions and a mutable view policy to Slowvote
Summary: Schema changes to modernize this app.

Test Plan: Ran schema changes, created a new slowvote. No real effects yet.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6453
2013-07-16 10:30:14 -07:00
epriestley
237ca095a6 Trim trailing whitespace from user realnames
Summary: Fixes T3069. Ref T3516. For a long time, LDAP import added trailing whitespace to real names when importing them (for example, `"John Smith"` would be imported as `"John Smith "`). We no longer do this, but should clean up the database since it seems highly unlikely that any user wants trailing whitespace in their real name.

Test Plan:
  $ ./bin/storage upgrade -f --apply phabricator:20130711.trimrealnames.php
  Applying patch 'phabricator:20130711.trimrealnames.php'...
  Trimming trailing whitespace from user real names...
  Trimming user 1 from 'Evan Priestley   ' to 'Evan Priestley'.
  User 4 is already trim.
  User 5 is already trim.
  User 6 is already trim.
  User 7 is already trim.
  User 8 is already trim.
  User 9 is already trim.
  User 10 is already trim.
  User 11 is already trim.
  User 12 is already trim.
  User 13 is already trim.
  User 14 is already trim.
  User 15 is already trim.
  User 21 is already trim.
  User 22 is already trim.
  User 26 is already trim.
  User 28 is already trim.
  User 29 is already trim.
  User 32 is already trim.
  User 33 is already trim.
  User 35 is already trim.
  User 39 is already trim.
  User 51 is already trim.
  User 53 is already trim.
  User 54 is already trim.
  User 55 is already trim.
  Done.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3069, T3516

Differential Revision: https://secure.phabricator.com/D6426
2013-07-11 10:10:06 -07:00
Bob Trahan
4fb3b27f1d Legalpad - add signature page
Summary: Fixes T3481. Sort of - this thing be very ugly. Also it assumes that you'll "always" want to sign terms. I was thinking in a future diff that should be optional as well as configurable, though it was unclear to me if either was worth pursuing...  Generally very hideous as the three main elements (PHUIDocument, AphrontErrorView, and AphrontForm with an AphrontFormInset) have never really played together before.

Test Plan: agreed to some test terms. noted UI displayed nicely. reloaded and noted UI told me I had signed it already. Went to different terms and filled them out wrong and got sensical errors.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3481

Differential Revision: https://secure.phabricator.com/D6399
2013-07-10 11:46:39 -07:00
Lauri-Henrik Jalonen
93e37e9060 Phabricator event timeline removed
Summary: Removed related files and references

Test Plan: Crossed my fingers

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2003

Differential Revision: https://secure.phabricator.com/D5744

Conflicts:
	src/__phutil_library_map__.php
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-07-09 18:07:42 -07:00
Bob Trahan
2d87bb29ac Legalpad - denormalize a field or two from DocumentBody to Document
Summary: this let's the list controller save a query. Fixes T3488. Note didn't bother denormalizing document body at all since I don't think we want to show a snippet.

Test Plan: viewed a list of legalpad documents - yay. viewed a legalpad document - yay. created a legalpad document - yay. edited a legalpad document - yay. edited with N authors - yay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3488

Differential Revision: https://secure.phabricator.com/D6382
2013-07-08 17:06:49 -07:00
Bob Trahan
2c03cd931b Legalpad V0.2 - add mail integration
Summary:
Supports !unsubscribe and commenting on replies. Subscribers get mailed something reasonable. Fixes T3480.

Sneaks in /LX/ support. In the near future I want to have that /LX/ be a clean "signature" page sans all the edit actions and other fluff... Will resolve this as part of T3481.

Test Plan: used the metamta console to add comments and unsubscribe. added a phlog() inside mail code to verify mail bodies looked okay.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3480

Differential Revision: https://secure.phabricator.com/D6369
2013-07-03 16:37:05 -07:00
epriestley
c3b2184977 Mostly modernize Conduit logs
Summary:
  - Add GC support to conduit logs.
  - Add Query support to conduit logs.
  - Record the actual user PHID.
  - Show client name.
  - Support querying by specific method, so I can link to this from a setup issue.

@wez, this migration may not be fast. It took about 8 seconds for me to migrate 800,000 rows in the `conduit_methodcalllog` table. This adds a GC which should keep the table at a more manageable size in the future.

You can safely delete all data older than 30 days from this table, although you should do it by `id` instead of `dateCreated` since there's no key on `dateCreated` until this patch.

Test Plan:
  - Ran GC.
  - Looked at log UI.
  - Ran Conduit methods.

Reviewers: btrahan

Reviewed By: btrahan

CC: wez, aran

Differential Revision: https://secure.phabricator.com/D6332
2013-07-01 12:37:34 -07:00
Bob Trahan
f15ed36d3c legalpad schema
Summary:
Ref T3116. This is a large amount of schema for V0 but it seems relatively complete to the desired features in T3116.

The only thing of note that is missing is documentSignatures should have some sort of "signedStatus". "Un-signing" seemed weird to me, though I could imagination "pending signature". "Pending signature" could be done via edges pretty easily.

Plan is to have "Document" be at the top level and own policy. "DocumentBody" will store a version of title and text for each and every "edit" on a larger Document. "Edges" are to be used to tie Authors => Document for V0ish. Transactions are going to be used to store all the various edits possible here. Oh and DocumentSignatures will do what you expect, but include documentVersion as part of the key.

Test Plan: just some schema. `storage update` worked though!

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T3116

Differential Revision: https://secure.phabricator.com/D6323
2013-06-28 13:56:16 -07:00
epriestley
f54a5d8087 Add DoorkeeperExternalObject
Summary:
Ref T2852. This table holds data about external objects and allows us to write edges to them.

Objects are identified with an `<applicationType, applicationDomain, objectType, objectID>` tuple. For example, Asana tasks will be, e.g., `<asana, asana.com, asana:task, 93829279873>` or similar.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6271
2013-06-24 15:54:36 -07:00
epriestley
3124838d65 Undo D6266 (DifferentialComment PHID migration)
Summary:
Ref T2222. My path forward here wasn't very good -- I was thinking I could set `transactionPHID` for the inline comments as I migrated, but it must be unique and an individual DifferentialComment may have more than one inline comment. Dropping the unique requirement just creates more issues for us, not fewer.

So the migration in D6266 isn't actually useful. Undo it -- this can't be a straight revert because some installs may already have upgraded.

Test Plan: Ran new migrations, verified the world ended up back in the same place as before (made comments, viewed reivsions).

Reviewers: btrahan

Reviewed By: btrahan

CC: wez, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D6269
2013-06-24 11:00:35 -07:00
epriestley
75fa580f3f Add PHIDs to DifferentialComments
Summary:
Ref T2222. This adds PHIDs to all Differential comments so I can migrate the inlinecommment table to transaction_comment in the next diff.

@wez, this will issue a few million queries for Facebook (roughly, one for each Differential comment ever made). It's safe to skip the `.php` half of the patch, bring Phabricator up normally, and then apply this patch with Phabricator running if that eases the migration, although the next few diffs will probably be downtime-required migrations so maybe it's easier to just schedule some downtime.

Test Plan: Ran migration locally. Verified existing comments and new comments received PHIDs.

Reviewers: btrahan

Reviewed By: btrahan

CC: wez, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D6266
2013-06-21 18:41:14 -07:00
epriestley
44302d2f07 Add storage for new Differential transactions and transaction comments
Summary:
Ref T2222. Ref T1460. Depends on D6260.

This creates the new tables, but doesn't start using them. I added three new fields for {T1460}, to represent fixed/done/replied states.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T1460, T2222

Differential Revision: https://secure.phabricator.com/D6261
2013-06-21 12:54:29 -07:00
epriestley
055535b462 Minor, fix the "password" auth default (should be enabled if not provided). 2013-06-20 11:20:41 -07:00
epriestley
3b9ccf11f2 Drive auth config with the database
Summary: Ref T1536. This is the last major migration. Moves us over to the DB and drops all the config stuff.

Test Plan:
  - Ran the migration.
  - Saw all my old config brought forward and respected, with accurate settings.
  - Ran LDAP import.
  - Grepped for all removed config options.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran, wez

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6243
2013-06-20 11:18:11 -07:00
epriestley
5f29ccaaca Add storage for Auth configuration in preparation for moving it into a web interface
Summary:
Ref T1536. Currently, we have about 40 auth-related configuration options. This is already roughly 20% of our config, and we want to add more providers. Additionally, we want to turn some of these auth options into multi-auth options (e.g., allow multiple Phabricator OAuth installs, or, theoretically multiple LDAP servers).

I'm going to move this into a separate "Auth" tool with a minimal CLI (`bin/auth`) interface and a more full web interface. Roughly:

  - Administrators will use the app to manage authentication providers.
  - The `bin/auth` CLI will provide a safety hatch if you lock yourself out by disabling all usable providers somehow.
  - We'll migrate existing configuration into the app and remove it.

General goals:

  - Make it much easier to configure authentication by providing an interface for it.
  - Make it easier to configure everything else by reducing the total number of available options.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6196
2013-06-17 10:48:41 -07:00
epriestley
8744cdb699 Migrate PhabricatorUserLDAPInfo to PhabricatorExternalAccount
Summary: Ref T1536. This is similar to D6172 but much simpler: we don't need to retain external interfaces here and can do a straight migration.

Test Plan: TBA

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6173
2013-06-16 09:55:55 -07:00
epriestley
8111dc74bf Migrate the OAuthInfo table to the ExternalAccount table
Summary: Ref T1536. Migrates the OAuthInfo table to ExternalAccount, and makes `PhabricatorUserOAuthInfo` a wrapper for an ExternalAccount.

Test Plan: Logged in with OAuth, registered with OAuth, linked/unlinked OAuth accounts, checked OAuth status screen, deleted an account with related OAuth.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6172
2013-06-14 07:04:41 -07:00
epriestley
8886416e30 Expand the "PhabricatorExternalAccount" table for new registration
Summary:
Ref T1536. This is the schema code for `PhabricatorExternalAccount` which was previously in D4647. I'm splitting it out so I can put it earlier in the sequence and because it's simple and standalone.

Expands `PhabricatorExternalAccount` to have everything we need for the rest of registration.

Test Plan: Implemented the remainder of new registration on top of this.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6169
2013-06-14 06:55:18 -07:00
epriestley
6ffbee115b Add ApplicationTransactions/CustomField based user profile editor
Summary:
Adds a profile edit controller (with just one field and on links to it) that uses ApplicationTransactions and CustomField.

{F45617}

My plan is to move the other profile fields to this interface and get rid of Settings -> Profile. Basically, these will be "settings":

  - Sex
  - Language
  - Timezone

These will be "profile":

  - Real Name
  - Title
  - Blurb
  - Profile Image (but I'm going to put this on a separate UI)
  - Other custom fields

Test Plan: Edited my realname using the new interface.

Reviewers: chad, seporaitis

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6152
2013-06-07 09:55:55 -07:00
epriestley
758586abda Allow builtin named queries to be disabled
Summary:
Applications come with builtin queries, but users might want to get rid of them. Allow users to disable named queries if they prefer.

This has one funky behavior, which is that the first time you disable a named query it goes to the top of your list. That will be fixed in the next diff, which will make them reorderable.

Test Plan: Added/edited/removed named queries, disabled/enabled builtin named queries.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6128
2013-06-05 05:28:25 -07:00
epriestley
d9848d3c46 Add a book controller and various amenities to Diviner's live view
Summary: Ref T988. Mostly backend changes, with a very rough frontend on top of them. See Conpherence discussion.

Test Plan: {F45010}

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6113
2013-06-04 11:15:34 -07:00
epriestley
3aae972406 Implement ApplicationSearch in Files
Summary: Ref T2625. Ref T1163. A couple of small generalization nudges, but this is almost entirely straightforward.

Test Plan: Executed various File queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1163, T2625

Differential Revision: https://secure.phabricator.com/D6091
2013-05-31 10:51:05 -07:00
epriestley
fb765b8c93 Add language and date ranges to Paste queries
Summary:
Ref T2625. Ref T3273. This is mostly a UI foil for T3273. Right now, to find tasks without owners or without projects you search for the magic strings "upforgrabs" and "noproject". Unsurprisingly, no users have ever figured this out. I want to get rid of it. Instead, these interfaces will look like:

      Assigned: [ Type a user name... ]
                [ X ] Find unassigned tasks.
      Projects: [ Type a project name... ]
                [ X ] Find tasks with no projects.

Seems reasonable, I think?

Test Plan: Searched for "rainbow, js", "rainbow + no language", "no language", date ranges, etc.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T2625, T3273

Differential Revision: https://secure.phabricator.com/D6085
2013-05-30 18:55:04 -07:00
epriestley
bccc1e1244 Add a date range query to Macros
Summary:
Ref T1163. Ref T2625. This could probably use some tweaks, but I kept things mostly-generic.

I added a new control for freeform dates so we can have it render help or whatever later on.

Test Plan: See screenshots.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625, T1163

Differential Revision: https://secure.phabricator.com/D6084
2013-05-30 17:32:12 -07:00
Jakub Vrana
32f91557f8 Store hash of session key
Summary:
This prevents security by obscurity.
If I have read-only access to the database then I can pretend to be any logged-in user.

I've used `PhabricatorHash::digest()` (even though we don't need salt as the hashed string is random) to be compatible with user log.

Test Plan:
Applied patch.
Verified I'm still logged in.
Logged out.
Logged in.

  $ arc tasks

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6080
2013-05-30 17:30:06 -07:00
epriestley
d63811d319 Store authors on image macros
Summary:
Currently, the author of an image macro is read from the attached file. This is messy and necessitates a join, and is not always correct. Instead, store the data when the macro is created.

This lays the groundwork for generalizing ApplicationSearch here. Ref T2625.

Test Plan: Migrated existing macros, created a new macro, checked web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6071
2013-05-29 15:05:44 -07:00
epriestley
2fd018ad92 Begin transacitonalizing repository edits and provide a more sensible edit interface
Summary:
Ref T2231, T603. Plan of attack here is pretty much:

  - Built out a new (currently not linked in the UI) edit interface in Diffusion which is transaction-based and has a sensible layout.
  - Build out a new create interface based on PagedForm which dumps into the new edit interface.
  - Throw the old stuff away.
  - Everyone lives happily ever after.

Test Plan:
{F44163}
{F44164}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D6029
2013-05-24 12:37:42 -07:00
Gareth Evans
183bb8c80f Stop writing empty strings to the ownerPHID column
Summary: If we're unassigning an owner from a task it should set the column to `NULL` rather than an empty string. Fixes T3239

Test Plan: Assigned and Unassigned a task. Make sure the db is doing as excpected. Ran the patch, checked the db.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T3239

Differential Revision: https://secure.phabricator.com/D6017
2013-05-23 09:22:18 -07:00
Bob Trahan
a56ca7a043 Conpherence - kill the image-based header UI
Summary:
removes the whole custom image thing, instead using a more standard application crumbs. Gives this glorious space back to the compose area which is now tens of pixels taller. Also defaults it to the people widget. Basically, fixes T3160.

For now, you **CAN NOT** edit the title of a conpherence. I didn't want to jam in too much here. Next diff will be to change the widget icons into the dropdown switcher, which will also bring back the editing of titles.

Test Plan: looked at conpherence and it was pretty. Resized it vigorously and it wasn't too bad.

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T3160

Differential Revision: https://secure.phabricator.com/D5998
2013-05-22 16:05:47 -07:00
epriestley
b0a5f42244 Add "live" publisher and storage to Diviner
Summary:
Ref T988. This adds basics for the non-static publishing target:

  - Storage (called "Live", e.g. `DivinerLiveAtom` to distinguish it from shared classes like `DivinerAtom`).
  - Mostly populate the storage.
  - Some minor fixes and improvements.

Test Plan: Generated docs, looked at DB, saw mostly-sensible output.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D5973
2013-05-20 10:18:26 -07:00
epriestley
eabe3a4d33 Begin improving the soundness of received mail
Summary:
We/I broke a couple of things here recently (see D5911) and are doing some work here in general (see D5912, etc.).

Generally, this code is pretty oldschool and not especially well architected for modern application-oriented Phabricator. It hardcodes a lot of stuff which should be applications' responsibilites.

Take the first steps toward making it more solid to reduce the risk here. In particular:

  - Factor out the "self mail" and "duplicate mail" checks and add unit tests.
  - Make Message-ID hash handling automatic.

Test Plan: Ran unit tests.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5915
2013-05-13 16:32:19 -07:00
Edward Speyer
fcb7286533 ReleephRequest xactions
Summary:
Migrate to `PhabricatorApplicationTransactions` (`ReleephRequestTransactions` applied by `ReleephRequestTransactionalEditor`, instead of `ReleephRequestEvents` created by `ReleephRequestEditor`) and migrate all the old events into transactions.  Email is supported in the standard way (no more `ReleephRequestMail`) as well.

This also collapses the Releeph request create and edit controllers into one class, as well as breaking everyone's subject-based mail rules by standardising them (but which should be more easily filtered by looking at headers.)

Test Plan:
* Make requests, then pick them.
* Pick and revert the same request so that discovery happens way after `arc` has told Releeph about what's been happening.
* Try to pick something that fails to pick in a project with pick instructions (and see the instructions are in the email.)
* Load all of FB's Releeph data into my DB and run the `storage upgrade` script.
* Request a commit via the "action" in a Differential revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: epriestley, aran, Korvin, wez

Maniphest Tasks: T3092, T2720

Differential Revision: https://secure.phabricator.com/D5868
2013-05-11 15:20:09 +01:00
Bryan Cuccioli
3c1f402da3 Add ability to name saved queries.
Summary: Can name saved queries.

Test Plan: Try naming some saved queries using the form.

Reviewers: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D5878

Conflicts:

	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-05-10 16:45:45 -07:00
Edward Speyer
58f0f37873 Add mail keys to ReleephRequests
Summary: Adding mail-keys; required for `PhabricatorApplicationTransaction` support.

Test Plan: Upgrade an old database with this patch, observe the matrix: {F42620}

Reviewers: wez, epriestley

Reviewed By: epriestley

CC: epriestley, aran

Maniphest Tasks: T2720

Differential Revision: https://secure.phabricator.com/D5852
2013-05-08 10:38:07 +01:00
Edward Speyer
5b2fc6a184 Simplify ReleephRequest schema
Summary:
Removing a bunch of cache-style columns from `ReleephRequest`, where it's actually much easier to just load the information at runtime.

This makes sense for migrating to `PhabricatorApplicationTransactions`, where each xaction changes one aspect of a `ReleephRequest` at a time (rather than multiple columns at once.)

Test Plan: Request something, run `arc releeph` and watch the picks, pass on some RQs, run `arc releeph` and watch the reverts.

Reviewers: wez, epriestley

Reviewed By: epriestley

CC: epriestley, aran

Maniphest Tasks: T2720

Differential Revision: https://secure.phabricator.com/D5851
2013-05-08 10:27:20 +01:00
Lauri-Henrik Jalonen
271d6605a9 Countdown revamp
Summary:
countdown_timer table named to countdown.
datepoint and related stuff renamed to epoch.
Countdowns now have phids.
Various UI items changed from timer to countdown.

Test Plan: Did run storage upgrade and added some countdowns.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2624

Differential Revision: https://secure.phabricator.com/D5812
2013-05-03 15:53:36 -07:00
Bryan Cuccioli
7ad2eae47f Implement saving queries.
Summary: Enable saved query objects to actually be saved to the database.

Test Plan: Insert a call to save() and check that the query is written correctly.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D5775

Conflicts:

	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-04-30 10:48:16 -07:00
Bob Trahan
11cb2f4f6c Conpherence - paginate thread list
Summary: this is D5750 but just the conpherence part. fixes a few random conpherence bugs / quirks as well. Also messes with ApplicationTransactionEditor to expose the xactions so Conpherence doesn't over-update participation rows. Fixes T2429.

Test Plan: set LIMIT to 3. verified I could scroll down all conpherences. next, picked a conpherence "in the middle" to load. verified I could page up and down. next, picked a conpherence in the middle then had another user update that conpherence. verified as I paged up the conpherence re-loaded properly selected

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin, vrana

Maniphest Tasks: T2429

Differential Revision: https://secure.phabricator.com/D5783
2013-04-26 10:30:41 -07:00
epriestley
7a5f622820 General cleanup for adding payment methods in Phortune
Summary:
This has no real behavioral changes (except better error handling), it just factors things out to be a bit cleaner. In particular:

  - Move more shared form behaviors into the common JS form component.
  - Move more error handling into shared pathways.
  - Make the specialized Stripe / Balanced methods do less work.

This needs some more polish for nontrival errors (especially on the Balanced side) but none of the error behavior is worse than it was and a lot of it is much better.

Ref T2787.

Test Plan: Hit all invalid form errors, added valid payment methods with Stripe and Balacned.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5771
2013-04-25 09:49:32 -07:00
Afaque Hussain
3f7ae27b58 Sql Patch to update the user_externalaccount table.
Summary: Sql Patch to rename the externalaccount table to user_externalaccount and to add dateCreated, dateModified fields to the updated table.

Test Plan: {F41442}

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5770
2013-04-24 15:14:26 -07:00
Afaque Hussain
dc6cfe6e01 Created PhabricatorExternalAccount Class and Sql patch to create an external_account table.
Summary: Created PhabricatorExternalAccount class with only data members. Will discuss with you regarding the necessary functions to be implemented in this class. Sql Patch to create a new table for external_accounts. Will I have to write unit tests the new storage object? Sending you this diff so that you can comment on this to further improve :).

Test Plan: {F40977}

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T1536, T1205

Differential Revision: https://secure.phabricator.com/D5724
2013-04-19 11:40:24 -07:00
Jakub Vrana
e31e998f3b Convert differential.revisionPHID commit detail to edge
Summary: Migration doesn't delete differential.revisionPHID but maybe it should?

Test Plan: Reparsed commit, ran the migration, deleted differential.revisionPHID, looked at task with attached commit with attached revision.

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5634
2013-04-12 22:48:16 -07:00
Bob Trahan
754705df4e Conpherence - get back-end prepped for loading less transactions all the time
Summary: this just does the back-end migration. I realized that we don't need to keep track of cacheTitle and cachePhoto since those are based off recent participation handles and dynamic relative to who is viewing it. Also kept the "last seen phid" as I think that will be useful to have auto-scroll to where you last read.  Ref T2867.

Test Plan: did the migration. observed sensical values in the database. created a new conpherence - again sensical values. updated a conpherence - more sensical values.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2867

Differential Revision: https://secure.phabricator.com/D5567
2013-04-04 16:57:38 -07:00
Bryan Cuccioli
2334f63c2c Paginate token leader board.
Summary: Add pagination to leader board. Add key on token count in db.

Test Plan: Set page size to 1 and give tokens to two tasks.

Reviewers: epriestley, AnhNhan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5513
2013-04-01 08:16:01 -07:00
James Rhodes
dd3b3bdd5e Added initial storage structure for Phrequent.
Summary:
Added the initial storage structure (DB tables and DAO classes)
for Phrequent.

Test Plan:
Apply the patch and run `bin/storage upgrade`.  It should
complete successfully.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2857

Differential Revision: https://secure.phabricator.com/D5476
2013-03-29 21:30:15 -07:00
epriestley
abdaf90239 Phortune v0.1: products
Summary:
Ref T2787. A product is the abstract representation of something you can buy or rent/subscribe to. Although the interface isn't locked down yet, this would ultimately be internal/administrative.

Products likely have some user-facing skin on top of them: plans would have a purchasing/comparison flow, physical goods would have a storefront, etc., so products don't have any information like descriptions or images, just the data that Phortune needs to correctly bill accounts.

Generally, this is very basic for the moment.

Test Plan:
{F37594}
{F37595}
{F37596}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5441
2013-03-28 09:13:07 -07:00
epriestley
4f3b5f0ea9 Phortune v0.1: add payment methods
Summary:
Hook @btrahan's Stripe form to the rest of Phortune.

  - Users can add payment methods.
  - They are saved to Stripe and associated with PhortunePaymentMethods on our side.
  - Payment methods appear on account overview.

Test Plan:
{F37548}
{F37549}
{F37550}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5438
2013-03-28 09:11:42 -07:00
epriestley
960ac3b2a6 Phortune v0
Summary:
Ref T2787. This does very little so far, but makes inroads on accounts and billing. This is mostly just modeled on what Stripe looks like. The objects are:

  - **Account**: Has one or more authorized users, who can make manage the account. An example might be "Phacility", and the three of us would be able to manage it. A user may be associated with more than one account (e.g., a corporate account and a personal account) but the UI tries to simplify the common case of a single account.
  - **Payment Method**: Something we can get sweet sweet money from; for now, a credit card registered with Stripe. Payment methods are associated with an account.
  - **Product**: A good (one time charge) or service (recurring charge). This might be "t-shirt" or "enterprise plan" or "hourly support" or whatever else.
  - **Purchase**: Represents a user purchasing a Product for an Account, using a Payment Method. e.g., you bought a shirt, or started a plan, or purchased support.
  - **Charge**: Actual charges against payment methods. A Purchase can create more than one charge if it's a plan, or if the first charge fails and we re-bill.

This doesn't fully account for stuff like coupons/discounts yet but they should fit into the model without any issues.

This only implements `Account`, and that only partially.

Test Plan: {F37531}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2787

Differential Revision: https://secure.phabricator.com/D5435
2013-03-28 09:10:34 -07:00
epriestley
696498934c Support edge transactions in ApplicationTransactions
Summary: Fixes T2655. Adds generic support for edge edits (e.g., membership or attached objects).

Test Plan: See next diff.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2655

Differential Revision: https://secure.phabricator.com/D5434
2013-03-28 08:34:34 -07:00
Bob Trahan
23dc686045 Conpherence - add per thread notification setting
Summary: Introduces a new settings panel for Conpherence specific settings.

Test Plan:
started a thread with a test user, thus two participants total. Replied to conpherence, toggling notification settings in between. Verified 1 or 2 emails were sent as appropos to the current toggle.

Toggled global setting and verified setting was updated in conpherences where nothing was specified. Verified setting conpherence setting overrides global setting.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2521

Differential Revision: https://secure.phabricator.com/D5391
2013-03-26 13:30:35 -07:00
Anh Nhan Nguyen
489f9e7dfe Added subscriptions to Phriction documents
Summary:
Fixes T2694

added edge infrastructure for Phriction

added mail subject prefix option for Phriction

added messy mail support for subscribers

adds edges to the phriction db, along with the subscriber interface
which gives us subscriptions for free.

simple display of subscribers, adequate to the current design and
sufficient fallbacks for exceptional cases. @chad may
be mailed about that one more UI element may be added to his redesign

mail support is messy. not generic at all. only sends to subscribed non-authors.

Test Plan:
tried out all kinds of stuff. applied patch, subscribed, unsubscribed with multiple
accs. verified proper

edited documents, verified that mail was sent in MetaMTA. Verified
contents, tos and stuff by looking into the db, comparing PHIDs etc.

functional testing per serious MTA (that is, AWS SES) worked wonderfully.

Here's how the subscription list looks like:
{F36320, layout=link}

Reviewers: epriestley, chad, btrahan

Reviewed By: epriestley

CC: hfcorriez, aran, Korvin

Maniphest Tasks: T2686, T2694

Differential Revision: https://secure.phabricator.com/D5372

Conflicts:

	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-03-21 10:28:22 -07:00
epriestley
9a515171f4 Phlux v0
Summary:
Yup, it's sitevars.

No conduit or caching or fancy stuff yet.

Ref T2793.

Test Plan:
{F36525}
{F36526}
{F36527}
{F36528}
{F36529}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2793

Differential Revision: https://secure.phabricator.com/D5397
2013-03-20 18:01:52 -07:00
Afaque Hussain
77a8765824 Schema patch to add a column to Phabricator_File.file table
Summary: Added a column called explicit_upload to Phabricator_File.file table

Test Plan: By chekcing locally if the the column has been added in table using mysql commands.

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5384
2013-03-19 15:00:44 -07:00
Edward Speyer
2497e5b5ed Releeph (Phabricator part)
Summary: A copy of the Releeph release tool.

Test Plan: Generally, click everything at least once.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2094

Differential Revision: https://secure.phabricator.com/D4932
2013-03-15 11:28:43 +00:00
vrana
1091dc7aa1 Save blame info to lint messages
Test Plan:
Applied the patch.
Looked at blame and plain blame of SVN and Git file.
Ran the lint saver.
Looked at lint messages list.
/diffusion/lint/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5218
2013-03-06 16:19:01 -08:00
James Rhodes
cd99850cb8 Increase Maniphest auxiliary storage size.
Test Plan:
Store large amounts of text into a string auxiliary field.  It should
be stored successfully rather than truncated.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2575

Differential Revision: https://secure.phabricator.com/D5246
2013-03-06 10:46:56 -08:00
epriestley
fe78944c9d Prepare Diffusion for hovercards
Summary:
Move Diffusion to be hovercard-ready, and expand our ability to resolve commit references.

  - Link unqualified hashes of 7 characters or more which match a commit.
  - Link qualified hashes of 5 characters or more which match a commit.
  - Support `{...}` syntax.

Test Plan: {F33896}

Reviewers: chad, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5121
2013-02-27 08:04:54 -08:00
epriestley
5de7774412 Un-breaking the update_phabricator.sh script for those that have data in their chatlog_event table
Summary:
The last commit broke update_phabricator.sh for me when it tried to migrate channel names into its own table.

It's a fairly straight forward patch and I'm almost certain I fixed it correctly. :)

Test Plan: Made small changes, ran update_phabricator.sh, repeat until the errors went away.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5116
2013-02-26 05:23:31 -08:00
epriestley
0f22854823 Minor, fix this patch so it applies after channel got deleted from the event table.
Test plan: Applied the patch explicitly with `--apply`.
2013-02-25 22:20:40 -08:00
Afaque Hussain
dd076a813f SQL patch to drop channel field.
Summary: Deleted the channel field and added a sql patch to drop the channel field.

Test Plan: I have messed up my local mysql:P, hence by storage upgrade is failing. Anyways, The chatlog_event table shouldn't contain the channel column now.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5011
2013-02-22 07:29:40 -08:00
vrana
8fcf4d5ac3 Use relatives in commit summary migration
Test Plan: Ran it on test DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5061
2013-02-21 16:26:14 -08:00
epriestley
b32bfb6541 Render commit summaries when rendering handles
Summary:
Fixes T2563. Instead of rendering "rPnnnnnn", render "rPnnnnnn: add feature X". Tweak Audit tables to accommodate.

@vrana / @nh, this migration might take a while. You could safely skip it when deploying and then run it after deployment.

I think I fixed all the other places where these render, but might have missed something.

Test Plan:
  - Ran first schema migration, clicked around to make sure nothing broke.
  - Ran `scripts/repository/reparse.php --message rXyyyyy`, verified summary populated.
  - Ran second migration.
  - Checked task/diffusion/audit/differential for weird rendering.

Reviewers: vrana

Reviewed By: vrana

CC: nh, aran, chrisbolt, allixsenos

Maniphest Tasks: T2563

Differential Revision: https://secure.phabricator.com/D5012
2013-02-21 15:09:35 -08:00
kwadwo
762ace810d Allow files to TTL and and be garbage collected
Summary: Added ttl field to files. Gabage collect files with expired ttl

Test Plan: created file with a ttl. Let garbage collector run

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4987
2013-02-20 13:38:36 -08:00
epriestley
8d79c7282d Accommodate long daemon command lines
Summary: Fixes T2559 with an incredibly original patch which I came up with myself.

Test Plan:
  $ ./bin/storage upgrade -f
  Applying patch 'phabricator:20130218.longdaemon.sql'...
  Storage is up to date. Use 'storage status' for details.

Reviewers: edward

Reviewed By: edward

CC: aran

Maniphest Tasks: T2559

Differential Revision: https://secure.phabricator.com/D5006
2013-02-18 11:51:42 -08:00
Afaque Hussain
61c26463bc Php schema patch to update channel id's of past events.
Summary: Php schema patch to update channel id's of past events.

Test Plan: Having some proxy issues here due to which connection is timing out and bot is not able to log into IRC. Bot connects to IRC in my home though ! So I wasn't able to quite to test this by running storage upgrade.

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5000
2013-02-18 10:53:53 -08:00
epriestley
26aac16346 Garbage collect TTL'd cache entries from the general cache
Summary: We currently garbage collect general cache entries after a set period of time (30 days by default), but the recent changes to DarkConsole have left us writing a lot of large, short-TTL data to the cache. In addition to a maximum age, GC cache entires after they TTL out.

Test Plan: Ran GC daemon, saw TTL'd entries get collected. Inserted a TTL'd entry, saw it get collected by GC. Saw non-ttl'd entries not get collected.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4990
2013-02-17 09:13:49 -08:00
epriestley
49c40d209d Tokens v1
Summary:
Features!

  - Giving tokens.
  - Taking tokens back.
  - Not giving tokens.

Test Plan: See screenshots.

Reviewers: chad, vrana

Reviewed By: chad

CC: aran, btrahan

Maniphest Tasks: T2541

Differential Revision: https://secure.phabricator.com/D4964
2013-02-15 07:47:14 -08:00
Afaque Hussain
73991bb262 Added channel ID to events
Summary: Added a column channelID column to phabricator_chatlog.chatlog_event

Test Plan: Checked through mysql to see if table is updated

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4956
2013-02-14 12:27:18 -08:00
Afaque Hussain
9baada1571 Schema Patch to Add a New Table
Summary: Added 20130214.chatlogchannel.sql in resources/sql/patches to add a new table

Test Plan: Hmmmmm .......

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4952
2013-02-14 11:37:05 -08:00
vrana
138da5a279 Kill some PhutilSafeHTML
Summary: Also couple of unrelated Ponder changes.

Test Plan: /Q5

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4858
2013-02-07 18:01:01 -08:00
Bob Trahan
1cde41b994 Conpherence - add crop
Summary:
mainly, this adds the image cropper - yay!

 - also removes the file image from the handle stuff I added in V1. now we do all this crazy photo stuff.

Test Plan:
 - uploaded a photo by dragging to header and noted 120 x 80 showed up on reload
 - uploaded a photo by dragging to edit dialogue spot and noted 120 x 80 showed up on reload
 - cropped a photo - noted it cropped right
 - cropped a photo again and again and again - seems like it crops okay

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2418, T2399

Differential Revision: https://secure.phabricator.com/D4790
2013-02-06 14:03:52 -08:00
vrana
8c99938aad Convert revision unsubscribers to edges
Test Plan: Ran the migration on a single revision, verified DB, called `loadUnsubscribedPHIDs()`.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4786
2013-02-04 11:36:55 -08:00
Lauri-Henrik Jalonen
5cb8787d91 Removed psth column from herald transcript
Summary: Added patch file to remove psth column in herald transcript tabe

Test Plan: Verified that column was removed with ./bin/storage upgrade

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2425

Differential Revision: https://secure.phabricator.com/D4672
2013-01-26 15:22:18 -08:00
Bob Trahan
b873f3f991 Conpherence V1
Summary: it's ugly. but it works. basically. See T2399 for a roughly prioritized list of what still needs to happen.

Test Plan:
- created a conpherence with myself from my profile
- created a conpherence with myself from "new conpherence"
- created a conphernece with another from "new conpherence"
- created a conpherence with several others
- created a conpherence with files in the initial post
 - verified files via comment text ("{F232} is awesome!") and via traditional attach
- edited a conpherence image
 - verified it showed up in the header and in the conpherence menu on the left
- edited a conpherence title
 - verified it showed up in the header and in the conpherence menu on the right
- verified each widget showed up when clicked and displayed the proper data
 - calendar being an exception since it sucks so hard right now.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, epriestley, chad, codeblock, Korvin

Maniphest Tasks: T2301

Differential Revision: https://secure.phabricator.com/D4620
2013-01-24 17:23:05 -08:00
vrana
e10fdbe77e Use write connection and transactions in SQL patches
Summary:
Patches often read from slaves (possibly stale data) and use that information to write on master.
It causes problems when applying more patches quickly after each other because data created in previous patch may not be replicated yet.

Test Plan:
  $ bin/storage upgrade

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4483
2013-01-17 12:07:16 -08:00
vrana
09ad34c34b Update quickstart.sql
Summary: It's faster a bit than applying patches.

Test Plan:
  $ bin/storage --trace upgrade --namespace x

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4481
2013-01-17 11:00:56 -08:00
Mailson Menezes
712e22208c Store width and height metadata of image files
Summary: Also provide a way to update old files metadata.

Test Plan: Create a revision which includes a image file. Check whether the widht, height metadata exists. Run `scripts/files/manage_files.php metadata --all` to update previously uploaded files.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2101

Differential Revision: https://secure.phabricator.com/D4347
2013-01-07 09:46:43 -08:00
Bob Trahan
3448781c40 de-duplicate emails received by phabricator multiple times
Summary: this can happen if you have Phabricator and email lists co-mingling such that Phabricator receives an email multiple times. we can prevent this from then spamming everyone or otherwise taking the action multiple times by storing a message id hash and dropping the message if we have more than one message that matches.

Test Plan: simulated sending the same email multiple times on the command line. noted only the first one made it through.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1726

Differential Revision: https://secure.phabricator.com/D4328
2013-01-03 17:04:30 -08:00
epriestley
32e4a7a37f Use transactions to show edit history for Configuration
Summary: Use ApplicationTransactions in Config to create an edit history. Resolves T2256.

Test Plan: {F28477}

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2256

Differential Revision: https://secure.phabricator.com/D4314
2013-01-01 18:14:41 -08:00
Ricky Elrod
a774620042 Start of a config web interface.
Summary:
This is somewhat clowny, particularly in how it handles JSON encode/decode, but
I've commented why I did things the way I did. The goal is to store minified JSON
but show pretty-printed JSON where possible, to the user editing it.

Test Plan:
* Went to /config/ and saw a list of keys from the `default` config.
* Clicked on one of them, submitted the default value successfully.
* Changed the value to invalid JSON and got a decent error.
* Changed the value to valid JSON and checked the DB to confirm it saved.
* Confirmed the DB values were minified.
* Confirmed the user-facing values were pretty-printed where they could be.
* Confirmed that PHIDs were getting assigned properly and that isDeleted
  properly defaulted to false/0.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2246

Differential Revision: https://secure.phabricator.com/D4290
2012-12-27 15:21:21 -08:00
epriestley
aae5f9efd3 Implement a more compact, general database-backed key-value cache
Summary:
See discussion in D4204. Facebook currently has a 314MB remarkup cache with a 55MB index, which is slow to access. Under the theory that this is an index size/quality problem (the current index is on a potentially-384-byte field, with many keys sharing prefixes), provide a more general index with fancy new features:

  - It implements PhutilKeyValueCache, so it can be a component in cache stacks and supports TTL.
  - It has a 12-byte hash-based key.
  - It automatically compresses large blocks of data (most of what we store is highly-compressible HTML).

Test Plan:
  - Basics:
    - Loaded /paste/, saw caches generate and save.
    - Reloaded /paste/, saw the page hit cache.
  - GC:
    - Ran GC daemon, saw nothing.
    - Set maximum lifetime to 1 second, ran GC daemon, saw it collect the entire cache.
  - Deflate:
    - Selected row formats from the database, saw a mixture of 'raw' and 'deflate' storage.
    - Used profiler to verify that 'deflate' is fast (12 calls @ 220us on my paste list).
  - Ran unit tests

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4259
2012-12-21 14:17:56 -08:00
epriestley
ba7723d905 Modernize Macro application
Summary: Adds feed, email, notifications, comments, partial editing, subscriptions, enable/disable, flags and crumbs to Macro.

Test Plan:
{F26839}
{F26840}
{F26841}
{F26842}
{F26843}
{F26844}
{F26845}

Reviewers: vrana, btrahan, chad

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2157, T175, T2104

Differential Revision: https://secure.phabricator.com/D4141
2012-12-11 14:01:03 -08:00
epriestley
7b6fa0db12 Genericize transactions in Pholio
Summary:
Split Pholio's transaction implementation into generic and application-specific parts. Moves us toward generic transactions, with support for:

  - Editing and deleting comments.
  - Setting visibility of individual comments (I'm not a fan of this feature but we'll see).

I want to move everything to a more generic piece of infrastructure but there's very little they can share right now so adding transactions to, e.g., Paste or Macros (T2157) means massive amounts of similar code.

Tons of work left to do here, but I think it basically works. Here's a screenshot:

{F26820}

Test Plan: Made transactions in Pholio.

Reviewers: btrahan, vrana, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2104

Differential Revision: https://secure.phabricator.com/D4136
2012-12-11 13:59:20 -08:00
vrana
4f615ad2a9 Allow excluding paths from package
Summary: Resolves T2149.

Test Plan:
  $ bin/storage upgrade

# /owners/ - saw +
# /owners/package/1/ - saw +
# /owners/edit/1/ - added exclude paths, saw correct e-mail
# /rPabc123 - included paths are still highlighted and excluded not
# /owners/view/search/?path=/included/ - found
# /owners/view/search/?path=/excluded/ - not found
# owners.query - path: /included/
# owners.query - path: /excluded/
# new unit test

  PhabricatorOwnersPackage::loadAffectedPackages(
    $repository,
    array('/excluded/b.php'));

  PhabricatorOwnersPackage::loadAffectedPackages(
    $repository,
    array('/included/a.php'));

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2149

Differential Revision: https://secure.phabricator.com/D4102
2012-12-07 16:33:16 -08:00
epriestley
fc9ad37b26 Add very basic scaffolding for Pholio
Summary:
I'm not going to land this until it's a bit more fleshed out since it would just confuse users, but this is probably more reviewable as a few diffs adding a couple features than one ULTRA-diff adding everything. Implement application basics for Pholio. This does more or less nothing, but adds storage, subscribe, flag, markup, indexing, query basics, PHIDs, handle loads, a couple of realy really basic controllers, etc.
Basic hierarchy is:

  - **Moleskine**: Top-level object like a Differential Revision, like "Ponder Feed Ideas".
  - **Image**: Each Moleskine has one or more images, like the unexpanded / expanded / mobile / empty states of feed.
  - **Transaction**: Comment or edit, like Maniphest. I generally want to move most apps to a transaction model so we can log edits.
  - **PixelComment**: Equivalent of an inline comment.

Test Plan: Created a fake object and viewed it.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran, davidreuss

Maniphest Tasks: T2097

Differential Revision: https://secure.phabricator.com/D3817
2012-11-21 17:22:36 -08:00
epriestley
ee2e85a0bb Fix several migration issues with the Task/Counter patch
Summary:
People hit three issues with D3914:

  - As per T2059, we applied a schema change from a `.php` patch, which currently does not work if you use a different user to make schema changes than for normal use.
    - Since the change in question is idempotent, just move it to a `.sql` patch. We'll follow up in T2059 and fix it properly.
  - Rogue daemons at several installs used old code (expecting autoincrement) to insert into the new table (no autoincrement), thereby creating tasks with ID 0.
    - Rename the table so they'll fail.
    - This also makes the code a little more consistent.
  - Some installs now have tasks with ID 0.
    - Use checks against null rather than against 0 so we can process these tasks.

The major issues this fixes are the schema upgrade failure in T2059, and the infinite loops in T2072 and elsewhere.

This isn't really a fully statisfactory fix. I'll discuss some next steps in T2072.

Test Plan: Created new tasks via MetaMTA/Differential. Ran tasks with `phd debug taskmaster`. Inserted a task 0 and verified it ran and archived correctly.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2072, T2059

Differential Revision: https://secure.phabricator.com/D3973
2012-11-16 10:19:22 -08:00
vrana
23a046b3cd Allow saving lint errors to database
Summary: This saves lint errors to the path change of current commit. It requires pushed revision. It doesn't save difference from previous commit mentioned in T2038#comment-4 - I don't plan doing it after all, everything would be much more complicated and the amount of data saved with this approach isn't that bad.

Test Plan: Applied patch, ran script, verified DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2038

Differential Revision: https://secure.phabricator.com/D3899
2012-11-08 15:39:43 -08:00
vrana
6cfc15ad6c Make Lisk counters patch more robust
Summary:
We need to revert this patch and we will need to re-apply it later.
We can't drop the table and delete these rows as we need to run both versions for a temporary period.

Test Plan: Applied it.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3924
2012-11-08 11:36:50 -08:00
Bob Trahan
9966af50dd Delete PhabricatorRemarkupRuleProxyImage
Summary: don't need it now that uploading files is so easy. Plus it made for some buggy jonx if / when there were bad image links coupled with caching. In theory this is a lot less pretty though if folks linked to a bunch of files served elsewhere using images.

Test Plan: http://does-not-exist.com/imaginary.jpg rendered as a link!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2000

Differential Revision: https://secure.phabricator.com/D3908
2012-11-07 14:31:43 -08:00
epriestley
7332599e03 Provide an IDS_COUNTER mechanism for ID assignment
Summary: See D3912 for discussion. InnoDB may reuse autoincrement IDs after restart; provide a way to avoid it.

Test Plan: Unit tests. Scheduled and executed tasks through `drydock lease --type host` and `phd debug taskmaster`.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3914
2012-11-07 13:33:07 -08:00
vrana
c6503f019f Start Ponder questions from 11
Summary: We don't link Q1 - Q4.

Test Plan: Created the table, insterted row, verified that the id is 11.

Reviewers: pieter, epriestley

Reviewed By: pieter

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3891
2012-11-05 13:21:24 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
89b37f0357 Make various Drydock improvements
Summary:
Tightens up a bunch of stuff:

  - In `drydock lease`, pull and print logs so the user can see what's happening.
  - Remove `DrydockAllocator`, which was a dumb class that did nothing. Move the tiny amount of logic it held directly to `DrydockLease`.
  - Move `resourceType` from worker task metadata directly to `DrydockLease`. Other things (like the web UI) can be more informative with this information available.
  - Pass leases to `allocateResource()`. We always allocate in response to a lease activation request, and the lease often has vital information. This also allows us to associate logs with leases correctly.

Test Plan: Ran `drydock lease --type host` and saw it perform a host allocation in EC2.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3870
2012-11-01 16:53:17 -07:00
epriestley
f0fdcf1a51 Undumb the Drydock resource allocator pipeline
Summary:
This was the major goal of D3859/D3855, and to a lesser degree D3854/D3852.

As Drydock is allocating a resource, it may need to allocate other resources first. For example, if it's allocating a working copy, it may need to allocate a host first.

Currently, we have the process basically queue up the allocation (insert a task into the queue) and sleep() until it finishes. This is problematic for a bunch of reasons, but the major one is that if allocation takes more resources (host, port, machine, DNS) than you have daemons, they could all end up sleeping and waiting for some other daemon to do their work. This is really stupid. Even if you only take up some of them, you're spending slots sleeping when you could be doing useful work.

To partially get around this and make the CLI experience less dumb, there's this goofy `synchronous` flag that gets passed around everywhere and pushes the workflow through a pile of special cases. Basically the `synchronous` flag causes us to do everything in-process. But this is dumb too because we'd rather do things in parallel if we can, and we have to have a lot of special case code to make it work at all.

Get rid of all of this. Instead of sleep()ing, try to work on the tasks that need to be worked on. If another daemon grabbed them already that's fine, but in the worst case we just gracefully degrade and do everything in process. So we get the best of both worlds: if we have parallelizable tasks and free daemons, things will execute in parallel. If we have nonparallelizable tasks or no free daemons, things will execute in process.

Test Plan: Ran `drydock_control.php --trace` and saw it perform cascading allocations without sleeping or special casing.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3861
2012-11-01 11:30:42 -07:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
Jakub Vrana
01de3dff81 Allow using StorageFixtureScopeGuard on Windows 2012-10-24 13:59:22 -07:00
Bob Trahan
60466d3bcc Create a status tool by giving /calendar/ some teeth
Summary: you can now add, edit, and delete status events. also added a "description" to status events and surface it in the big calendar view on mouse hover. some refactoring changes as well to make validation logic centralized within the storage class.

Test Plan: added, edited, deleted. yay.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T407

Differential Revision: https://secure.phabricator.com/D3810
2012-10-24 13:22:24 -07:00
epriestley
a50b8e39b1 Make posts 1:1 with blogs and implement policy controls
Summary:
This leaves the UI in a pretty rough state, but implements blog policy controls and queries, and 1:1 relationships between posts and blogs. Needs a bunch more cleanup but seemed like an okayish breaking point in terms of cohesiveness.

Posts have these rules:

  - Drafts are visible only to the author.
  - Published posts are visible to anyone who can see the blog they appear on.
  - Posts are only editable by the author.

...so we don't need any special policy UI or state to accommodate these rules.

Posts may have no blog if they're grandfathered in or you write a post to a blog and then lose the ability to see the blog. This is the messiest edge case -- specifically:

  - You write a post to blog A.
  - You publish the post.
  - I edit the "Visible To:" for blog A and set it to exclude you.

What we do in this case is let you see the post in "My Posts", but you can no longer see the blog and you'll see the post as not being part of a blog. We can maybe give you some UI to let you move it later or something.

Test Plan: Hit all (I think?) of the interfaces without issues. Definitely some UI problems still right now.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3694
2012-10-15 14:50:04 -07:00
epriestley
dbcf2e44e8 Make PhameBlogs respect policies
Summary:
Adds "can view" and "can edit" policies to blogs. Replaces "bloggers" with "can join".

This doesn't fully remove "bloggers" because I didn't want this to get too crazy/huge.

Test Plan: Created, edited, deleted blogs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3693
2012-10-15 14:49:52 -07:00
epriestley
2532cb9613 Add mail keys to Ponder questions
Summary:
We need to go slightly farther to stub reply handler functionality for Ponder in at least some configurations, where we rely on the presence of a unique random key to generate per-object or per-object+user reply addresses.

This should probably be formalized in an interface since it's currently pretty ad-hoc.

Test Plan:
  - Made comments in Ponder under a per-user email configuration.
  - Ran migration, verified mail keys were generated.
  - Ran migration again (with --apply), verified existing questions were skipped.
  - Created a new question, verified mail key generation.

Reviewers: pieter

Reviewed By: pieter

CC: aran

Maniphest Tasks: T1873

Differential Revision: https://secure.phabricator.com/D3665
2012-10-08 20:14:58 -07:00
vrana
cbde56cdce Properly create xhpast database
Summary:
It isn't deleted by `storage destroy`.

This should be a no-op on current storage because we execute `CREATE DATABASE IF NOT EXISTS`.

Test Plan:
  $ bin/storage destroy --dryrun

Reviewers: nh, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3659
2012-10-08 16:09:20 -07:00
Bob Trahan
9e1b643896 Phame - allow blogs to specify custom URIs
Summary: this then enables people to create blog.theircompany.com. And for us, blog.phacility.com...!

Test Plan:
 - created custom URIs of various goodness and verified the error messages were sensical.
 - verified if "false" in configuration then custom uri stuff disappears

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3542
2012-09-30 17:10:27 -07:00
vrana
d119ac672f Remember action in Differential comment draft
Summary:
It happens to me quite often that I leave the window with revision (by closing it or by visiting a link from it).
When I return then the comment draft is there so I clowncopterize it but forget that I wanted to take some other action than Comment.

Test Plan: Selected "Add Reviewers", added some reviewers, closed the window, opened it - the action and reviewers were still there.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3530
2012-09-21 13:05:09 -07:00
Nick Harper
5978bbfc64 Do sampled profiling of requests
Summary:
People have occasionally complained about phabricator being slow. We have
the access log to look at to see when slowness happens, but it doesn't tell
us much about why it happened. Since it's usually a sporadic issue that's
reported, it's hard to reproduce and then profile. This change will allow us
to collect sampled profiles so we can look at them when slowness occurs.

Test Plan:
checking that sampling works correctly:
- set rate to 0; do several page loads; check no new entries in table
- set rate to 1; check that there's a new row in the table for each page load
- set rate to 10; check that some requests write to table and some don't
check new ui for samples:
- load /xhprof/list/all/, see a list with a lot of samples
- load /xhprof/list/sampled/, see only sampled runs
- load /xhprof/list/manual/, see only non-sampled runs
- load /xhprof/list/my-runs/, se only my manual runs

Reviewers: vrana, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3458
2012-09-17 10:53:45 -07:00
epriestley
b39175342d Add paste policy storage
Summary: Add storage to Pastes for view policies.

Test Plan: Set policies on pastes, see next diff.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3474
2012-09-13 10:11:14 -07:00
Pieter Hooimeijer
5883b4f50c adding comments to ponder
Summary: This is pretty spartan, but it does the job.

Test Plan:
Patch, update storage, add some comment
to your favorite question or answer.

Reviewers: nh, vrana, epriestley

Reviewed By: epriestley

CC: aran, Korvin, starruler, syrneus, me.here, victorzarate7

Maniphest Tasks: T1645

Differential Revision: https://secure.phabricator.com/D3471
2012-09-11 12:13:20 -07:00
vrana
f770900983 Save edge type as number
Summary: We use numbers here and I see no reason for strings.

Test Plan:
  $ bin/storage upgrade

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3303
2012-08-16 14:43:03 -07:00
Nick Harper
3908f7db2e Show list of non-exited daemons
Summary: This is arguably a more useful view than listing all daemons.

Test Plan: Looked at list, only saw daemons that haven't exited

Reviewers: vrana, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3286
2012-08-14 18:01:15 -07:00
epriestley
bd0be1c650 Add View, Edit and Join policies to PhabricatorProject
Summary:
  - In ProjectQuery, always load the viewer's membership in the project because we need it to perform a CAN_VIEW test.
  - Add storage for the view, edit and join policies.
  - A user can always view a project if they are a member.
  - A user can always join a project if they can edit it.
  - Editing a project requires both "view" and "edit" permissions, and edit does not imply view.
  - This has no effect on the application yet.

Test Plan: See next diff.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3219
2012-08-11 07:05:01 -07:00
Pieter Hooimeijer
64472dd7b8 Adding Ponder-related files.
Summary:
Ponder is similar in spirit to the Wiki tool, but uses a Q&A
format and up/downvotes to signal user sentiment. Popular
questions are moved to the top of the feed on a 5-minute
cycle based on age (younger is better) and vote count (higher
is better).

Pre-apologies for noob diff.

Test Plan:
- `./bin/phd list` Should include `PonderHeatDaemon`; phd launch it
  if necessary.

- Navigate to /ponder/ ; observe sanity when adding questions,
  voting on them, and adding answers.

- Confirm that questions and answers are linkable using Q5 / Q5#A5 formatted object links.

- Confirm that searching for Ponder Questions works using built-in
  search.

Feedback on code / schema / whatever organization very welcome.

Reviewers: nh, vrana, epriestley

Reviewed By: epriestley

CC: gmarcotte, aran, Korvin, starruler

Differential Revision: https://secure.phabricator.com/D3136
2012-08-10 10:44:04 -07:00
epriestley
d32926e5f7 Work-in-progress schema for Facts app
Summary: See discussion in D3078 for why I've separated this. Pretty sure it's not quite ready yet -- I want to build a couple of things on it so we have a better idea of what we need (autoincrement ID? <factType, objectA, epoch> primary key? objectB column? valueZ?) and don't need to do a ton of schema patches.

Test Plan: Applied patches, ran D3078.

Reviewers: vrana, btrahan, majak

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1581, T1562

Differential Revision: https://secure.phabricator.com/D3088
2012-08-09 08:40:56 -07:00
epriestley
f9fcaa1f84 Migrate project membership to edges
Summary:
  - Store project members in edges.
  - Migrate existing members to edge storage.
  - Delete PhabricatorProjectAffiliation.
  - I left the actual underlying data around just in case something goes wrong; we can delete it evenutally.

Test Plan:
  - Ran migration.
  - Created a new project.
  - Joined and left a project.
  - Added and removed project members.
  - Manually called PhabricatorOwnersOwner::loadAffiliatedUserPHIDs() to verify its behavior.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3186
2012-08-07 18:02:05 -07:00
Alan Huang
bcb9de4ea1 Add a context field to symbol objects
Summary:
See T1602.

This is just the minimal functional patch; the scripts will continue
working because of the `DEFAULT ''`.

Test Plan:
Can't fully test this until I get more code working, but
nothing broke horribly yet.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3147
2012-08-06 12:20:45 -07:00
Nick Harper
88caa45854 Save daemon state to database
Summary:
To make it easier to monitor daemons, let's store their current state
(running, died, exited, or unknown) to the db. The purpose of this is to
provide more information on the daemon console about the status of daemons,
especially when they are running on multiple machines. This is mostly backend
work, with only a few frontend changes. (It is also dependent on a change
to libphutil.)

These changes will make dead or stuck daemons more obvious, and will allow
more work on the frontend to hide daemons (and logs) that have exited cleanly,
i.e. ones we don't care about any more.

Test Plan:
- run db migration, check in db that all daemons were marked as exited
- start up a daemon, check in db that it is marked as running
- open web interface, check that daemon is listed as running
- after daemon has been running for a little bit, check in db that dateModified
  is being updated (indicating daemon is properly sending heartbeat)
- kill -9 daemon (but don't run bin/phd yet), and check that db still shows it
  as running
- edit daemon db entry to show it as being on a different host, and backdate
  dateModified field by 3 minutes, and check the web ui to show that the status
  is unknown.
- change db entry to have proper host, check in web ui that daemon status is
  displayed as dead. Check db to see that the status was saved.
- run bin/phd stop, and see that the formerly dead daemon is now exited.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3126
2012-08-01 17:06:04 -07:00
epriestley
9be12551a9 Move Task <=> Revision storage to Edges
Summary:
  - Add edges for this relationship.
  - Use edges to store this data.
  - Migrate old data.
  - Fix some warnings with generating feed stories about Aux and Edge transactions.
  - Fix a task-task edge issue with "Create Subtask".

Test Plan:
  - Migrated data, verified reivsions showed up.
  - Attached and detached tasks to revisions and vice versa.
  - Created a new revision with attached tasks.
  - Created a subtask.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3018
2012-07-20 08:59:39 -07:00
Bob Trahan
ae13d33859 Phame - introduce blogs
Summary:
blogs are collections of posts. a blog also has metadata like a name, description and "bloggers" that can edit the metadata of the blog and contribute posts.

changes include the post edit flow where bloggers can now select which blogs to publish to. also made various small tweaks throughout the UI to make things sensical and clean as the concept of blogs is introduced.

there's edges powering this stuff.  bloggers <=> blogs and posts <=> blogs in particular.

Test Plan:
made blogs, deleted blogs, tried to make blogs with no bloggers. all went well.
verified ui to publish only showed up for public posts, published posts to blogs, un-published posts to blogs, re-published posts to blogs, deleted posts and verified they disappeared from blogs.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3003
2012-07-19 09:03:10 -07:00
epriestley
ee709a0543 Use Edges to store dependencies between revisions in Differential
Summary: See D3006. Move this data to the edge store.

Test Plan:
  - Created dependencies, migrated, verified dependencies were preserved.
  - Added new dependencies, they worked.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3007
2012-07-18 20:42:06 -07:00
epriestley
9196a6bd9f Use Edges to store dependencies between tasks in Maniphest
Summary:
  - Use edges to store "X depends on Y" information in Maniphest.
  - Show both "Depends On" and "Dependent Tasks".
  - Migrate all the old edges.

Test Plan:
  - Added some relationships, migrated, verified they were preserved.
  - Added some new valid relationships, verified tasks got updated with sensible transactions and sent reasonable emails.
  - Tried to add a cycle, got an ugly but effective error.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3006
2012-07-18 20:41:42 -07:00
epriestley
5d8b75b4da Use the unified markup cache for Maniphest
Summary:
  - See D2945.
  - Drop `cache` field from ManiphestTransaction.
  - Render task descriptions and transactions through PhabricatorMarkupEngine.
  - Also pull the list of macros more lazily.

Test Plan:
  - Verified transactions and transaction preview work correctly and interact with cache correctly.
  - Verified tasks descriptions and task preview work correctly.
  - Verified we don't hit the imagemacro table when we're rendering everything from cache anymore.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2946
2012-07-11 11:40:10 -07:00
epriestley
2b0b9a1573 Add a generic multistep Markup cache
Summary:
The immediate issue this addresses is T1366, adding a rendering cache to Phriction. For wiki pages with code blocks especially, rerendering them each time is expensive.

The broader issue is that out markup caches aren't very good right now. They have three major problems:

**Problem 1: the data is stored in the wrong place.** We currently store remarkup caches on objects. This means we're always loading it and passing it around even when we don't need it, can't genericize cache management code (e.g., have one simple script to drop/GC caches), need to update authoritative rows to clear caches, and can't genericize rendering code since each object is different.

To solve this, I created a dedicated cache database that I plan to move all markup caches to use.

**Problem 2: time-variant rules break when cached.** Some rules like `**bold**` are time-invariant and always produce the same output, but some rules like `{Tnnn}` and `@username` are variant and may render differently (because a task was closed or a user is on vacation). Currently, we cache the raw output, so these time-variant rules get locked at whatever values they had when they were first rendered. This is the main reason Phriction doesn't have a cache right now -- I wanted `{Tnnn}` rules to reflect open/closed tasks.

To solve this, I split markup into a "preprocessing" phase (which does all the parsing and evaluates all time-invariant rules) and a "postprocessing" phase (which evaluates time-variant rules only). The preprocessing phase is most of the expense (and, notably, includes syntax highlighting) so this is nearly as good as caching the final output. I did most of the work here in D737 / D738, but we never moved to use it in Phabricator -- we currently just do the two operations serially in all cases.

This diff splits them apart and caches the output of preprocessing only, so we benefit from caching but also get accurate time-variant rendering.

**Problem 3: cache access isn't batched/pipelined optimally.** When we're rendering a list of markup blocks, we should be able to batch datafetching better than we do. D738 helped with this (fetching is batched within a single hunk of markup) and this improves batching on cache access. We could still do better here, but this is at least a step forward.

Also fixes a bug with generating a link in the Phriction history interface ($uri gets clobbered).

I'm using PHP serialization instead of JSON serialization because Remarkup does some stuff with non-ascii characters that might not survive JSON.

Test Plan:
  - Created a Phriction document and verified that previews don't go to cache (no rows appear in the cache table).
  - Verified that published documents come out of cache.
  - Verified that caches generate/regenerate correctly, time-variant rules render properly and old documents hit the right caches.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1366

Differential Revision: https://secure.phabricator.com/D2945
2012-07-09 15:20:56 -07:00
epriestley
7cf6313be9 Add a generic object for unit tests
Summary:
A later diff adds unit tests against edges, but we need real objects to connect with edges. Add some trivial objects to the Harbormaster database to compliment the similar HarbormasterScratchTable.
On its own, this does nothing interesting.

Test Plan: Built unit tests on this in a followup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2937
2012-07-09 10:39:14 -07:00
dschleimer
86fa4fd97f [Phabricator] track Mercurial bookmarks for differential diffs
Summary:
This adds all the changes necessary to track the active Mercurial
bookmark for differential diffs.  We render both branch and bookmark
information in the branch field of the Differential revison view, as
seen in
https://secure.phabricator.com/file/data/kzpmu3evfkukxdjyxrfz/PHID-FILE-eqorsqupxvwirqi2s5lo/bookmark_differential.jpg

The Arcanist half of this is https://secure.phabricator.com/D2896

Test Plan:
Mostly D2896.

Additionally, loaded a diff created with a bookmark, as per the link in the summary.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1331

Differential Revision: https://secure.phabricator.com/D2897
2012-06-30 15:41:58 -07:00
vrana
48ebcf0679 Allow user override translation and implement PhutilPerson
Test Plan:
Altered database.
Wrote a custom translation and selected it in preferences.
Verified that the text is custom translated.
Set language back to default.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D2757
2012-06-14 18:33:00 -07:00
vrana
892a2d1b61 Make Thread-Topic human readable
Summary:
Some e-mail clients display this header and it needs to be constant.

This is somehow involved but I doubt that there is a simpler solution.

Test Plan:
Applied SQL patch.
Commented on revision, commented on commit, changed package.
Verified that the `Thread-Topic` has constant and human readable value.

Reviewers: epriestley

Reviewed By: epriestley

CC: ola, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2745
2012-06-14 11:36:34 -07:00
Espen Volden
726041584f Made it possible to login using LDAP
Summary: Made it possible to link and unlink LDAP accounts with  Phabricator accounts.

Test Plan:
I've tested this code locally and in production where I work.
I've tried creating an account from scratch by logging in with LDAP and linking and unlinking an LDAP account with an existing account. I've tried to associate the same LDAP account with different Phabricator accounts and it failed as expected.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, auduny, svemir

Maniphest Tasks: T742

Differential Revision: https://secure.phabricator.com/D2722
2012-06-13 08:58:46 -07:00
Keebuhm Park
207f101aee SQL patch for notification
Summary: Added `PhabricatorBuiltinPatchList` entry so that "storage upgrade" will update the database. Renamed and numbered the notification.sql patch.

Test Plan: Drop phabricator_feed.feed_storynotification table if it exists and run bin/storage upgrade to check if the patch is correctly applied.

Reviewers: epriestley, btrahan, allenjohnashton

Reviewed By: epriestley

CC: ddfisher, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2687
2012-06-08 12:42:59 -07:00
John-Ashton Allen
3a6ee79190 Adds base notification application
Summary: First diff in a series of diffs to add notifications to Phabricator. This is the notification application ONLY. This commit does not include the changes to other applications that makes them add notifications. As such, no notifications will be generated beyond the initial database import.

Test Plan: This is part of the notifications architecture which has been running on http://theoryphabricator.com for the past several months.

Reviewers: epriestley, btrahan, ddfisher

Reviewed By: epriestley

CC: allenjohnashton, keebuhm, aran, Korvin, jungejason, nh

Maniphest Tasks: T974

Differential Revision: https://secure.phabricator.com/D2571
2012-06-08 06:32:02 -07:00
epriestley
f6fbe40bd5 Minor, completely remove references to PHID from schema patches so upgrade-from-scratch works. 2012-05-24 13:59:12 -07:00
epriestley
a89cef8e39 Remove PHID database, add Harbormaster database
Summary:
  - We currently write every PHID we generate to a table. This was motivated by two concerns:
    - **Understanding Data**: At Facebook, the data was sometimes kind of a mess. You could look at a random user in the ID tool and see 9000 assocs with random binary data attached to them, pointing at a zillion other objects with no idea how any of it got there. I originally created this table to have a canonical source of truth about PHID basics, at least. In practice, our data model has been really tidy and consistent, and we don't use any of the auxiliary data in this table (or even write it). The handle abstraction is powerful and covers essentially all of the useful data in the app, and we have human-readable types in the keys. So I don't think we have a real need here, and this table isn't serving it if we do.
    - **Uniqueness**: With a unique key, we can be sure they're unique, even if we get astronomically unlucky and get a collision. But every table we use them in has a unique key anyway. So we actually get pretty much nothing here, except maybe some vague guarantee that we won't reallocate a key later if the original object is deleted. But it's hard to imagine any install will ever have a collision, given that the key space is 36^20 per object type.
  - We also currently use PHIDs and Users in tests sometimes. This is silly and can break (see D2461).
  - Drop the PHID database.
  - Introduce a "Harbormaster" database (the eventual CI tool, after Drydock).
  - Add a scratch table to the Harbormaster database for doing unit test meta-tests.
  - Now, PHID generation does no writes, and unit tests are isolated from the application.
  - @csilvers: This should slightly improve the performance of the large query-bound tail in D2457.

Test Plan: Ran unit tests. Ran storage upgrade.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: csilvers, aran, nh, edward

Differential Revision: https://secure.phabricator.com/D2466
2012-05-20 14:46:01 -07:00
epriestley
95017e14e9 Minor, derrrp. 2012-05-07 10:34:07 -07:00
epriestley
58efe52b3a Minor, fix method call fatal. 2012-05-07 10:32:29 -07:00
epriestley
87207b2f4e Allow users to have multiple email addresses, and verify emails
Summary:
  - Move email to a separate table.
  - Migrate existing email to new storage.
  - Allow users to add and remove email addresses.
  - Allow users to verify email addresses.
  - Allow users to change their primary email address.
  - Convert all the registration/reset/login code to understand these changes.
  - There are a few security considerations here but I think I've addressed them. Principally, it is important to never let a user acquire a verified email address they don't actually own. We ensure this by tightening the scoping of token generation rules to be (user, email) specific.
  - This should have essentially zero impact on Facebook, but may require some minor changes in the registration code -- I don't exactly remember how it is set up.

Not included here (next steps):

  - Allow configuration to restrict email to certain domains.
  - Allow configuration to require validated email.

Test Plan:
This is a fairly extensive, difficult-to-test change.

  - From "Email Addresses" interface:
    - Added new email (verified email verifications sent).
    - Changed primary email (verified old/new notificactions sent).
    - Resent verification emails (verified they sent).
    - Removed email.
    - Tried to add already-owned email.
  - Created new users with "accountadmin". Edited existing users with "accountadmin".
  - Created new users with "add_user.php".
  - Created new users with web interface.
  - Clicked welcome email link, verified it verified email.
  - Reset password.
  - Linked/unlinked oauth accounts.
  - Logged in with oauth account.
  - Logged in with email.
  - Registered with Oauth account.
  - Tried to register with OAuth account with duplicate email.
  - Verified errors for email verification with bad tokens, etc.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2393
2012-05-07 10:29:33 -07:00
vrana
416e4e7b67 Allowing setting user status
Summary:
I will use it for highlighting users which are not currently available.

Maybe I will also use it in the nagging tool.

I don't plan creating a UI for it as API is currently enough for us.
Maybe I will visualize it at /calendar/ later.

I plan creating `user.deletestatus` method when this one will be done.

Test Plan:
`storage upgrade`
Call Conduit `user.addstatus`.
Verify DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2382
2012-05-03 18:24:30 -07:00
vrana
73c82e5a94 Display holidays
Summary:
We will need it for two purposes:

- Status tool.
- Nagging tool - @aran suggested using "3 business days" and I don't want it to fall on New Year's Eve or such.

I don't plan working on any interface for editing this as this kind of data should be always imported.

Test Plan:
`bin/storage upgrade`
`scripts/calendar/import_us_holidays.php`
/calendar/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2375
2012-05-03 09:22:52 -07:00
epriestley
68b597ff75 SQL Patch Management: SQL Changes
Summary:
Splits out the SQL changes. These are most of the changes, but primarily mechanical:

  - Moved "initialize.sql" to "0000.legacy.sql" and partially reverted to an older version, such that patches 0000 + 000 + 001 + ... + 137 put us in the right state when applied sequentially.
  - Removed "create database" commands from all SQL. These are handled by separate DB patches now, so we have the data to do operations like "storage databases" (list databases) and "storage destroy" (drop databases).
  - Removed "phabricator_" namespace from all SQL, and replaced with "{$NAMESPACE}_" token so we can namespace databases.
  - Shortened some column lengths so patches apply correctly if originally created as InnoDB; also a few similar tweaks elsewhere.

Test Plan: See D2323 for discussion and test plan.

Reviewers: edward, vrana, btrahan, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T140, T345

Differential Revision: https://secure.phabricator.com/D2329
2012-04-30 07:53:53 -07:00
David Reuss
42b1c73f41 Allow CC's/Auditors added to audits
Test Plan:
Added CC's/Auditors, clicked the form elements, and saw correct
behaviour. Verified that metadata was present in the detail table.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, 20after4, Koolvin

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D2002
2012-04-23 13:50:25 -07:00
vrana
549a86cd96 Add sex
Summary:
We will need it for intl.

I've put it to User instead of UserProfile to be easier accessible.

Test Plan:
Apply SQL patch.
Change sex to Male.
Change sex to Unknown.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D2287
2012-04-19 16:05:25 -07:00
vrana
1f2cf78c1b Display committed date in Revision Status field
Summary:
This is slightly more complicated for this reason:

- We don't set `dateCommitted` for normal commits, only for markcommitted.
-- We need to add this date to old revisions now.

Test Plan:
Reparse a revision - commit date was set.
Conduit `markcommitted` - commit date was set.
Run SQL script.
Display closed revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2282
2012-04-19 15:05:09 -07:00
vrana
204b6694af Improve performance of empty search
Summary:
Searching for empty query kills us because whole `search_document` table is read in this case.

This diff adds an index just for this query.

Other solution would be to disable searching for empty string. But it can be actually useful (listing newest documents of any type).

Test Plan:
  lang=sql
  EXPLAIN SELECT document.phid, document.documentType, document.documentTitle, document.documentCreated
  FROM `search_document` document
  GROUP BY document.phid
  ORDER BY documentCreated DESC
  LIMIT 0, 21;

Reviewers: epriestley

Reviewed By: epriestley

CC: jungejason, aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2277
2012-04-18 14:51:09 -07:00
vrana
1f1c7a34b7 Improve image macros
Summary:
Couple of small improvements:

- Delete `randomon` macro.
- Make name unique (deleting current conflicts randomly).
- Image macro must be alone on the line.
- Filter by name.

Test Plan:
Run SQL.
/file/macro/
/file/macro/?name=imagemacro
Try to create conflicting name.
Write this comment:

  Test imagemacro.
  imagemacro

Reviewers: aran, epriestley

Reviewed By: epriestley

CC: epriestley, Koolvin

Differential Revision: https://secure.phabricator.com/D2230
2012-04-17 12:16:58 -07:00
Nick Harper
1ea8bd3ab7 Fix patch 131 for db with lots of revisions
Summary:
The old version of this loads all differential revisions at once, but that much
can't all be loaded into memory when there are close to 500,000 revisions. This
diff splits up loading the revisions.

Test Plan: Ran this to run the migration in our install

Reviewers: jungejason, epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2243
2012-04-16 12:35:48 -07:00
Bob Trahan
51418900f7 Phame V1 - Phabricator blogging software
Summary:
'cuz we need to be phamous!

V1 feature set

- posts
-- standard thing you'd expect - a title and a remarkup-powered body and...
-- "phame" title - a short string that can be used to reference the story. this gets auto-updated when you mess with the title.
-- configuration - for now, do you want Facebook, Disqus or no comments? this is a per-post thing but feeds from an instance-wide configuration

Please do toss out any must have features or changes.

Test Plan: played around with this bad boy like whoa

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, vrana

Maniphest Tasks: T1111

Differential Revision: https://secure.phabricator.com/D2202
2012-04-12 13:09:04 -07:00
epriestley
fe9ba6bc67 Improve DifferentialRevisionQuery and add the ability to query by arcanist project
Summary:
  - We currently post-filter by branches, but should do this in SQL. See T799.
  - We currently identify branch-name-matches as being in the working copy even if they belong to a different project (e.g., two different projects with commits on the branch "master"). See T1100.
  - Denormalize branch and project information into DifferentialRevision.
  - Expose project information in the API.

Test Plan: Ran conduit API queries with branches and arc project IDs, got reasonable results.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1100, T799

Differential Revision: https://secure.phabricator.com/D2190
2012-04-10 12:51:34 -07:00
epriestley
488b1cf641 Allow Maniphest queries to be saved
Summary:
There have been a couple of requests for this since bookmarks are "out this year like woah" and "totally uncool dude".

Allow users to save named custom queries and make them the /maniphest/ default if they so desire.

A little messy. :/

Test Plan: Saved, edited, deleted custom queries. Made custom query default; made 'no default' default. Verified default behavior. Issued a modified search from a custom query.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley, 20after4

Maniphest Tasks: T923, T1034

Differential Revision: https://secure.phabricator.com/D1964
2012-04-10 09:46:04 -07:00
vrana
32d2395a45 Unify links to www.phabricator.com and phabricator.com
Test Plan:
  scripts/sql/upgrade_schema.php

Verify links at /directory/2/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Maniphest Tasks: T1096

Differential Revision: https://secure.phabricator.com/D2172
2012-04-09 14:32:03 -07:00
vrana
6aa729b1c9 Support MySQL 5.5
Summary:
`116.utf8.sql` throws this under MySQL 5.5:

> Column length too big for column 'keyBody' (max = 21845); use BLOB or TEXT instead

I guess that's because MySQL 5.5 changed maximum length of UTF-8 character from 3 bytes to 4.

I've updated also `116.utf8.sql` for people with new installs.

Test Plan:
  upgrade_schema.php

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2117
2012-04-06 09:55:58 -07:00
epriestley
877cb136e8 Add an assocations-like "Edges" framework
Summary:
We have a lot of cases where we store object relationships, but it's all kind of messy and custom. Some particular problems:

  - We go to great lengths to enforce order stability in Differential revisions, but the implementation is complex and inelegant.
  - Some relationships are stored on-object, so we can't pull the inverses easily. For example, Maniphest shows child tasks but not parent tasks.
  - I want to add more of these and don't want to continue building custom stuff.
  - UIs like the "attach stuff to other stuff" UI need custom branches for each object type.
  - Stuff like "allow commits to close tasks" is notrivial because of nonstandard metadata storage.

Provide an association-like "edge" framework to fix these problems. This is nearly identical to associations, with a few differences:

  - I put edge metadata in a separate table and don't load it by default, to keep edge rows small and allow large metadata if necessary. The on-edge metadata seemed to get abused a lot at Facebook.
  - I put a 'seq' column on the edges to ensure they have an explicit, stable ordering within a source and type.

This isn't actually used anywhere yet, but my first target is attaching commits to tasks for T904.

Test Plan: Made a mock page that used Editor and Query. Verified adding and removing edges, overwriting edges, writing and loading edge data, sequence number generation.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, 20after4

Differential Revision: https://secure.phabricator.com/D2088
2012-04-04 15:30:21 -07:00
vrana
5885764728 Allow IPv6
Test Plan:
/people/logs/
Search for `2620:0:1cfe:`.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, jungejason

Differential Revision: https://secure.phabricator.com/D2089
2012-04-03 11:36:50 -07:00
epriestley
b028920a5e Minor, subpriority default previously reverse ordered eveything
Auditors: btrahan
2012-04-02 12:26:50 -07:00
epriestley
e7853e4801 Allow tasks to be subprioritized by drag-and-drop
Summary:
Like the title says, similar to Facebook Tasks.

Not sure how I really feel about this, but I guess it's kind of OK? I never used
this feature in Facebook Tasks but I think some people like it.

The drag-and-drop to repri across priorities feels okayish.

Because subpriority is a double and we just split the difference when
reprioritizing, you lose ~a bit of precision every time you repri two tasks
against each other and so you can break it by swapping the priorities of two
tasks ~50 times. This case is pretty silly and pathological. We can add some
code to deal with this at some point if necessary.

I think this also fixes the whacky task layout widths once and for all.

(There are a couple of minor UI glitches like headers not vanishing and header
counts not updating that I'm not fixing because I am lazy.)

Test Plan: Dragged and dropped tasks around.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley, mgummelt

Maniphest Tasks: T859

Differential Revision: https://secure.phabricator.com/D1731
2012-04-02 12:12:04 -07:00
epriestley
698ec68327 General Herald refactoring pass
Summary:
**Who can delete global rules?**: I discussed this with @jungejason. The current behavior is that the rule author or any administrator can delete a global rule, but this
isn't consistent with who can edit a rule (anyone) and doesn't really make much sense (it's an artifact of the global/personal split). I proposed that anyone can delete a
rule but we don't actually delete them, and log the deletion. However, when it came time to actually write the code for this I backed off a bit and continued actually
deleting the rules -- I think this does a reasonable job of balancing accountability with complexity. So the new impelmentation is:

  - Personal rules can be deleted only by their owners.
  - Global rules can be deleted by any user.
  - All deletes are logged.
  - Logs are more detailed.
  - All logged actions can be viewed in aggregate.

**Minor Cleanup**

  - Merged `HomeController` and `AllController`.
  - Moved most queries to Query classes.
  - Use AphrontFormSelectControl::renderSelectTag() where appropriate (this is a fairly recent addition).
  - Use an AphrontErrorView to render the dry run notice (this didn't exist when I ported).
  - Reenable some transaction code (this works again now).
  - Removed the ability for admins to change rule authors (this was a little buggy, messy, and doesn't make tons of sense after the personal/global rule split).
  - Rules which depend on other rules now display the right options (all global rules, all your personal rules for personal rules).
  - Fix a bug in AphrontTableView where the "no data" cell would be rendered too wide if some columns are not visible.
  - Allow selectFilter() in AphrontNavFilterView to be called without a 'default' argument.

Test Plan:
  - Browsed, created, edited, deleted personal and gules.
  - Verified generated logs.
  - Did some dry runs.
  - Verified transcript list and transcript details.
  - Created/edited all/any rules; created/edited once/every time rules.
  - Filtered admin views by users.

Reviewers: jungejason, btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2040
2012-03-30 10:49:55 -07:00
epriestley
7ad68e63e4 Add "Flags" to allow users to collect the things they love
Summary:
Flags are a personal collection of things you want to take a look at later. You can use several different colors and add notes.

Not really sure if this is actually a good idea or not but it was easy to build.

Planned features:

  - Allow Herald rules to add flags.
  - In the "edit flag" dialog, have a "[x] Subscribe Me" checkbox that CCs you.
  - Support Diffusion.
  - Support Phriction.
  - Always show flags on an object if you have them (in every view)?
  - Edit dialog feels a little heavy?
  - More filtering in /flag/ tool.
  - Add a top-level links somewhere?

Test Plan: Added, edited and removed flags from things. Viewed flags in flag view.

Reviewers: aran, btrahan

Reviewed By: btrahan

CC: aran, epriestley, Koolvin

Maniphest Tasks: T1041

Differential Revision: https://secure.phabricator.com/D2024
2012-03-27 16:22:40 -07:00
epriestley
914f044b62 More Drydock Stuff
Summary:
  - Still really really rough.
  - Adds a full synchronous mode for debugging.
  - Adds some logging.
  - It can now allocate EC2 machines and put webroots on them in a hacky, terrible way.
  - Adds a base query class.

Test Plan: oh hey look a test page? http://ec2-50-18-65-151.us-west-1.compute.amazonaws.com:2011/

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D2026
2012-03-26 20:54:26 -07:00
epriestley
0a4cbdff5e Straighten out Diffusion file integration
Summary:
This is in preparation for getting the "View Options" dropdown working on audits.

  - Use Files to serve raw data so we get all the security benefits of the alternate file domain. Although the difficulty of exploiting this is high (you need commit access to the repo) there's no reason to leave it dangling.
  - Add a "contentHash" to Files so we can lookup files by content rather than adding some weird linker table. We can do other things with this later, potentially.
  - Don't use 'data' URIs since they're crazy and we can just link to the file URI.
  - When showing a binary file or an image, don't give options like "show highlighted text with blame" or "edit in external editor" since they don't make any sense.
  - Use the existing infrastructure to figure out if things are images or binaries instead of an ad-hoc thing in this class.

Test Plan: Looked at text, image and binary files in Diffusion. Verified we reuse existing files if we've already generated them.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1899
2012-03-19 19:52:24 -07:00
epriestley
900190b2fe Add inline comments to Diffusion/Audit
Summary:
  - Add inline comments to Audits, like Differential.
  - Creates new storage for the comments in the Audits database.
  - Creates a new PhabricatorAuditInlineComment class, similar to DifferentialInlineComment.
  - Defines an Interface which Differential and Audit comments conform to.
  - Makes consumers of DifferentialInlineComments consume objects which implement that interface instead.
  - Adds save

NOTE: Some features are still missing! Wanted to cut this off before it got crazy:

  - Inline comments aren't shown in the main comment list.
  - Inline comments aren't shown in the emails.
  - Inline comments aren't previewed.

I'll followup with those but this was getting pretty big.

@vrana, does the SQL change look correct?

Test Plan:
  - Created, edited, deleted, replied to, reloaded and saved inline comments in Diffusion, on the left and right side of diffs.
  - Created, edited, deleted, replied to, reloaded and saved inline comments in Differentila, on the left and right side of primary and diff-versus-diff diffs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1898
2012-03-14 12:56:01 -07:00
vrana
f4be64792e Drop unused column
Summary: https://secure.phabricator.com/D1830?id=3203#inline-2058

Test Plan:
Run script on db containing this column.
Run script on db not containing this column.
Visit /repository/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1854
2012-03-12 12:11:36 -07:00
vrana
4c1e356658 Convert database to UTF-8
Summary: This is the script used for conversion: P319

Test Plan:
Update diff with UTF-8 characters in description.
`sql/upgrade_schema.php`
Verify data in DB and that it looks good on web.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T327

Differential Revision: https://secure.phabricator.com/D1830
2012-03-12 12:11:02 -07:00
vrana
d5bf30bb48 Prepare database for UTF-8
Summary: D1830#8

Test Plan:
`scripts/sql/upgrade_schema.php`
Try adding duplicate SSH Public Key - failed.
Try adding new SSH Public Key - succeeded.

Reviewers: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1852
2012-03-09 18:56:22 -08:00
epriestley
d94129b739 Migrate "PhabricatorOwnersPackageCommitRelationship" to "PhabricatorRepositoryAuditRequest"
Summary:
  - Move table to Repository, since we have no Owners joins in the application anymore but would like to do a Repository join.
  - Rename "packagePHID" to "auditorPHID", since this column may contain package, project, or user PHIDs.

Test Plan:
  - Browsed Owners, Audit, and Differential interfaces to the Audit tool.
  - Made comments and state changes.
  - Ran "reparse.php --herald --owners" on several commits.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley, nh, vrana

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1787
2012-03-05 13:17:29 -08:00
epriestley
1e4e3d1fef Minor, prevent patch 113 from breaking in some environments(?) 2012-03-05 12:34:33 -08:00
vrana
f5f7987013 Revert rP87c60abbd02d, apply D1772
Test Plan:
Apply SQL patch.
Visit /differential/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1781
2012-03-05 11:04:55 -08:00
Bob Trahan
0327a5fc69 OAuthServer polish and random sauce
Summary:
This diff makes the OAuthServer more compliant with the spec by
- making it return well-formatted error codes with error types from the spec.
- making it respect the "state" variable, which is a transparent variable the
client passes and the server passes back
- making it be super, duper compliant with respect to redirect uris
-- if specified in authorization step, check if its valid relative to the client
registered URI and if so save it
-- if specified in authorization step, check if its been specified in the access
step and error if it doesn't match or doesn't exist
-- note we don't make any use of it in the access step which seems strange but
hey, that's what the spec says!
This diff makes the OAuthServer suck less by
- making the "cancel" button do something in the user authorization flow
- making the client list view and client edit view be a bit more usable around
client secrets
- fixing a few bugs I managed to introduce along the way

Test Plan:
- create a test phabricator client, updated my conf, and then linked and
unlinked phabricator to itself
- wrote some tests for PhabricatorOAuthServer -- they pass!
-- these validate the various validate URI checks
- tried a few important authorization calls
--
http://phabricator.dev/oauthserver/auth/?client_id=X&state=test&redirect_uri=http://www.evil.com
--- verified error'd from mismatching redirect uri's
--- verified state parameter in response
--- verified did not redirect to client redirect uri
-- http://phabricator.dev/oauthserver/auth/?client_id=X w/ existing
authorization
--- got redirected to proper client url with error that response_type not
specified
-- http://phabricator.dev/oauthserver/auth/?client_id=X&response_type=code w/
existing authorization
--- got redirected to proper client url with pertinent code!
- tried a few important access calls
-- verified appropriate errors if missing any required parameters
-- verified good access code with appropriate other variables resulted in an
access token
- verified that if redirect_uri set correctly in authorization required for
access and errors if differs at all / only succeeds if exactly the same

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, ajtrichards

Maniphest Tasks: T889, T906, T897

Differential Revision: https://secure.phabricator.com/D1727
2012-03-01 14:46:18 -08:00
epriestley
8a0a00f118 Make PhabricatorRepositoryCommmit schema changes for audit
Summary:
  - Add a proper mailKey field to make these things mailable. Backfill all
existing objects.
  - Denormalize authorPHID to the commit object so we can query by it
efficiently in a future diff. We currently use the search engine to drive
"commits by author" but that's not so good for audit, which needs more
constraints.
  - Add an overall audit status field so we can efficiently query "commits that
needs your attention".
  - Add enough code to convince myself that these fields are basically
reasonable and work correctly.

Test Plan:
  - Ran schema upgrades. Checked database state afterward.
  - Ran "reparse.php --owners --herald" to verify worker changes.
  - Looked at a commit, altered aggregate status via audits / reparse.php,
verified it responded correctly.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley, nh

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1706
2012-02-28 21:06:34 -08:00
Bob Trahan
3c4070a168 OAuth Server -- add controllers to RUD client authorizations and CRUD clients
Summary:
beyond the title, this diff tweaks the test console to have a bit more
functionality.  also makes a small change to CSS for AphrontFormControlMarkup,
which IMO fixes a display issue on
https://secure.phabricator.com/settings/page/profile/ where the Profile URI is
all up in the air and whatnot

I think this is missing pagination.  I am getting tired of the size though and
will add later.  See T905.

Test Plan:
viewed, updated and deleted client authorizations.  viewed, created,
updated and deleted clients

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T849, T850, T848

Differential Revision: https://secure.phabricator.com/D1683
2012-02-24 14:56:18 -08:00
Bob Trahan
af295e0b26 OAuth Server enhancements -- more complete access token response and groundwork
for scope

Summary:
this patch makes the access token response "complete" relative to spec by
returning when it expires AND that the token_type is in fact 'Bearer'.

This patch also lays the groundwork for scope by fixing the underlying data
model and adding the first scope checks for "offline_access" relative to expires
and the "whoami" method.   Further, conduit is augmented to open up individual
methods for access via OAuth generally to enable "whoami" access.   There's also
a tidy little scope class to keep track of all the various scopes we plan to
have as well as strings for display (T849 - work undone)

Somewhat of a hack but Conduit methods by default have SCOPE_NOT_ACCESSIBLE.  We
then don't even bother with the OAuth stuff within conduit if we're not supposed
to be accessing the method via Conduit.   Felt relatively clean to me in terms
of additional code complexity, etc.

Next up ends up being T848 (scope in OAuth) and T849 (let user's authorize
clients for specific scopes which kinds of needs T850).  There's also a bunch of
work that needs to be done to return the appropriate, well-formatted error
codes.  All in due time...!

Test Plan:
verified that an access_token with no scope doesn't let me see
anything anymore.  :(  verified that access_tokens made awhile ago expire.  :(

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T888, T848

Differential Revision: https://secure.phabricator.com/D1657
2012-02-21 16:33:06 -08:00
Bob Trahan
7a3f33b5c2 OAuth - Phabricator OAuth server and Phabricator client for new Phabricator OAuth Server
Summary:
adds a Phabricator OAuth server, which has three big commands:
 - auth - allows $user to authorize a given client or application.  if $user has already authorized, it hands an authoization code back to $redirect_uri
 - token - given a valid authorization code, this command returns an authorization token
 - whoami - Conduit.whoami, all nice and purdy relative to the oauth server.
Also has a "test" handler, which I used to create some test data.  T850 will
delete this as it adds the ability to create this data in the Phabricator
product.

This diff also adds the corresponding client in Phabricator for the Phabricator
OAuth Server.  (Note that clients are known as "providers" in the Phabricator
codebase but client makes more sense relative to the server nomenclature)

Also, related to make this work well
 - clean up the diagnostics page by variabilizing the provider-specific
information and extending the provider classes as appropriate.
 - augment Conduit.whoami for more full-featured OAuth support, at least where
the Phabricator client is concerned

What's missing here...   See T844, T848, T849, T850, and T852.

Test Plan:
- created a dummy client via the test handler.   setup development.conf to have
have proper variables for this dummy client.  went through authorization and
de-authorization flows
- viewed the diagnostics page for all known oauth providers and saw
provider-specific debugging information

Reviewers: epriestley

CC: aran, epriestley

Maniphest Tasks: T44, T797

Differential Revision: https://secure.phabricator.com/D1595
2012-02-19 14:00:13 -08:00
epriestley
7200040479 Add a basic chatlog
Summary:
This is pretty simple and unpolished, but it's getting pretty big and it seems
like a reasonable starting point.

  - Log chat in various "channels".
  - Conduit record and query methods.
  - IRCBot integration for IRC logging

Major TODO:

  - Web UI is really unpolished and has no search, paging, anchor-linking, etc.
Basically all presentation stuff, though.
  - I think the bot should have a map of channels to log with channel aliases?
  - The "channels" should probably be in a separate table.
  - The "authors" should probably be correlated to Phabricator accounts somehow,
where possible.

Test Plan: Used phabotlocal to log #phabricator.

Reviewers: kdeggelman, btrahan, Koolvin

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T837

Differential Revision: https://secure.phabricator.com/D1625
2012-02-17 10:21:38 -08:00
epriestley
d7bb1f183c Trim "\n" from mimeType field in phabricator_file.file
Summary: See patch comment, which explains the situation.

Test Plan: Ran upgrade, verified mimeType field is \n-free in database.

Reviewers: davidreuss, btrahan

Reviewed By: davidreuss

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1624
2012-02-16 07:25:56 -08:00
epriestley
35c5852d3f Add a safeguard against multiple patches with the same version
Summary:
I accidentally added two "104" patches. This actually works OK for the most part
but is fundamentally bad and wrong.

Merge the patches (installs applied both as "104", so we can't move one to
"105") and add a safeguard.

Test Plan: Ran upgrade_schema.php with two "104" patches, got error'd. Ran
without, got successs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1614
2012-02-14 16:24:02 -08:00
epriestley
4caa684724 Simplify Project status field
Summary:
This was a sort of speculative feature added by a contributor some time ago and
just serves as a label; for now, simplify it into "active" and "archived" and
remove "archived" projects from the "active" list.

  - Fix a bug where we'd publish a "renamed from X to X" transaction that had no
effect.
  - Publish stories about status changes.
  - Remove the "edit affiliation" controller, which has no links in the UI
(effectively replaced by join/leave links).
  - Add query/conduit support.

Test Plan: Edited the status of several projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T681

Differential Revision: https://secure.phabricator.com/D1573
2012-02-07 14:59:38 -08:00
epriestley
a5f8846f47 Use a unique random key to identify queries, not a sequential ID
Summary:
We save search information and then redirect to a "/search/<query_id>/" URI in
order to make search URIs short and bookmarkable, and save query data for
analysis/improvement of search results.

Currently, there's a vague object enumeration security issue with using
sequential IDs to identify searches, where non-admins can see searches other
users have performed. This isn't really too concerning but we lose nothing by
using random keys from a large ID space instead.

  - Drop 'authorPHID', which was unused anyway, so searches can not be
personally identified, even by admins.
  - Identify searches by random hash keys, not sequential IDs.
  - Map old queries' keys to their IDs so we don't break any existing bookmarked
URIs.

Test Plan: Ran several searches, got redirected to URIs with random hashes from
a large ID space rather than sequential integers.

Reviewers: arice, btrahan

Reviewed By: arice

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1587
2012-02-07 14:58:46 -08:00
awyler
56df2bc7be Add basic edit history to herald rules
Summary:
Add a very basic edit history table to herald rules.  This table is updated
whenever saving a herald rule.  The contents of the save are not examined, and
the edit history contains no information about the rule itself *yet*.  Edit
history can be viewed by anyone through /herald/history/<rule id>/.

Task ID: #

Blame Rev:

Test Plan:
Made a test rule, saved some stuff.

Revert Plan:

Tags:

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: zizzy, aran, xela, epriestley

Differential Revision: https://secure.phabricator.com/D1387
2012-01-30 11:52:44 -08:00
epriestley
5b463e634c Write fewer "applied" rows and clean up excess historical rows
Summary:
  - Only write the <ruleID, phid> row if the rule is a one-time rule.
  - Delete all the rows for rules which aren't one-time.

NOTE: This is probably like several million rows for Facebook and could take a
while.

Test Plan:
Added some one-time and every-time rules, ran them against objects, verified
only relevant rows were inserted.
Ran upgrade script against a database with one-time and every-time "ruleapplied"
rows, got the irrelevant rows removed.

Reviewers: nh, btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1484
2012-01-25 11:53:39 -08:00
epriestley
3142fe4419 Remove massive "rule applied" query
Summary:
Herald rules may be marked as "one-time". We track this by writing a row with
<ruleID, phid> when we apply a rule.

However, the current test for rule application involves loading every <ruleID,
*> pair. We also always write this row even for rules which are not one-time, so
if there are 100 rules, we'll load 1,000,000 rows after processing 10,000
objects.

Instead, load only the <phid, *> pairs, which are guaranteed to be bounded to at
most the number of rules.

I'll follow up with a diff that causes us to write rows only for one-time rules,
and deletes all historic rows which are not associated with one-time rules.

Test Plan:
Grepped for callsites to loadAllByContentTypeWithFullData(). Ran
rules in test console.

Reviewers: nh, btrahan, jungejason

Reviewed By: nh

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1483
2012-01-24 19:29:54 -08:00
epriestley
b43eb5aa7c Add transaction-oriented editing to projects
Summary:
  - Make some editing operations transaction-oriented, like Maniphest. (This
seems to be a good model, particularly for extensibility.) I'll move the rest of
the editing operations to transactions in future diffs.
  - Make transaction-oriented operations publish feed stories.

Test Plan:
  - Created a new project.
  - Edited an existing project.
  - Created a new project via quick create flow from Maniphest.
  - Verified feed stories publish correctly.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T681

Differential Revision: https://secure.phabricator.com/D1477
2012-01-24 09:44:35 -08:00
epriestley
d1ee08b2df Drydock Rough Cut
Summary:
Rough cut of Drydock. This is very basic and doesn't do much of use yet (it
//does// allocate EC2 machines as host resources and expose interfaces to them),
but I think the overall structure is more or less reasonable.

== Interfaces

Vision: Applications interact with Drydock resources through DrydockInterfaces,
like **command**, **filesystem** and **httpd** interfaces. Each interface allows
applications to perform some kind of operation on the resource, like executing
commands, reading/writing files, or configuring a web server. Interfaces have a
concrete, specific API:

  // Filesystem Interface
  $fs = $lease->getInterface('filesystem'); // Constants, some day?
  $fs->writeFile('index.html', 'hello world!');

  // Command Interface
  $cmd = $lease->getInterface('command');
  echo $cmd->execx('uptime');

  // HTTPD Interface
  $httpd = $lease->getInterface('httpd');
  $httpd->restart();

Interfaces are mostly just stock, although installs might add new interfaces if
they expose different ways to interact with resources (for instance, a resource
might want to expose a new 'MongoDB' interface or whatever).

Currently: We have like part of a command interface.

== Leases

Vision: Leases keep track of which resources are in use, and what they're being
used for. They allow us to know when we need to allocate more resources (too
many sandcastles on the existing hosts, e.g.) and when we can release resources
(because they are no longer being used). They also give applications something
to hold while resources are being allocated.

  // EXAMPLE: How this should work some day.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('sandcastle');
  $allocator->setAttributes(
    array(
      'diffID' => $diff->getID(),
    ));
  $lease = $allocator->allocate();
  $diff->setSandcastleLeaseID($lease->getID());

  // ...

  if ($lease->getStatus() == DrydockLeaseStatus::STATUS_ACTIVE) {
    $sandcastle_link = $lease->getInterface('httpd')->getURI('/');
  } else {
    $sandcastle_link = 'Still building your sandcastle...';
  }
  echo "Sandcastle for this diff: ".$sandcastle_link;

  // EXAMPLE: How this actually works now.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('host');
  // NOTE: Allocation is currently synchronous but will be task-driven soon.
  $lease = $allocator->allocate();

Leases are completely stock, installs will not define new lease types.

Currently: Leases exist and work but are very very basic.

== Resources

Vision: Resources represent some actual thing we've put somewhere, whether it's
a host, a block of storage, a webroot, or whatever else. Applications interact
through resources by acquiring leases to them, and then getting interfaces
through these leases. The lease acquisition process has a side effect of
allocating new resources if a lease can't be acquired on existing resources
(e.g., the application wants storage but all storage resources are full) and
things are configured to autoscale.

Resources may themselves acquire leases in order to allocate. For instance, a
storage resource might first acquire a lease to a host resource. A 'test
scaffold' resource might lease a storage resource and a mysql resource.

Not all resources are auto-allocate: the entry-level version of Drydock is that
you manually allocate a couple boxes and configure them through the web console.
Then, e.g.,  'storage' / 'webroot' resources allocate on top of them, but the
host pool itself does not autoscale.

Resources are completely stock, they are abstract shells representing any
arbitrary thing.

Currently: Resource exist ('host' only) but are very very basic.

== Blueprints

Vision: Blueprints contain instructions for building interfaces to, (possibly)
allocating, updating, managing, and destroying a specific type of resource in a
specific location. One way to think of them is that they are scripts for
creating and deleting resources. For example, the LocalHost, RemoteHost and
EC2Host blueprints can all manage 'host' resources.

Eventually, we will support more types of resources (storage, webroot,
sandcastle, test scaffold, phacility deployment) and more providers for resource
types, some of which will be in the Phabricator mainline and some of which will
be custom.

Blueprints are very custom and specific to application types, so installs will
define new blueprints if they are making significant use of Drydock.

Currently: They exist but have few capabilities. The stock blueprints do nearly
nothing useful. There is a technically functional blueprint for host allocation
in EC2.

== Allocator

This is just the actual code to execute the lease acquisition process.

Test Plan: Ran "drydock_control.php" script, it allocated a machine in EC2,
acquired a lease on it, interfaced with it, and then released the lease. Ran it
again, got a fresh lease on the existing resource.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D1454
2012-01-19 21:12:57 -08:00
awyler
6080d74112 Created personal vs. global herald rule distingtion
Summary:
A personal rule only has actions targeting the owner.  Likewise, only they can
edit the rule. OTOH, a global may affect any target and is editable by anyone.

There are no new action types.  Instead, type of the rule modifies the available
targets and the messaging in the ui.  This is beneficial because herald rule
adapters don't need to be aware of the difference between emailing the owner of
a personal rule and emailing an arbitrary user.

This diff sets up the logic and ui for creating personal/global rules.  All
existing rules have been defaulted to global.

TODO: Filter all existing rules into personal/global
TODO: Create a UI for surfacing (relevant?) global rules.

Test Plan:
1. Created a personal rule to email myself.  Created a dumby revision satisfying
the conditions of that rule.  Verified that I recieved a herald email.
2. Removed my adminship, change the owner of a personal rule. verified that I
couldn't edit the rule.
3.Changed rule type to global. verified that I could edit the rule.
4. Verified that admins can edit both global and personal rules.

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: aran, zizzy

Differential Revision: https://secure.phabricator.com/D1449
2012-01-19 11:21:49 -08:00
vrana
6472dbe168 Change fileName to filename
Summary: There are lots of callsites to $changeset->getFilename() so it seemed
easier to rename getFileName() to getFilename() even if it includes database
change. Plus I think that getFilename() is better.

Test Plan:
Alter database.
Open revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D1437
2012-01-17 10:50:14 -08:00
vrana
59380582d8 Delete /xhprof/ from directory, mark /mail/ as Admin Only
Summary: /xhprof/ doesn't exist at all, /mail/ is only for admins

Test Plan: Display directory

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, vrana

Differential Revision: https://secure.phabricator.com/D1405
2012-01-15 20:27:55 -08:00
epriestley
95cde81daf Minor, reduce slug column size to 128, see D1391. 2012-01-13 17:16:37 -08:00
epriestley
13bf353f14 Shorten Phriction slug field to 256 to avoid key length issues in UTF-8 default
databases

Summary: This is a stopgap patch until we fix T345/T327 properly, but you can't
add a 512-character key to a UTF-8 column because it creates an index on more
than 767 bytes, which is the key length limit.

Test Plan: Ran schmea upgrade script.

Reviewers: indiefan, btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley, xela, jungejason

Maniphest Tasks: T327

Differential Revision: https://secure.phabricator.com/D1391
2012-01-13 15:25:18 -08:00
epriestley
efb0fa739f Make tracked git repositories use an implicit 'origin' remote
Summary:
See T624. I originally wrote this to require an explicit remote, but this
creates an ugly "origin:" in all the URIs and makes T270 more difficult.

Treat all branch names as implying 'origin/'.

Test Plan:
  - Pulled and imported a fresh copy of libphutil without issues.
  - Browsed various git repositories.
  - Browsed Javelin's various branches.
  - Ran upgrade script, got a bunch of clean 'origin/master' -> 'master'
conversions.
  - Tried to specify an explicit remote in a default branch name.
  - Unit tests.

Reviewers: nh, jungejason, btrahan

Reviewed By: btrahan

CC: aran, btrahan

Maniphest Tasks: T624

Differential Revision: https://secure.phabricator.com/D1269
2011-12-29 08:35:32 -08:00
epriestley
f30f4fa713 Issue CREATE DATABASE before ALTER TABLE in patch 88
Summary:
Sometimes the SQL account may not have CREATE DATABASE privileges, so try to
fail this script immediately rather than having it work halfway if the
permission is missing.

This is just a minor tweak to try to slightly improve some cases, the long term
fix is T345 but that's a good deal more work than this.

Test Plan: Visually inspected patch; trivial change.

Reviewers: jungejason, btrahan, skrul

Reviewed By: skrul

CC: aran, skrul

Differential Revision: https://secure.phabricator.com/D1287
2011-12-27 17:50:23 -08:00
epriestley
bdbe9df65e Remove support for GitHub post-receive notifications
Summary:
  - These never actually did anything.
  - I don't even really remember why I built them, maybe the Open Source team
was pushing for more GitHub integration or something? I really have no idea.
  - Anyway, repository tailers do everything these could do (and much more).

Test Plan:
  - Ran tailers off GitHub for many months without needing post-receive hooks.
  - Grepped for relevant strings, couldn't find any references.
  - Used "Repository" edit interface for a Git repository.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, jungejason

Maniphest Tasks: T706

Differential Revision: https://secure.phabricator.com/D1273
2011-12-24 09:00:08 -08:00
epriestley
21ba07d5bd Provide wiki pages for projects
Summary:
Provide tighter integration between Projects and Phriction. Partly, I have most
of a rewrite for the Projects homepage ready but it's not currently possible to
publish feed stories about a project so all the feeds are empty/boring. This
partly makes them more useful and partly just provides a tool integration point.

  - When you create a project, all the wiki pages in projects/<project_name>/*
are associated with it.
  - Publish updates to those pages as being related to the project so they'll
show up in project feeds.
  - Show a project link on those pages.

This is very "convention over configuration" but I think it's the right
approach. We could provide some sort of, like, "@project=derp" tag to let you
associated arbitrary pages to projects later, but just letting you move pages is
probably far better.

Test Plan:
  - Ran upgrade scripts against stupidly named projects ("der", "  der", "  der
", "der (2)", "  der (2) (2)", etc). Ended up with uniquely named projects.
  - Ran unit tests.
  - Created /projects/ wiki documents and made sure they displayed correctly.
  - Verified feed stories publish as project-related.
  - Edited projects, including perfomring a name-colliding edit.
  - Created projects, including performing a name-colliding create.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley, btrahan

Maniphest Tasks: T681

Differential Revision: 1231
2011-12-20 14:03:12 -08:00
jungejason
c80d1480d5 Add Basic Auditing Functionalities
Summary:
add basic auditing functionalities. For the related commits for a
package, we detect the following conditions which might be suspicious to the
owners of the package:

* no revision specified
* revision not found
* author not match
* reviewedby not match
* owners not involved
* commit author not recognized

The owners of the package can change the status of the audit entries by
accepting it or specify concern.

The owner can turn on/off the auditing for a package.

Test Plan:
*  verified that non-owner cannot see the details of the audit and cannot modify
it
*  verified that all the audit reasons can be detected
*  tested dropdown filtering and package search
*  verified really normal change not detected
*  verified accept/concern a commit
*  tested enable/disable a package for auditing
*  verified one audit applies to all <commit, packages> to the packages the
auditor owns
*  verified that re-parsing a commit won't have effect if there exists a
 relationship for <commit, package> already

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, benmathews, btrahan, mpodobnik, prithvi, TomL, epriestley

Differential Revision: 1242
2011-12-20 13:36:53 -08:00
epriestley
afc2f8526b Allow Phriction documents to be deleted
Summary:
  - Add a "delete" operation. Delete is just a special edit which removes the
page from indexes and shows a notice that the document has been deleted.
  - When a user deletes all the content on a page, treat it as a delete.
  - When a conduit call deletes all the content on a page, treat it as a delete.
  - Add page status to Conduit.
  - Add change type field to history.
  - Added a couple of constants to support a future 'move' change, which would
move content from one document to another.

Test Plan:
  - Verified deleted pages vanish from the document index (and restoring them
puts them back).
  - Verified deleted pages show "This page has been deleted...".
  - Created, edited and deleted a document via Conduit.
  - Deleted pages via "delete" button.
  - Deleted pages via editing content to nothing.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: skrul, aran, btrahan, epriestley

Maniphest Tasks: T680

Differential Revision: 1230
2011-12-17 11:45:25 -08:00
epriestley
93d5d29541 Remove "Former" project members
Summary:
This is a needlessly confusing/complex feature that I originally wrote sort of
speculativley. I think we can better serve what little need may exist here with
project feeds.

I'm probably going to get rid of or deemphasize "role" too and just add "Join
Project" and "Leave Project" buttons.

Test Plan: Viewed project list, project profile. Edited project profile and
affiliation.

Reviewers: btrahan, jungejason, zeeg

Reviewed By: btrahan

CC: aran, btrahan

Maniphest Tasks: T681

Differential Revision: 1228
2011-12-16 17:46:02 -08:00
jungejason
c13b7da290 Add Related Commits for Owners
Summary:
For each commit, find the affected packages, and provide a way to
search by package.

Test Plan:
create commits that touch and don't touch two packages, and verify
that they display correctly in all the UI pages.

Reviewers: epriestley, blair, nh, tuomaspelkonen

Reviewed By: epriestley

CC: benmathews, aran, epriestley, btrahan, jungejason, mpodobnik, prithvi

Maniphest Tasks: T83

Differential Revision: 1208
2011-12-14 22:48:57 -08:00
Bob Trahan
d739d8ccd5 Paste - upgrade scheme to support queries by authorPHID
Test Plan: ran upgrade_schema.php on my local install.   re-examined SQL query
"explain select * from phabricator_pastebin.pastebin_paste where authorPHID =
"PHID-USER-xee4ju2teq7mflitwfcs" ORDER BY id DESC;" on my local install and
noted fewer rows scanned, index used, etc

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1218
2011-12-14 19:48:47 -08:00
epriestley
4fd81150be Remove "Updated" view from Differential
Summary:
This landed during my review drama embargo and is a generally good idea but had
some implementation issues.

@elynde reports it has been broken for some time, although it still works on
secure.phabricator.com so I'm guessing it's just taking a zillion years to run
at Facebook. It's up to more than a second for me on secure.phabricator.com:

https://secure.phabricator.com/file/view/PHID-FILE-v4ql4c66u3xnkarmrpm4/

The basic problem is that some of the data architecture around this
implementation is hard to scale. I want to pursue a similar feature eventually,
but drive it off notifications that we'll ship through real-time infrastructure
too.

I'm also trying to get rid of DifferentialRevisionListData and this simplifies
that somewhat.

Test Plan:
  - Grepped for table name, table constant, query constant, and class name; no
hits.
  - Applied SQL patch.
  - Verified that Differential no longer shows "Updated".

Reviewers: elynde, btrahan, jungejason

Reviewed By: elynde

CC: aran, elynde

Differential Revision: 1178
2011-12-07 06:55:03 -08:00
epriestley
720af267a7 Add a missing key to the ManiphestTransaction table
Summary:
Caught a bad (8ms) query that didn't have a good key available. Add an
appropriate key.

The query in question is:

   SELECT * FROM `maniphest_transaction` WHERE taskID = 262 ORDER BY id ASC;

...which is used in the task detail view.

Test Plan: Ran EXPLAIN on this query before/after upgrading, it now uses the
newly available key.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran

Differential Revision: 1037
2011-10-23 14:43:03 -07:00
epriestley
0669abc5f0 Use a proper entropy source to generate file keys
Summary:
See T549. Under configurations where files are served from an alternate domain
which does not have cookie credentials, we use random keys to prevent browsing,
similar to how Facebook relies on pseudorandom information in image URIs (we
could some day go farther than this and generate file sessions on the alternate
domain or something, I guess).

Currently, we generate these random keys in a roundabout manner. Instead, use a
real entropy source and store the key on the object. This reduces the number of
sha1() calls in the codebase as per T547.

Test Plan: Ran upgrade scripts, verified database was populated correctly.
Configured alternate file domain, uploaded file, verified secret generated and
worked properly. Changed secret, was given 404.

Reviewers: jungejason, benmathews, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley

Differential Revision: 1036
2011-10-23 14:42:23 -07:00
epriestley
ddce177d81 Add a name token table so on-demand typeaheads can match last names
Summary: See T585. We currently don't match middle/last/nth names in on-demand
tokenizers. Build a table so we can match them.

Test Plan:
Ran upgrade script, verified table looks sensible. Searched for "priestley" in a
tokenizer, got a bunch of test account hits.

  mysql> select * from user_nametoken;
  +-------------------+--------+
  | token             | userID |
  +-------------------+--------+
  | evan              |      1 |
  | priestley         |      1 |
  | epriestley        |      1 |
  | epriestley2       |      2 |
  | ducks             |      4 |
  | epriestley3       |      4 |
  | asdf              |      6 |
  | epriestley99      |      6 |
  ...

Reviewers: bh, nh, jungejason, tuomaspelkonen, aran

Reviewed By: aran

CC: aran

Differential Revision: 1034
2011-10-23 14:25:26 -07:00
epriestley
9a4bb3901e Allow bugs@ addresses to blanket-accept tasks
Summary: Allow configuration of a default author for bugs@ emails which don't
correspond to a known system user.

Test Plan: Configured a default author, sent some mails from nonsense addresses,
tasks were created.

Reviewers: davidreuss, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley, ide

Differential Revision: 1013
2011-10-20 14:26:19 -07:00
Marek Sapota
cb0700a99c Allow databases to be created outside upgrade_schema scrpt.
Summary:
Current scripts make it hard to administer Phabricator instance while not having
direct (priviledged) access to the database.  This change allows scenario where
DB administrator creates the databases for you before you run update_schema
script.

Test Plan:
Create the databases before running the update_schema script - it shouldn't
complain that the databases already exist.

Reviewers: aran, epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1014
2011-10-17 13:59:56 -07:00
epriestley
254f606e89 Tie all the pieces for symbol cross-references together
Summary:
This makes symbol cross-references work in Differential. You need to do a little
legwork but I'll document that once the change has baked for a little while.

Basically:

  - Projects are annotated with indexed languages, and "shared library" projects
(for example, symbols in Phabricator should be searched for in Arcanist and
libphutil).
  - When we render a changeset, we check if its language is an indexed one. If
it is, we invoke the decorator Javascript.
  - The Javascript takes you to a lookup page, which either gives you a list of
matching symbols (if several match) or redirects you instantly to the
definition.

Test Plan: Clicked class and function symbols in a diff, got jumped into
sensible sorts of places in Diffusion.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 980
2011-10-09 17:58:17 -07:00
epriestley
016b060aea Add a relation table for Revisions to local commit hashes
Summary:
This allows us to performantly query for diffs related to a given local hash.
Immediate applications are:

  - Commit detection in Mercurial and Git-Immutable workflows.
  - Some async unit test stuff @mgummelt was doing.

Test Plan:
Diffed locally under SVN/Git/hg, checked the table, got sensible output.

  mysql> select * from differential_revisionhash;
  +------------+------+------------------------------------------+
  | revisionID | type | hash                                     |
  +------------+------+------------------------------------------+
  |         40 | gtcm | 8c6fb2f95598a50f7aac64a5f4cc6c12b5db42f5 |
  |         40 | gttr | 54710e361a465f4ff39565a93b2a221b6e7dd07c |
  |         41 | hgcm | c29cb69aec14                             |
  |         41 | hgcm | e7309be4eabb                             |
  |         41 | hgcm | 4e885caeff60                             |
  |         41 | hgcm | 213ee1cd30ea                             |
  |         41 | hgcm | b4050fb3490f                             |
  |         41 | hgcm | 72a76bd7ffa2                             |
  |         41 | hgcm | 06c2687e63fb                             |
  |         41 | hgcm | 2b464bde6b48                             |
  +------------+------+------------------------------------------+
  10 rows in set (0.00 sec)

NOTE: Mercurial hashes are short-form but I'll shoot out a separate Arcanist
diff to fix this.

Reviewers: Makinde, fratrik, mgummelt, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 961
2011-09-26 15:02:37 -07:00
epriestley
43a3f4d234 Build an "affected path" index when attaching diffs to revisions
Summary: See T262. This creates the index on the Differential side which we need in order to execute this query efficiently on the Diffusion side.

Also renames "DiffusionGitPathIDQuery" to "DiffusionPathIDQuery", this query object has nothing to do with git.

Test Plan: Attached top-level and sub-level diffs to revisions and verified they populated the table with sensible data.

Reviewers: bmaurer, aravindn, fmoo, jungejason, nh, tuomaspelkonen, aran

CC:

Differential Revision: 931
2011-09-15 07:45:14 -07:00
epriestley
cd05c960ff Add storage for repository symbol tracking
Summary: See T315 for an extensive description of this feature. Adds the
descibed storage table.

Test Plan: Used phpsh to read/write symbol objects.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: tuomaspelkonen

CC: aran, epriestley, tuomaspelkonen

Differential Revision: 897
2011-09-13 08:49:44 -07:00
epriestley
e875c81f6d Remove blameRevision and revertPlan from the DifferentialRevision schema
Summary:
These fields use auxiliary storage now. Migrate the data and get rid of the
columns in the main table.

  - This might take a little while to run, although there are <500k rows so
probably not too long.
  - Maybe grab a backup of the table first, if I screwed something up this will
delete the data in these fields.

Test Plan:
  - Ran migration locally.
  - Browsed Differential.
  - Grepped for "revertPlan" and "blameRevision".

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason, epriestley

Differential Revision: 832
2011-09-04 16:19:12 -07:00
epriestley
69445222f7 Track content sources (email, web, conduit, mobile) for replies
Summary:
When an object is updated, record the content source for the update. This mostly
isn't terribly useful but one concrete thing I want to do with it is let admins
audit via-email replies more easily since there are a bunch of options which let
you do hyjinx if you intentionally configure them insecurely. I think having a
little more auditability around this feature is generally good. At some point
I'm going to turn this into a link admins can click to see details.

It also allows us to see how frequently different mechanisms are used, and lets
you see if someone is at their desk or on a mobile or whatever, at least
indirectly.

The "tablet" and "mobile" sources are currently unused but I figured I'd throw
them in anyway. SMS support should definitely happen at some point.

Not 100% sure about the design for this, I might change it to plain text at some
point.

Test Plan: Updated objects and saw update sources rendered.

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, epriestley, jungejason

Differential Revision: 844
2011-08-30 11:08:27 -07:00
epriestley
dd74903cae Add basic auxiliary field storage for Differential
Summary:
Precursor to building this out to solve T343. This is similar to the Maniphest
fields we landed recently, although I think they're dissimilar enough that it
isn't worth going crazy trying to make them share code, at least for now.

This doesn't really do anything yet, just adds a storage object and a couple of
selector/field indirection classes.

Test Plan: Ran SQL upgrade script, created an aux field.
Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 798
2011-08-14 10:04:21 -07:00
epriestley
879431fb50 Improve GC performance for Herald Transcripts
Summary: This has to table scan a ginormous table right now, give it a fighting
chance with a more usable key.
Test Plan:
  - Launched GC daemon, no errors.
  - Used test console to create a new transcript.
  - Viewed some old transcripts.
  - Ran EXPLAIN on the SELECT and verified it was utilizing the garbageCollected
key.

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, epriestley, jungejason
Differential Revision: 735
2011-07-28 18:50:54 -07:00
hunterbridges
aeae33b7d6 Key Value Store for ManiphestTask
Test Plan: Look at a task detail. Some dummy attributes are automatically added in ManiphestTaskDetailController.
Reviewed By: epriestley

Differential Revision: 730
2011-07-25 19:11:55 -07:00
epriestley
6e08a9215d Move "Preferences" to "Settings"
Summary:
It makes more sense to just make this a settings panel rather than a standalone
app, particularly since setting panels are relatively well separated now.

Also default-disabled the SSH Keys interface since it won't currently be useful
for most installs.

Test Plan: Edited preferences.
Reviewed By: jungejason
Reviewers: tuomaspelkonen, jungejason, aran
CC: aran, jungejason
Differential Revision: 716
2011-07-24 12:25:43 -07:00
Hua Wang
2a894438ae Add a 'description' field to Phriction
Summary:
Add a new column to PhrictionContent called 'comment' or 'description' or
something
        Add an optional field to the Phriction document editing interface that
allows you to add a comment

Test Plan:
Run the sql patch to modify phriction_content table
           Create a new wiki page in Phriction
           Type in words in description field and save the page
           Visit this page and click "Edit Page" button
           The content in the desciption field is saved

Reviewed By: epriestley
Reviewers: epriestley, hsb, codeblock
Commenters: codeblock
CC: aran, codeblock, hwang, epriestley
Differential Revision: 709
2011-07-23 21:11:42 -07:00
epriestley
8df62d5352 Allow users to associate SSH Public Keys with their accounts
Summary:
With the sshd-vcs thing I hacked together, this will enable Phabricator to host
repositories without requiring users to have SSH accounts.

I also fixed "subporjects" and added an explicit ENGINE to it.

Test Plan: Created, edited and deleted public keys. Attempted to add the same
public key twice. Attempted to add invalid and unnamed public keys.
Reviewed By: aran
Reviewers: jungejason, tuomaspelkonen, aran, cadamo, codeblock
CC: aran, epriestley
Differential Revision: 711
2011-07-23 09:15:20 -07:00
Cristian Adamo
8cca3079fc Add subprojects to a main project.
Summary:
Quite basic subproject attachment.

Test Plan:
Go to some proj
ect and add another as a subproject.
Play around the ui.

Reviewers:	epr
iestley
CC:
2011-07-19 15:50:15 -03:00
Ricky Elrod
95d6e0a3c4 Store parents of forked pastes, and list child pastes if there are any.
Summary:
Added a 'parent' field which stores a PHID of another paste. If it is not NULL show a list of children pastes on view.
Also did some misc. refactoring to clean up the code a bit, specifically in the Create controller.

Test Plan:
- Checked old pastes, they were not affected.
- Added a paste, successfully.
- Forked it, successfully.
- Went to the original paste, saw the child paste listed.
- Forked it again, saw the new one added to the list.

Reviewers:
epriestley

CC:

Differential Revision: 672
2011-07-15 18:42:08 -04:00
epriestley
4c44c44964 Add Phriction to the main nav menu
Summary:
A few minor things:

  - Add Phriction to the homepage.
  - Link to remarkup reference on the edit screen.
  - Add a help tab with a super basic help document.

Test Plan: Cursory inspection.
Reviewed By: codeblock
Reviewers: hsb, codeblock, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, codeblock
Differential Revision: 655
2011-07-12 09:26:51 -07:00
epriestley
5704b2bc70 Basic edit/create workflow for Phriction
Summary:
This is another chunk of D636, I just simplified it a bit and added slugs.

When you go to a page like /w/pokemon/, it allows you to create or edit the
page.

Title vs slug stuff is a little funky but I think mostly-reasonable.

Test Plan: Created and edited /w/, /w/pokemon/, etc.
Reviewed By: hsb
Reviewers: hsb, codeblock, jungejason, aran, tuomaspelkonen
CC: aran, hsb
Differential Revision: 643
2011-07-11 14:47:33 -07:00
epriestley
f95913ec47 Phriction basics
Summary:
Basically a copy/paste of parts of D636, but with two changes:

  - Fully separate the index table ("document") from the content table
("content"). I think this will be a cleaner solution in the long run.
  - Build slugs into the document structure.

This doesn't do anything useful, it just normalizes slugs and lays some
groundwork.

Test Plan:
  - Visited various /w/ pages and saw them normalize correctly.
  - Verified the DAO works by inserting dummy rows.

Reviewed By: codeblock
Reviewers: hsb, codeblock, jungejason, aran, tuomaspelkonen
CC: aran, codeblock, epriestley
Differential Revision: 638
2011-07-11 11:42:50 -07:00
epriestley
467308dd12 Script to selectively convert MyISAM tables to InnoDB
Summary:
A bunch of the .sql patch files don't explicitly specify the table engine, but
we should always use InnoDB with the exception of one table which needs MyISAM
for FULLTEXT.

MySQL doesn't no-op an ALTER TABLE statment that changes the engine back to
itself and converting large tables can be time consuming, so convert only the
required tables.

Test Plan: Ran on secure.phabricator.com and my local box, it fixed all the
issues in about 3 seconds on secure.phabricator.com and <<1 second on my local.
Reviewed By: codeblock
Reviewers: codeblock, tuomaspelkonen, jungejason, aran
CC: aran, epriestley, codeblock
Differential Revision: 641
2011-07-11 11:42:28 -07:00
epriestley
233a2055f1 Add missing keys to some tables
Summary: I wrote a little query plan plugin for "Services", it's not ready yet
but it found a few cases where we're doing table scans.
Test Plan:
https://secure.phabricator.com/file/view/PHID-FILE-374fd7e846a0ce27d5fb/
https://secure.phabricator.com/file/view/PHID-FILE-2f680de9b8c3c3b7089e/
https://secure.phabricator.com/file/view/PHID-FILE-742f645fcf011252ff25/

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 627
2011-07-09 10:55:15 -07:00
epriestley
51c2726a34 Add Differential parse cache to the GC daemon
Summary:
Add the differential parse cache to the GC. This is the largest object in the
system by a wide margin, I think.

This table is potentially gigantic which is why the script truncates it before
doing a schema change.

Test Plan: Ran the GC daemon, it cleaned up some parse caches.
Reviewed By: jungejason
Reviewers: tuomaspelkonen, jungejason, aran
Commenters: tuomaspelkonen
CC: aran, jungejason, tuomaspelkonen, epriestley
Differential Revision: 620
2011-07-08 17:31:25 -07:00