1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-24 15:52:41 +01:00
Commit graph

452 commits

Author SHA1 Message Date
epriestley
d1983560a6 Show when objects have a non-default policy
Summary:
Fixes T6787. I'm kind of cheating a little bit here by not unifying default selection with `initializeNew(...)` methods, but I figure we can let this settle for a bit and then go do that later. It's pretty minor.

Since we're not doing templates I kind of want to swap the `'template'` key to `'type'` so maybe I'll do that too at some point.

@chad, freel free to change these, I was just trying to make them pretty obvious. I //do// think it's good for them to stand out, but my approach is probably a bit inconsistent/heavy-handed in the new design.

Test Plan:
{F525024}

{F525025}

{F525026}

{F525027}

Reviewers: btrahan, chad

Reviewed By: btrahan

Subscribers: johnny-bit, joshuaspence, chad, epriestley

Maniphest Tasks: T6787

Differential Revision: https://secure.phabricator.com/D13387
2015-06-22 11:46:59 -07:00
epriestley
bb58a123e6 Modularize Celerity postprocessors
Summary: Not sure if we want this, but it seems to work fine.

Test Plan: {F516736}

Reviewers: joshuaspence, chad

Reviewed By: joshuaspence, chad

Subscribers: joshuaspence, epriestley

Differential Revision: https://secure.phabricator.com/D13363
2015-06-20 06:10:42 -07:00
Chad Little
801607381d [Redesign] PhabricatorApplicationSearchResultView
Summary: Ref T8099. This adds a new class which all search engines return for layout. I thought about this a number of ways, and I think this is the cleanest path. Each Engine can return whatever UI bits they needs, and AppSearch or Dashboard picks and lays the bits out as needed. In the AppSearch case, interfaces like Notifications, Calendar, Legalpad all need more custom layouts. I think this also leaves a resonable path forward for NUX as well. Also, not sure I implemented the class correctly, but assume thats easy to fix?

Test Plan: Review and do a search in each application changed. Grep for all call sites.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T8099

Differential Revision: https://secure.phabricator.com/D13332
2015-06-19 11:46:20 +01:00
epriestley
53ef057b1b Merge branch 'master' into redesign-2015 2015-06-15 08:06:23 -07:00
Joshua Spence
1239cfdeaf Add a bunch of tests for subclass implementations
Summary: Add a bunch of tests to ensure that subclasses behave.

Test Plan: `arc unit`

Reviewers: eadler, #blessed_reviewers, epriestley

Reviewed By: eadler, #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13272
2015-06-15 18:13:27 +10:00
Joshua Spence
b6d745b666 Extend from Phobject
Summary: All classes should extend from some other class. See D13275 for some explanation.

Test Plan: `arc unit`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D13283
2015-06-15 18:02:27 +10:00
epriestley
52f8756c3c Add a "template" parameter to application default policies
Summary:
Ref T5681. Ref T6860. This doesn't do anything interesting on its own, just makes the next diff smaller.

In the next diff, policies become aware of the types of objects they're acting on. We need to specify which object type all the "Default View/Edit" settings are for so they get the right rules.

For example, a rule like "Allow task author" is OK for "View Policy" on a task, and also OK for "Default View Policy" on ManiphestApplication. But it's not OK for "Can Create Tasks" on ManiphestApplication.

So annotate all the "template"/"default" policies with their types. The next diff will use these to let you select appropriate rules for the given object type.

Test Plan:
  - Used `grep` to find these.
  - This change has no effect.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5681, T6860

Differential Revision: https://secure.phabricator.com/D13251
2015-06-11 13:25:30 -07:00
epriestley
57b898af9a Merge branch 'master' into redesign-2015 2015-06-10 07:44:58 -07:00
epriestley
7f5919de1c Don't load attached files for profile images
Summary:
Ref T8478. I think the cycle is:

  - Conpherence Thread > Loads handle for participant > loads file for profile image > loads attached files to check visibility > loads conpherence thread > ...

So, specifically, someone attached their profile image to a thread or message somewhere.

This breaks the cycle by stopping the attached-files visibility check from happening, since we don't need it. This seemed like the easiest link in the chain to break.

//Ideally//, I think the longer-term and more complete fix here is to stop Conpherence from requiring handles in order to load thread handles (and, generally, having a "handles must not load other handles" rule), but that's not trivial and might not be especially practical.

Test Plan: Will test in production.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8478

Differential Revision: https://secure.phabricator.com/D13216
2015-06-08 15:39:47 -07:00
Joshua Spence
af1b586990 Fix method visibilities
Summary: See also D13186.

Test Plan: Ran `arc unit --everything`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D13201
2015-06-09 07:17:44 +10:00
epriestley
bf87976d25 Support ordering in SearchField
Summary:
Ref T8441. Ref T7715. Automatically generate a modern "Order" control in ApplicationSearch for engines which fully support SearchField.

Notably, this allows the standard "Order" control to automatically support custom field orders. We do this in Maniphest today, but in an ad-hoc way.

Test Plan: Performed order-by queries in Almanac (Services), Pholio, Files, People, Projects, and Paste.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13193
2015-06-08 12:21:48 -07:00
epriestley
3cdaf52ce9 Make Subscribers automatically provide working SearchFields
Summary:
Ref T8441. Ref T7715. For modern Query classes, automatically make subscriber queries and SearchField integrations work.

In particular, we can just drive this query with EdgeLogic and don't need to do anything specific on these Query classes beyond making sure they're implemented in a way that picks up all of the EdgeLogic clauses.

Test Plan:
  - Searched for subscribers in Pholio, Files, Paste, and Projects.
  - Searched for all other fields in Projects to check that Query changes are OK.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13191
2015-06-08 12:20:53 -07:00
epriestley
cdef3e8bc8 Convert Files to SearchFields
Summary:
Ref T8441. Ref T7715.

  - Update FileSearchEngine.
  - Nothing too special/fancy.

Test Plan:
  - Searched for various files.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7715, T8441

Differential Revision: https://secure.phabricator.com/D13175
2015-06-07 07:32:09 -07:00
epriestley
bfca11dbba Merge branch 'master' into redesign-2015 2015-05-22 12:57:32 -07:00
Joshua Spence
36e2d02d6e phtize all the things
Summary: `pht`ize a whole bunch of strings in rP.

Test Plan: Intense eyeballing.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12797
2015-05-22 21:16:39 +10:00
epriestley
d70ca6b7c8 When file transforms race and lose, accept defeat gracefully
Summary: Fixes T8277. Transforming files can race; resolve the race after we lose.

Test Plan:
  - Added `sleep(10)` near the bottom of the transform controller.
  - Transformed a file in two browser windows at the same time; got something like this (exception corresponds to the loser of the race):

{F412526}

  - Applied patch.
  - Repeated process, got this:

{F412527}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T8277

Differential Revision: https://secure.phabricator.com/D12965
2015-05-21 09:42:20 -07:00
epriestley
f99c7beb90 Fully remove all the public-create-mail settings
Summary: Fixes T5703. These have been unused in production for a while and the new stuff seems good.

Test Plan: Mostly `grep`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5703

Differential Revision: https://secure.phabricator.com/D12949
2015-05-20 14:21:33 -07:00
Chad Little
a4784e03ff [Redesign] Add Table, Collapse support to ObjectBox
Summary: Converts most all tables to be directly set via `setTable` to an ObjectBox. I think this path is more flexible design wise, as we can change the box based on children, and not just CSS. We also already do this with PropertyList, Forms, ObjectList, and Header. `setCollapsed` is added to ObjectBox to all children objects to bleed to the edges (like diffs).

Test Plan: I did a grep of `appendChild($table)` as well as searches for `PHUIObjectBoxView`, also with manual opening of hundreds of files. I'm sure I missed 5-8 places. If you just appendChild($table) nothing breaks, it just looks a little funny.

Reviewers: epriestley, btrahan

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12955
2015-05-20 12:48:43 -07:00
Joshua Spence
c896aeb62e Various linter fixes
Summary: Apply various linter fixes.

Test Plan: Unit tests + eyeballing.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12390
2015-05-20 07:27:41 +10:00
Joshua Spence
16a8ed72bd Modernize search engine selection
Summary: Remove the `PhabricatorDefaultSearchEngineSelector` class. This is quite similar to D12053.

Test Plan: Went to `/view/PhabricatorSearchApplication/` and saw the storage engine configuration. Set `search.elastic.host` and saw the highlighted storage engine change.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12670
2015-05-20 06:59:59 +10:00
Bob Trahan
81a475d5a6 Transactions - make implementing TYPE_XXXX_POLICY transactions optional
Summary: Ref T6403. This was actually simple stuff.

Test Plan: changed the edit policy of a paste. changed the edit and join policy of a phame blog.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6403

Differential Revision: https://secure.phabricator.com/D12933
2015-05-19 12:58:18 -07:00
Bob Trahan
3ef0721ada Reduce PhabricatorUser::getOmnipotentUser calls by adding a getViewer method to PhbaricatorDestructionEngine
Summary:
Fixes T6956. Before this change, we called PhabricatorUser::getOmnipotentUser in the various delete methods to query the data. Now, we use $engine->getViewer(), since its always a good thing to have less calls to PhabricatorUser::getOmnipotentUser thrown around the codebase.

I used the "codemod" tool to audit the existing calls to PhabricatorDestructorEngine (all of them) so ostensibly this gets all the spots. If I missed something though, its still going to work, so this change is very low risk.

Test Plan: ./bin/remove destroy P1; visit P1 and get a 404

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6956

Differential Revision: https://secure.phabricator.com/D12866
2015-05-15 14:07:17 -07:00
Joshua Spence
61b178f44e Use PhutilInvalidStateException
Summary: Use `PhutilInvalidStateException`. Depends on D12803.

Test Plan: Unit tests pass.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12829
2015-05-14 07:53:52 +10:00
Joshua Spence
acb45968d8 Use __CLASS__ instead of hard-coding class names
Summary: Use `__CLASS__` instead of hard-coding class names. Depends on D12605.

Test Plan: Eyeball it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12806
2015-05-14 07:21:13 +10:00
epriestley
b36a8fa885 Implement a user profile image cache
Summary:
Ref T7707. The general form of this can probably be refined somewhat over time as we have more use cases.

I put this cache on the user object itself because we essentially always need this data and it's trivial to invalidate the cache (we can do it implicilty during reads).

Also fix an issue with short, wide images not thumbnailing properly after recent changes.

Test Plan:
  - Loaded some pages; saw caches write; saw good pictures.
  - Reloaded; saw cache reads; saw good pictures.
  - Changed profile picture; saw immediate update.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7707

Differential Revision: https://secure.phabricator.com/D12826
2015-05-13 11:38:51 -07:00
epriestley
47b14c9bde Convert inline profile image transforms to new transformations
Summary:
Ref T7707. Fixes T7879. Fixes T4406. When creating profile images:

  - Use the new transforms;
  - mark them as "profile" images so they're forced to the most-open policies.

Test Plan:
  - Set restrictive default file policies.
  - Changed profile picture, project pictures, etc. Verified they were visible to logged-out users.
  - Registered via OAuth.
  - Updated a Conpherence thread image.
  - Browsed around looking for profile images, fixed sizing on everything I could find.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7879, T7707, T4406

Differential Revision: https://secure.phabricator.com/D12821
2015-05-13 11:38:46 -07:00
epriestley
7e365eb8ae Convert "profile" image transforms to the new pathway
Summary:
Ref T7707. This ends up being sort of complicated: to support 100x100 images in T4406, we need to scale small images //up// so they look OK when we scale them back down with `background-size` in CSS.

The rest of it is mostly straightforward.

Test Plan:
  - Did an OAuth handshake and saw a scaled-up, scaled-down profile picture that looked correct.
  - Used Pholio, edited pholio, embedded pholio.
  - Uploaded a bunch of small/weird/big images and regenerated all their transforms.
  - Uploaded some text files into Pholio.
  - Grepped for removed methods, etc.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7707

Differential Revision: https://secure.phabricator.com/D12818
2015-05-13 11:38:46 -07:00
epriestley
75f6211233 Convert "preview" image transforms to new pathway
Summary: Ref T7707. Move the 220px (file uploads) and 100px (Pholio thumbgrid) previews over to the new stuff.

Test Plan: Uploaded a bunch of images to remarkup and Pholio; they generated reasonable results in the web UI.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7707

Differential Revision: https://secure.phabricator.com/D12814
2015-05-13 11:38:46 -07:00
epriestley
200e525df1 Support imagemagick on new image transform pathway
Summary: Ref T7707. For animated GIFs, use imagemagick if it is available.

Test Plan: Generated small versions of a bunch of different GIFs.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7707

Differential Revision: https://secure.phabricator.com/D12813
2015-05-13 11:38:46 -07:00
epriestley
fd1e0bc4d3 Implement error-checked "preview" transforms
Summary: Ref T7707. These transforms have a single maximum dimension instead of fixed X and Y dimensions.

Test Plan: Transformed a bunch of files with different sizes/aspect ratios, got sensible results.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7707

Differential Revision: https://secure.phabricator.com/D12812
2015-05-13 11:38:45 -07:00
epriestley
2bdb5404c7 Implement new profile transform with amazing "error handling" feature
Summary:
Ref T7707. Ref T4406. Ref T2479. This implements the profile-style (fixed width and height) transforms in a modern way.

  - Added a "regnerate" feature to the support UI to make testing easier and surface errors.
  - Laboriously check errors from everything.
  - Fix the profile thumbnailing so it crops properly instead of leaving margins.
  - Also defuses the "gigantic white PNG" attack.

This doesn't handle the imagemagick case (for animated GIFs) yet.

Test Plan:
  - Uploaded a variety of wide/narrow/small/large files and converted them into sensible profile pictures.
  - Tried to thumbnail some text files.
  - Set the pixel-size and file-size limits artificially small and hit them.
  - Used "regenerate" a bunch while testing the rest of this stuff.
  - Verified that non-regenerate flows still produce a default/placeholder image.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4406, T2479, T7707

Differential Revision: https://secure.phabricator.com/D12811
2015-05-13 11:38:45 -07:00
epriestley
a19b0029e2 Fix bad aspect ratio on some file previews
Summary: Ref T7707. Fixes T4724. I misread the report on T4724; this is trivial. We're just reading the wrong properties in setting "width" and "height" attributes, the actual thumbnailing logic is fine.

Test Plan: Uploaded image from T4724, saw it have a proper aspect ratio.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: spicyj, epriestley

Maniphest Tasks: T4724, T7707

Differential Revision: https://secure.phabricator.com/D12810
2015-05-13 11:38:45 -07:00
epriestley
65ff40844b Make modular transforms handle exceptions gracefully
Summary:
Ref T7707. Ref T2479. Ref T5258.

The thumbnailing code is some of the only code in the codebase which doesn't use exceptions to handle errors. I'm going to convert it to use exceptions; make sure they do something reasonable at top level.

Strategy here is:

  - By default, we just fall back to a placeholder image if anything goes wrong.
  - Later, I'll likely add a "debug" workflow from the new "Transforms" UI which will surface the specific exception instead (the code can't really raise any interesting exceptions right now).

Test Plan: Faked an exception and saw some reasonable default images.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5258, T2479, T7707

Differential Revision: https://secure.phabricator.com/D12809
2015-05-13 11:38:45 -07:00
epriestley
c998e44b5a Modularize file transforms and provide a "transforms" UI
Summary:
Ref T7707. Available transforms are currently relatively hard-coded and don't really have any support UI.

Modularize them so we can build some support UI.

This doesn't actually //use// any of the new stuff yet: I want to make a clean cutover once I fix the aspect ratio stuff so I can pick up a cachekey/URI change as a side effect.

Test Plan: {F400524}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: chad, epriestley

Maniphest Tasks: T7707

Differential Revision: https://secure.phabricator.com/D12808
2015-05-13 11:38:45 -07:00
epriestley
ae32d1afb8 Remove unused file transforms
Summary:
Ref T7707.

  - Modernize the file transform endpoint a bit.
  - Delete transforms which are no longer used in the product.

Test Plan:
  - Used Pholio (navigation, inline thumbs).
  - Uploaded images (embed thumb).
  - Changed profile picture (profile thumb).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7707

Differential Revision: https://secure.phabricator.com/D12807
2015-05-13 11:38:45 -07:00
Joshua Spence
4ea9d76f66 Add some missing type hints
Summary: Add some typehints for Remarkup rules.

Test Plan: Browsed around some Remarkup text.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12694
2015-05-05 07:33:00 +10:00
lkassianik
55ff197f2a File names should be editable.
Summary: Fixes T7480, File names should be editable and the event should show up in feed.

Test Plan: Upload a file, view file details, edit file, change file name by adding a space and a word to the name, save changes, file name should retain space and not normalize the name, file details should show the edit event, install feed should correctly show an event for the action.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T7480

Differential Revision: https://secure.phabricator.com/D12561
2015-04-26 15:24:29 -07:00
Bob Trahan
e40aa8f782 Quicksand - make things work correctly with global drag and drop upload
Summary: Fixes T7685. This required making the global drag and drop behavior able to "uninstall" itself so to speak, and then it re-installs it self as necessary.

Test Plan:
Did the following all successfully

 - uploaded a file to homepage
 - homepage -> differential -- no way to upload via drag and drop
 - homepage -> differential -> homepage -- uploaded a file
 - homepage -> differential -> browser back button to homepage -- uploaded a file

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7685

Differential Revision: https://secure.phabricator.com/D12534
2015-04-23 15:08:35 -07:00
epriestley
55e49d7e31 Provide more buildXClause() and buildXClauseParts() on PolicyAwareQuery
Summary:
Ref T4100. Ref T5595. These functions are trivial for now, but move us toward being able to define more default query behavior by default.

Future changes will give these methods meaningful, nontrivial behaviors.

Test Plan: `arc unit --everything`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5595, T4100

Differential Revision: https://secure.phabricator.com/D12454
2015-04-20 10:06:10 -07:00
epriestley
f5580c7a08 Make buildWhereClause() a method of AphrontCursorPagedPolicyAwareQuery
Summary:
Ref T4100. Ref T5595.

To support a unified "Projects:" query across all applications, a future diff is going to add a set of "Edge Logic" capabilities to `PolicyAwareQuery` which write the required SELECT, JOIN, WHERE, HAVING and GROUP clauses for you.

With the addition of "Edge Logic", we'll have three systems which may need to build components of query claues: ordering/paging, customfields/applicationsearch, and edge logic.

For most clauses, queries don't currently call into the parent explicitly to get default components. I want to move more query construction logic up the class tree so it can be shared.

For most methods, this isn't a problem, but many subclasses define a `buildWhereClause()`. Make all such definitions protected and consistent.

This causes no behavioral changes.

Test Plan: Ran `arc unit --everything`, which does a pretty through job of verifying this statically.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: yelirekim, hach-que, epriestley

Maniphest Tasks: T4100, T5595

Differential Revision: https://secure.phabricator.com/D12453
2015-04-20 10:06:09 -07:00
epriestley
156b156e77 Give Conduit params/return/errors protected visibility
Summary:
Ref T7803. Ref T5873. I want to drive Conduit through more shared infrastructure, but can't currently add parameters automatically.

Put a `getX()` around the `defineX()` methods so the parent can provide default behaviors.

Also like 60% of methods don't define any special error types; don't require them to implement this method. I want to move away from this in general.

Test Plan:
  - Ran `arc unit --everything`.
  - Called `conduit.query`.
  - Browsed Conduit UI.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: hach-que, epriestley

Maniphest Tasks: T5873, T7803

Differential Revision: https://secure.phabricator.com/D12380
2015-04-13 11:58:35 -07:00
epriestley
4fba6e7730 Remove trivial implementations of getPagingColumn()
Summary:
Ref T7803. Some Query subclasses implement getPagingColumn() in a trivial way, usually to provide a table alias.

Formalize the concept of a primary table alias, and remove obsoleted getPagingColumn() implementations.

Test Plan: Issued affected queries.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7803

Differential Revision: https://secure.phabricator.com/D12356
2015-04-13 11:58:19 -07:00
Joshua Spence
ea376685ae Fix some odd looking arrays
Summary: These arrays looks a little odd, most likely due to the autofix applied by `ArcanistXHPASTLinter::LINT_ARRAY_SEPARATOR`. See D12296 in which I attempt to improve the autocorrection from this linter rule.

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D12281
2015-04-05 22:29:39 +10:00
epriestley
eb81fd1562 Expose all application mail receivers
Summary:
Fixes T7199. This still isn't a shining example of perfect code, but the raw amount of copy/paste is much lower than it used to be.

  - Reduce code duplication between existing receivers.
  - Expose receiving objects in help menus where appropriate.
  - Connect some "TODO" receivers.

Test Plan:
  - Sent mail to every supported object type.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12249
2015-04-01 11:52:02 -07:00
epriestley
c32fee0e48 Fully modularize mail commands
Summary: Ref T7199. Everyone can have a mail command! You can have a mail command! You can have a mail command! Mail commands for everyone!

Test Plan: Used `bin/mail receive-test` to issue commands against files and pastes.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12238
2015-04-01 08:40:00 -07:00
epriestley
7c5f71b691 Subclass most ReplyHandlers from TransactionReplyHandler
Summary: Ref T7199. Half of these aren't even reachable, but make some progress toward reducing the amount of nonsense and garbage in mail handling.

Test Plan: Tested all reachable handlers with `bin/mail receive-test`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12237
2015-04-01 08:39:50 -07:00
epriestley
0d99c84bd7 Modernize email command parsing
Summary:
Ref T7199. This prepares for an exciting new world of more powerful "!action" commands. In particular:

  - We parse multiple commands per mail.
  - We parse command arguments (these are currently not used).
  - We parse commands at the beginning or end of mail.

Additionally:

  - Do a quick modernization pass on all handlers.
  - Break legacy compatibility with really hacky Facebook stuff (see T1992). They've theoretically been on notice for a year and a half, and their setup relies on calling very old reply handler APIs directly.
  - Some of these handlers had some copy/paste fluff.
  - The Releeph handler is unreachable, but fix it //in theory//.

Test Plan:
- Sent mail to a file; used "!unsubscribe".
- Sent mail to a legalpad document; used "!unsubscribe".
- Sent mail to a task; used various "!close", "!claim", "!assign", etc.
- Sent mail to a paste.
- Sent mail to a revision; used various "!reject", "!claim", etc.
- Tried to send mail to a pull request but it's not actually reachable.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12230
2015-03-31 16:48:27 -07:00
epriestley
030e05aa4c Remove reply handler instructions from email
Summary:
Ref T7199. Although this is useful for discovery, it's un-useful enough that we already have an option to disable it, and most applications do not provide any meaningful instructions.

Throwing it away makes it easier to move forward and lets us get rid of a config option.

This is becoming a more advanced/power-user feature anyway, and the new syntax will be significantly more complex and hard to explain with a one-liner. I'm currently thinking that I'll maybe make the "help" menu a dropdown and give it some options like:

  +---+
  | O |
  +---+---------------------+
  | Maniphest Documentation |
  | Maniphest Email Actions |
  +-------------------------+

Then you click the "Email Actions" thing and get a runtime-derived list of available options. Not sure if I'll actually build that, but I think we can fairly throw the in-mail instructions away even if we don't go in that specific direction.

Test Plan: Grepped for `replyHandlerInstructions`, got no hits.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12229
2015-03-31 16:48:17 -07:00
epriestley
7cf726c7f7 Fix an issue with FileMailReceiver not working well
Summary: Ref T7199. Guess no one has ever tried to reply to file mail.

Test Plan: Used `bin/mail receive-test` to send mail to files.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7199

Differential Revision: https://secure.phabricator.com/D12228
2015-03-31 16:47:58 -07:00
epriestley
d403700e1f Convert all tokenizers to take token/scalar inputs
Summary: Ref T7689. Ref T4100. This advances the goals of removing `loadViewerHandles()` (only 67 callsites remain!) and letting tokenizers some day take token functions like `viewer()` and `members(differential)`.

Test Plan:
- Sent a new message; used "To".
  - I simplified the cancel URI construction slightly because it's moot in all normal cases.
- Edited a thread; used "Add Participants".
- Searched rooms; used "Participants".
- Searched countdowns; used "Authors".
- Created a diff; used "Repository".
- Edited a revision; edited "Projects"; edited "Reveiwers"; edited "Subscribers".
- Searched for revisions; edited "responsible users"; "authors"; "reviwers"; "subscribers"; "repositories".
- Added revision comments; edited "Add Reveiwers"; "Add Subscribers".
- Commented on a commit; edited "Add Auditors"; "Add subscribers".
- Edited a commit; edited "Projects".
- Edited a repository; edited "Projects".
- Searched feed, used "include Users"; "include Proejcts".
- Searched files, used "authors".
- Edited initiative; edited "Projects".
- Searched backers; used "Backers".
- Searched initiatives; used "Owners".
- Edited build plans; edited "Run Command".
- Searched Herald; used "Authors".
- Added signature exemption in Legalpad.
- Searhced legalpad; used "creators"; used "contributors".
- Searched signatures; used "documents"; used "signers".
- Created meme.
- Searched macros; used "Authors".
- Used "Projects" in Maniphest reports.
- Used Maniphest comment actions.
- Edited Maniphest tasks; edited "Assigned To"; edited "CC"; edited "projects".
- Used "parent" in Maniphest task creation workflow.
- Searched for projects; used "assigned to"; "in any projec"; "in all projects"; "not in projects"; "in users' projects"; "authors"; "subscribers".
- Edited Maniphest bug filing domains, used "Default Author".
- Searched for OAuth applications, used "Creators".
- Edited Owners pacakge; edited "Primary Owner"; edited "Owners".
- Searched for Owners packages; used "Owner".
  - OMG this UI is OLD
- Edited a paste; edited "Projects".
- Searched for paste; used "Authors".
- Searched user activity log; used "Actors"; used "Users".
- Edited a mock; edited "Projects"; edited "CC".
- Searched for mocks; used "Authors".
- Edited Phortune account; edited "Members".
- Edited Phortune merchant account; edited "Members".
- Searched Phrequent; used "Users".
- Edited Ponder question; sued "projects".
- Searched Ponder; used "Authors"; used "Answered By".
- Added project members.
- Searched for projects; used "Members".
- Edited a Releeph product; edited "Pushers".
- Searched pull requests; searched "Requestors".
- Edited an arcanist project; used "Uses Symbols From".
- Searhced push logs; used "Repositories"; used "Pushers".
- Searched repositories; used "In nay project".
- Used global search; used Authors/owners/Subscribers/In Any Project.
- Edited a slowvote; used "Projects".
- Searched slovotes; used "Authors".
- Created a custom "Users" field; edited and searched for it.
- Made a whole lot of typos in this list. ^^^^^^

Did not test:

- Lint is nontrivial to test locally, I'll test it in production.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4100, T7689

Differential Revision: https://secure.phabricator.com/D12224
2015-03-31 14:10:55 -07:00
epriestley
a8271ecd40 Remove most callsites to Controller->renderHandlesForPHIDs()
Summary: Ref T7689. This moves most of the easy/testable callsites off `Controller->renderHandlesForPHIDs()`.

Test Plan:
- Viewed a file; viewed author; viewed "attached" tab.
- Viewed a mock; viewed attached tasks.
- Viewed a credential; viewed "Used By".
- Viewed a paste; viewed author; viewed forks; viewed forked from.
- Viewed a dashboard; viewed panel list.
- Viewed a dashboard panel; viewed "Appears On".
- Viewed a Phortune account; viewed "Members"; viewed payment methods.
- Viewed a Phortune merchant account; viewed "Members".
- Viewed Phortune account switcher; viewed "Accounts".
  - I just removed "Members:" here since it felt kind of out-of-place anyway.
- Viewed a Phragment fragment, viewed "Latest Version", viewed "Snapshots".
- Viewed a Phargment snapshot, viewed "Fragment".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: hach-que, epriestley

Maniphest Tasks: T7689

Differential Revision: https://secure.phabricator.com/D12207
2015-03-31 05:48:19 -07:00
epriestley
40fb0f98df Mostly defuse DNS rebinding attack for outbound requests
Summary: Ref T6755. I'll add some notes there about specifics.

Test Plan:
  - Made connections to HTTP and HTTPS URIs.
  - Added some debugging code to verify that HTTP URIs were pre-resolved.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6755

Differential Revision: https://secure.phabricator.com/D12169
2015-03-26 11:12:22 -07:00
epriestley
2e72e9ff31 Rate limit outbound requests in Macros
Summary:
Ref T6755. Although we do not return response bodies, it is possible to perform crude portscanning if you can execute a DNS rebinding attack (which, for now, remains theoretical).

Limit users to 60 requests / hour to make it less feasible. This would require ~30 years to portscan all ports on a `/32` netblock.

Users who can guess that services may exist can confirm their existence more quickly than this, but if the attacker already had a very small set of candidate services it seems unlikely that portscanning would be of much use in executing the attack.

This protection should eventually be applied to T4190, too (that task also has other considerations).

Test Plan: Set rate limit very low, hit rate limit.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6755

Differential Revision: https://secure.phabricator.com/D12168
2015-03-26 11:11:52 -07:00
epriestley
6ce4044bfa Lock MIME type configuration
Summary:
Ref T6755. This mitigates an attack where you:

  - compromise an administrative account;
  - configure "text/plain" as an "image" MIME type; and
  - create a new macro sourced from a sensitive resource which is locally accessible over HTTP GET, using DNS rebinding.

You can then view the content of the resource in Files. By preventing the compromised account from reconfiguring the MIME types, the server will instead destroy the response and prevent the attacker from seeing it.

In general, these options should change very rarely, and they often sit just beyond the edge of security vulnerabilities anyway.

For example, if you ignore the warnings about an alternate file domain and elect to serve content from the primary domain, it's still somewhat difficult for an attacker to exploit the vulnerability. If they can add "text/html" or "image/svg+xml" as image MIME types, it becomes trivial. In this case not having an alternate domain is the main issue, but easy modification of this config increases risk/exposure.

Test Plan: Viewed affected config and saw that it is locked.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6755

Differential Revision: https://secure.phabricator.com/D12154
2015-03-25 10:16:22 -07:00
epriestley
4f8147dbb8 Improve protection against SSRF attacks
Summary:
Ref T6755. This improves our resistance to SSRF attacks:

  - Follow redirects manually and verify each component of the redirect chain.
  - Handle authentication provider profile picture fetches more strictly.

Test Plan:
  - Tried to download macros from various URIs which issued redirects, etc.
  - Downloaded an actual macro.
  - Went through external account workflow.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6755

Differential Revision: https://secure.phabricator.com/D12151
2015-03-24 18:49:01 -07:00
epriestley
22b2b8eb89 Fix a bad call in file chunk destruction
Summary: This signature changed at some point after I tested things and I didn't catch it.

Test Plan: Destroyed a chunked large file with `bin/remove`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D12152
2015-03-24 18:48:51 -07:00
epriestley
1c32c9b965 Improve granluarity and defaults of security.allow-outbound-http
Summary:
Ref T6755. This is a partial fix, but:

  - Allow netblocks to be blacklisted instead of making the feature all-or-nothing.
  - Default to disallow requests to all reserved private/local/special IP blocks. This should generally be a "safe" setting.
  - Explain the risks better.
  - Improve the errors rasied by Macro when failing.
  - Removed `security.allow-outbound-http`, as it is superseded by this setting and is somewhat misleading.
    - We still make outbound HTTP requests to OAuth.
    - We still make outbound HTTP requests for repositories.

From a technical perspective:

  - Separate URIs that are safe to link to or redirect to (basically, not "javascript://") from URIs that are safe to fetch (nothing in a private block).
  - Add the default blacklist.
  - Be more careful with response data in Macro fetching, and don't let the user see it if it isn't ultimately valid.

Additionally:

  - I want to do this check before pulling repositories, but that's enough of a mess that it should go in a separate diff.
  - The future implementation of T4190 needs to perform the fetch check.

Test Plan:
  - Fetched a valid macro.
  - Fetched a non-image, verified it didn't result in a viewable file.
  - Fetched a private-ip-space image, got an error.
  - Fetched a 404, got a useful-enough error without additional revealing response content (which is usually HTML anyway and not useful).
  - Fetched a bad protocol, got an error.
  - Linked to a local resource, a phriction page, a valid remote site, all worked.
  - Linked to private IP space, which worked fine (we want to let you link and redierect to other private services, just not fetch them).
  - Added and executed unit tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6755

Differential Revision: https://secure.phabricator.com/D12136
2015-03-23 10:44:03 -07:00
epriestley
b7fa55ff93 Fix improper selection of the chunk engine as a writable engine
Summary:
Fixes T7621. The engine selection code started out making sense, but didn't make as much sense by the time I was done with it.

Specifically, from the vanilla file upload, we may incorrectly try to write directly to the chunk storage engine. This is incorrect, and produces a confusing/bad error.

Make chunk storage engines explicit and don't try to do single-file one-shot writes to them.

Test Plan:
  - Tried to upload a large file with vanilla uploader, got better error message.
  - Uploaded small and large files with drag and drop.
  - Viewed {nav Files > Help/Options}.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7621

Differential Revision: https://secure.phabricator.com/D12110
2015-03-18 19:06:39 -07:00
epriestley
7a93b443c3 Make file upload policies more consistent
Summary:
Ref T7149. Currently, global drag and drop always uses the most open visibility policy on the install. This was appropriate before the application preference was introduced, but default to the application preference now.

In particular, this supports a default value of "Administrators" in the Phacility cluster.

Also simplify/clean up some code.

Test Plan:
  - Set application default policy to "Adminstrators".
  - Uploaded file via drag-and-drop, saw "administrators" policy.
  - Uploaded file via `arc upload`, saw "administrators" policy.
    - Saw better URI for a text file upload after patch.
  - Uploaded file via drag-and-drop-to-textarea, saw "only viewer" policy.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12093
2015-03-17 06:33:30 -07:00
epriestley
7482d260b0 Rewrite file documentation to be chunk-aware
Summary:
Ref T7149. We can simplify configuration somewhat by removing the upload limit setting, now that we support arbitrarily large files.

  - Merge configuration documentation.
  - Tell users to set things to at least 32MB. This is 8MB maximum one-shot file + 4x headroom. Chunk sizes are 4MB.

Test Plan:
  - Faked all the setup warnings.
  - Read documentation.
  - Uploaded some files.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12083
2015-03-15 11:37:47 -07:00
epriestley
1773af6ada Enable the chunk storage engine
Summary: Ref T7149. This works now, so enable it.

Test Plan:
  - Uploaded large and small files in Firefox, Safari and Chrome.
  - Uploaded large files with `arc upload`.
  - Stopped/resumed large files with all clients.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12079
2015-03-15 11:37:05 -07:00
epriestley
6b69bc3fbb Delete all "force chunking" file upload code
Summary: Ref T7149. This was just to make testing easier, but chunking substantially works now.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12076
2015-03-15 11:32:18 -07:00
epriestley
c19bb57730 Stream chunks when sending chunked files
Summary: Ref T7149. Return a real iterator from the Chunk engine, which processes chunks sequentially.

Test Plan:
This is a bit hard to read, but shows the underlying chunks being accessed one at a time and only some being accessed when requesting a range of a file:

```
$ ./bin/files cat F878 --trace --begin 100 --end 256
...
>>> [10] <query> SELECT * FROM `file_storageblob` WHERE `id` = 85
<<< [10] <query> 240 us
 better software.

Phabricat>>> [11] <query> SELECT * FROM `file_storageblob` WHERE `id` = 84
<<< [11] <query> 205 us
or includes applications for:

 >>> [12] <query> SELECT * FROM `file_storageblob` WHERE `id` = 83
<<< [12] <query> 226 us
 - reviewing and auditing source>>> [13] <query> SELECT * FROM `file_storageblob` WHERE `id` = 82
<<< [13] <query> 203 us
 code;
  - hosting and browsing >>> [14] <query> SELECT * FROM `file_storageblob` WHERE `id` = 81
<<< [14] <query> 231 us
repositories;
  - tracking bugs;
```

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12073
2015-03-14 08:29:30 -07:00
epriestley
81d88985a0 Prepare file responses for streaming chunks
Summary:
Ref T7149. This still buffers the whole file, but is reaaaaal close to not doing that.

Allow Responses to be streamed, and rewrite the range stuff in the FileResponse so it does not rely on having the entire content available.

Test Plan:
  - Artificially slowed down downloads, suspended/resumed them (works in chrome, not so much in Safari/Firefox?)
  - Played sounds in Safari/Chrome.
  - Viewed a bunch of pages and files in every browser.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12072
2015-03-14 08:29:12 -07:00
epriestley
2aefb43843 Support a file data iteration interface for large files
Summary: Ref T7149. A couple diffs down the line, this will let us emit chunked files without doing all the work up front or holding the entire file in RAM.

Test Plan:
(Some newlines added for clarity.)

```
$ ./bin/files cat F942
ABCDEFGHIJKLMNOPQRSTUVWXYZ
$ ./bin/files cat F942 --begin 1
BCDEFGHIJKLMNOPQRSTUVWXYZ
$ ./bin/files cat F942 --end 10
ABCDEFGHIJ
$ ./bin/files cat F942 --begin 3 --end 5
DE
$
```

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12071
2015-03-14 08:28:59 -07:00
epriestley
32d8d67535 Support resuming JS uploads of chunked files
Summary: Ref T7149. We can't compute hashes of large files efficiently, but we can resume uploads by the same author, with the same name and file size, which are only partially completed. This seems like a reasonable heuristic that is unlikely to ever misfire, even if it's a little magical.

Test Plan:
  - Forced chunking on.
  - Started uploading a chunked file.
  - Closed the browser window.
  - Dropped it into a new window.
  - Upload resumed //(!!!)//
  - Did this again.
  - Downloaded the final file, which successfully reconstructed the original file.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, chad, epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12070
2015-03-14 08:28:46 -07:00
epriestley
135280be9e Support HTML5 / Javascript chunked file uploads
Summary:
Ref T7149. This adds chunking support to drag-and-drop uploads. It never activates right now unless you hack things up, since the chunk engine is still hard-coded as disabled.

The overall approach is the same as `arc upload` in D12061, with some slight changes to the API return values to avoid a few extra HTTP calls.

Test Plan:
  - Enabled chunk engine.
  - Uploaded some READMEs in a bunch of tiny 32 byte chunks.
  - Worked out of the box in Safari, Chrome, Firefox.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12066
2015-03-13 11:30:36 -07:00
epriestley
aa4adf3ab8 Add support for partially uploaded files
Summary:
Ref T7149. This flags allocated but incomplete files and doesn't explode when trying to download them.

Files are marked complete when the last chunk is uploaded.

I added a key on `<authorPHID, isPartial>` so we can show you a list of partially uploaded files and prompt you to resume them at some point down the road.

Test Plan: Massaged debugging settings and uploaded README.md very slowly in 32b chunks. Saw the file lose its "Partial" flag when the last chunk finished.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12063
2015-03-13 11:30:24 -07:00
epriestley
6c3552f939 Add bin/files cat to print a file to stdout
Summary:
Ref T7149. This makes debugging some of this stuff a bit easier by removing the HTTP part in the middle.

Particularly, I anticipate having this stream data chunk-by-chunk in the near future.

Test Plan: Ran `files cat F23`, got output.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12062
2015-03-13 11:30:13 -07:00
epriestley
4aed453b06 Add a chunking storage engine for files
Summary:
Ref T7149. This isn't complete and isn't active yet, but does basically work. I'll shore it up in the next few diffs.

The new workflow goes like this:

> Client, file.allocate(): I'd like to upload a file with length L, metadata M, and hash H.

Then the server returns `upload` (a boolean) and `filePHID` (a PHID). These mean:

| upload | filePHID | means |
|---|---|---|
| false | false | Server can't accept file.
| false | true | File data already known, file created from hash.
| true | false | Just upload normally.
| true | true | Query chunks to start or resume a chunked upload.

All but the last case are uninteresting and work like exising uploads with `file.uploadhash` (which we can eventually deprecate).

In the last case:

> Client, file.querychunks(): Give me a list of chunks that I should upload.

This returns all the chunks for the file. Chunks have a start byte, an end byte, and a "complete" flag to indicate that the server already has the data.

Then, the client fills in chunks by sending them:

> Client, file.uploadchunk(): Here is the data for one chunk.

This stuff doesn't work yet or has some caveats:

  - I haven't tested resume much.
  - Files need an "isPartial()" flag for partial uploads, and the UI needs to respect it.
  - The JS client needs to become chunk-aware.
  - Chunk size is set crazy low to make testing easier.
  - Some debugging flags that I'll remove soon-ish.
  - Downloading works, but still streams the whole file into memory.
  - This storage engine is disabled by default (hardcoded as a unit test engine) because it's still sketchy.
  - Need some code to remove the "isParital" flag when the last chunk is uploaded.
  - Maybe do checksumming on chunks.

Test Plan:
  - Hacked up `arc upload` (see next diff) to be chunk-aware and uploaded a readme in 18 32-byte chunks. Then downloaded it. Got the same file back that I uploaded.
  - File UI now shows some basic chunk info for chunked files:

{F336434}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: joshuaspence, epriestley

Maniphest Tasks: T7149

Differential Revision: https://secure.phabricator.com/D12060
2015-03-13 11:30:02 -07:00
epriestley
e2296a0ff7 Modernize file storage engine selection
Summary:
Fixes T5843. File storage engines use a very old "selector" mechanism which makes them difficult to extend.

This mechanism predates widespread use of `PhutilSymbolLoader` to discover available implementations at runtime. Runtime discovery has generally proven more flexible and easier to use than explicit selection (although it sometimes needs more UI to support it in cases where order or enabled/disabled flags can not be directly determined).

Use a modern runtime discovery mechanism instead of an explicit selector. This might break any installs which subclassed the `Selector`, but I believe almost no such installs exist, and they'll receive a meaningful exception upon upgrading (any custom engines will no longer implement all of the required methods).

Looking forward, this modernizes infrastructure to prepare for new "virtual" chunked-storage engines, with the eventual goal of supporting very large file uploads and data import into the Phacility cluster.

This uses D12051 to add UI to make it easier to understand the state of storage engines.

Test Plan:
Used new UI panel to assess storage engines:

{F336270}

  - Uploaded a small file, saw it go to MySQL engine.
  - Uploaded a larger file, saw it go to S3 engine.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5843

Differential Revision: https://secure.phabricator.com/D12053
2015-03-12 13:28:53 -07:00
Chad Little
6a036f32b2 Move Macro image height/width to CSS
Summary: This makes macros and memes grow to 100% of their container //at most//, instead of showing a scrollbar. This is useful for overly large macros, smaller spaces like Feed and Conpherences, and Inline Comments. Fixes T7528

Test Plan: Tested a very large macro, a very large meme, and a very very tiny macro. It looks like memes get cached though, unsure if we should clean them up or just leave them

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7528

Differential Revision: https://secure.phabricator.com/D12045
2015-03-11 17:35:55 -07:00
epriestley
ed49b41e91 When redirecting to acquire file access tokens, retain 'download' parameter
Summary: Fixes T7398. Previously, we would redirect to get a token and then redirect back to make use of it, but lose "download" in the process, and thus not get the correct "Content-Disposition" header.

Test Plan: Clicked "Download" on a lightboxed file.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7398

Differential Revision: https://secure.phabricator.com/D11915
2015-03-01 12:12:45 -08:00
epriestley
d1eda610fa Identify builtin files and give them open policies
Summary:
Fixes T7379. Currently, builtin files generate with a "users" view policy even if an install is public.

Because these files TTL after 7 days, there's no migration here. Installs won't see the fix actually happen for up to 7 days after updating, though.

Test Plan:
  - Deleted a builtin.
  - Loaded projects page to regenerate it.
  - Saw new builtin had most open policy and was marked as a builtin.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7379

Differential Revision: https://secure.phabricator.com/D11917
2015-03-01 12:12:38 -08:00
epriestley
3469265e17 Improve config option documentation for Imagemagick
Summary: Fixes T7306. Fixes a typo and improves the text.

Test Plan: reading

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7306

Differential Revision: https://secure.phabricator.com/D11797
2015-02-17 15:31:20 -08:00
epriestley
3a8cd60bab When cluster.instance is defined, use it to namespace S3 objects
Summary: Ref T7163. This isn't //technically// necessary but seems generally desirable.

Test Plan: Will deploy S3 in production.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7163

Differential Revision: https://secure.phabricator.com/D11770
2015-02-16 11:30:37 -08:00
epriestley
5b1ea8c8d5 Pass instance through file transform URIs
Summary:
This makes thumbnail URIs work on instanced, CDN'd installs like Phacility cluster instances.

Some of these transforms can proabably be removed, but the underlying code to generate the transform should be cleaned up too and we have some other tasks filed elsewhere about this anyway.

Test Plan: CDN'd local install now loads thumbnails properly.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11719
2015-02-09 15:31:47 -08:00
Chad Little
ae7dc8b9d2 Add getGroup to ConfigOptions
Summary: Adds core and apps grouping to configuration options, makes it somewhat easier to browse config options.

Test Plan: Set each option, review list. Breakdown is nearly 50/50 apps/core.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11722
2015-02-09 13:10:56 -08:00
epriestley
74b860519d Remarkup: Correctly render inline embed layout
Summary:
The generated HTML is like `<p>some text <div …>…</div> more text</p>`, and HTML `<p/>` tags may not contain block content like `<div/>` tags. Browsers actually parse this as if it was `<p>some text </p><div …>…</div> more text<p></p>` (sic).

The layout CSS class already has `display: inline` set, but this is not sufficient. Browser's HTML parser doesn't care what CSS rules will be applied, it only deals with the meanings of tags.

Fixes T7201.

Test Plan:
Verify that the following displays the image inline:

`some text {Fnnn,layout=inline} more text`

Reviewers: chad, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Projects: #remarkup

Maniphest Tasks: T7201

Differential Revision: https://secure.phabricator.com/D11706
2015-02-09 07:52:46 -08:00
Bob Trahan
5a9df1a225 Policy - filter app engines where the user can't see the application from panel editing
Summary: Fixes T7118. This does the basic "filter the list" thing, though it ends up being a little manual since I guess this hasn't come up before? There is also potential weird behavior if the user was using an app and lost access to it - they will have nothing selected on edit - but I think this is actually correct behavior in this circumstance.

Test Plan:
used a user who couldn't get access to the "quick create" apps and noted that the dropdown list on dashboard panel create was missing the expected engines

ran `arc unit --everything` to verify abstract method implemented everywhere

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7118

Differential Revision: https://secure.phabricator.com/D11687
2015-02-04 15:47:48 -08:00
epriestley
e6fb1dc1e9 When an install is instanced, include the instance identifier in the URI for file data
Summary:
This allows us to CDN the cluster.

General problem is that we can't easily give each instance its own CDN URI (`giraffe.phcdn.net`) in Cloudfront, because it requires that you enumerate all aliases (and there's a limit of 100) and depends on SNI (a newish feature of SSL which allows one server to serve multiple certificates, but which doesn't have full support everywhere yet).

It's //possible// that we could eventually work around this, or use Cloudflare instead (which has a different model that seems like a slightly easier fit for CDN-domain-per-instance), but I don't want to sink a ton of work into this and want to keep things on AWS insofar as we reasonably can.

The easiest way to fix this is just to put the instance identity into URIs, then read it out when handling CDN requests. This has no effect on installs without cluster instance configuration, which is all of them except ours.

It's also slightly desirable to share this stuff, since we get to share the cache for static resources, which are always identical across instances.

So requests go from the Cloudfront gateway ("xyz.cloudfront.com") to the LB with a hard-coded instance name ("cdn.phacility.com"), which gets them routed to a balanced web machine. The web machine picks the correct instance name out of the URI, acts as that instance, and does the correct thing.

The messiest part of this is that we need "cdn.phacility.com" to be a real instance so it can serve static resources, but that's not a big deal. We have a few other hard-codes which have to be real resources for now, like we must have a merchant named "Phacility".

Test Plan:
  - Viewed files with `security.alternate-file-domain` off (i.e., no file tokens).
  - Viewed pages and files with `security.alternate-file-domain` on. Saw correct resource behavior, @isntance generation of URIs, and correct token redirect behavior for files.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11668
2015-02-03 14:55:46 -08:00
Chad Little
99292c5c6a Use icons with Config Options page
Summary: This sets an icon for each config, makes it easier to scan.

Test Plan:
Reload Config page, see all new icons

{F281089}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11619
2015-02-02 10:17:25 -08:00
Chad Little
8b06804394 Remove getIconName from all applications
Summary: Not used anymore

Test Plan: grep for 'getIconName'

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11582
2015-01-30 12:11:21 -08:00
Bob Trahan
c89dc19976 Application emails - move over paste and files
Summary: Fixes T3404 (post D11565), fixes T5952. This infrastructure has been getting deployed against Maniphest and its time to get these other two applications going on it.

Test Plan: created an email address for paste and used `./bin/mail receive-test` ; a paste was successfully created

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5952, T3404

Differential Revision: https://secure.phabricator.com/D11570
2015-01-29 14:47:32 -08:00
Chad Little
5d8bb61dde Add FontIcon bridge to AppIcons
Summary: Select a similar or better FontAwesome icon to represent each application

Test Plan: Visual inspection

Reviewers: epriestley, btrahan

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11489
2015-01-24 23:43:01 -08:00
Chad Little
3bc54c2041 Project revamp part 2: Edit
Summary:
Taking a pass at revamping the edit pages in Projects. Specifically:

 - Remove EditMainController
 - Move actions from EditMain to Profile
 - Move properties from EditMain to Profile
 - Move timeline from EditMain to Profile
 - Move Open Tasks from Profile to sidenavicon
 - Add custom icons and colors to timeline

Feel free to bang on this a bit and give feedback, feels generally correct to me.

Test Plan: Edit everything I could on various projects. Check links, timelines, actions.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11421
2015-01-19 10:14:27 -08:00
Joshua Spence
daadf95537 Fix visibility of PhutilArgumentWorkflow::didConstruct methods
Summary: Ref T6822.

Test Plan: `grep`. This method is only called from within `PhutilArgumentWorkflow::__construct`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11415
2015-01-16 07:42:07 +11:00
Joshua Spence
c2ac63e9ad Increase visibility of PhabricatorController::buildApplicationMenu methods
Summary: Ref T6822. This method needs to be `public` because it is called from `PhabricatorApplicationSearchController::buildApplicationMenu()`.

Test Plan: I wouldn't expect //increasing// method visibility to break anything.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11416
2015-01-16 07:41:26 +11:00
Joshua Spence
36b760cd8a Fix method visiblity for PhabricatorFileTestCase::getPhabricatorTestCaseConfiguration
Summary: Ref T6822.

Test Plan: `grep` for `->getPhabricatorTestCaseConfiguration(`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11366
2015-01-14 07:04:36 +11:00
Joshua Spence
d6b882a804 Fix visiblity of LiskDAO::getConfiguration()
Summary: Ref T6822.

Test Plan: `grep`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: hach-que, Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11370
2015-01-14 06:54:13 +11:00
Chad Little
790d250967 Move ActionList mobile links to better location
Summary: Ref T5752, moves mobile action menus to the object box instead of crumbs.

Test Plan: View action menus at tablet, desktop, and mobile break points. Verify clicking buttons works as expected opening menu.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T5752

Differential Revision: https://secure.phabricator.com/D11340
2015-01-12 07:24:35 -08:00
lkassianik
9853ff2cff T6856 Make Project Images that match Typeahead choices
Summary: Ref T6856, Make matching Project picture to Project icon easy.

Test Plan: Edit Project, edit Project picture, click "Use Project Icon", Project picture should now match Project icon.

Reviewers: chad, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: btrahan, Korvin, epriestley

Maniphest Tasks: T6856

Differential Revision: https://secure.phabricator.com/D11300
2015-01-11 09:40:53 -08:00
Joshua Spence
e7f8e79742 Fix method visibility for PhabricatorController subclasses
Summary: Ref T6822.

Test Plan: Visual inspection. These methods are only called from within `PhabricatorController` subclasses.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11241
2015-01-07 07:34:59 +11:00
Joshua Spence
e448386d39 Fix method visibility for PhabricatorApplicationSearchEngine methods
Summary: Ref T6822.

Test Plan: Visual inspection. These methods are only called from within the `PhabricatorApplicationSearchEngine` class.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6822

Differential Revision: https://secure.phabricator.com/D11242
2015-01-07 07:34:52 +11:00
Joshua Spence
7c2a7d0365 Modernize remaining edge types
Summary: Modernize remaining edges to subclass `PhabricatorEdgeType`. Largely based on D11045.

Test Plan: Browsed around and performed various actions include subscribing, unsubscribing and watching.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11116
2015-01-03 10:58:20 +11:00
Joshua Spence
8dee37a132 Fix some linter violations
Summary: Self-explanatory.

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11142
2015-01-03 09:11:41 +11:00
Fabian Stelzer
6132d8012b show the current size limit when a file upload fails
Summary: Show the php.ini setting for max upload or alterantively the in phabricator configured one. Fixes T6663

Test Plan: changed php.ini and alternatively phabricator file upload size settings to minimal values and try to upload a larger file

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, btrahan, chad, epriestley

Maniphest Tasks: T6663

Differential Revision: https://secure.phabricator.com/D11031
2014-12-23 05:17:16 -08:00
Chad Little
b925f42c55 Remove docs sprite, replace with FontAwesome
Summary: Removes the docs sprite in Conpherence with FontAwesome, adds additional icons. Unsure what happens if someone customized this config option.

Test Plan: Added images and files to a Conpherence, saw new icons.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11028
2014-12-20 20:59:24 -08:00
Bob Trahan
6ab3f06b6e Transactions - adding willRenderTimeline to handle tricky cases
Summary: Fixes T6693.

Test Plan:
Made a bunch of comments on a diff with differential, being sure to leave inlines here and there. This reproduced the issue in T6693. With this patch this issue no longer reproduces!

Successfully "showed older changes" in Maniphest too.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6693

Differential Revision: https://secure.phabricator.com/D10931
2014-12-04 13:58:52 -08:00
Bob Trahan
5e8600442d Transactions - land buildTransactionTimeline in a bunch more apps
Summary:
Ref T4712. Specifically...

 - Dashboards
  - two objects needed PhabricatorApplicationTransactionInterface
 - Macros
 - Paste
 - Phlux
  - one object needed PhabricatorApplicationTransactionInterface
  - added setShouldTerminate(true)
 - Files
  - one object needed PhabricatorApplicationTransactionInterface
 - Passphrase
  - one object needed PhabricatorApplicationTransactionInterface
  - added setShouldTerminate(true)
 - Drydock
  - one object needed PhabricatorApplicationTransactionInterface
  - added setShouldTerminate(true)

Test Plan: foreach application, verify that the timeline(s) showed up correctly, including with appropriate setShouldTerminate-ness

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T4712

Differential Revision: https://secure.phabricator.com/D10924
2014-12-03 13:16:15 -08:00