1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2025-01-11 15:21:03 +01:00
Commit graph

3347 commits

Author SHA1 Message Date
epriestley
7aa12f192a Add PhabricatorQueryIterator, for buffered iteration over a CursorPagedPolicyAwareQuery
Summary:
See D19446. This should make it easier to process larger, more complex result sets in constant memory.

Today, `LiskMigrationIterator` takes constant memory but can't apply `needX()` reqeusts or `withY(...)` constraints.

Using a raw `Query` can handle this stuff, but requires memory proportional to the size of the result set.

Offer the best of both worlds: constant memory and full access to the power of `Query` classes.

Test Plan:
Used this script to iterate over every commit, saw sensible behavior:

```name=list-commits.php
<?php

require_once 'scripts/init/init-script.php';

$viewer = PhabricatorUser::getOmnipotentUser();

$query = id(new DiffusionCommitQuery())
  ->setViewer($viewer);

$iterator = new PhabricatorQueryIterator($query);
foreach ($iterator as $commit) {
  echo $commit->getID()."\n";
}
```

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19450
2018-05-14 12:13:51 -07:00
epriestley
5b640a434c Support an "Ancestors Of: ..." constraint in commit queries
Summary:
Ref T13137. See PHI609. An install would like to filter audit requests on a particular branch, e.g. "master".

This is difficult in the general case because we can not apply this constraint efficiently under every conceivable data shape, but we can do a reasonable job in most practical cases.

See T13137#238822 for more detailed discussion on the approach here.

This is a bit rough, but should do the job for now.

Test Plan:
- Filtered commits by various branches, e.g. "master"; "lfs". Saw correct-seeming results.
- Stubbed out the "just list everything" path to hit the `diffusion.internal.ancestors` path, saw the same correct-seeming results.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13137

Differential Revision: https://secure.phabricator.com/D19431
2018-05-08 15:51:42 -07:00
epriestley
397645b273 Export task point values as double, not int
Summary:
See <https://discourse.phabricator-community.org/t/maniphest-non-integer-point-values-in-csv-export/1443>.

We currently export the Maniphest "points" field as an integer, but allow it to accept decimal values (e.g. "6.25").

Also fix a bug where we wouldn't roll over from "..., X, Y, Z, AA, AB, ..." correctly for Excel column names if sheet had more than 26 columns.

Test Plan:
  - Set a task point value to 6.25.
  - Exported to text, JSON, XLS.
  - Saw 6.25 represented accurately in exports.
  - Exported an excel sheet with 27+ columns.
  - Manually printed the first 200 column names to check that the algorithm looks correct.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19434
2018-05-08 15:49:40 -07:00
epriestley
304c6a4597 Improve UI and documentation for "Ignore Attributes" in Owners slightly
Summary:
See PHI251. Ref T13137.

  - Replace the perplexing text box with a checkbox that explains what it does.
  - Mention this feature in the documentation.

Test Plan:
  - Clicked/unclicked checkbox.
  - Read documentation.
  - Used an existing checkbox control in Slowvote to make sure I didn't break it.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13137

Differential Revision: https://secure.phabricator.com/D19433
2018-05-08 14:03:30 -07:00
epriestley
4a98e0ff65 Allow Owners packages to be configured to ignore generated paths in Differential
Summary:
Depends on D19427. Ref T13130. See PHI251. Support configuring owners packages so they ignore generated paths.

This is still a little rough. A couple limitations:

  - It's hard to figure out how to use this control if you don't know what it's for, but we don't currently have a "CheckboxesEditField". I may add that soon.
  - The attribute ignore list doesn't apply to Diffusion, only Differential, which isn't obvious. I'll either try to make it work in Diffusion or note this somewhere.
  - No documentation yet (which could mitigate the other two issues a bit).

But the actual behavior seems to work fine.

Test Plan:
  - Set a package to ignore paths with the "generated" attribute. Saw the package stop matching generated paths in Differential.
  - Removed the attribute from the ignore list.
  - Tried to set invalid attributes, got sensible errors.
  - Queried a package with Conduit, got the ignored attribute list.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13130

Differential Revision: https://secure.phabricator.com/D19428
2018-05-05 08:47:29 -07:00
Austin McKinley
9a0dd55442 Extend PhabricatorPolicyCodex interface to handle "interesting" policy defaults
Summary:
Fixes T13128. Ref PHI590. This is a rough-and-ready implementation of a new `PhabricatorPolicyCodex->compareToDefaultPolicy()` method that subclasses can override to handle special cases of policy defaults. Also implements a `PolicyCodex` for Phriction documents, because the default policy of a Phriction document is the policy of the root document.

I might break this change into two parts, one of which maintains the current behavior and another which implements `PhrictionDocumentPolicyCodex`.

Test Plan: Created some Phriction docs, fiddled with policies, observed expected colors in the header. Will test more comprehensively after review for basic reasonable-ness.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, swisspol

Maniphest Tasks: T13128

Differential Revision: https://secure.phabricator.com/D19409
2018-04-27 16:56:11 -07:00
epriestley
b4796d2837 Add "Content type" and "Rule type" fields to Herald rules for Herald rules
Summary:
Depends on D19400. Ref T13130. Currently, when you write Herald rules about other Herald rules, you can't pick a rule type or content type, so there's no way to get notified about edits to just global rules (which is the primary driving use case).

Add a "Content type" field to let the rule match rules that affect revisions, tasks, commits, etc.

Add a "Rule type" field to let the rule match global, personal, or object rules.

Test Plan:
  - Wrote a global rule for other rules about global Herald rules:

{F5540307}

{F5540308}

  - Ran it against itself which matched:

{F5540309}

  - Ran it against another rule (not a global rule about Herald rules), which did not match:

{F5540311}

  - Also reviewed the fields in those transcripts in more detail to make sure they were extracting matching correctly.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13130

Differential Revision: https://secure.phabricator.com/D19403
2018-04-25 06:54:48 -07:00
epriestley
cac41d1e48 Support Herald rules for Herald rules
Summary:
Depends on D19399. Ref T13130. This adds basic support for writing Herald rules against Herald rules. See T13130 for a lot more detail.

This needs a bit more work to be useful: for example, there's no way to specify the rule type or subject, so you can't say "notify me when global rules are edited" or "notify me when Maniphest rules are edited". I'll add some fields for that in followup changes to actually solve the original use case.

Test Plan:
  - Wrote Herald rules against Herald rules.
  - Ran them by editing rules and in the test console.
  - Verified they sent some mail with `bin/mail list-outbound`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13130

Differential Revision: https://secure.phabricator.com/D19400
2018-04-25 06:47:19 -07:00
epriestley
1b24b486f5 Manage object mailKeys automatically in Mail instead of storing them on objects
Summary:
Ref T13065. `mailKey`s are a private secret for each object. In some mail configurations, they help us ensure that inbound mail is authentic: when we send you mail, the "Reply-To" is "T123+456+abcdef".

  - The `T123` is the object you're actually replying to.
  - The `456` is your user ID.
  - The `abcdef` is a hash of your user account with the `mailKey`.

Knowing this hash effectively proves that Phabricator has sent you mail about the object before, i.e. that you legitimately control the account you're sending from. Without this, anyone could send mail to any object "From" someone else, and have comments post under their username.

To generate this hash, we need a stable secret per object. (We can't use properties like the PHID because the secret has to be legitimately secret.)

Today, we store these in `mailKey` properties on the actual objects, and manually generate them. This results in tons and tons and tons of copies of this same ~10 lines of code.

Instead, just store them in the Mail application and generate them on demand. This change also anticipates possibly adding flags like "must encrypt" and "original subject", which are other "durable metadata about mail transmission" properties we may have use cases for eventually.

Test Plan:
  - See next change for additional testing and context.
  - Sent mail about Herald rules (next change); saw mail keys generate cleanly.
  - Destroyed a Herald rule with a mail key, saw the mail properties get nuked.
  - Grepped for `getMailKey()` and converted all callsites I could which aren't the copy/pasted boilerplate present in 50 places.
  - Used `bin/mail receive-test --to T123` to test normal mail receipt of older-style objects and make sure that wasn't broken.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065

Differential Revision: https://secure.phabricator.com/D19399
2018-04-25 06:46:58 -07:00
epriestley
843bfb4fd8 Add a "commits" attachment to "differential.diff.search" for retrieving local commit information
Summary:
Ref T13124. See PHI593.

When you `arc diff` in a Git or Mercurial repository, we upload some information about the local commits in your working copy which the change was generated from.

In the future (for example, with T1508) we may increase the prominence of this feature.

Provide a stable way to read this information back via the API. This roughly mirrors the information we provide about commits in "diffusion.commit.search", although the latter is less fleshed-out today.

Test Plan: Used `differential.diff.search` to retrieve commit information about Git, Mercurial, and Subversion diffs. (There's no info for Subversion, but it doesn't crash or anything.)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13124

Differential Revision: https://secure.phabricator.com/D19386
2018-04-19 17:25:06 -07:00
Austin McKinley
e81b2173ad Add edge tables for Phlux
Summary: Fixes T13129. This at least makes the existing UI work again before we banish Phlux to the shadow realm.

Test Plan: Edited the visibility for a Phlux variable, didn't get an error. Nothing showed up in the edge tables when I made those changes, but at least it doesn't error out anymore.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13129

Differential Revision: https://secure.phabricator.com/D19387
2018-04-19 15:49:08 -07:00
epriestley
a817aa6c71 Add an "Abort Older Builds" build step to Harbormaster
Summary:
Ref T13124. See PHI531. When a revision is updated, builds against the older diff tend to stop being relevant. Add an option to abort outstanding older builds automatically.

At least for now, I'm adding this as a build step instead of some kind of special checkbox. An alternate implementation would be some kind of "Edit Options" action on plans with a checkbox like `[X] When this build starts, abort older builds.`

I think adding it as a build step is a bit simpler, and likely to lead to greater consistency and flexibility down the road, make it easier to add options, etc., and since we don't really have any other current use cases for "a bunch of checkboxes". This might change eventually if we add a bunch of checkboxes for some other reason.

The actual step activates //before// the build queues, so it doesn't need to wait in queue before it can actually act. T13088 discusses some plans here if this sticks.

Test Plan:
  - Created a "Sleep for 120 seconds" build plan and triggered it with Herald.
  - Added an "Abort Older Builds" step.
  - Updated a revision several times in a row, saw older builds abort.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13124

Differential Revision: https://secure.phabricator.com/D19376
2018-04-17 14:59:47 -07:00
epriestley
ea9187ea92 Allow Almanac properties to be set and deleted via Conduit
Summary:
Depends on D19342. Ref T12414. Ref T13120. This adds an EditEngine extension for editing Almanac properties.

The actual wire format is a little weird. Normally, we'd have a transaction for each property, but since you can pick any property names you want we can't really do that (we'd have to generate infinite transactions).

The transaction wire format anticipates that transactions may eventually get some kind of metadata -- each transaction looks like this:

```
{
  "type": "title",
  "value": "Example title"
}
```

...and we can add more keys there. For example, I could have made this transaction look like this:

```
{
  "type": "property.set",
  "almanac.property.key": "some-key",
  "value": "some-value"
}
```

However, I don't want to just accept any possible key freely, and it might be a decent chunk of work to formalize this better. It also doesn't feel great.

I just built special transaction types intead, so you:

```
{
  "type": "property.set",
  "value": {
   "some-key": "some-value",
   ...
  }
}
```

Internally, we may generate more than one transaction as a result (if the "value" has more than one key).

This feels a bit more natural and is probably easier for clients to use anyway.

Test Plan: Set and deleted Service, Device and Binding properties via the API.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19343
2018-04-11 10:42:10 -07:00
epriestley
208504a5e3 Provide "almanac.binding.search" and "almanac.binding.edit"
Summary:
Depends on D19338. Ref T13120. Ref T12414. These are the last of the new API methods.

This stuff still doesn't work:

  - You can't actually enable/disable bindings yet. I want to take a look at the use cases and consider changing "disabled" to "status", or providing a different way to solve the problem.
  - You can't edit properties via the API. I expect to enable this for all `AlmanacPropertyInterface` objects with an extension in a future change.

Test Plan:
  - Searched for bindings via API.
  - Viewed binding web UI for API methods.
  - Created bindings via API.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19340
2018-04-11 10:37:38 -07:00
epriestley
e502df509d Implement "almanac.interface.search" and "almanac.interface.edit"
Summary: Depends on D19337. Ref T13120. Ref T12414. These are slightly more substantive than namespace/network, but pretty much standard fare.

Test Plan:
  - Searched for interfaces with "almanac.interface.search".
  - Created and edited interfaces with "almanac.interface.edit".
  - Created and edited interfaces with web UI since some stuff got tweaked.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19338
2018-04-11 10:35:03 -07:00
epriestley
10947c8684 Add "almanac.namespace.edit" and "almanac.namespace.search" API methods
Summary: Depends on D19336. Ref T13120. Ref T12414. These are simple, straightforward, and uninteresting.

Test Plan:
  - Searched for namespaces with "almanac.namespace.search".
  - Created and edited namespaces with "almanac.namespace.edit".

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19337
2018-04-11 10:34:30 -07:00
epriestley
9022e14082 Use a more conventional spelling of "Almanac" for "almanac.service.edit" class
Summary: Depends on D19335. Ref T13120. Ref T12414. There are many good ways to spell "almanac", but stick with convention here.

Test Plan: (O_O)

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19336
2018-04-11 10:34:04 -07:00
epriestley
a8c4da13c0 Add "almanac.network.edit" and "almanac.network.search" API methods
Summary: Depends on D19334. Ref T13120. Ref T12414. These are pretty straightforward, but no one really has a use case for them anyway today so they're primarily just for completeness.

Test Plan:
  - Queried networks with `almanac.network.search`.
  - Created and edited networks with `almanac.network.edit`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19335
2018-04-11 10:33:41 -07:00
epriestley
4bce3fc8e6 Modularize Almanac property transactions
Summary:
Depends on D19329. Ref T13120. Ref T12414. Recent changes have mostly modularized Almanac transactions, but the "property" transactions remained written in an older style with the logic on the Editor/Transaction classes.

This moves them to modern modular transactions. These end up being a little bit copy-pastey, but it doesn't feel too terribly bad.

Test Plan: Created, edited, and deleted properties on services, devices and bindings. Grepped for removed constants.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19334
2018-04-11 10:33:18 -07:00
epriestley
71c77fcc3a Modularize transactions for Almanac Device
Summary:
Depends on D19328. Ref T13120. Ref T12414.

Prior work has left us with just a NAME transaction here, which is straightforward to modularize.

Test Plan:
  - Created and renamed devices.
  - Tried to set no name, a bad name, a duplicate name (got errors).
  - Tried to create/rename into a namespace I could not edit (got an error).
  - Grepped for `AlmanacDeviceTransaction::`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19329
2018-04-11 10:31:46 -07:00
epriestley
d240969e47 Use Interface transactions, not Device transactions, to destroy Interfaces
Summary:
Depends on D19324. Ref T13120. Ref T12414.

This moves "Destroy Interface" to use Interface transactions instead of Device transactions, so we can ultimately get rid of the complex and difficult-to-modernize `AlmanacDeviceTransaction::TYPE_INTERFACE`.

This transaction is a bit weird since it makes the interface delete itself, but this should work OK for now. At some point in the future I'd probably want to change this into more of a "disable" action, but I don't think we face any immediate peril by retaining this behavior for now.

Test Plan:
  - Destroyed interfaces on devices using the web UI, saw them vanish.
  - Ran daemons, nothing fataled/exploded even though the transaction is weird and destroys the object it affects.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19325
2018-04-11 10:30:15 -07:00
epriestley
6ccf35f9a2 Edit Interfaces in Almanac with EditEngine
Summary:
Depends on D19323. Ref T13120. Ref T12414.

Move editing to modern stuff and fix some implementation errors from D19323 (mostly copy/paste stuff).

Test Plan:
  - Created and edited interfaces.
  - Tried to create/edit an interface with a bogus/empty address/port, got errors.
  - Tried to create an interface on a bogus device, got an error.
  - Tried to create an interface on a device I could not edit, got an error.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19324
2018-04-11 10:29:50 -07:00
epriestley
f9c6a69d9c Add skeleton code for Almanac Interfaces to have real transactions
Summary:
Depends on D19322. Ref T13120. Ref T12414.

Currently, `AlmanacDevice` has a bit of a beast of a `TYPE_INTERFACE` transaction that fully creates a complex Interface object. This isn't very flexible or consistent, and Interfaces are complex enough to reasonably have their own object behaviors (for example, they have their own PHIDs).

The complexity of this transaction makes modularizing `AlmanacDevice` transactions tricky. To simplify this, move Interface toward having its own set of normal transactions.

This change just adds some reasonable-looking transactions; it doesn't actually hook them up in the UI or make them reachable. I'll test that they actually work as I swap the UI over.

We may also have some code using the `TYPE_INTERFACE` transaction in Phacility support stuff, so that may need to wait a week to actually phase out.

Test Plan: Ran `bin/storage upgrade` and `arc liberate`. This code isn't reachable yet.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19323
2018-04-11 10:29:26 -07:00
epriestley
580409b562 Modularize Almanac Network transactions
Summary: Depends on D19321. Ref T13120. Ref T12414. Move transactions for Almanac Networks (just "name") to ModularTransactions.

Test Plan:
  - Created a new network.
  - Renamed a network.
  - Tried to create a network with no name (got an error).
  - Grepped for `AlmanacNetworkTransaction::`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19322
2018-04-11 10:29:05 -07:00
epriestley
f62494355d Modularize Almanac Binding transactions
Summary: Depends on D19320. Ref T13120. Ref T12414. Move transactions for Almanac Bindings to ModularTransactions.

Test Plan:
  - Created a new binding.
  - Tried to create a duplicate binding, got an error.
  - Edited a binding to rebind it to a different device.
  - Disabled and enabled bindings.
  - Grepped for `AlmanacBindingTransaction::` constants.

When a binding is created, it currently renders a bad "changed the interface from ??? to X" transaction. This is because creation isn't currently using EditEngine. I plan to swap it shortly, which will turn this into a real "Create" transaction and fix the issue.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19321
2018-04-11 10:28:42 -07:00
epriestley
5ada1211cd Modularize Almanac Namespace transactions
Summary: Depends on D19318. Ref T13120. Ref T12414. Move transactions for Almanac Namespaces ("name" is the only meaningful one) to ModularTransactions.

Test Plan:
  - Created a new namespace.
  - Edited a namespace.
  - Tried to choose no name, an invalid name, a duplicate name, and a name in a namespace I can't edit; got appropriate errors.
  - Grepped for `AlmanacNamespaceTransaction::TYPE_NAME`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19320
2018-04-11 10:24:10 -07:00
epriestley
6983479e4f Allow "almanac.service.edit" to create services
Summary:
Depends on D19317. Ref T13120. Ref T12414. See PHI145. See PHI473.

This adds a Conduit-only "type" transaction for Almanac services. This is very similar to the approach in D18849 for Drydock blueprints.

Test Plan:
  - Tried to create an empty service via "almanac.service.edit", was told to pick a type.
  - Tried to pick a bad type, was told to pick a good type.
  - Created a new Almanac service via "almanac.service.edit".
  - Tried to edit the service to change the type, wasn't allowed to.
  - Created and edited via the web UI, nothing changed from before.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19318
2018-04-11 10:23:50 -07:00
epriestley
c428f60a97 Partially modularize AlmanacService transactions
Summary:
Ref T13120. Ref T12414. See PHI145. See PHI473. This partially modernizes AlmanacService transactions by moving them to ModularTransactions.

This isn't complete because the "update property" and "remove property" transactions aren't modularized. They still //work//, since the parent Editor implements them, but they no longer render properly on the timeline since the `Transaction` object no longer has rendering logic for them.

Tentatively, I'm going to try to convert the rest of the Almanac objects and then modularize those transactions. (Currently, all of Binding, Device, Namespace and Service support properties, although they can only actually be edited on Service, Device and Binding.)

If that turns out to be really tricky for some reason I can just copy/paste the timeline rendering for now, but I think it won't be too hard.

Test Plan:
  - Created and edited Services.
  - Tried to create a service with: a bad name, no name, a name which put it in a namespace I can't edit (got errors in all cases).
  - Edited and removed properties. The edits worked, the timeline just renders a generic story now ('X edited this object (transaction type "almanac:property:update").').

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13120, T12414

Differential Revision: https://secure.phabricator.com/D19317
2018-04-11 10:22:34 -07:00
Austin McKinley
0755482bf0 Add transactions for installing/uninstalling applications
Summary: Fixes T11476.

Test Plan:
 - Installed/uninstalled the Conpherence application
 - Observed correct timeline stories
 - Observed correct config in database
 - Observed 404 for application page

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T11476

Differential Revision: https://secure.phabricator.com/D19339
2018-04-11 08:54:55 -07:00
epriestley
09c6d42b95 Mostly make blame work with DocumentEngine
Summary: Ref T13105. This needs refinement but blame sort of works again, now.

Test Plan: Viewed files in Diffusion and Files; saw blame in Diffusion when viewing in source mode.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19309
2018-04-09 04:48:21 -07:00
epriestley
1fde4a9450 Move Diffusion browse rendering to DocumentEngine, breaking almost all features
Summary:
Ref T13105. This breaks about 9,000 features but moves Diffusion to DocumentEngine for rendering. See T13105 for a more complete list of all the broken stuff.

But you can't bake a software without breaking all the features every time you make a change, right?

Test Plan: Viewed various files in Diffusion, used DocumentEngine features like highlighting and rendering engine selection.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Subscribers: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19302
2018-04-09 04:46:26 -07:00
epriestley
245132a0b2 Pull file Document Engine rendering out of "Files" application controllers
Summary:
Ref T13105. This separates document rendering from the Controllers which trigger it so it can be reused elsewhere (notably, in Diffusion).

This shouldn't cause any application behavior to change, it just pulls the rendering logic out so it can be reused elsewhere.

Test Plan: Viewed various types of files in Files; toggled rendering, highlighting, and encoding.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19301
2018-04-09 04:45:58 -07:00
epriestley
8c8e7f07b5 Add a standalone view for browsing changesets of very large revisions
Summary: Ref T13110. Installs have various reasons for sending unreviewable changes (changes where the text of the change will never be reviewed by a human) through Differential anyway. Prepare for accommodating this more gracefully by building a standalone changeset list page which paginates the changesets.

Test Plan: Clicked the new "Changeset List" button on a revision, was taken to a separate page.

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19295
2018-04-05 06:35:06 -07:00
epriestley
51461f18c1 When publishing buildables in Differential, ignore autobuilds (local lint and unit)
Summary:
Depends on D19280. Ref T13110. Although Harbormaster cares about all builds, Differential does not practically care about local lint and unit results in determining build status.

In Differential, orient publishing around "remote builds" instead of "builds".

This does not yet change any of the draft logic, it just makes the timeline story use newer logic.

Test Plan: Used `bin/harbormaster publish` (with some guard-clause removal) to publish some buildables to revisions without anything crashing.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19281
2018-04-03 11:02:12 -07:00
epriestley
ada0c9126c Provide a modular buildable transaction in Diffusion
Summary:
Depends on D19279. Ref T13110. This implements the existing publishing logic for buildables, but does so via ModularTransactions instead of a core transaction type.

Since each application is implementing build transactions independently, this removes the core type.

Next, Differential will get a similar treatment.

Test Plan: Used `bin/harbormaster publish` (with some commenting-out-guard-clauses) to publish a commit Buildable; saw unchanged feed behavior.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19280
2018-04-03 11:01:37 -07:00
epriestley
c20b4e365b Move structural build publishing logic to BuildEngine, provide "bin/harbormaster publish"
Summary:
Depends on D19278. Ref T13110. This moves most of the structural logic for publishing builds to BuildableEngine and provides a `bin/harbormaster publish` to make publishing easy to retry/debug.

This intentionally removes the bit which actually does anything when builds publish. Followup changes will implement application-specific versions of the publishing logic in Differential and Diffusion.

Test Plan: Ran `bin/harbormaster publish Bxxx`, saw it do nothing (but not crash).

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19279
2018-04-03 10:58:27 -07:00
epriestley
95c9d403f4 Make objects implementing BuildableInterface produce a BuildableEngine
Summary:
Ref T13110. Currently, build status is published the same way for every Buildable by the BuildEngine.

I want to change this to delegate publishing to each Buildable, particularly so that Differential may use more detailed rules for handling builds and drafts.

Rather than add additional methods to the existing `BuildableInterface`, add an engine generator method instead. This is a pattern which has seen more use recently (e.g., in Ferret) and lets us pay a little more upfront to pull complex pieces of logic out of the main class and let them use inheritence more easily. If we had Traits that might cover this to some degree.

I'd expect to eventually reduce the size of `BuildableInterface` and move the `CircleCI` and `BuildKite` interfaces so that the `BuildableEngine` implements them instead of the main object.

Here, this new engine does nothing and is never instantiated. In upcoming changes, publishing logic will move into it so that Differential can handle publishing differently.

Test Plan: Ran `arc liberate`, loaded pages, grepped for `BuildableInterface`.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19278
2018-04-03 10:57:51 -07:00
epriestley
66392e5b8b Add a rough "bin/repository unpublish" workflow to attempt to cleanup improperly published repositories
Summary:
Ref T13114. See PHI514. This makes some attempt to undo the damage caused by incorrectly publishing a repository.

Don't run this.

Test Plan: Yikes.

Maniphest Tasks: T13114

Differential Revision: https://secure.phabricator.com/D19271
2018-03-30 08:46:11 -07:00
epriestley
7f9a9bc800 Make Harbormaster objects destructible
Summary:
Ref T13114. See PHI511. Ref T13072. This makes Buildables, Builds, Targets and Artifacts destructible with `bin/remove destroy`.

This might not be totally exhaustive. In particular:

  - File artifacts won't destroy the file. This is sort of okay because file artifacts are currently just a file reference, but probably shouldn't be how things work in the long term.
  - `BuildCommand` doesn't get cleaned up, but `BuildMessage` does on `Build`. See T13072 for more.

Test Plan: Used `bin/remove destroy` to nuke a bunch of builds, buildables, etc. Loaded stuff in the web UI and it all looked like it got nuked properly.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13114, T13072

Differential Revision: https://secure.phabricator.com/D19269
2018-03-29 13:01:14 -07:00
epriestley
bba1b185f8 Improve minor client behaviors for document rendering
Summary:
Ref T13105. This adds various small client-side improvements to document rendering.

  - In the menu, show which renderer is in use.
  - Make linking to lines work.
  - Make URIs persist information about which rendering engine is in use.
  - Improve the UI feedback for transitions between document types.
  - Load slower documents asynchronously by default.
  - Discard irrelevant requests if you spam the view menu.

Test Plan: Loaded files, linked to lines, swapped between modes, copy/pasted URLs.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19256
2018-03-23 14:09:31 -07:00
epriestley
4906364751 Add a JSON document rendering engine
Summary: Depends on D19254. This engine just formats JSON files in a nicer, more readable way.

Test Plan: Looked at some JSON files, saw them become formatted nicely.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Differential Revision: https://secure.phabricator.com/D19255
2018-03-23 12:29:05 -07:00
epriestley
d2727d24da Add an abstract "Text" document engine and a "Source" document engine
Summary: Ref T13105. Allow normal text files to be rendered as documents, and add a "source code" rendering engine.

Test Plan: Viewed some source code.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19254
2018-03-23 12:28:43 -07:00
epriestley
cbf3d3c371 Add a very rough, proof-of-concept Jupyter notebook document engine
Summary:
Depends on D19252. Ref T13105. This very roughly renders Jupyter notebooks.

It's probably better than showing the raw JSON, but not by much.

Test Plan:
  - Viewed various notebooks with various cell types, including markdown, code, stdout, stderr, images, HTML, and Javascript.
  - HTML and Javascript are not live-fired since they're wildly dangerous.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19253
2018-03-23 07:14:45 -07:00
epriestley
fb4ce851c4 Add a PDF document "rendering" engine
Summary:
Depends on D19251. Ref T13105. This adds rendering engine support for PDFs.

It doesn't actually render them, it just renders a link which you can click to view them in a new window. This is much easier than actually rendering them inline and at least 95% as good most of the time (and probably more-than-100%-as-good some of the time).

This makes PDF a viewable MIME type by default and adds a narrow CSP exception for it. See also T13112.

Test Plan:
  - Viewed PDFs in Files, got a link to view them in a new tab.
  - Clicked the link in Safari, Chrome, and Firefox; got inline PDFs.
  - Verified primary CSP is still `object-src 'none'` with `curl ...`.
  - Interacted with the vanilla lightbox element to check that it still works.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19252
2018-03-23 07:14:17 -07:00
epriestley
8b658706a8 Add a basic Remarkup document rendering engine
Summary:
Ref T13105. Although Markdown is trickier to deal with, we can handle Remarkup easily.

This may need some support for encoding options.

Test Plan: Viewed `.remarkup` files, got remarkup document presentation by default. Viewed other text files, got an option to render as remarkup.

Reviewers: avivey

Reviewed By: avivey

Subscribers: mydeveloperday, avivey

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19251
2018-03-23 07:07:50 -07:00
epriestley
f646153f4d Add an async driver for document rendering and a crude "Hexdump" document engine
Summary: Depends on D19237. Ref T13105. This adds a (very basic) "Hexdump" engine (mostly just to have a second option to switch to) and a selector for choosing view modes.

Test Plan: Viewed some files, switched between audio/video/image/hexdump.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19238
2018-03-19 15:18:05 -07:00
epriestley
01f22a8d06 Roughly modularize document rendering in Files
Summary:
Ref T13105. This change begins modularizing document rendering. I'm starting in Files since it's the use case with the smallest amount of complexity.

Currently, we hard-coding the inline rendering for images, audio, and video. Instead, use the modular engine pattern to make rendering flexible and extensible.

There aren't any options for switching modes yet and none of the renderers do anything fancy. This API is also probably very unstable.

Test Plan: Viewwed images, audio, video, and other files. Saw reasonable renderings, with "nothing can render this" for any other file type.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19237
2018-03-19 15:17:04 -07:00
epriestley
df8d4dff67 Raise a warning when mentioning a user in a comment on a draft revision
Summary: See PHI433. Ref T13102. Users in the wild have mixed expecations about exactly what "draft" means. Recent changes have tried to make behavior more clear. As part of clarifying messaging, make it explicit that `@mention` does not work on drafts by showing users a warning when they try to `@mention` a user.

Test Plan:
  - Mentioned users on drafts, got a warning.
  - Posted normal comments on drafts, no warning.
  - Posted normal/mention comments on non-drafts, no warning.

Maniphest Tasks: T13102

Differential Revision: https://secure.phabricator.com/D19210
2018-03-12 17:03:14 -07:00
epriestley
3c4f31e4b9 Dynamically composite favicons from customizable sources
Summary: Ref T13103. Make favicons customizable, and perform dynamic compositing to add marker to indicate things like "unread messages".

Test Plan: Viewed favicons in Safari, Firefox and Chrome. With unread messages, saw pink dot composited into icon.

Maniphest Tasks: T13103

Differential Revision: https://secure.phabricator.com/D19209
2018-03-12 15:28:41 -07:00
epriestley
c7408f2797 PhabricatorMemeEngine HA HA HA HA
Summary:
Depends on D19198. Ref T13101. Ref T5258. Pull compositing logic out of the `Controller`.

This is moving toward fixing memes in email.

Test Plan: Used new and old memes. Used API memes.

Maniphest Tasks: T13101, T5258

Differential Revision: https://secure.phabricator.com/D19200
2018-03-08 11:06:52 -08:00