1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-30 02:32:42 +01:00
Commit graph

3085 commits

Author SHA1 Message Date
epriestley
fc103f71e9 Fix very odd wrapping / linebreaking for Remarkup code block headers in Safari
Summary: Fixes T13118. Ref T13120. This construction is a little odd; I'm not entirely sure why Safari is doing what it's doing, but this appears to fix it.

Test Plan: Viewed blocks like those in T13118 in Safari. Before the patch, weird last-letter wrapping. After the patch, sensible behavior.

Maniphest Tasks: T13118, T13120

Differential Revision: https://secure.phabricator.com/D19303
2018-04-08 06:15:58 -07:00
epriestley
e70c9f72a4 Show revision sizes using a perplexing, inexplicable symbol code
Summary: Ref T13110. See PHI230. Show revision sizes on a roughly logarithmic scale from 1-7 stars. See D16322 for theorycrafting on this element.

Test Plan: Looked at some revisions, saw plausible-looking size markers.

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19294
2018-04-03 12:49:27 -07:00
epriestley
615d27c8e9 Show an additional "Draft" tag on non-broadcasting revisions in a non-draft state
Summary:
Depends on D19284. Ref T13110. It's now possible to get a revision into a "Abandoned + But, Never Promoted From Draft" state. Show this in the header and provide the draft hint above the comment area.

Also, remove `shouldBroadcast()`. The method `getShouldBroadcast()` now has the same meaning.

Finally, migrate existing drafts to `shouldBroadcast = false` and default `shouldBroadcast` to `true`. If we don't do this, every older revision becomes a non-broadcasting revision because this flag was not explicitly set on revision creation before, only on promotion out of draft.

Test Plan: Ran migration; abandoned draft revisions and ended up in a draft + abandoned state.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13110

Differential Revision: https://secure.phabricator.com/D19285
2018-04-03 11:09:49 -07:00
epriestley
7189cb7ba8 Support text encoding and syntax highlighting options in document rendering
Summary: Depends on D19273. Ref T13105. Adds "Change Text Encoding..." and "Highlight As..." options when rendering documents, and makes an effort to automatically detect and handle text encoding.

Test Plan:
  - Uploaded a Shift-JIS file, saw it auto-detect as Shift-JIS.
  - Converted files between encodings.
  - Highlighted various things as "Rainbow", etc.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19274
2018-03-30 11:28:52 -07:00
epriestley
7eaa27683e Make closed/disabled results in the remarkup autocomplete more visually clear
Summary:
Ref T13114. See PHI522. Although it looks like results are already ordered correctly, the override rendering isn't accommodating disabled results gracefully.

Give closed results a distinctive look (grey + strikethru) so it's clear when you're autocompleting `@mention...` into a disabled user.

Test Plan: {F5497621}

Maniphest Tasks: T13114

Differential Revision: https://secure.phabricator.com/D19272
2018-03-30 08:47:00 -07:00
epriestley
b7d3101e7c Minor document rendering fixes: dropdown for synchronous files, URI normalization for default renderers
Summary:
Depends on D19258. Ref T13105.

  - When the default renderer is an Ajax renderer, don't replace the URI. For example, when viewing a Jupyter notebook, the URI should remain `/F123`, not instantly change to `/view/123/jupyter/`.
  - Fix an issue where non-ajax renderers could fail to display the dropdown menu properly.

Test Plan:
  - Viewed a Jupyter notebook, stayed on the same URI.
  - Changed rendering, got different URIs.
  - Viewed a JSON file and toggled renderers via dropdown.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19259
2018-03-28 15:07:21 -07:00
epriestley
f583406ba9 Drop uniqueness constraint on PushEvent request ID
Summary: See <https://discourse.phabricator-community.org/t/pushing-to-mercurial-repository-fails/1275/1>. Mercurial may invoke hooks multiple times per push.

Test Plan: Pushed to Mercurial, saw key constraint failure.

Differential Revision: https://secure.phabricator.com/D19257
2018-03-26 07:02:15 -07:00
epriestley
bba1b185f8 Improve minor client behaviors for document rendering
Summary:
Ref T13105. This adds various small client-side improvements to document rendering.

  - In the menu, show which renderer is in use.
  - Make linking to lines work.
  - Make URIs persist information about which rendering engine is in use.
  - Improve the UI feedback for transitions between document types.
  - Load slower documents asynchronously by default.
  - Discard irrelevant requests if you spam the view menu.

Test Plan: Loaded files, linked to lines, swapped between modes, copy/pasted URLs.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19256
2018-03-23 14:09:31 -07:00
epriestley
d2727d24da Add an abstract "Text" document engine and a "Source" document engine
Summary: Ref T13105. Allow normal text files to be rendered as documents, and add a "source code" rendering engine.

Test Plan: Viewed some source code.

Reviewers: mydeveloperday

Reviewed By: mydeveloperday

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19254
2018-03-23 12:28:43 -07:00
epriestley
cbf3d3c371 Add a very rough, proof-of-concept Jupyter notebook document engine
Summary:
Depends on D19252. Ref T13105. This very roughly renders Jupyter notebooks.

It's probably better than showing the raw JSON, but not by much.

Test Plan:
  - Viewed various notebooks with various cell types, including markdown, code, stdout, stderr, images, HTML, and Javascript.
  - HTML and Javascript are not live-fired since they're wildly dangerous.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19253
2018-03-23 07:14:45 -07:00
epriestley
fb4ce851c4 Add a PDF document "rendering" engine
Summary:
Depends on D19251. Ref T13105. This adds rendering engine support for PDFs.

It doesn't actually render them, it just renders a link which you can click to view them in a new window. This is much easier than actually rendering them inline and at least 95% as good most of the time (and probably more-than-100%-as-good some of the time).

This makes PDF a viewable MIME type by default and adds a narrow CSP exception for it. See also T13112.

Test Plan:
  - Viewed PDFs in Files, got a link to view them in a new tab.
  - Clicked the link in Safari, Chrome, and Firefox; got inline PDFs.
  - Verified primary CSP is still `object-src 'none'` with `curl ...`.
  - Interacted with the vanilla lightbox element to check that it still works.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19252
2018-03-23 07:14:17 -07:00
epriestley
8b658706a8 Add a basic Remarkup document rendering engine
Summary:
Ref T13105. Although Markdown is trickier to deal with, we can handle Remarkup easily.

This may need some support for encoding options.

Test Plan: Viewed `.remarkup` files, got remarkup document presentation by default. Viewed other text files, got an option to render as remarkup.

Reviewers: avivey

Reviewed By: avivey

Subscribers: mydeveloperday, avivey

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19251
2018-03-23 07:07:50 -07:00
epriestley
df3c937dab Record lock timing information on PushEvents
Summary:
Depends on D19249. Ref T13109. Add timing information to the `PushEvent`:

  - `writeWait`: Time spent waiting for a write lock.
  - `readWait`: Time spent waiting for a read lock.
  - `hostWait`: Roughly, total time spent on the leaf node.

The primary goal here is to see if `readWait` is meaningful in the wild. If it is, that motivates smarter routing, and the value of smarter routing can be demonstrated by looking for a reduction in read wait times.

Test Plan: Pushed some stuff, saw reasonable timing values in the table. Saw timing information in "Export Data".

Maniphest Tasks: T13109

Differential Revision: https://secure.phabricator.com/D19250
2018-03-22 13:46:01 -07:00
epriestley
69bff489d4 Generate a random unique "Request ID" for SSH requests so processes can coordinate better
Summary:
Depends on D19247. Ref T13109. When we receive an SSH request, generate a random unique ID for the request. Then thread it down through the process tree.

The immediate goal is to let the `ssh-exec` process coordinate with `commit-hook` process and log information about read and write lock wait times. Today, there's no way for `ssh-exec` to interact with the `PushEvent`, but this is the most helpful place to store this data for users.

Test Plan: Made pushes, saw the `PushEvent` table populate with a random request ID. Exported data and saw the ID preserved in the export.

Maniphest Tasks: T13109

Differential Revision: https://secure.phabricator.com/D19249
2018-03-22 13:44:30 -07:00
epriestley
e010aaca43 accidentally a word
Summary: Sometimes I dream I am a small turtle.

Test Plan: squeak squeak

Differential Revision: https://secure.phabricator.com/D19248
2018-03-22 13:43:10 -07:00
epriestley
c8583b016d When workflow dialog buttons are clicked, disable the button
Summary:
Depends on D19245. Fixes T11145. Ref T13108. See PHI488. Disable workflow buttons when they're clicked to prevent accidental client-side double submission.

This might have some weird side effects but we should normally never need to re-use a workflow dialog form so it's not immediately obvious that this can break anything.

Test Plan:
  - Added `sleep(1)` to the Mute controller and the Maniphest task controller.
  - Added `phlog(...)` to the Mute controller.
  - Opened the mute dialog, mashed the button a thousand times.
    - Before: Saw a bunch of logs.
    - After: Button immediately disables, saw only one log.

Maniphest Tasks: T13108, T11145

Differential Revision: https://secure.phabricator.com/D19246
2018-03-21 11:58:13 -07:00
epriestley
9e278a89ba If a Workflow form receives a redirect response, don't re-enable the submit buttons
Summary:
See PHI488. Ref T13108. Currently, there is a narrow window between when the response returns and when the browser actually follows the redirect where the form is live and you can click the button again.

This is relativey easy if Phabricator is running //too fast// since the button may be disabled only momentarily. This seems to be easier in Firefox/Chrome than Safari.

Test Plan:
  - In Firefox and Chrome, spam-clicked a comment submit button.
    - Before: could sometimes get a double-submit.
    - After: couldn't get a double-submit.
    - This could probably be reproduced more reliabily by adding a `sleep(1)` to whatever we're redirecting //to//.
  - Submitted an empty comment, got a dialog plus a still-enabled form (so this doesn't break the non-redirect case).

Maniphest Tasks: T13108

Differential Revision: https://secure.phabricator.com/D19245
2018-03-21 11:56:21 -07:00
epriestley
4aafce6862 Add filesize limits for document rendering engines and support partial/complete rendering
Summary:
Depends on D19238. Ref T13105. Give document engines some reasonable automatic support for degrading gracefully when someone tries to hexdump a 100MB file or similar.

Also, make "Video" sort above "Audio" for files which could be rendered either way.

Test Plan: Viewed audio, video, image, and other files. Adjusted limits and saw full, partial, and fallback/error rendering.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19239
2018-03-19 15:18:34 -07:00
epriestley
f646153f4d Add an async driver for document rendering and a crude "Hexdump" document engine
Summary: Depends on D19237. Ref T13105. This adds a (very basic) "Hexdump" engine (mostly just to have a second option to switch to) and a selector for choosing view modes.

Test Plan: Viewed some files, switched between audio/video/image/hexdump.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19238
2018-03-19 15:18:05 -07:00
epriestley
01f22a8d06 Roughly modularize document rendering in Files
Summary:
Ref T13105. This change begins modularizing document rendering. I'm starting in Files since it's the use case with the smallest amount of complexity.

Currently, we hard-coding the inline rendering for images, audio, and video. Instead, use the modular engine pattern to make rendering flexible and extensible.

There aren't any options for switching modes yet and none of the renderers do anything fancy. This API is also probably very unstable.

Test Plan: Viewwed images, audio, video, and other files. Saw reasonable renderings, with "nothing can render this" for any other file type.

Maniphest Tasks: T13105

Differential Revision: https://secure.phabricator.com/D19237
2018-03-19 15:17:04 -07:00
epriestley
dbc72a05bc Correct the behavior of "Desktop Only" in Notifications preferences
Summary:
See <https://discourse.phabricator-community.org/t/desktop-only-notifications-mode-is-broken/1234>. Ref T13102. The "Desktop Only" mode for notifications currently shows both desktop and web notifications.

In fact, `JX.Notification` currently has no ability to render notifications as desktop-only. Make this work.

Note that many of the variables and parameters here, including `showAnyNotification`, `web_ready`, and `desktop_ready`, are named in an incorrect or misleading way. However, the new behavior appears to be correct.

Test Plan:
  - Emitted test notifications in "No Notifications", "Web Only", "Web and Desktop", and "Desktop" modes.
  - Saw appropriate notifications appear in the UI.

Maniphest Tasks: T13102

Differential Revision: https://secure.phabricator.com/D19233
2018-03-16 15:17:49 -07:00
epriestley
2b5c73fc3d In "Analyze Query Plans" mode, collect service call stack traces in DarkConsole
Summary: Ref T13106. When profiling service queries, there's no convenient way to easily get a sense of why a query was issued. Add a mode to collect traces for each query to make this more clear. This is rough, but works well enough to be useful.

Test Plan: Clicked "Analyze Query Plans", got stack traces for each service call.

Maniphest Tasks: T13106

Differential Revision: https://secure.phabricator.com/D19221
2018-03-14 20:34:34 -07:00
epriestley
dc7e40ff3f Fix the DarkConsole inline error log stack trace expansion behavior for Content-Security-Policy
Summary:
See PHI451. Ref T13102. DarkConsole uses an ancient inline "onclick" handler to expand the stack traces for errors.

The new Content-Security-Policy prevents this from functioning.

Replace this with a more modern behavior-driven action instead.

Test Plan:
  - Clicked some errors in DarkConsole, saw stack traces appear.
  - Grepped for `onclick` and `jsprintf()` to see if I could find any more of these, but came up empty.

Maniphest Tasks: T13102

Differential Revision: https://secure.phabricator.com/D19218
2018-03-13 16:45:20 -07:00
epriestley
0bf8e33bb6 Issue setup guidance recommending MySQLi and MySQL Native Driver
Summary:
Fixes T12994. We need `MYSQLI_ASYNC` to implement client-side query timeouts, and we need MySQLi + MySQL Native Driver to get `MYSQLI_ASYNC`.

Recommend users install MySQLi and MySQL Native Driver if they don't have them. These are generally the defaults and best practice anyway, but Ubuntu makes it easy to use the older stuff.

All the cases we're currently aware of stem from `apt-get install php5-mysql` (which explicitly selects the non-native driver) so issue particular guidance about `php5-mysqlnd`.

Test Plan:
  - Faked both issues locally, reviewed the text.
  - Will deploy to `secure`, which currently has the non-native driver.

Maniphest Tasks: T12994

Differential Revision: https://secure.phabricator.com/D19216
2018-03-13 12:38:09 -07:00
epriestley
2b19f91936 Allow Doorkeeper references to have multiple display variations (full, short, etc.)
Summary:
Ref T13102. An install has a custom rule for bridging JIRA references via Doorkeeper and would like to be able to render them as `JIRA-123` instead of `JIRA JIRA-123 Full JIRA title`.

I think it's reasonable to imagine future support upstream for `JIRA-123`, `{JIRA-123}`, and so on, although we do not support these today. We can take a small step toward eventual support by letting the rendering pipeline understand different view modes.

This adds an optional `name` (the default text rendered before we do the OAuth sync) and an optional `view`, which can be `short` or `full`.

Test Plan:
I tested this primarily with Asana, since it's less of a pain to set up than JIRA. The logic should be similar, hopefully.

I changed `DoorkeeperAsanaRemarkupRule` to specify `name` and `view`, e.g `'view' => (mt_rand(0, 1) ? 'short' : 'full')`. Then I made a bunch of Asana references in a comment and saw them randomly go short or long.

Maniphest Tasks: T13102

Differential Revision: https://secure.phabricator.com/D19215
2018-03-13 11:29:52 -07:00
epriestley
1e93b49b1b Allow custom actions in Differential to explicitly override "accept" stickiness
Summary:
See PHI431. Ref T13102. An install is interested in a custom "non-sticky" accept action, roughly.

On the one hand, this is a pretty hacky patch. However, I suspect it inches us closer to T731, and I'm generally comfortable with exploring the realms of "Accept Next Update", "Unblock Without Accepting", etc., as long as most of it doesn't end up enabled by default in the upstream.

Test Plan:
  - Accepted and updated revisions normally, saw accepts respect global stickiness.
  - Modified the "Accept" action to explicitly be unsticky, saw nonsticky accept behavior after update.

Maniphest Tasks: T13102

Differential Revision: https://secure.phabricator.com/D19211
2018-03-12 17:10:43 -07:00
epriestley
3c4f31e4b9 Dynamically composite favicons from customizable sources
Summary: Ref T13103. Make favicons customizable, and perform dynamic compositing to add marker to indicate things like "unread messages".

Test Plan: Viewed favicons in Safari, Firefox and Chrome. With unread messages, saw pink dot composited into icon.

Maniphest Tasks: T13103

Differential Revision: https://secure.phabricator.com/D19209
2018-03-12 15:28:41 -07:00
epriestley
3e992c6713 Add audit, review, and dominion information to "owners.search" API method
Summary:
See PHI439. This fills in additional information about Owners packages.

Also removes dead `primaryOwnerPHID`.

Test Plan: Called `owners.search` and reviewed the results. Grepped for `primaryOwnerPHID`.

Differential Revision: https://secure.phabricator.com/D19207
2018-03-09 12:11:13 -08:00
epriestley
e83cfa295b Fix image prev/next cycling in lightboxes
Summary:
See <https://discourse.phabricator-community.org/t/lightbox-not-working/1201/3>. The lightbox code is fragile and currently relies on simulating a click on the actual "<a />" tag surrounding other images in the document.

This breaks the prev/next links which ignore the event because it there's no "<img />".

Instead, don't simulate clicks and just call the code we want directly.

Test Plan: Added several images to a page, used lightbox prev/next buttons to cycle between them.

Differential Revision: https://secure.phabricator.com/D19197
2018-03-08 08:28:04 -08:00
epriestley
9d3a722eb1 When proxying an "{image ...}" image fails, show the user an error message
Summary:
Depends on D19192. Ref T4190. Ref T13101. Instead of directly including the proxy endpoint with `<img src="..." />`, emit a placeholder and use AJAX to make the request. If the proxy fetch fails, replace the placeholder with an error message.

This isn't the most polished implementation imaginable, but it's much less mysterious about errors.

Test Plan: Used `{image ...}` for valid and invalid images, got images and useful error messages respectively.

Maniphest Tasks: T13101, T4190

Differential Revision: https://secure.phabricator.com/D19193
2018-03-08 07:03:26 -08:00
epriestley
a4cc1373d3 Use a tokenizer, not a gigantic poorly-ordered "<select />", to choose repositories in Owners
Summary: Depends on D19190. Fixes T12590. Ref T13099. Replaces the barely-usable, gigantic, poorly ordered "<select />" control with a tokenizer. Attempts to fix various minor issues.

Test Plan:
  - Edited paths: include/exclude paths, from different repositories, different actual paths.
  - Used "Add New Path" to add rows, got repository selector prepopulated with last value.
  - Used "remove".
  - Used validation typeahead, got reasonable behaviors?

The error behavior if you delete the repository for a path is a little sketchy still, but roughly okay.

Maniphest Tasks: T13099, T12590

Differential Revision: https://secure.phabricator.com/D19191
2018-03-07 20:57:24 -08:00
epriestley
ab0ac7f61b Remove very old "owners-default-path" code from Owners
Summary: Ref T12590. This is ancient code which was used to prefill `/trunk/tfb/www/` or similar at Facebook. I don't think it ever had a UI and no install has asked for this feature since 2011.

Test Plan: Grepped for affected symbols, edited paths in Owners.

Maniphest Tasks: T12590

Differential Revision: https://secure.phabricator.com/D19189
2018-03-07 18:25:27 -08:00
epriestley
229d467770 Restore lightbox behavior for thumbnailed images
Summary: Ref T13099. See <https://discourse.phabricator-community.org/t/lightbox-not-working/1201>. The Content-Security-Policy changes rewrote some of this code and the handling for "Download" links is incorrectly catching clicks on thumbnailed images.

Test Plan: Clicked a thumbnailed image, got a lightbox. Command-clicked a download link, still got link behavior instead of a lightbox.

Maniphest Tasks: T13099

Differential Revision: https://secure.phabricator.com/D19188
2018-03-07 07:33:43 -08:00
epriestley
df1e9ce646 Treat Owners paths like "/src/backend" and "/src/backend/" identically
Summary:
Depends on D19183. Ref T11015. Currently, adding a trailing slash works great and omitting it mysteriously doesn't work.

Store a normalized version with an unconditional trailing slash for the lookup logic to operate on, and a separate display version which tracks what the user actually typed.

Test Plan:
  - Entered "/src/main.c", "/src/main.c/", saw them de-duplicate.
  - Entered "/src/main.c", saw it stay that way in the UI but appear as "/src/main.c/" internally.
  - Added a rule for "/src/applications/owners" (no slash), created a revision touching paths in that directory, saw Owners fire for it.
  - Changed the display value of a path only ("/src/main.c" to "/src/main.c/"), saw the update reflected in the UI without any beahvioral change.

Maniphest Tasks: T11015

Differential Revision: https://secure.phabricator.com/D19184
2018-03-06 20:31:46 -08:00
epriestley
adde4089b4 Allow owners paths to be arbitrarily long and add storage for display paths
Summary:
Depends on D19182. Ref T11015. This changes `path` from `text255` to `longtext` because paths may be arbitrarily long.

It adds `pathDisplay` to prepare for display paths and storage paths having different values. For now, `pathDisplay` is copied from `path` and always has the same value.

Test Plan:
  - Ran migration, checked database for sanity (all `pathDisplay` and `path` values identical).
  - Added new paths, saw `pathDisplay` and `path` get the same values.
  - Added an unreasonably enormous path with far more than 255 characters.

Maniphest Tasks: T11015

Differential Revision: https://secure.phabricator.com/D19183
2018-03-06 20:31:22 -08:00
epriestley
8cb273a053 Add a unique key to OwnersPath on "<packageID, repositoryPHID, pathIndex>"
Summary:
Depends on D19181. Ref T11015. This nukes duplicates from the table if they exist, then adds a unique key.

(Duplicates should not exist and can not be added with any recent version of the web UI.)

Test Plan:
  - Tried to add duplicates with web UI, didn't have any luck.
  - Explicitly added duplicates with manual `INSERT`s.
  - Viewed packages in web UI and saw duplicates.
  - Ran migrations, got a clean purge and a nice unique key.
  - There's still no way to actually hit a duplicate key error in the UI (unless you can collide hashes, I suppose), this is purely a correctness/robustness change.

Maniphest Tasks: T11015

Differential Revision: https://secure.phabricator.com/D19182
2018-03-06 20:30:59 -08:00
epriestley
1bf4422c74 Add and populate a pathIndex column for OwnersPath
Summary: Ref T11015. This supports making path names arbitrarily long and putting a proper unique key on the table.

Test Plan:
  - Migrated, checked database, saw nice digested indexes.
  - Edited a package, saw new rows update with digested indexes.

Maniphest Tasks: T11015

Differential Revision: https://secure.phabricator.com/D19181
2018-03-06 20:30:33 -08:00
epriestley
dbccfb234f Perform a client-side redirect after OAuth server authorization
Summary:
Ref T13099. See that task for discussion. Chrome is unhappy with an MFA form submitting to an endpoint which redirects you to an OAuth URI.

Instead, do the redirect entirely on the client.

Chrome's rationale here isn't obvious, so we may be able to revert this at some point.

Test Plan: Went through the OAuth flow locally, was redirected on the client. Will verify in production.

Maniphest Tasks: T13099

Differential Revision: https://secure.phabricator.com/D19177
2018-03-06 12:18:27 -08:00
epriestley
743d1ac426 Mostly modularize the Differential "update" transaction
Summary: Ref T13099. Move most of the "Update" logic to modular transactions

Test Plan: Created and updated revisions. Flushed the task queue. Grepped for `TYPE_UPDATE`. Reviewed update transactions in the timeline and feed.

Maniphest Tasks: T13099

Differential Revision: https://secure.phabricator.com/D19175
2018-03-06 09:10:32 -08:00
epriestley
44f0664d2c Add a "lock log" for debugging where locks are being held
Summary: Depends on D19173. Ref T13096. Adds an optional, disabled-by-default lock log to make it easier to figure out what is acquiring and holding locks.

Test Plan: Ran `bin/lock log --enable`, `--disable`, `--name`, etc. Saw sensible-looking output with log enabled and daemons restarted. Saw no additional output with log disabled and daemons restarted.

Maniphest Tasks: T13096

Differential Revision: https://secure.phabricator.com/D19174
2018-03-05 17:55:34 -08:00
epriestley
1f40e50f7e Improve live Harbormaster log follow behaviors
Summary:
Depends on D19166. Ref T13088. When the user scrolls away from a followed log, break the focus lock.

Let users stop following a live log.

Show when lines are added more clearly.

Don't refresh quite as quickly give users a better shot at clicking the stop button.

These behaviors can probably be refined but are at least more plausible and less actively user-hostile than the first version of this behavior was.

Test Plan: Used `write-log --rate` to write a large log slowly. Clicked "Follow Log", followed for a bit. Scrolled away, still got live updates but no more scroll lock. Clicked stop, no more updates.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19167
2018-03-01 13:11:22 -08:00
epriestley
4e91ad276d Prevent copying Harbormaster build log line numbers with CSS psuedocontent instead of ZWS
Summary:
Depends on D19165. Ref T13088. Currently, in other applications, we use Zero Width Spaces and Javascript "copy" listeners to prevent line numbers from being copied. This isn't terribly elegant.

Modern browsers support a second approach: using psuedo-elements with `content`. Try this in Harbormaster since it's conceptually cleaner, at least. One immediate drawback is that Command-F can't find this text either.

Test Plan: In Safari, Chrome and Firefox, highlighted ranges of lines and copy/pasted text. Got just text (no line numbers) in all cases.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19166
2018-03-01 13:03:40 -08:00
epriestley
73619c4643 Share the Paste line highlighting behavior for Harbormaster build logs
Summary: Depends on D19164. Ref T13088. Now that the JS behaviors are generic, use them on the Harbormaster standalone page.

Test Plan: Clicked lines and dragged across line ranges. Reloaded pages. Saw expected highlighting behavior in the client and on the server across reloads.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19165
2018-03-01 12:57:30 -08:00
epriestley
fe3de5dd58 Make Paste source code line highlighting behavior more generic
Summary: Depends on D19163. Ref T13088. Increase the generality of this code so it can be shared with Harbormaster.

Test Plan: Clicked individual lines, clicked-and-dragged, etc., in Paste. Got sensible URI and highlight behaviors.

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19164
2018-03-01 12:46:36 -08:00
epriestley
49af4165bc Support rendering arbitrary sections in the middle of a Harbormaster build log so links to line 3500 work
Summary:
Depends on D19162. Ref T13088. When a user links to `$1234`, we need to render a default view of the log with a piece at the head, a piece at the end, and a piece in the middle.

We also need to figure out the offset for line 1234, or multiple offsets for "1234-2345".

Since the logic views/reads mostly anticipated this it isn't too much of a mess, although there are a couple of bugs this exposes with view specifications that use combinations of parameters which were previously impossible.

Test Plan: Viewed a large log with no line marker. Viewed `$1`. Viewed `$end`. Viewed `$35-40`, etc. Expanded context around logs.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19163
2018-03-01 11:18:21 -08:00
epriestley
a2fdf14275 Stop using forms to download files in file embed and lightbox elements
Summary: Depends on D19156. Ref T13094. This replaces the remaining forms in the file embed view and lightbox with normal download links.

Test Plan: Clicked "Download" and lightbox -> download for embedded files.

Maniphest Tasks: T13094

Differential Revision: https://secure.phabricator.com/D19157
2018-02-28 17:21:07 -08:00
epriestley
ab579f2511 Never generate file download forms which point to the CDN domain, tighten "form-action" CSP
Summary:
Depends on D19155. Ref T13094. Ref T4340.

We can't currently implement a strict `form-action 'self'` content security policy because some file downloads rely on a `<form />` which sometimes POSTs to the CDN domain.

Broadly, stop generating these forms. We just redirect instead, and show an interstitial confirm dialog if no CDN domain is configured. This makes the UX for installs with no CDN domain a little worse and the UX for everyone else better.

Then, implement the stricter Content-Security-Policy.

This also removes extra confirm dialogs for downloading Harbormaster build logs and data exports.

Test Plan:
  - Went through the plain data export, data export with bulk jobs, ssh key generation, calendar ICS download, Diffusion data, Paste data, Harbormaster log data, and normal file data download workflows with a CDN domain.
  - Went through all those workflows again without a CDN domain.
  - Grepped for affected symbols (`getCDNURI()`, `getDownloadURI()`).
  - Added an evil form to a page, tried to submit it, was rejected.
  - Went through the ReCaptcha and Stripe flows again to see if they're submitting any forms.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13094, T4340

Differential Revision: https://secure.phabricator.com/D19156
2018-02-28 17:20:12 -08:00
epriestley
f114b2dd7d When viewing a live build log, trap users in a small personal hell where nothing but slavish devotion to the log exists
Summary: Depends on D19152. Ref T13088. This adds live log tailing. It is probably not the final version of this feature because it prevents escape once you begin tailing a log.

Test Plan: Used `bin/harbormaster write-log --rate ...` to write a log slowly. Viewed it in the web UI. Clicked "Follow Log". Followed the log until the write finished, a lifetime later.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19153
2018-02-28 12:38:41 -08:00
epriestley
143033dc1f When showing a small piece of a Harbormaster build log, load a small piece of data instead of the entire log
Summary: Depends on D19148. Ref T13088. The new rendering always executes range requests for data it needs, and we can satisfy these requests by loading the smallest number of chunks which span that range.

Test Plan: Piped 50,000 lines of Apache log into Harbormaster, viewed it in the new UI, got sensible rendering times and a reasonable amount of data actually going over the wire.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19149
2018-02-28 12:32:26 -08:00
epriestley
dba4c4bdf6 Emit a "Content-Security-Policy" HTTP header
Summary:
See PHI399. Ref T4340. This header provides an additional layer of protection against various attacks, including XSS attacks which embed inline `<script ...>` or `onhover="..."` content into the document.

**style-src**: The "unsafe-inline" directive affects both `style="..."` and `<style>`. We use a lot of `style="..."`, some very legitimately, so we can't realistically get away from this any time soon. We only use one `<style>` (for monospaced font preferences) but can't disable `<style>` without disabling `style="..."`.

**img-src**: We use "data:" URIs to inline small images into CSS, and there's a significant performance benefit from doing this. There doesn't seem to be a way to allow "data" URIs in CSS without allowing them in the document itself.

**script-src** and **frame-src**: For a small number of flows (Recaptcha, Stripe) we embed external javascript, some of which embeds child elements (or additional resources) into the document. We now whitelist these narrowly on the respective pages.

This won't work with Quicksand, so I've blacklisted it for now.

**connect-src**: We need to include `'self'` for AJAX to work, and any websocket URIs.

**Clickjacking**: We now have three layers of protection:

  - X-Frame-Options: works in older browsers.
  - `frame-ancestors 'none'`: does the same thing.
  - Explicit framebust in JX.Stratcom after initialization: works in ancient IE.

We could probably drop the explicit framebust but it wasn't difficult to retain.

**script tags**: We previously used an inline `<script>` tag to start Javelin. I've moved this to `<data data-javelin-init ...>` tags, which seems to work properly.

**`__DEV__`**: We previously used an inline `<script>` tag to set the `__DEV__` mode flag. I tried using the "initialization" tags for this, but they fire too late. I moved it to `<html data-developer-mode="1">`, which seems OK everywhere.

**CSP Scope**: Only the CSP header on the original request appears to matter -- you can't refine the scope by emitting headers on CSS/JS. To reduce confusion, I disabled the headers on those response types. More headers could be disabled, although we're likely already deep in the land of diminishing returns.

**Initialization**: The initialization sequence has changed slightly. Previously, we waited for the <script> in bottom of the document to evaluate. Now, we go fishing for tags when domcontentready fires.

Test Plan:
  - Browsed around in Firefox, Safari and Chrome looking for console warnings. Interacted with various Javascript behaviors. Enabled Quicksand.
  - Disabled all the framebusting, launched a clickjacking attack, verified that each layer of protection is individually effective.
  - Verified that the XHProf iframe in Darkconsole and the PHPAST frame layout work properly.
  - Enabled notifications, verified no complaints about connecting to Aphlict.
  - Hit `__DEV__` mode warnings based on the new data attribute.
  - Tried to do sketchy stuff with `data:` URIs and SVGs. This works but doesn't seem to be able to do anything dangerous.
  - Went through the Stripe and Recaptcha workflows.
  - Dumped and examined the CSP headers with `curl`, etc.
  - Added a raw <script> tag to a page (as though I'd found an XSS attack), verified it was no longer executed.

Maniphest Tasks: T4340

Differential Revision: https://secure.phabricator.com/D19143
2018-02-27 10:17:30 -08:00