Summary:
Ref T7149. We can simplify configuration somewhat by removing the upload limit setting, now that we support arbitrarily large files.
- Merge configuration documentation.
- Tell users to set things to at least 32MB. This is 8MB maximum one-shot file + 4x headroom. Chunk sizes are 4MB.
Test Plan:
- Faked all the setup warnings.
- Read documentation.
- Uploaded some files.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12083
Summary: Ref T7149. This works now, so enable it.
Test Plan:
- Uploaded large and small files in Firefox, Safari and Chrome.
- Uploaded large files with `arc upload`.
- Stopped/resumed large files with all clients.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12079
Summary: Ref T7149. This was just to make testing easier, but chunking substantially works now.
Test Plan: `grep`
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12076
Summary:
Right now, if a daemon dies it can leave the setup warning around for like 10 minutes or something until we reap it.
Tighten the warning so we only care about actively running daemons.
Test Plan: Checked setup issues.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D12088
Summary: Normalizes size and colors. Default to non-bold names, grey text, and hide the time in column. Also re-evaluated header spacing in Photoshop.
Test Plan: Lots of photoshop, tested full in desktop, mobile and tablet and normal durable column. This was sadly 2 hours of work.
Reviewers: epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12087
Summary:
- Don't show a loading state on the whole column while sending chat. We could show some kind of minor loading state, but standard JX.Busy stuff will kick in after a couple seconds anyway.
- Blank the textarea immediately on submit so you can start typing more text.
- Don't disable the form while submiting; disabling it prevents you from typing more text.
- Hide the placeholder while the textarea is focused. If we don't do this, the placeholder reappearing after submitting text feels weird to me.
Test Plan:
- Sent a lot of text.
- Real fast.
- Focused and unfocused the area.
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D12086
Summary: Ref T7538. We can figure out whether to backport this to main Conpherence later and/or remove buttons, etc., but this behavior seems pretty clearly good.
Test Plan:
- Pressed enter (sent message).
- Pressed shift+enter (newline).
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T7538
Differential Revision: https://secure.phabricator.com/D12085
Summary: Fixes T7529. I think stylesheet order got juggled at some point and made this more specific, but we don't actually need it.
Test Plan:
- Column now looks reasonable.
- Everything else does too.
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T7529
Differential Revision: https://secure.phabricator.com/D12084
Summary: Currently we punch down Dashboard columns on smaller displays. This adds another set of rules for if durable-column is present.
Test Plan: Test breakpoints at 1300 and 1000 pixel wide.
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7531
Differential Revision: https://secure.phabricator.com/D12056
Summary: Ref T7149. Return a real iterator from the Chunk engine, which processes chunks sequentially.
Test Plan:
This is a bit hard to read, but shows the underlying chunks being accessed one at a time and only some being accessed when requesting a range of a file:
```
$ ./bin/files cat F878 --trace --begin 100 --end 256
...
>>> [10] <query> SELECT * FROM `file_storageblob` WHERE `id` = 85
<<< [10] <query> 240 us
better software.
Phabricat>>> [11] <query> SELECT * FROM `file_storageblob` WHERE `id` = 84
<<< [11] <query> 205 us
or includes applications for:
>>> [12] <query> SELECT * FROM `file_storageblob` WHERE `id` = 83
<<< [12] <query> 226 us
- reviewing and auditing source>>> [13] <query> SELECT * FROM `file_storageblob` WHERE `id` = 82
<<< [13] <query> 203 us
code;
- hosting and browsing >>> [14] <query> SELECT * FROM `file_storageblob` WHERE `id` = 81
<<< [14] <query> 231 us
repositories;
- tracking bugs;
```
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12073
Summary:
Ref T7149. This still buffers the whole file, but is reaaaaal close to not doing that.
Allow Responses to be streamed, and rewrite the range stuff in the FileResponse so it does not rely on having the entire content available.
Test Plan:
- Artificially slowed down downloads, suspended/resumed them (works in chrome, not so much in Safari/Firefox?)
- Played sounds in Safari/Chrome.
- Viewed a bunch of pages and files in every browser.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12072
Summary: Ref T7149. A couple diffs down the line, this will let us emit chunked files without doing all the work up front or holding the entire file in RAM.
Test Plan:
(Some newlines added for clarity.)
```
$ ./bin/files cat F942
ABCDEFGHIJKLMNOPQRSTUVWXYZ
$ ./bin/files cat F942 --begin 1
BCDEFGHIJKLMNOPQRSTUVWXYZ
$ ./bin/files cat F942 --end 10
ABCDEFGHIJ
$ ./bin/files cat F942 --begin 3 --end 5
DE
$
```
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12071
Summary: Ref T7149. We can't compute hashes of large files efficiently, but we can resume uploads by the same author, with the same name and file size, which are only partially completed. This seems like a reasonable heuristic that is unlikely to ever misfire, even if it's a little magical.
Test Plan:
- Forced chunking on.
- Started uploading a chunked file.
- Closed the browser window.
- Dropped it into a new window.
- Upload resumed //(!!!)//
- Did this again.
- Downloaded the final file, which successfully reconstructed the original file.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, chad, epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12070
Summary: Changes the text to just "Stay", which is still obvious what it means, with less copy. Fixes T7027
Test Plan: Now works on mobile.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7027
Differential Revision: https://secure.phabricator.com/D12075
Summary: Fixes T7556, adds margin instead of padding to links.
Test Plan: Set my status as away, see proper spacing.
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7556
Differential Revision: https://secure.phabricator.com/D12069
Summary:
Ref T7149. This adds chunking support to drag-and-drop uploads. It never activates right now unless you hack things up, since the chunk engine is still hard-coded as disabled.
The overall approach is the same as `arc upload` in D12061, with some slight changes to the API return values to avoid a few extra HTTP calls.
Test Plan:
- Enabled chunk engine.
- Uploaded some READMEs in a bunch of tiny 32 byte chunks.
- Worked out of the box in Safari, Chrome, Firefox.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12066
Summary:
Ref T7149. This flags allocated but incomplete files and doesn't explode when trying to download them.
Files are marked complete when the last chunk is uploaded.
I added a key on `<authorPHID, isPartial>` so we can show you a list of partially uploaded files and prompt you to resume them at some point down the road.
Test Plan: Massaged debugging settings and uploaded README.md very slowly in 32b chunks. Saw the file lose its "Partial" flag when the last chunk finished.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12063
Summary:
Ref T7149. This makes debugging some of this stuff a bit easier by removing the HTTP part in the middle.
Particularly, I anticipate having this stream data chunk-by-chunk in the near future.
Test Plan: Ran `files cat F23`, got output.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12062
Summary:
Ref T7149. This isn't complete and isn't active yet, but does basically work. I'll shore it up in the next few diffs.
The new workflow goes like this:
> Client, file.allocate(): I'd like to upload a file with length L, metadata M, and hash H.
Then the server returns `upload` (a boolean) and `filePHID` (a PHID). These mean:
| upload | filePHID | means |
|---|---|---|
| false | false | Server can't accept file.
| false | true | File data already known, file created from hash.
| true | false | Just upload normally.
| true | true | Query chunks to start or resume a chunked upload.
All but the last case are uninteresting and work like exising uploads with `file.uploadhash` (which we can eventually deprecate).
In the last case:
> Client, file.querychunks(): Give me a list of chunks that I should upload.
This returns all the chunks for the file. Chunks have a start byte, an end byte, and a "complete" flag to indicate that the server already has the data.
Then, the client fills in chunks by sending them:
> Client, file.uploadchunk(): Here is the data for one chunk.
This stuff doesn't work yet or has some caveats:
- I haven't tested resume much.
- Files need an "isPartial()" flag for partial uploads, and the UI needs to respect it.
- The JS client needs to become chunk-aware.
- Chunk size is set crazy low to make testing easier.
- Some debugging flags that I'll remove soon-ish.
- Downloading works, but still streams the whole file into memory.
- This storage engine is disabled by default (hardcoded as a unit test engine) because it's still sketchy.
- Need some code to remove the "isParital" flag when the last chunk is uploaded.
- Maybe do checksumming on chunks.
Test Plan:
- Hacked up `arc upload` (see next diff) to be chunk-aware and uploaded a readme in 18 32-byte chunks. Then downloaded it. Got the same file back that I uploaded.
- File UI now shows some basic chunk info for chunked files:
{F336434}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12060
Summary: D12058 for all its brilliance should have applied this css rule to all instances of this, not just once the column is toggled
Test Plan: tried to make the flickering happen WITHOUT the column toggled and failed; uploaded successfully
Reviewers: chad, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12059
Summary:
Ref T7539. This stuff is pretty whack - the "dragEnter" and "dragLeave" fire over and over and over despite not moving the mouse sometimes from JX.Mask interaction, causing that neat flickering effect. Fix this mess by disabling pointer events for the mask.
Test Plan: dragged a file to the durable column textarea. it uploaded and inlined the Fxxx into the message. tried uploading a few times and it worked repeatedly.
Reviewers: epriestley, chad
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7539
Differential Revision: https://secure.phabricator.com/D12058
Summary: Fixes T7539. We need to set the "with-column" css class on the document body to make things like the jx-mask style-able. Also, make the global upload control only do it for the standard phabrcator page and not the document body.
Test Plan: dragged a file to conpherence column and it worked! uploaded a file to homepage with column open and it worked! uploaded a file to /file/ with column open and it worked!
Reviewers: chad, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7539
Differential Revision: https://secure.phabricator.com/D12055
Summary:
Fixes T5843. File storage engines use a very old "selector" mechanism which makes them difficult to extend.
This mechanism predates widespread use of `PhutilSymbolLoader` to discover available implementations at runtime. Runtime discovery has generally proven more flexible and easier to use than explicit selection (although it sometimes needs more UI to support it in cases where order or enabled/disabled flags can not be directly determined).
Use a modern runtime discovery mechanism instead of an explicit selector. This might break any installs which subclassed the `Selector`, but I believe almost no such installs exist, and they'll receive a meaningful exception upon upgrading (any custom engines will no longer implement all of the required methods).
Looking forward, this modernizes infrastructure to prepare for new "virtual" chunked-storage engines, with the eventual goal of supporting very large file uploads and data import into the Phacility cluster.
This uses D12051 to add UI to make it easier to understand the state of storage engines.
Test Plan:
Used new UI panel to assess storage engines:
{F336270}
- Uploaded a small file, saw it go to MySQL engine.
- Uploaded a larger file, saw it go to S3 engine.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5843
Differential Revision: https://secure.phabricator.com/D12053
Summary:
Ref T7149. This is a few steps away, but:
- Generally, I'd like to reduce the amount of "Config" configuration we have.
- One good way to do this is to move it into UIs in Application configuration. We did this with email recently.
- I think this was a great change and I'd like to keep moving in this direction.
- T7149 touches configuration related to file storage engines. Although I'm not planning to fully move configuration into applications yet, it would be easier to debug and test if I could drop a read-only panel there to show engines.
- So, modularize the config stuff so I can add a new panel without hard-coding it.
Test Plan:
- Added, edited, and deleted application emails.
- Viewed non-email application detail pages.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7149
Differential Revision: https://secure.phabricator.com/D12051
Summary: I left in an opacity change by mistake, and fix language on threads.
Test Plan: review in sandbox
Reviewers: epriestley, btrahan
Reviewed By: btrahan
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12054
Summary:
Ref T7014. This makes it so
- you can't invoke the column from a device
- if you are in desktop size and resize to tablet or phone, the column closes.
- if you resize desktop -> device -> desktop, the column closes at device size and reopens at desktop size
- if you load, then resize device -> desktop the column opens if the user has that preference
- there is a brief flicker when you load on 'device' with the column open preference. it lasts as long as the js stack takes to calculate the device css rule.
Test Plan: see summary but i did stuff to do all that
Reviewers: chad, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7014
Differential Revision: https://secure.phabricator.com/D12052
Summary: This adds a parameter for time only on Conpherence Transactions, although grepping around, Conpherence might be the only user of this View at this point. Since we have the date markers separately, we can use just the timestamp for a cleaner feel. Also updated a bit of the spacing and colors to match Conpherence Full. Ref T7531
Test Plan:
A lot of Photoshop, and different types of chats.
{F336204}
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7531
Differential Revision: https://secure.phabricator.com/D12049
Summary: I don't know the names of all the Conpherences I have ongoing and all my test icons are Psyducks. haha ha ...
Test Plan: Hover over icons, now it's all Psyduck and [No Title].
Reviewers: epriestley, btrahan
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12050
Summary: Correct minor spacing issues, makes long Conpherence Titles fallback to ellipsis when overflow. Fixes T7541
Test Plan:
Test a long and short title Conpherence.
{F336174}
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7541
Differential Revision: https://secure.phabricator.com/D12048
Summary: This makes macros and memes grow to 100% of their container //at most//, instead of showing a scrollbar. This is useful for overly large macros, smaller spaces like Feed and Conpherences, and Inline Comments. Fixes T7528
Test Plan: Tested a very large macro, a very large meme, and a very very tiny macro. It looks like memes get cached though, unsure if we should clean them up or just leave them
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7528
Differential Revision: https://secure.phabricator.com/D12045
Summary: Ref T7014. This got broken in today's action. For whatever reason the only way I can get the CSS to show up correctly is to move the require statement to where it was before rP5ef99dba2afc9f9ed3ca77707366a78be15f4871. Otherwise, this feature massages the UI a bit to make sure the "loading" stuff is set correctly in this state.
Test Plan: toggled conpherence open and it looked good. reloaded and it looked good.
Reviewers: chad, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7014
Differential Revision: https://secure.phabricator.com/D12047
Summary: Ref T7014. This changes the title and selected icon right as the user clicks it. This could //maybe// be in the "willLoadThread" callback hook, but it doesn't happen every time we load a thread, just **this** time so keep it right in the listener for now.
Test Plan: switched some threads and liked what I saw
Reviewers: epriestley, chad
Subscribers: Korvin, epriestley
Maniphest Tasks: T7014
Differential Revision: https://secure.phabricator.com/D12043
Summary: Ref T7014. This just makes it so there's almost no UI and a simple "You have no messages. <button>Send a message.</button>" UI
Test Plan: hacked the code such that should_404 and conpherence were false and null respectively. verified i got the right ui in the durable column. verified send a message button worked, ending up with me in main conpherence view on the right message
Reviewers: chad, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7014
Differential Revision: https://secure.phabricator.com/D12042
Summary: Numerous visual updates to the Durable Column, mostly to emulate current Conpherence look and feel.
Test Plan: Lots of little pixel chasing. Also Chrome, Firefox.
Reviewers: epriestley, btrahan
Reviewed By: btrahan
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12041
Summary:
This is cool in theory, but has broken like 5 times and is broken now too. The CSS magic just isn't robust enough to keep up with CSS changes.
Just strip it out for now; if we come up with some more durable replacement we can put that back in its place.
Test Plan: Typed konami code, page didn't break horribly.
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D12039
Summary: When you open the column, keep it open on future requests.
Test Plan: Opened column, clicked to Conpherence (no column), clicked elsewhere (column again), reloaded page (column), closed column, clicked something (no column).
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D12038
Summary:
Fixes T7060. Removes some hard-coding.
This assumes that "pages with no durable column" and "pages with no Quicksand" are the same, but that's correct today and I can't come up with a use case where they'd be different offhand.
Test Plan:
- Clicked a revision with column open, got Quicksand navigation.
- Clicked into Conpherence with column open, got real navigation.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7060
Differential Revision: https://secure.phabricator.com/D12036
Summary: Ref T7380. This does the most basic thing ever and sticks up to 6 icons in there.
Test Plan: clicked the icons and noted new conpherences loaded in nicely
Reviewers: epriestley, chad
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7380
Differential Revision: https://secure.phabricator.com/D12037
Summary:
Ref T5369. New HTML5 version without flash dependencies.
This doesn't play any sounds.
Test Plan: Did not play any sounds.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T5369
Differential Revision: https://secure.phabricator.com/D9535
Summary: Ref T7014. The main conpherence view is kind of broken without this in subtle ways because of /conpherence/ versus /conpherence/x/ init'ing things differently; this fixes that. Moves more normal view conpherence logic into threadManager. Makes all the display code happen outside of threadManager, setting us up for some display manager later maybe.
Test Plan: sent messages, updated title, etc and the messages pane auto scrolled correctly!
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7014
Differential Revision: https://secure.phabricator.com/D12035
