Summary:
Fixes T10750. Files have some outdated cache/key code which prevents recording an edit history on file comments.
Remove this ancient cruft.
(Users must `bin/storage adjust` after upgrading to this patch to reap the benefits.)
Test Plan:
- Ran `bin/storage adjust`.
- Edited a comment in Files.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10750
Differential Revision: https://secure.phabricator.com/D16312
Summary:
Fixes T11307. Fixes T8124. Currently, builtin files are tracked by using a special transform with an invalid source ID.
Just use a dedicated column instead. The transform thing is too clever/weird/hacky and exposes us to issues with the "file" and "transform" tables getting out of sync (possibly the issue in T11307?) and with race conditions.
Test Plan:
- Loaded profile "edit picture" page, saw builtins.
- Deleted all builtin files, put 3 second sleep in the storage engine write, loaded profile page in two windows.
- Before patch: one of them failed with a race.
- After patch: both of them loaded.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T8124, T11307
Differential Revision: https://secure.phabricator.com/D16271
Summary:
Fixes T10907. As written, this workflow will incorrectly reuse a temporary file if one exists.
Instead, make a new permanent file.
(Storage is still shared, so this usually will not actually create a copy of the file's data.)
Test Plan:
- Set a project's icon by clicking first button in "Use Picture" row.
- Before patch: temporary image was reused.
- After patch: new permanent file is generated.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10907
Differential Revision: https://secure.phabricator.com/D16270
Summary:
Ref T10628. Currently, tabs are part of ObjectBoxes. However, the code is a bit of a mess and I want to use them in some other contexts, notably the "prose diff" dialog to show "old raw, new raw, diff".
Pull them out, and update Files to use the new stuff. My plan is:
- Update all callsites to this stuff.
- Remove the builtin-in ObjectBox integration to simplify ObjectBox a bit.
- Move forward with T10628.
This is pretty straightforward. A couple of the sigils are a little weird, but I'll update the JS later. For now, the same JS can drive both old and new tabs.
Test Plan: Viewed files, everything was unchanged.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10628
Differential Revision: https://secure.phabricator.com/D16205
Summary:
Fixes T11242. See that task for detailed discussion.
Previously, it didn't particularly matter that we don't MIME detect chunked files since they were all just big blobs of junk (PSDs, zips/tarballs, whatever) that we handled uniformly.
However, videos are large and the MIME type also matters.
- Detect the overall mime type by detecitng the MIME type of the first chunk. This appears to work properly, at least for video.
- Skip mime type detection on other chunks, which we were performing and ignoring. This makes uploading chunked files a little faster since we don't need to write stuff to disk.
Test Plan:
Uploaded a 50MB video locally, saw it as chunks with a "video/mp4" mime type, played it in the browser in Phabricator as an embedded HTML 5 video.
{F1706837}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11242
Differential Revision: https://secure.phabricator.com/D16204
Summary:
Ref T11140. This makes encryption actually work:
- Provide a new configuation option, `keyring`, for specifying encryption keys.
- One key may be marked as `default`. This activates AES256 encryption for Files.
- Add `bin/files generate-key`. This is helps when generating valid encryption keys.
- Add `bin/files encode`. This changes the storage encoding of a file, and helps test encodings and migrate existing data.
- Add `bin/files cycle`. This re-encodes the block key with a new master key, if your master key leaks or you're just paraonid.
- Document all these options and behaviors.
Test Plan:
- Configured a bad `keyring`, hit a bunch of different errors.
- Used `bin/files generate-key` to try to generate bad keys, got appropriate errors ("raw doesn't support keys", etc).
- Used `bin/files generate-key` to generate an AES256 key.
- Put the new AES256 key into the `keyring`, without `default`.
- Uploaded a new file, verified it still uploaded as raw data (no `default` key yet).
- Used `bin/files encode` to change a file to ROT13 and back to raw. Verified old data got deleted and new data got stored properly.
- Used `bin/files encode --key ...` to explicitly convert a file to AES256 with my non-default key.
- Forced a re-encode of an AES256 file, verified the old data was deleted and a new key and IV were generated.
- Used `bin/files cycle` to try to cycle raw/rot13 files, got errors.
- Used `bin/files cycle` to cycle AES256 files. Verified metadata changed but file data did not. Verified file data was still decryptable with metadata.
- Ran `bin/files cycle --all`.
- Ran `encode` and `cycle` on chunked files, saw commands fail properly. These commands operate on the underlying data blocks, not the chunk metadata.
- Set key to `default`, uploaded a file, saw it stored as AES256.
- Read documentation.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11140
Differential Revision: https://secure.phabricator.com/D16127
Summary:
Ref T11140. This doesn't do anything yet since there's no way to enable it and no way to store master keys.
Those are slightly tougher problems and I'm not totally satisfied that I have an approach I really like for either problem, so I may wait for a bit before tackling them. Once they're solved, this does the mechanical encrypt/decrypt stuff, though.
This design is substantially similar to the AWS S3 server-side encryption design, and intended as an analog for it. The decisions AWS has made in design generally seem reasonable to me.
Each block of file data is encrypted with a unique key and a unique IV, and then that key and IV are encrypted with the master key (and a distinct, unique IV). This is better than just encrypting with the master key directly because:
- You can rotate the master key later and only need to re-encrypt a small amount of key data (about 48 bytes per file chunk), instead of re-encrypting all of the actual file data (up to 4MB per file chunk).
- Instead of putting the master key on every server, you can put it on some dedicated keyserver which accepts encrypted keys, decrypts them, and returns plaintext keys, and can send it 32-byte keys for decryption instead of 4MB blocks of file data.
- You have to compromise the master key, the database, AND the file store to get the file data. This is probably not much of a barrier realistically, but it does make attacks very slightly harder.
The "KeyRing" thing may change once I figure out how I want users to store master keys, but it was the simplest approach to get the unit tests working.
Test Plan:
- Ran unit tests.
- Dumped raw data, saw encrypted blob.
- No way to actually use this in the real application yet so it can't be tested too extensively.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11140
Differential Revision: https://secure.phabricator.com/D16124
Summary:
Ref T11140. When reading and writing files, we optionally apply a "storage format" to them.
The default format is "raw", which means we just store the raw data.
This change modularizes formats and adds a "rot13" format, which proves formatting works and is testable. In the future, I'll add real encryption formats.
Test Plan:
- Added unit tests.
- Viewed files in web UI.
- Changed a file's format to rot13, saw the data get rotated on display.
- Set default format to rot13:
- Uploaded a small file, verified data was stored as rot13.
- Uploaded a large file, verified metadata was stored as "raw" (just a type, no actual data) and blob data was stored as rot13.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11140
Differential Revision: https://secure.phabricator.com/D16122
Summary: Ref T11142. H264 video in a Quicktime container works in Safari and Firefox for me (although not Chrome), so include it in the default video mime types.
Test Plan: Uploaded video file from T11142 locally, saw it render with `<video />` properly in Safari.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11142
Differential Revision: https://secure.phabricator.com/D16121
Summary: Ref T6523. Allows you to click stuff instead of using drag-and-drop.
Test Plan: On iOS simulator, created and updated a mock.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T6523
Differential Revision: https://secure.phabricator.com/D16088
Summary: Ref T6916. Added video to remarkup using D7156 as reference.
Test Plan:
- Viewed video files (MP4, Ogg) in Safari, Chrome, Firefox (some don't work, e.g., OGG in Safari, but nothing we can really do about that).
- Used `alt`.
- Used `autoplay`.
- Used `loop`.
- Used `media=audio`.
- Viewed file detail page.
Reviewers: nateguchi2, chad, #blessed_reviewers
Reviewed By: chad, #blessed_reviewers
Subscribers: asherkin, ivo, joshuaspence, Korvin, epriestley
Tags: #remarkup
Maniphest Tasks: T6916
Differential Revision: https://secure.phabricator.com/D11297
Summary:
Ref T4103. Ref T10078. This puts a user cache in front of notification and message counts.
This reduces the number of queries issued on every page by 4 (2x building the menu, 2x building Quicksand data).
Also fixes some minor issues:
- Daemons could choke on sending mail in the user's translation.
- No-op object updates could fail in the daemons.
- Questionable data access pattern in the file query coming out of the profile file cache.
Test Plan:
- Sent myself notifications. Saw count go up.
- Cleared them by visiting objects and clearing all notifications. Saw count go down.
- Sent myself messages. Saw count go up.
- Cleared them by visiting threads. Saw count go down.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103, T10078
Differential Revision: https://secure.phabricator.com/D16041
Summary:
Ref T5187. This definitely feels a bit flimsy and I'm going to hold it until I cut the release since it changes a couple of things about Workflow in general, but it seems to work OK and most of it is fine.
The intent is described in T5187#176236.
In practice, most of that works like I describe, then the `phui-file-upload` behavior gets some weird glue to figure out if the input is part of the form. Not the most elegant system, but I think it'll hold until we come up with many reasons to write a lot more Javascript.
Test Plan:
Used both drag-and-drop and the upload dialog to upload files in Safari, Firefox and Chrome.
{F1653716}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T5187
Differential Revision: https://secure.phabricator.com/D15953
Summary: Seems to work ok, if you give `size=wide` to an image file, we blow it out a bit in DocumentPro mode.
Test Plan:
Test in Phame and Maniphest.
{F1256717}
{F1256718}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15840
Summary:
Ref T7789. Currently, we use different viewers if you have `security.alternate-file-domain` configured vs if you do not.
This is largely residual from the days of one-time-tokens, and can cause messy configuration-dependent bugs like the one in T7789#172057.
Instead, always use the omnipotent viewer. Knowledge of the secret key alone is sufficient to access a file.
Test Plan:
- Disabled `security.alternate-file-domain`.
- Reproduced an issue similar to the one described on T7789.
- Applied change.
- Clean LFS interaction.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7789
Differential Revision: https://secure.phabricator.com/D15784
Summary: Ref T10262. Instead of dumping an unhelpful 403 "ACCESS DENIED" page on users, explain the most likely cause of the issue and give them a link to return to the file detail page to learn more or get an up-to-date link.
Test Plan: Hit both errors, had a lovely experience with the helpful dialog text.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10262
Differential Revision: https://secure.phabricator.com/D15650
Summary:
Ref T10262. Currently, we always render a tag like this when you `{F123}` an image in remarkup:
```
<img src="/xform/preview/abcdef/" />
```
This either generates the preview or redirects to an existing preview. This is a good behavior in general, because the preview may take a while to generate and we don't want to wait for it to generate on the server side.
However, this flickers a lot in Safari. We might be able to cache this, but we really shouldn't, since the preview URI isn't a legitimately stable/permanent one.
Instead, do a (cheap) server-side check to see if the preview already exists. If it does, return a direct URI. This gives us a stable thumbnail in Safari.
Test Plan:
- Dragged a dog picture into comment box.
- Typed text.
- Thing didn't flicker like crazy all the time in Safari.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10262
Differential Revision: https://secure.phabricator.com/D15646
Summary:
Ref T10262. This removes one-time tokens and makes file data responses always-cacheable (for 30 days).
The URI will stop working once any attached object changes its view policy, or the file view policy itself changes.
Files with `canCDN` (totally public data like profile images, CSS, JS, etc) use "cache-control: public" so they can be CDN'd.
Files without `canCDN` use "cache-control: private" so they won't be cached by the CDN. They could still be cached by a misbehaving local cache, but if you don't want your users seeing one anothers' secret files you should configure your local network properly.
Our "Cache-Control" headers were also from 1999 or something, update them to be more modern/sane. I can't find any evidence that any browser has done the wrong thing with this simpler ruleset in the last ~10 years.
Test Plan:
- Configured alternate file domain.
- Viewed site: stuff worked.
- Accessed a file on primary domain, got redirected to alternate domain.
- Verified proper cache headers for `canCDN` (public) and non-`canCDN` (private) files.
- Uploaded a file to a task, edited task policy, verified it scrambled the old URI.
- Reloaded task, new URI generated transparently.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10262
Differential Revision: https://secure.phabricator.com/D15642
Summary:
Ref T10262. Files have an internal secret key which is partially used to control access to them, and determines part of the URL you need to access them. Scramble (regenerate) the secret when:
- the view policy for the file itself changes (and the new policy is not "public" or "all users"); or
- the view policy or space for an object the file is attached to changes (and the file policy is not "public" or "all users").
This basically means that when you change the visibility of a task, any old URLs for attached files stop working and new ones are implicitly generated.
Test Plan:
- Attached a file to a task, used `SELECT * FROM file WHERE id = ...` to inspect the secret.
- Set view policy to public, same secret.
- Set view policy to me, new secret.
- Changed task view policy, new secret.
- Changed task space, new secret.
- Changed task title, same old secret.
- Added and ran unit tests which cover this behavior.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10262
Differential Revision: https://secure.phabricator.com/D15641
Summary: Modernize Files a bit, use newPage
Test Plan: New file, drag and drop file, view file, edit file
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15631
Summary:
Ref T7789. Ref T10604. This implements the `upload` action, which streams file data into Files.
This makes Git LFS actually work, at least roughly.
Test Plan:
- Tracked files in an LFS repository.
- Pushed LFS data (`git lfs track '*.png'; git add something.png; git commit -m ...; git push`).
- Pulled LFS data (`git checkout master^; rm -rf .git/lfs; git checkout master; open something.png`).
- Verified LFS refs show up in the gitlfsref table.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7789, T10604
Differential Revision: https://secure.phabricator.com/D15492
Summary: Fixes T10603. This is the last of the ad-hoc temporary tokens.
Test Plan:
- Used a file token.
- Viewed type in {nav Config > Temporary Tokens}.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15481
Summary:
Ref T10603. This makes minor updates to temporary tokens:
- Rename `objectPHID` (which is sometimes used to store some other kind of identifier instead of a PHID) to `tokenResource` (i.e., which resource does this token permit access to?).
- Add a `userPHID` column. For LFS tokens and some other types of tokens, I want to bind the token to both a resource (like a repository) and a user.
- Add a `properties` column. This makes tokens more flexible and supports custom behavior (like scoping LFS tokens even more tightly).
Test Plan:
- Ran `bin/storage upgrade -f`, got a clean upgrade.
- Viewed one-time tokens.
- Revoked one token.
- Revoked all tokens.
- Performed a one-time login.
- Performed a password reset.
- Added an MFA token.
- Removed an MFA token.
- Used a file token to view a file.
- Verified file token was removed after viewing file.
- Linked my account to an OAuth1 account (Twitter).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15478
Summary:
Every caller returns `true`. This was added a long time ago for Projects, but projects are no longer subscribable.
I don't anticipate needing this in the future.
Test Plan: Grepped for this method.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15409
Summary: This makes small cover images full-width instead of teeny tiny dots in the middle of an island of whitespace.
Test Plan: Uploaded a small cover image.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15203
Summary:
No way to set photos yet, but if you magic them in they work.
Primarily, this consolidates rendering logic so the move + edit + view controllers all run the same code to do tags / cover photos.
Test Plan: {F1095870}
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15201
Summary: Fixes T10273. The threshold is `null` if no chunk engines are available, but the code didn't handle this properly.
Test Plan: Disabled all chunk engines, reloaded, hit issue described in task. Applied patch, got clean file content.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10273
Differential Revision: https://secure.phabricator.com/D15179
Summary: Mostly for consistency, we're not using other forms of icons and this makes all classes that use an icon call it in the same way.
Test Plan: tested uiexamples, lots of other random pages.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15125
Summary: I moved history to manage and missed this callsite.
Test Plan: Use present icon/color quick select.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15099
Summary:
First pass at a new Project Home page. This is starting to sprawl, so punting this up now before it gets too large.
- Project homes now have "large header"
- Custom Fields / Descriptions are in the main column
- Feed is simpler visually
- new "Background" option for PHUIObjectBoxView
- move header buttons globally to "Grey" instead of "Simple"
- New color and hover states for "Grey"
- Transitions on Buttons haha
- Edit Icon on Nav is now under "Manage" panel
- New "Manage" Panel
TODO:
- More testing of bad cases of Custom Fields
- Members Page in flux, needs design
- Um still not sure how to make Custom Field not show UI
Test Plan:
Lots of random Project page visits. Save project, watch project, edit project, etc.
{F1068191}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15097
Summary:
Fixes T10186. After D14970, `diffusion.filecontentquery` puts the content in a file and returns the file PHID.
However, it does this in a way that doesn't go through the chunking engine, so it will fail for files larger than the chunk threshold (generally, 8MB).
Instead, stream the file from the underlying command directly into chunked storage.
Test Plan:
- Made a commit including a really big file: 4dcd4c492b
- Used `diffusion.filecontentquery` to load file content.
- Parsed/imported commit locally.
- Used `diffusion.filecontentquery` to load content for smaller files (README, etc).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10186
Differential Revision: https://secure.phabricator.com/D15072
Summary:
Ref T6183. Ref T10054. Historically, only members could watch projects because there were some weird special cases with policies. These policy issues have been resolved and Herald is generally powerful enough to do equivalent watches on most objects anyway.
Also puts a "Watch Project" button on the feed panel to make the behavior and meaning more obvious.
Test Plan:
- Watched a project I was not a member of.
- Clicked the feed watch/unwatch button.
{F1064909}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T6183, T10054
Differential Revision: https://secure.phabricator.com/D15063
Summary:
Ref T10054. This shuffles some stuff around to move us closer to mocks in M1450 in terms of what information is on which pages.
Home now has feed, members, watchers, link to "edit project / project edit history".
History now has edit history, edit details, edit picture, archive/unarchive.
Test Plan:
New home page:
{F1064889}
New edit/history page:
{F1064890}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15060
Summary: These are old project image choices, remove and only go with FontAwesome related images.
Test Plan: Project -> Edit Picture -> Save
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15051
Summary:
Fixes T6856. Fixes T10164.
- Make the profile image composer code use the underlying icon name instead of the top-level icon key, so it works instead of 404'ing.
- Change the button to show a preview of the profile icon instead of the text "Use Icon and Color".
- When creating a new non-milestone project, automatically set the profile image to the icon + color image.
Test Plan:
- Created several new projects, saw appropriate default icons.
- Edited projects, saw icon previews.
- Clicked icon buttons to set icons.
- Poked around other applications which use builtins (Pholio, user profiles) to look for anything I broke, but everything seemed fine.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T6856, T10164
Differential Revision: https://secure.phabricator.com/D15050
Summary: Ref T10054. Just simplifying this a bit before I start laying in the new profile menus.
Test Plan:
- Viewed Diviner on desktop and checked the mobile menu.
- Viewed Files on desktop and checked the mobile menu.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10054
Differential Revision: https://secure.phabricator.com/D15015
Summary:
Ref T5155. Swaps Phabricator over to the new first-party S3 client using the v4 authentication API so it works in all regions.
The API requires an explicit region, so the new `amazon-s3.region` is now required. I'll write guidance about this.
Test Plan:
- Uploaded files to S3.
- Migrated ~1GB of files to S3.
- Loaded a bunch of files off S3.
- Browsed around the S3 bucket.
- Deleted a file, verified the data on S3 was destroyed.
- Hit new setup warning.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T5155
Differential Revision: https://secure.phabricator.com/D14982
Summary: Ref T10110. If an image had `width` or `height`, we would accidentally not give it an `$image_class`.
Test Plan:
{F1057988}
{F1057989}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10110
Differential Revision: https://secure.phabricator.com/D14983
Summary: Ref T9828. Mostly just does a minor modernization pass, but also doesn't migrate chunked files since it's not meaningful (they don't have data, directly).
Test Plan: Ran `bin/files migrate` with various flags. Migrated S3 -> Blob and Blob -> S3.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9828
Differential Revision: https://secure.phabricator.com/D14981
Summary:
Ref T8612. If a change affects more than 10K paths + hunks, tell the user it's too big and don't bother trying to write it. We're mostly bounded by INSERTs here.
Also, fix an issue with file upload errors. The keys are real PHP constants, but were accidentally converted to strings in D12797, causing every error to show as "unknown error".
Test Plan: {F1057509}
Reviewers: joshuaspence
Reviewed By: joshuaspence
Maniphest Tasks: T8612
Differential Revision: https://secure.phabricator.com/D14977
Summary:
Ref T10010. Ref T5819. General alignment of the stars:
- There were some hacks in Conduit around stripping `fa-...` off icons when reading and writing that I wanted to get rid of.
- We probably have room for a subtitle in the new heavy nav, and using the icon name is a good starting point (and maybe good enough on its own?)
- The project list was real bad looking with redundant tag/names, now it is very slightly less bad looking with non-redundant types?
- Some installs will want to call Milestones something else, and this gets us a big part of the way there.
- This may slightly help to reinforce "tag" vs "policy" vs "group" stuff?
---
I'm letting installs have enough rope to shoot themselves in the foot (e.g., define 100 icons). It isn't the end of the world if they reuse icons, and is clearly their fault.
I think the cases where 100 icons will break down are:
- Icon selector dialog may get very unwieldy.
- Query UI will be pretty iffy/huge with 100 icons.
We could improve these fairly easily if an install comes up with a reasonable use case for having 100 icons.
---
The UI on the icon itself in the list views is a little iffy -- mostly, it's too saturated/bold.
I'd ideally like to try either:
- rendering a "shade" version (i.e. lighter, less-saturated color); or
- rendering a "shade" tag with just the icon in it.
However, there didn't seem to be a way to do the first one right now (`fa-example sh-blue` doesn't work) and the second one had weird margins/padding, so I left it like this for now. I figure we can clean it up once we build the thick nav, since that will probably also want an identical element.
(I don't want to render a full tag with the icon + name since I think that's confusing -- it looks like a project/object tag, but is not.)
Test Plan:
{F1049905}
{F1049906}
Reviewers: chad
Reviewed By: chad
Subscribers: 20after4, Luke081515.2
Maniphest Tasks: T5819, T10010
Differential Revision: https://secure.phabricator.com/D14918
Summary: Ref T9156. This makes the UX a little more modern/standard/safe.
Test Plan:
```
epriestley@orbital ~/dev/phabricator $ ./bin/lipsum generate
Choose which type or types of test data you want to generate, or select "all".
- Differential Revisions
- Files
- Maniphest Tasks
- Pastes
- Pholio Mocks
- Projects
- User Accounts
```
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9156
Differential Revision: https://secure.phabricator.com/D14873
Summary: Ref T10032, adds "Basic" NUX to more applications.
Test Plan: Visit each with ?nux=true and click on the create link. T10032 is tracking which apps need general modernization to pick up these changes.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T10032
Differential Revision: https://secure.phabricator.com/D14847
Summary:
Ref T10004. After D14804, we get this behavior by default and no longer need to set it explicitly.
(If some endpoint did eventually need to set it explicitly, it could just change what it passes to `setHref()`, but I believe we currently have no such endpoints and do not foresee ever having any.)
Test Plan:
- As a logged out user, clicked various links in Differential, Maniphest, Files, etc., always got redirected to a sensible place after login.
- Grepped for `setObjectURI()`, `getObjectURI()` (there are a few remaining callsites, but to a different method with the same name in Doorkeeper).
Reviewers: chad
Reviewed By: chad
Subscribers: hach-que
Maniphest Tasks: T10004
Differential Revision: https://secure.phabricator.com/D14805
Summary:
Ref T9992. This is a step on the path to getting EditEngine working in Badges, Projects and Calendar.
This doesn't add a new `EditField` for icons yet, just standardizes the old stuff. New stuff is more general and I saved 150 lines of code.
I put the endpoint in Files because the similar "choose a profile picture" endpoint will definitely go there, and this endpoint might eventually feature, like, "draw your own icon~~" or something.
Test Plan:
- Created events, projects and badges with custom icons.
- Edited events, projects and badges, changing their icons.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9992
Differential Revision: https://secure.phabricator.com/D14799
Summary: Also increase the timeout for the external process to complete the transform.
Test Plan: Careful inspection
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: joshuaspence, cburroughs, chad, Korvin
Differential Revision: https://secure.phabricator.com/D14528
Summary:
Fixes T9494. This:
- Removes all the random GC.x.y.z config.
- Puts it all in one place that's locked and which you use `bin/garbage set-policy ...` to adjust.
- Makes every TTL-based GC configurable.
- Simplifies the code in the actual GCs.
Test Plan:
- Ran `bin/garbage collect` to collect some garbage, until it stopped collecting.
- Ran `bin/garbage set-policy ...` to shorten policy. Saw change in web UI. Ran `bin/garbage collect` again and saw it collect more garbage.
- Set policy to indefinite and saw it not collect garabge.
- Set policy to default and saw it reflected in web UI / `collect`.
- Ran `bin/phd debug trigger` and saw all GCs fire with reasonable looking queries.
- Read new docs.
{F857928}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9494
Differential Revision: https://secure.phabricator.com/D14219
Summary:
Ref T7148. I can do most of the export stuff by only modifying the Instances codebase, but want to upload all the backups and exports as temporary files and can't currently do this via the API.
Make the necessary API changes so that the export workflow can use them when it gets built out.
Test Plan: See next diff. Uploaded files with `arc upload --temporary` and saw them upload as temporary files.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7148
Differential Revision: https://secure.phabricator.com/D14055
Summary:
This enables CORGI.
Currently, `AphrontSite` subclasses can't really have their own routes. They can do this sort of hacky rewriting of paths, but that's a mess and not desirable in the long run.
Instead, let subclasses build their own routing maps. This will let CORP and ORG have their own routing maps.
I was able to get rid of the `PhameBlogResourcesSite` since it can really just share the standard resources site.
Test Plan:
- With no base URI set, and a base URI set, loaded main page and resources (from main site).
- With file domain set, loaded resources from main site and file site.
- Loaded a skinned blog from a domain.
- Loaded a skinned blog from the main site.
- Viewed "Request" tab of DarkConsole to see site/controller info.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D14008
Summary: Use `PhutilClassMaQuery` instead of `PhutilSymbolLoader`, mostly for consistency. Depends on D13588.
Test Plan: Poked around a bunch of pages.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13589
Summary:
Fixes T8984. Because of how drag-and-drop upload works, the text file with content `code` is interpreted as a forbidden variable. Disable this check for the drop upload controller.
(The risk here is a general one where the controller redirects and bundles paramters; this controller does not do that, so it's safe to make this change.)
Test Plan: Uploaded a text file containing only the string "code" (no quotes) by using drag-and-drop.
Reviewers: chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T8984
Differential Revision: https://secure.phabricator.com/D13744
Summary: Use `PhutilClassMapQuery` where appropriate.
Test Plan: Browsed around the UI to verify things seemed somewhat working.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13429
Summary: Move some `PhabricatorSearchField` subclasses to be adjacent to the application to which they belong. This seems generally better to me than lumping them all together in the `src/applications/search/field/` directory. I was also wondering if it makes sense to rename these subclasses as `PhabricatorXSearchField` rather than `PhabricatorSearchXField` (as per T5655), but wasn't really sure if these objects are meant to be search-fields, or just fields belonging to the #search application.
Test Plan: N/A.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13374
Summary:
Fixes T6787. I'm kind of cheating a little bit here by not unifying default selection with `initializeNew(...)` methods, but I figure we can let this settle for a bit and then go do that later. It's pretty minor.
Since we're not doing templates I kind of want to swap the `'template'` key to `'type'` so maybe I'll do that too at some point.
@chad, freel free to change these, I was just trying to make them pretty obvious. I //do// think it's good for them to stand out, but my approach is probably a bit inconsistent/heavy-handed in the new design.
Test Plan:
{F525024}
{F525025}
{F525026}
{F525027}
Reviewers: btrahan, chad
Reviewed By: btrahan
Subscribers: johnny-bit, joshuaspence, chad, epriestley
Maniphest Tasks: T6787
Differential Revision: https://secure.phabricator.com/D13387
Summary: Not sure if we want this, but it seems to work fine.
Test Plan: {F516736}
Reviewers: joshuaspence, chad
Reviewed By: joshuaspence, chad
Subscribers: joshuaspence, epriestley
Differential Revision: https://secure.phabricator.com/D13363
Summary: Ref T8099. This adds a new class which all search engines return for layout. I thought about this a number of ways, and I think this is the cleanest path. Each Engine can return whatever UI bits they needs, and AppSearch or Dashboard picks and lays the bits out as needed. In the AppSearch case, interfaces like Notifications, Calendar, Legalpad all need more custom layouts. I think this also leaves a resonable path forward for NUX as well. Also, not sure I implemented the class correctly, but assume thats easy to fix?
Test Plan: Review and do a search in each application changed. Grep for all call sites.
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T8099
Differential Revision: https://secure.phabricator.com/D13332
Summary: All classes should extend from some other class. See D13275 for some explanation.
Test Plan: `arc unit`
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13283
Summary:
Ref T5681. Ref T6860. This doesn't do anything interesting on its own, just makes the next diff smaller.
In the next diff, policies become aware of the types of objects they're acting on. We need to specify which object type all the "Default View/Edit" settings are for so they get the right rules.
For example, a rule like "Allow task author" is OK for "View Policy" on a task, and also OK for "Default View Policy" on ManiphestApplication. But it's not OK for "Can Create Tasks" on ManiphestApplication.
So annotate all the "template"/"default" policies with their types. The next diff will use these to let you select appropriate rules for the given object type.
Test Plan:
- Used `grep` to find these.
- This change has no effect.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5681, T6860
Differential Revision: https://secure.phabricator.com/D13251
Summary:
Ref T8478. I think the cycle is:
- Conpherence Thread > Loads handle for participant > loads file for profile image > loads attached files to check visibility > loads conpherence thread > ...
So, specifically, someone attached their profile image to a thread or message somewhere.
This breaks the cycle by stopping the attached-files visibility check from happening, since we don't need it. This seemed like the easiest link in the chain to break.
//Ideally//, I think the longer-term and more complete fix here is to stop Conpherence from requiring handles in order to load thread handles (and, generally, having a "handles must not load other handles" rule), but that's not trivial and might not be especially practical.
Test Plan: Will test in production.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8478
Differential Revision: https://secure.phabricator.com/D13216
Summary: See also D13186.
Test Plan: Ran `arc unit --everything`
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D13201
Summary:
Ref T8441. Ref T7715. Automatically generate a modern "Order" control in ApplicationSearch for engines which fully support SearchField.
Notably, this allows the standard "Order" control to automatically support custom field orders. We do this in Maniphest today, but in an ad-hoc way.
Test Plan: Performed order-by queries in Almanac (Services), Pholio, Files, People, Projects, and Paste.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7715, T8441
Differential Revision: https://secure.phabricator.com/D13193
Summary:
Ref T8441. Ref T7715. For modern Query classes, automatically make subscriber queries and SearchField integrations work.
In particular, we can just drive this query with EdgeLogic and don't need to do anything specific on these Query classes beyond making sure they're implemented in a way that picks up all of the EdgeLogic clauses.
Test Plan:
- Searched for subscribers in Pholio, Files, Paste, and Projects.
- Searched for all other fields in Projects to check that Query changes are OK.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7715, T8441
Differential Revision: https://secure.phabricator.com/D13191
Summary: Fixes T8277. Transforming files can race; resolve the race after we lose.
Test Plan:
- Added `sleep(10)` near the bottom of the transform controller.
- Transformed a file in two browser windows at the same time; got something like this (exception corresponds to the loser of the race):
{F412526}
- Applied patch.
- Repeated process, got this:
{F412527}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8277
Differential Revision: https://secure.phabricator.com/D12965
Summary: Fixes T5703. These have been unused in production for a while and the new stuff seems good.
Test Plan: Mostly `grep`.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5703
Differential Revision: https://secure.phabricator.com/D12949
Summary: Converts most all tables to be directly set via `setTable` to an ObjectBox. I think this path is more flexible design wise, as we can change the box based on children, and not just CSS. We also already do this with PropertyList, Forms, ObjectList, and Header. `setCollapsed` is added to ObjectBox to all children objects to bleed to the edges (like diffs).
Test Plan: I did a grep of `appendChild($table)` as well as searches for `PHUIObjectBoxView`, also with manual opening of hundreds of files. I'm sure I missed 5-8 places. If you just appendChild($table) nothing breaks, it just looks a little funny.
Reviewers: epriestley, btrahan
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12955
Summary: Remove the `PhabricatorDefaultSearchEngineSelector` class. This is quite similar to D12053.
Test Plan: Went to `/view/PhabricatorSearchApplication/` and saw the storage engine configuration. Set `search.elastic.host` and saw the highlighted storage engine change.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12670
Summary: Ref T6403. This was actually simple stuff.
Test Plan: changed the edit policy of a paste. changed the edit and join policy of a phame blog.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T6403
Differential Revision: https://secure.phabricator.com/D12933
Summary:
Fixes T6956. Before this change, we called PhabricatorUser::getOmnipotentUser in the various delete methods to query the data. Now, we use $engine->getViewer(), since its always a good thing to have less calls to PhabricatorUser::getOmnipotentUser thrown around the codebase.
I used the "codemod" tool to audit the existing calls to PhabricatorDestructorEngine (all of them) so ostensibly this gets all the spots. If I missed something though, its still going to work, so this change is very low risk.
Test Plan: ./bin/remove destroy P1; visit P1 and get a 404
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T6956
Differential Revision: https://secure.phabricator.com/D12866
Summary: Use `__CLASS__` instead of hard-coding class names. Depends on D12605.
Test Plan: Eyeball it.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: hach-que, Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12806
Summary:
Ref T7707. The general form of this can probably be refined somewhat over time as we have more use cases.
I put this cache on the user object itself because we essentially always need this data and it's trivial to invalidate the cache (we can do it implicilty during reads).
Also fix an issue with short, wide images not thumbnailing properly after recent changes.
Test Plan:
- Loaded some pages; saw caches write; saw good pictures.
- Reloaded; saw cache reads; saw good pictures.
- Changed profile picture; saw immediate update.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7707
Differential Revision: https://secure.phabricator.com/D12826
Summary:
Ref T7707. Fixes T7879. Fixes T4406. When creating profile images:
- Use the new transforms;
- mark them as "profile" images so they're forced to the most-open policies.
Test Plan:
- Set restrictive default file policies.
- Changed profile picture, project pictures, etc. Verified they were visible to logged-out users.
- Registered via OAuth.
- Updated a Conpherence thread image.
- Browsed around looking for profile images, fixed sizing on everything I could find.
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7879, T7707, T4406
Differential Revision: https://secure.phabricator.com/D12821
Summary:
Ref T7707. This ends up being sort of complicated: to support 100x100 images in T4406, we need to scale small images //up// so they look OK when we scale them back down with `background-size` in CSS.
The rest of it is mostly straightforward.
Test Plan:
- Did an OAuth handshake and saw a scaled-up, scaled-down profile picture that looked correct.
- Used Pholio, edited pholio, embedded pholio.
- Uploaded a bunch of small/weird/big images and regenerated all their transforms.
- Uploaded some text files into Pholio.
- Grepped for removed methods, etc.
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7707
Differential Revision: https://secure.phabricator.com/D12818
Summary: Ref T7707. Move the 220px (file uploads) and 100px (Pholio thumbgrid) previews over to the new stuff.
Test Plan: Uploaded a bunch of images to remarkup and Pholio; they generated reasonable results in the web UI.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7707
Differential Revision: https://secure.phabricator.com/D12814
Summary: Ref T7707. For animated GIFs, use imagemagick if it is available.
Test Plan: Generated small versions of a bunch of different GIFs.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7707
Differential Revision: https://secure.phabricator.com/D12813
Summary: Ref T7707. These transforms have a single maximum dimension instead of fixed X and Y dimensions.
Test Plan: Transformed a bunch of files with different sizes/aspect ratios, got sensible results.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7707
Differential Revision: https://secure.phabricator.com/D12812
Summary:
Ref T7707. Ref T4406. Ref T2479. This implements the profile-style (fixed width and height) transforms in a modern way.
- Added a "regnerate" feature to the support UI to make testing easier and surface errors.
- Laboriously check errors from everything.
- Fix the profile thumbnailing so it crops properly instead of leaving margins.
- Also defuses the "gigantic white PNG" attack.
This doesn't handle the imagemagick case (for animated GIFs) yet.
Test Plan:
- Uploaded a variety of wide/narrow/small/large files and converted them into sensible profile pictures.
- Tried to thumbnail some text files.
- Set the pixel-size and file-size limits artificially small and hit them.
- Used "regenerate" a bunch while testing the rest of this stuff.
- Verified that non-regenerate flows still produce a default/placeholder image.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4406, T2479, T7707
Differential Revision: https://secure.phabricator.com/D12811
