1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-15 03:12:41 +01:00
phorge-phorge/src/applications/files
epriestley 6a62797056 Fix some issues with Diffusion file data limits
Summary:
See <https://discourse.phabricator-community.org/t/files-created-from-repository-contents-slightly-over-one-chunk-in-size-are-truncated-to-exactly-one-chunk-in-size/988/1>. Three issues here:

  - When we finish reading `git cat-file ...` or whatever, we can end up with more than one chunk worth of bytes left in the internal buffer if the read is fast. Use `while` instead of `if` to make sure we write the whole buffer.
  - Limiting output with `setStdoutSizeLimit()` isn't really a reliable way to limit the size if we're also reading from the buffer. It's also pretty indirect and confusing. Instead, just let the `FileUploadSource` explicitly implement a byte limit in a straightforward way.
  - We weren't setting the time limit correctly on the main path.

Overall, this could cause >4MB files to "write" as 4MB files, with the rest of the file left in the UploadSource buffer. Since these files were technically under the limit, they could return as valid. This was intermittent.

Test Plan:
  - Pushed a ~4.2MB file.
  - Reloaded Diffusion a bunch, sometimes saw the `while/if` buffer race and produce a 4MB file with a prompt to download it. (Other times, the buffer worked right and the page just says "this file is too big, sorry").
  - Applied patches.
  - Reloaded Diffusion a bunch, no longer saw bad behavior or truncated files.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18885
2018-01-22 11:52:37 -08:00
..
action Rate limit outbound requests in Macros 2015-03-26 11:11:52 -07:00
application Move Files editing and commenting to EditEngine 2017-04-04 16:15:11 -07:00
applicationpanel Allow diffusion.filecontentquery to load data for arbitrarily large files 2016-01-21 09:52:43 -08:00
builtin Don't require the "gd" extension be installed in order to run unit tests 2017-11-30 13:51:31 -08:00
capability Add default policy to Files application 2014-11-21 11:17:20 -08:00
conduit Stop trying to assess the image dimensions of large files and file chunks 2017-12-18 09:17:32 -08:00
config Prevent enormous changes from being pushed to repositoires by default 2018-01-04 10:02:29 -08:00
constants Prettier file embeds 2016-11-27 14:57:06 -08:00
controller Don't show personalized menu items until users establish a full session 2017-11-28 10:01:58 -08:00
edge Modernize remaining edge types 2015-01-03 10:58:20 +11:00
editor Make daemons perform file deletion 2017-04-18 11:09:41 -07:00
engine Kick off indexing for File objects on creation 2017-04-18 08:38:34 -07:00
exception Store and verify content integrity checksums for files 2017-04-05 11:12:31 -07:00
format Support HMAC+SHA256 with automatic key generation and management 2017-04-06 15:42:59 -07:00
garbagecollector Implement ngram search for File objects 2017-04-17 17:37:20 -07:00
iconset Convert all calls to 'IconFont' to just 'Icon' 2016-01-27 20:59:27 -08:00
keyring Support AES256 at-rest encryption in Files 2016-06-16 08:08:56 -07:00
lipsum Make bin/lipsum generate hanldle generator keys and arguments more clearly 2017-02-27 09:09:28 -08:00
mail phtize all the things 2015-05-22 21:16:39 +10:00
management Fix spelling 2017-10-09 10:48:04 -07:00
markup Add download link to embedded files 2017-01-03 10:50:26 -08:00
phid Mark PhabricatorPHIDType::getPHIDTypeApplicationClass() as abstract 2015-11-03 06:47:12 +11:00
query Add a cache purger for builtin files 2017-06-22 11:13:23 -07:00
storage Stop trying to assess the image dimensions of large files and file chunks 2017-12-18 09:17:32 -08:00
transform Add unit tests for file thumbnail generation 2017-04-23 11:02:21 -07:00
typeahead Add a basic icon typeahead 2017-02-03 09:19:29 -08:00
uploadsource Fix some issues with Diffusion file data limits 2018-01-22 11:52:37 -08:00
view Allow users to drop .ics files on calendar views to import them 2016-10-18 15:26:44 -07:00
worker Make daemons perform file deletion 2017-04-18 11:09:41 -07:00
xaction Switch File deletion to use ModularTransactions 2017-04-18 13:01:51 -07:00
PhabricatorImageTransformer.php Make the Files "TTL" API more structured 2017-04-04 16:16:28 -07:00