1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-19 03:50:54 +01:00

Detect the MIME type of large files by examining the first chunk

Summary:
Fixes T11242. See that task for detailed discussion.

Previously, it didn't particularly matter that we don't MIME detect chunked files since they were all just big blobs of junk (PSDs, zips/tarballs, whatever) that we handled uniformly.

However, videos are large and the MIME type also matters.

  - Detect the overall mime type by detecitng the MIME type of the first chunk. This appears to work properly, at least for video.
  - Skip mime type detection on other chunks, which we were performing and ignoring. This makes uploading chunked files a little faster since we don't need to write stuff to disk.

Test Plan:
Uploaded a 50MB video locally, saw it as chunks with a "video/mp4" mime type, played it in the browser in Phabricator as an embedded HTML 5 video.

{F1706837}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11242

Differential Revision: https://secure.phabricator.com/D16204
This commit is contained in:
epriestley 2016-06-30 13:26:21 -07:00
parent 7a315780b4
commit 01862b8f23
2 changed files with 26 additions and 5 deletions

View file

@ -51,6 +51,16 @@ final class FileUploadChunkConduitAPIMethod
$start, $start,
$start + $length); $start + $length);
// If this is the initial chunk, leave the MIME type unset so we detect
// it and can update the parent file. If this is any other chunk, it has
// no meaningful MIME type. Provide a default type so we can avoid writing
// it to disk to perform MIME type detection.
if (!$start) {
$mime_type = null;
} else {
$mime_type = 'application/octet-stream';
}
// NOTE: These files have a view policy which prevents normal access. They // NOTE: These files have a view policy which prevents normal access. They
// are only accessed through the storage engine. // are only accessed through the storage engine.
$chunk_data = PhabricatorFile::newFromFileData( $chunk_data = PhabricatorFile::newFromFileData(
@ -58,13 +68,26 @@ final class FileUploadChunkConduitAPIMethod
array( array(
'name' => $file->getMonogram().'.chunk-'.$chunk->getID(), 'name' => $file->getMonogram().'.chunk-'.$chunk->getID(),
'viewPolicy' => PhabricatorPolicies::POLICY_NOONE, 'viewPolicy' => PhabricatorPolicies::POLICY_NOONE,
'mime-type' => $mime_type,
)); ));
$chunk->setDataFilePHID($chunk_data->getPHID())->save(); $chunk->setDataFilePHID($chunk_data->getPHID())->save();
$needs_update = false;
$missing = $this->loadAnyMissingChunk($viewer, $file); $missing = $this->loadAnyMissingChunk($viewer, $file);
if (!$missing) { if (!$missing) {
$file->setIsPartial(0)->save(); $file->setIsPartial(0);
$needs_update = true;
}
if (!$start) {
$file->setMimeType($chunk_data->getMimeType());
$needs_update = true;
}
if ($needs_update) {
$file->save();
} }
return null; return null;

View file

@ -270,10 +270,8 @@ final class PhabricatorFile extends PhabricatorFileDAO
$file->setByteSize($length); $file->setByteSize($length);
// TODO: We might be able to test the first chunk in order to figure // NOTE: Once we receive the first chunk, we'll detect its MIME type and
// this out more reliably, since MIME detection usually examines headers. // update the parent file. This matters for large media files like video.
// However, enormous files are probably always either actually raw data
// or reasonable to treat like raw data.
$file->setMimeType('application/octet-stream'); $file->setMimeType('application/octet-stream');
$chunked_hash = idx($params, 'chunkedHash'); $chunked_hash = idx($params, 'chunkedHash');