1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-02 03:32:42 +01:00

When migrating files between storage engines with "bin/files migrate ...", skip expired temporary files

Summary:
See T7148. This just cheats us out of a weird sort of race where we:

  - Dump an instance, including some `F123` which is a temporary file which expires in 3 minutes.
  - A few minutes later, the daemons delete the data for that file.
  - A few minutes after that, we try to `bin/files migrate --copy` to copy the data from S3 into the MySQL blob store.
  - This fails since the data is already gone.

Instead, just skip these files since they're already dead to us.

Test Plan: Faked this locally, will migrate the PHI769 instance on `aux001`.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19536
This commit is contained in:
epriestley 2018-07-25 16:05:34 -07:00
parent ee7879e626
commit 682c3bc9ee

View file

@ -85,6 +85,32 @@ final class PhabricatorFilesManagementMigrateWorkflow
foreach ($iterator as $file) { foreach ($iterator as $file) {
$monogram = $file->getMonogram(); $monogram = $file->getMonogram();
// See T7148. When we export data for an instance, we copy all the data
// for Files from S3 into the database dump so that the database dump is
// a complete, standalone archive of all the data. In the general case,
// installs may have a similar process using "--copy" to create a more
// complete backup.
// When doing this, we may run into temporary files which have been
// deleted between the time we took the original dump and the current
// timestamp. These files can't be copied since the data no longer
// exists: the daemons on the live install already deleted it.
// Simply avoid this whole mess by declining to migrate expired temporary
// files. They're as good as dead anyway.
$ttl = $file->getTTL();
if ($ttl) {
if ($ttl < PhabricatorTime::getNow()) {
echo tsprintf(
"%s\n",
pht(
'%s: Skipping expired temporary file.',
$monogram));
continue;
}
}
$engine_key = $file->getStorageEngine(); $engine_key = $file->getStorageEngine();
$engine = idx($engines, $engine_key); $engine = idx($engines, $engine_key);