From 012ace763422f75b77dfaa265d8d1f16df36f5cc Mon Sep 17 00:00:00 2001
From: epriestley <git@epriestley.com>
Date: Wed, 25 Jul 2018 16:05:34 -0700
Subject: [PATCH] When migrating files between storage engines with "bin/files
 migrate ...", skip expired temporary files

Summary:
See T7148. This just cheats us out of a weird sort of race where we:

  - Dump an instance, including some `F123` which is a temporary file which expires in 3 minutes.
  - A few minutes later, the daemons delete the data for that file.
  - A few minutes after that, we try to `bin/files migrate --copy` to copy the data from S3 into the MySQL blob store.
  - This fails since the data is already gone.

Instead, just skip these files since they're already dead to us.

Test Plan: Faked this locally, will migrate the PHI769 instance on `aux001`.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19536
---
 ...bricatorFilesManagementMigrateWorkflow.php | 26 +++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php b/src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php
index eda43e2ef3..909e45c31f 100644
--- a/src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php
+++ b/src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php
@@ -85,6 +85,32 @@ final class PhabricatorFilesManagementMigrateWorkflow
     foreach ($iterator as $file) {
       $monogram = $file->getMonogram();
 
+      // See T7148. When we export data for an instance, we copy all the data
+      // for Files from S3 into the database dump so that the database dump is
+      // a complete, standalone archive of all the data. In the general case,
+      // installs may have a similar process using "--copy" to create a more
+      // complete backup.
+
+      // When doing this, we may run into temporary files which have been
+      // deleted between the time we took the original dump and the current
+      // timestamp. These files can't be copied since the data no longer
+      // exists: the daemons on the live install already deleted it.
+
+      // Simply avoid this whole mess by declining to migrate expired temporary
+      // files. They're as good as dead anyway.
+
+      $ttl = $file->getTTL();
+      if ($ttl) {
+        if ($ttl < PhabricatorTime::getNow()) {
+          echo tsprintf(
+            "%s\n",
+            pht(
+              '%s: Skipping expired temporary file.',
+              $monogram));
+          continue;
+        }
+      }
+
       $engine_key = $file->getStorageEngine();
       $engine = idx($engines, $engine_key);