Provide "builtin" files and use them to fix Pholio when files are deleted

Summary:
Fixes T3132. Currently, if a user deletes a file which is present in a mock, that mock throws an exception when loading. If the file is also the cover photo, the mock list throws an exception as well.

In other applications, we can sometimes deal with this (a sub-object vanishing) by implicitly hiding the parent object (for example, we can just vanish feed stories about objects which no longer exist). We can also sometimes deal with it by preventing sub-objects from being directly deleted.

However, neither approach is reasonable in this case.

If we vanish the whole mock, we'll lose all the comments and it will generally be weird. Vanishing a mock is a big deal compared to vanishing a feed story. We'll also need to load more data on the list view to prevent showing a mock on the list view and then realizing we need to vanish it on the detail view (because all of its images have been deleted).

We permit total deletion of files to allow users to recover from accidentally uploading sensitive files (which has happened a few times), and I'm hesitant to remove this capability because I think it serves a real need, so we can't prevent sub-objects from being deleted.

So we're left in a relatively unique situation. To solve this, I've added a "builtin" mechanism, which allows us to expose some resource we ship with as a PhabricatorFile. Then we just swap it out in place of the original file and proceed forward normally, as though nothing happened. The user sees a placeholder image instead of the original, but everything else works reasonably and this seems like a fairly acceptable outcome.

I believe we can use this mechanism to simplify some other code too, like default profile pictures.

Test Plan: Deleted a Pholio mock cover image's file. Implemented change, saw functional Pholio again with beautiful life-affirming "?" art replacing soul-shattering exception.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T3132

Differential Revision: https://secure.phabricator.com/D5870

src/applications/files/query/PhabricatorFileQuery.php

@@ -7,6 +7,7 @@ final class PhabricatorFileQuery
   private $phids;
   private $authorPHIDs;
   private $explicitUploads;
+  private $transforms;
 
   public function withIDs(array $ids) {
     $this->ids = $ids;
@@ -23,6 +24,21 @@ final class PhabricatorFileQuery
     return $this;
   }
 
+  public function withTransforms(array $specs) {
+    foreach ($specs as $spec) {
+      if (!is_array($spec) ||
+          empty($spec['originalPHID']) ||
+          empty($spec['transform'])) {
+        throw new Exception(
+          "Transform specification must be a dictionary with keys ".
+          "'originalPHID' and 'transform'!");
+      }
+    }
+
+    $this->transforms = $specs;
+    return $this;
+  }
+
   public function showOnlyExplicitUploads($explicit_uploads) {
     $this->explicitUploads = $explicit_uploads;
     return $this;
@@ -34,8 +50,9 @@ final class PhabricatorFileQuery
 
     $data = queryfx_all(
       $conn_r,
-      'SELECT * FROM %T f %Q %Q %Q',
+      'SELECT * FROM %T f %Q %Q %Q %Q',
       $table->getTableName(),
+      $this->buildJoinClause($conn_r),
       $this->buildWhereClause($conn_r),
       $this->buildOrderClause($conn_r),
       $this->buildLimitClause($conn_r));
@@ -43,6 +60,19 @@ final class PhabricatorFileQuery
     return $table->loadAllFromArray($data);
   }
 
+  private function buildJoinClause(AphrontDatabaseConnection $conn_r) {
+    $joins = array();
+
+    if ($this->transforms) {
+      $joins[] = qsprintf(
+        $conn_r,
+        'JOIN %T t ON t.transformedPHID = f.phid',
+        id(new PhabricatorTransformedFile())->getTableName());
+    }
+
+    return implode(' ', $joins);
+  }
+
   private function buildWhereClause(AphrontDatabaseConnection $conn_r) {
     $where = array();
 
@@ -51,31 +81,47 @@ final class PhabricatorFileQuery
     if ($this->ids) {
       $where[] = qsprintf(
         $conn_r,
-        'id IN (%Ld)',
+        'f.id IN (%Ld)',
         $this->ids);
     }
 
     if ($this->phids) {
       $where[] = qsprintf(
         $conn_r,
-        'phid IN (%Ls)',
+        'f.phid IN (%Ls)',
         $this->phids);
     }
 
     if ($this->authorPHIDs) {
       $where[] = qsprintf(
         $conn_r,
-        'authorPHID IN (%Ls)',
+        'f.authorPHID IN (%Ls)',
         $this->authorPHIDs);
     }
 
     if ($this->explicitUploads) {
       $where[] = qsprintf(
         $conn_r,
-        'isExplicitUpload = true');
+        'f.isExplicitUpload = true');
+    }
+
+    if ($this->transforms) {
+      $clauses = array();
+      foreach ($this->transforms as $transform) {
+        $clauses[] = qsprintf(
+          $conn_r,
+          '(t.originalPHID = %s AND t.transform = %s)',
+          $transform['originalPHID'],
+          $transform['transform']);
+      }
+      $where[] = qsprintf($conn_r, '(%Q)', implode(') OR (', $clauses));
     }
 
     return $this->formatWhereClause($where);
   }
 
+  protected function getPagingColumn() {
+    return 'f.id';
+  }
+
 }

src/applications/files/storage/PhabricatorFile.php

@@ -682,6 +682,89 @@ final class PhabricatorFile extends PhabricatorFileDAO
   }
 
 
+  /**
+   * Load (or build) the {@class:PhabricatorFile} objects for builtin file
+   * resources. The builtin mechanism allows files shipped with Phabricator
+   * to be treated like normal files so that APIs do not need to special case
+   * things like default images or deleted files.
+   *
+   * Builtins are located in `resources/builtin/` and identified by their
+   * name.
+   *
+   * @param  PhabricatorUser                Viewing user.
+   * @param  list<string>                   List of builtin file names.
+   * @return dict<string, PhabricatorFile>  Dictionary of named builtins.
+   */
+  public static function loadBuiltins(PhabricatorUser $user, array $names) {
+    $specs = array();
+    foreach ($names as $name) {
+      $specs[] = array(
+        'originalPHID' => PhabricatorPHIDConstants::PHID_VOID,
+        'transform'    => 'builtin:'.$name,
+      );
+    }
+
+    $files = id(new PhabricatorFileQuery())
+      ->setViewer($user)
+      ->withTransforms($specs)
+      ->execute();
+
+    $files = mpull($files, null, 'getName');
+
+    $root = dirname(phutil_get_library_root('phabricator'));
+    $root = $root.'/resources/builtin/';
+
+    $build = array();
+    foreach ($names as $name) {
+      if (isset($files[$name])) {
+        continue;
+      }
+
+      // This is just a sanity check to prevent loading arbitrary files.
+      if (basename($name) != $name) {
+        throw new Exception("Invalid builtin name '{$name}'!");
+      }
+
+      $path = $root.$name;
+
+      if (!Filesystem::pathExists($path)) {
+        throw new Exception("Builtin '{$path}' does not exist!");
+      }
+
+      $data = Filesystem::readFile($path);
+      $params = array(
+        'name' => $name,
+        'ttl'  => time() + (60 * 60 * 24 * 7),
+      );
+
+      $unguarded = AphrontWriteGuard::beginScopedUnguardedWrites();
+        $file = PhabricatorFile::newFromFileData($data, $params);
+        $xform = id(new PhabricatorTransformedFile())
+          ->setOriginalPHID(PhabricatorPHIDConstants::PHID_VOID)
+          ->setTransform('builtin:'.$name)
+          ->setTransformedPHID($file->getPHID())
+          ->save();
+      unset($unguarded);
+
+      $files[$name] = $file;
+    }
+
+    return $files;
+  }
+
+
+  /**
+   * Convenience wrapper for @{method:loadBuiltins}.
+   *
+   * @param PhabricatorUser   Viewing user.
+   * @param string            Single builtin name to load.
+   * @return PhabricatorFile  Corresponding builtin file.
+   */
+  public static function loadBuiltin(PhabricatorUser $user, $name) {
+    return idx(self::loadBuiltins($user, array($name)), $name);
+  }
+
+
 /* -(  PhabricatorPolicyInterface Implementation  )-------------------------- */
 
 

src/applications/phid/PhabricatorPHIDConstants.php

@@ -44,4 +44,7 @@ final class PhabricatorPHIDConstants {
   const PHID_TYPE_XCMT    = 'XCMT';
   const PHID_TYPE_XUSR    = 'XUSR';
 
+  const PHID_TYPE_VOID    = 'VOID';
+  const PHID_VOID         = 'PHID-VOID-00000000000000000000';
+
 }

src/applications/pholio/query/PholioMockQuery.php

@@ -129,7 +129,11 @@ final class PholioMockQuery
     }
 
     foreach ($all_images as $image) {
-      $image->attachFile($all_files[$image->getFilePHID()]);
+      $file = idx($all_files, $image->getFilePHID());
+      if (!$file) {
+        $file = PhabricatorFile::loadBuiltin($this->getViewer(), 'missing.png');
+      }
+      $image->attachFile($file);
       if ($this->needInlineComments) {
         $inlines = idx($all_images, $image->getID(), array());
         $image->attachInlineComments($inlines);
@@ -151,7 +155,11 @@ final class PholioMockQuery
       $cover_file_phids), null, 'getPHID');
 
     foreach ($mocks as $mock) {
-      $mock->attachCoverFile($cover_files[$mock->getCoverPHID()]);
+      $file = idx($cover_files, $mock->getCoverPHID());
+      if (!$file) {
+        $file = PhabricatorFile::loadBuiltin($this->getViewer(), 'missing.png');
+      }
+      $mock->attachCoverFile($file);
     }
   }