revi-archive/phorge-phorge
1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-09-22 10:18:48 +02:00
Code Issues Releases Wiki Activity
phorge-phorge/src/applications
History
epriestley f43355855c Add bin/files compact for sharing file data storage 
Summary:
Fixes T5912. When we write files, we attempt to share storage if two files have the same content.

In some cases, we may not share storage. Examples include:

  - Files migrated with `bin/files migrate` (it's simpler not to try to dedupe them).
  - Old files, from before storage was sharable (the mechanism did not exist).
  - Files broken by the bug fixed in T5912.

Add a script to compact files by pointing files with the same content hash at the same file contnet.

In the particular case of files broken by the bug in T5912, we know the hash of the file's content and will only point them at a file that we can load the data for, so this fixes them.

Compaction is not hugely useful in general, but this script isn't too complex and the ability to fix damage from the bug in T5912 is desirable. We could remove this capability eventually.

Test Plan:
  - Ran `files compact --all --dry-run` and sanity checked a bunch of the duplicates for actually being duplicates.
  - Migrated individual files with `files compact Fnnn --trace` and verified the storage compacted and all files survived the process.
  - Verified unused storage was correctly destroyed after removing the last reference to it.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5912

Differential Revision: https://secure.phabricator.com/D10327
2014-08-21 11:47:59 -07:00
..
aphlict/management Try nodejs before node when starting notification server 2014-06-07 13:56:23 -07:00
arcanist/conduit Rename Conduit classes 2014-07-25 10:54:15 +10:00
audit Include all audit states in audit.query 2014-08-21 06:14:23 -07:00
auth Be more strict about "Location:" redirects 2014-08-18 14:11:06 -07:00
base Fix security.require-https by marking redirect as external 2014-08-21 21:35:29 +10:00
cache Rename PhutilKeyValueCache subclasses 2014-08-06 08:12:28 +10:00
calendar Events - add a byline to event list 2014-08-06 15:04:12 -07:00
chatlog Fix some missing renames of Application classes 2014-07-24 18:03:59 -07:00
conduit Fix string construction in Conduit exceptions 2014-08-11 12:08:06 -07:00
config HTML emails 2014-08-15 08:12:21 -07:00
conpherence Touch up notification/messages panels 2014-08-14 17:19:01 -07:00
countdown Rename PhutilRemarkupRule subclasses 2014-08-05 00:55:43 +10:00
daemon Don't SIGINT ourselves on bad daemon data 2014-08-20 13:18:17 -07:00
dashboard Implement DestructibleInterface for dashboards and panels 2014-08-18 13:15:13 -07:00
differential Allow Herald "diff" rules to reject content before it is written 2014-08-20 14:26:29 -07:00
diffusion Improve UX for repository updates 2014-08-21 11:30:12 -07:00
diviner Rename PhutilKeyValueCache subclasses 2014-08-06 08:12:28 +10:00
doorkeeper Rename PhutilRemarkupRule subclasses 2014-08-05 00:55:43 +10:00
draft/storage Differential - add DifferentialDraft to track whether revisions have draft feedback or not 2014-02-18 16:25:16 -08:00
drydock Remove localhost Drydock allocator 2014-08-20 08:29:32 +10:00
fact Send graceful shutdown signals to daemons in Phabricator 2014-08-11 20:18:31 -07:00
feed Show projects on feed stories 2014-08-13 11:24:56 -07:00
files Add bin/files compact for sharing file data storage 2014-08-21 11:47:59 -07:00
flag Rename Conduit classes 2014-07-25 10:54:15 +10:00
harbormaster Fix an issue with build generations not being set for strict MySQL 2014-08-21 09:23:48 -07:00
help Implement the getName method in PhabricatorApplication subclasses 2014-07-23 23:52:50 +10:00
herald Fix herald "Repository" rule for Revisions and Diffs 2014-08-21 10:54:07 -07:00
home Implement the getName method in PhabricatorApplication subclasses 2014-07-23 23:52:50 +10:00
legalpad Modularize mail tags 2014-08-12 12:28:41 -07:00
lipsum Change double quotes to single quotes. 2014-06-09 11:36:50 -07:00
macro Flag meme redirect as external 2014-08-20 06:59:08 -07:00
mailinglists Rename AphrontQueryException subclasses 2014-08-06 07:51:21 +10:00
maniphest Don't render a grip on tasks returned over AJAX to non-draggable UIs 2014-08-16 11:12:43 -07:00
meta Rename PHIDType classes 2014-07-24 08:05:46 +10:00
metamta Minor tweaks to bin/mail send-test 2014-08-21 11:25:44 -07:00
notification Move notifications to ApplicationSearch 2014-08-16 11:14:32 -07:00
nuance Modularize mail tags 2014-08-12 12:28:41 -07:00
oauthserver Simplify the implementation of PhabricatorPolicyCapability subclasses 2014-07-25 08:25:42 +10:00
owners Rename AphrontQueryException subclasses 2014-08-06 07:51:21 +10:00
passphrase Don't leave temporary files around when trying to use credentials with destroyed secrets 2014-08-21 11:26:02 -07:00
paste Paste - fix caching mechanism for S3-stored files 2014-08-19 12:01:17 -07:00
people Add autocomplete=off to all non-login password forms 2014-08-13 10:06:48 -07:00
phame Remove "Edit" text on Phame 2014-08-08 10:02:22 -07:00
phid Remove PHID_TYPE_ACMT 2014-08-05 12:02:22 -07:00
phlux Modularize mail tags 2014-08-12 12:28:41 -07:00
pholio Modularize mail tags 2014-08-12 12:28:41 -07:00
phortune Be more strict about "Location:" redirects 2014-08-18 14:11:06 -07:00
phpast Add some missing application names 2014-07-23 08:15:44 -07:00
phragment Correct typo: security.alter[n]ate-file-domain 2014-08-07 09:41:20 -07:00
phrequent Account for preempting events on the Phrequent list view 2014-08-11 12:30:48 -07:00
phriction Rename PhutilRemarkupRule subclasses 2014-08-05 00:55:43 +10:00
policy Simplify the implementation of PhabricatorPolicyCapability subclasses 2014-07-25 08:25:42 +10:00
ponder Remarkup - add a regex to blacklist what objects get link 2014-08-14 15:20:45 -07:00
project Allow columns to have a point limit 2014-08-15 11:16:08 -07:00
releeph Modularize mail tags 2014-08-12 12:28:41 -07:00
remarkup/conduit Rename Conduit classes 2014-07-25 10:54:15 +10:00
repository Improve UX for repository updates 2014-08-21 11:30:12 -07:00
search Provide a setup warning about using the default MySQL stopword file 2014-08-13 15:34:09 -07:00
settings Allow users to recover from a missing password hasher 2014-08-21 11:30:05 -07:00
slowvote Modularize mail tags 2014-08-12 12:28:41 -07:00
subscriptions Rename PHIDType classes 2014-07-24 08:05:46 +10:00
support/application Implement the getName method in PhabricatorApplication subclasses 2014-07-23 23:52:50 +10:00
system When destroying an object, destroy its Herald transcripts too 2014-08-20 15:04:34 -07:00
tokens Rename Conduit classes 2014-07-25 10:54:15 +10:00
transactions Login to Comment - kill intermediary dialogue 2014-08-20 13:17:14 -07:00
typeahead Projects - tokenize projects more aggressively with respect to '-' 2014-08-14 12:28:11 -07:00
uiexample Replace ActionHeader minicons with Font icons 2014-08-15 11:08:28 -07:00
xhprof Implement the getName method in PhabricatorApplication subclasses 2014-07-23 23:52:50 +10:00