epriestley f43355855c Add bin/files compact for sharing file data storage ... Summary: Fixes T5912. When we write files, we attempt to share storage if two files have the same content. In some cases, we may not share storage. Examples include: - Files migrated with `bin/files migrate` (it's simpler not to try to dedupe them). - Old files, from before storage was sharable (the mechanism did not exist). - Files broken by the bug fixed in T5912. Add a script to compact files by pointing files with the same content hash at the same file contnet. In the particular case of files broken by the bug in T5912, we know the hash of the file's content and will only point them at a file that we can load the data for, so this fixes them. Compaction is not hugely useful in general, but this script isn't too complex and the ability to fix damage from the bug in T5912 is desirable. We could remove this capability eventually. Test Plan: - Ran `files compact --all --dry-run` and sanity checked a bunch of the duplicates for actually being duplicates. - Migrated individual files with `files compact Fnnn --trace` and verified the storage compacted and all files survived the process. - Verified unused storage was correctly destroyed after removing the last reference to it. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T5912 Differential Revision: https://secure.phabricator.com/D10327		2014-08-21 11:47:59 -07:00
..
aphlict/management	Try nodejs before node when starting notification server	2014-06-07 13:56:23 -07:00
arcanist/conduit	Rename Conduit classes	2014-07-25 10:54:15 +10:00
audit	Include all audit states in audit.query	2014-08-21 06:14:23 -07:00
auth	Be more strict about "Location:" redirects	2014-08-18 14:11:06 -07:00
base	Fix security.require-https by marking redirect as external	2014-08-21 21:35:29 +10:00
cache	Rename PhutilKeyValueCache subclasses	2014-08-06 08:12:28 +10:00
calendar	Events - add a byline to event list	2014-08-06 15:04:12 -07:00
chatlog	Fix some missing renames of Application classes	2014-07-24 18:03:59 -07:00
conduit	Fix string construction in Conduit exceptions	2014-08-11 12:08:06 -07:00
config	HTML emails	2014-08-15 08:12:21 -07:00
conpherence	Touch up notification/messages panels	2014-08-14 17:19:01 -07:00
countdown	Rename PhutilRemarkupRule subclasses	2014-08-05 00:55:43 +10:00
daemon	Don't SIGINT ourselves on bad daemon data	2014-08-20 13:18:17 -07:00
dashboard	Implement DestructibleInterface for dashboards and panels	2014-08-18 13:15:13 -07:00
differential	Allow Herald "diff" rules to reject content before it is written	2014-08-20 14:26:29 -07:00
diffusion	Improve UX for repository updates	2014-08-21 11:30:12 -07:00
diviner	Rename PhutilKeyValueCache subclasses	2014-08-06 08:12:28 +10:00
doorkeeper	Rename PhutilRemarkupRule subclasses	2014-08-05 00:55:43 +10:00
draft/storage	Differential - add DifferentialDraft to track whether revisions have draft feedback or not	2014-02-18 16:25:16 -08:00
drydock	Remove localhost Drydock allocator	2014-08-20 08:29:32 +10:00
fact	Send graceful shutdown signals to daemons in Phabricator	2014-08-11 20:18:31 -07:00
feed	Show projects on feed stories	2014-08-13 11:24:56 -07:00
files	Add bin/files compact for sharing file data storage	2014-08-21 11:47:59 -07:00
flag	Rename Conduit classes	2014-07-25 10:54:15 +10:00
harbormaster	Fix an issue with build generations not being set for strict MySQL	2014-08-21 09:23:48 -07:00
help	Implement the getName method in PhabricatorApplication subclasses	2014-07-23 23:52:50 +10:00
herald	Fix herald "Repository" rule for Revisions and Diffs	2014-08-21 10:54:07 -07:00
home	Implement the getName method in PhabricatorApplication subclasses	2014-07-23 23:52:50 +10:00
legalpad	Modularize mail tags	2014-08-12 12:28:41 -07:00
lipsum	Change double quotes to single quotes.	2014-06-09 11:36:50 -07:00
macro	Flag meme redirect as external	2014-08-20 06:59:08 -07:00
mailinglists	Rename AphrontQueryException subclasses	2014-08-06 07:51:21 +10:00
maniphest	Don't render a grip on tasks returned over AJAX to non-draggable UIs	2014-08-16 11:12:43 -07:00
meta	Rename PHIDType classes	2014-07-24 08:05:46 +10:00
metamta	Minor tweaks to bin/mail send-test	2014-08-21 11:25:44 -07:00
notification	Move notifications to ApplicationSearch	2014-08-16 11:14:32 -07:00
nuance	Modularize mail tags	2014-08-12 12:28:41 -07:00
oauthserver	Simplify the implementation of PhabricatorPolicyCapability subclasses	2014-07-25 08:25:42 +10:00
owners	Rename AphrontQueryException subclasses	2014-08-06 07:51:21 +10:00
passphrase	Don't leave temporary files around when trying to use credentials with destroyed secrets	2014-08-21 11:26:02 -07:00
paste	Paste - fix caching mechanism for S3-stored files	2014-08-19 12:01:17 -07:00
people	Add autocomplete=off to all non-login password forms	2014-08-13 10:06:48 -07:00
phame	Remove "Edit" text on Phame	2014-08-08 10:02:22 -07:00
phid	Remove PHID_TYPE_ACMT	2014-08-05 12:02:22 -07:00
phlux	Modularize mail tags	2014-08-12 12:28:41 -07:00
pholio	Modularize mail tags	2014-08-12 12:28:41 -07:00
phortune	Be more strict about "Location:" redirects	2014-08-18 14:11:06 -07:00
phpast	Add some missing application names	2014-07-23 08:15:44 -07:00
phragment	Correct typo: security.alter[n]ate-file-domain	2014-08-07 09:41:20 -07:00
phrequent	Account for preempting events on the Phrequent list view	2014-08-11 12:30:48 -07:00
phriction	Rename PhutilRemarkupRule subclasses	2014-08-05 00:55:43 +10:00
policy	Simplify the implementation of PhabricatorPolicyCapability subclasses	2014-07-25 08:25:42 +10:00
ponder	Remarkup - add a regex to blacklist what objects get link	2014-08-14 15:20:45 -07:00
project	Allow columns to have a point limit	2014-08-15 11:16:08 -07:00
releeph	Modularize mail tags	2014-08-12 12:28:41 -07:00
remarkup/conduit	Rename Conduit classes	2014-07-25 10:54:15 +10:00
repository	Improve UX for repository updates	2014-08-21 11:30:12 -07:00
search	Provide a setup warning about using the default MySQL stopword file	2014-08-13 15:34:09 -07:00
settings	Allow users to recover from a missing password hasher	2014-08-21 11:30:05 -07:00
slowvote	Modularize mail tags	2014-08-12 12:28:41 -07:00
subscriptions	Rename PHIDType classes	2014-07-24 08:05:46 +10:00
support/application	Implement the getName method in PhabricatorApplication subclasses	2014-07-23 23:52:50 +10:00
system	When destroying an object, destroy its Herald transcripts too	2014-08-20 15:04:34 -07:00
tokens	Rename Conduit classes	2014-07-25 10:54:15 +10:00
transactions	Login to Comment - kill intermediary dialogue	2014-08-20 13:17:14 -07:00
typeahead	Projects - tokenize projects more aggressively with respect to '-'	2014-08-14 12:28:11 -07:00
uiexample	Replace ActionHeader minicons with Font icons	2014-08-15 11:08:28 -07:00
xhprof	Implement the getName method in PhabricatorApplication subclasses	2014-07-23 23:52:50 +10:00