1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-24 07:42:40 +01:00
phorge-phorge/resources/sql/autopatches/20140212.dx.1.armageddon.php

223 lines
5.9 KiB
PHP
Raw Normal View History

Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
<?php
$conn_w = id(new DifferentialRevision())->establishConnection('w');
$rows = new LiskRawMigrationIterator($conn_w, 'differential_comment');
$content_source = PhabricatorContentSource::newForSource(
PhabricatorContentSource::SOURCE_LEGACY,
array())->serialize();
echo pht('Migrating Differential comments to modern storage...')."\n";
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
foreach ($rows as $row) {
$id = $row['id'];
echo pht('Migrating comment %d...', $id)."\n";
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
$revision = id(new DifferentialRevision())->load($row['revisionID']);
if (!$revision) {
echo pht('No revision, continuing.')."\n";
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
continue;
}
$revision_phid = $revision->getPHID();
$comments = queryfx_all(
$conn_w,
'SELECT * FROM %T WHERE legacyCommentID = %d',
'differential_transaction_comment',
$id);
$main_comments = array();
$inline_comments = array();
foreach ($comments as $comment) {
if ($comment['changesetID']) {
$inline_comments[] = $comment;
} else {
$main_comments[] = $comment;
}
}
$metadata = json_decode($row['metadata'], true);
if (!is_array($metadata)) {
$metadata = array();
}
$key_cc = 'added-ccs';
$key_add_rev = 'added-reviewers';
$key_rem_rev = 'removed-reviewers';
$key_diff_id = 'diff-id';
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
$xactions = array();
// Build the main action transaction.
switch ($row['action']) {
case DifferentialAction::ACTION_COMMENT:
case DifferentialAction::ACTION_ADDREVIEWERS:
case DifferentialAction::ACTION_ADDCCS:
case DifferentialAction::ACTION_UPDATE:
case DifferentialTransaction::TYPE_INLINE:
// These actions will have their transactions created by other rules.
break;
default:
// Otherwise, this is a normal action (like an accept or reject).
$xactions[] = array(
'type' => DifferentialTransaction::TYPE_ACTION,
'old' => null,
'new' => $row['action'],
);
break;
}
// Build the diff update transaction, if one exists.
$diff_id = idx($metadata, $key_diff_id);
if (!is_scalar($diff_id)) {
$diff_id = null;
}
if ($diff_id || $row['action'] == DifferentialAction::ACTION_UPDATE) {
$xactions[] = array(
'type' => DifferentialTransaction::TYPE_UPDATE,
'old' => null,
'new' => $diff_id,
);
}
// Build the add/remove reviewers transaction, if one exists.
$add_rev = idx($metadata, $key_add_rev, array());
if (!is_array($add_rev)) {
$add_rev = array();
}
$rem_rev = idx($metadata, $key_rem_rev, array());
if (!is_array($rem_rev)) {
$rem_rev = array();
}
if ($add_rev || $rem_rev) {
$old = array();
foreach ($rem_rev as $phid) {
if (!is_scalar($phid)) {
continue;
}
$old[$phid] = array(
'src' => $revision_phid,
'type' => DifferentialRevisionHasReviewerEdgeType::EDGECONST,
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
'dst' => $phid,
);
}
$new = array();
foreach ($add_rev as $phid) {
if (!is_scalar($phid)) {
continue;
}
$new[$phid] = array(
'src' => $revision_phid,
'type' => DifferentialRevisionHasReviewerEdgeType::EDGECONST,
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
'dst' => $phid,
);
}
$xactions[] = array(
'type' => PhabricatorTransactions::TYPE_EDGE,
'old' => $old,
'new' => $new,
'meta' => array(
'edge:type' => DifferentialRevisionHasReviewerEdgeType::EDGECONST,
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
),
);
}
// Build the CC transaction, if one exists.
$add_cc = idx($metadata, $key_cc, array());
if (!is_array($add_cc)) {
$add_cc = array();
}
if ($add_cc) {
$xactions[] = array(
'type' => PhabricatorTransactions::TYPE_SUBSCRIBERS,
'old' => array(),
'new' => array_fuse($add_cc),
);
}
// Build the main comment transaction.
foreach ($main_comments as $main) {
$xactions[] = array(
'type' => PhabricatorTransactions::TYPE_COMMENT,
'old' => null,
'new' => null,
'phid' => $main['transactionPHID'],
'comment' => $main,
);
}
// Build inline comment transactions.
foreach ($inline_comments as $inline) {
$xactions[] = array(
'type' => DifferentialTransaction::TYPE_INLINE,
'old' => null,
'new' => null,
'phid' => $inline['transactionPHID'],
'comment' => $inline,
);
}
foreach ($xactions as $xaction) {
// Generate a new PHID, if we don't already have one from the comment
// table. We pregenerated into the comment table to make this a little
// easier, so we only need to write to one table.
$xaction_phid = idx($xaction, 'phid');
if (!$xaction_phid) {
$xaction_phid = PhabricatorPHID::generateNewPHID(
PhabricatorApplicationTransactionTransactionPHIDType::TYPECONST,
DifferentialRevisionPHIDType::TYPECONST);
Migrate Differential comments to ApplicationTransactions Summary: Ref T2222. This is the big one. This migrates each `DifferentialComment` to one or more ApplicationTransactions (action, cc, reviewers, update, comment, inlines), and makes `DifferentialComment` a double-reader for ApplicationTransactions. The migration is pretty straightforward: - If a comment took an action not otherwise covered, it gets an "action" transaction. This is something like "epriestley abandoned this revision.". - If a comment updated the diff, it gets an "updated diff" transaction. Very old transactions of this type may not have a diff ID (probably only at Facebook). - If a comment added or removed reviewers, it gets a "changed reviewers" transaction. - If a comment added CCs, it gets a "subscribers" transaction. - If a comment added comment text, it gets a "comment" transaction. - For each inline attached to a comment, we generate an "inline" transaction. Most comments generate a small number of transactions, but a few generate a significant number. At HEAD, the code is basically already doing this, so comments in the last day or two already obey these rules, roughly, and will all generate only one transaction (except inlines). Because we've already preallocated PHIDs in the comment text table, we only need to write to the transaction table. NOTE: This significantly degrades Differential, making inline comments pretty much useless (they each get their own transaction, and don't show line numbers or files). The data is all fine, but the UI is garbage now. This needs to be fixed before we can deploy this to users, but it's easily separable since it's all just display code. Specifically, they look like this: {F112270} Test Plan: I've migrated locally and put things through their paces, but it's hard to catch sketchy stuff locally because most of my test data is nonsense and bad migrations wouldn't necessarily look out of place. IMPORTANT: I'm planning to push this to a branch and then shift production over to the branch, and run it for a day or two before bringing it to master. I generally feel good about this change: it's not that big since we were able to separate a lot of pieces out of it, and it's pretty straightforward. That said, it's still one of the most scary/dangerous changes we've ever made. Reviewers: btrahan CC: chad, aran Maniphest Tasks: T2222 Differential Revision: https://secure.phabricator.com/D8210
2014-02-12 00:36:58 +01:00
}
unset($xaction['phid']);
$comment_phid = null;
$comment_version = 0;
if (idx($xaction, 'comment')) {
$comment_phid = $xaction['comment']['phid'];
$comment_version = 1;
}
$old = idx($xaction, 'old');
$new = idx($xaction, 'new');
$meta = idx($xaction, 'meta', array());
queryfx(
$conn_w,
'INSERT INTO %T (phid, authorPHID, objectPHID, viewPolicy, editPolicy,
commentPHID, commentVersion, transactionType, oldValue, newValue,
contentSource, metadata, dateCreated, dateModified)
VALUES (%s, %s, %s, %s, %s, %ns, %d, %s, %ns, %ns, %s, %s, %d, %d)',
'differential_transaction',
// PHID, authorPHID, objectPHID
$xaction_phid,
(string)$row['authorPHID'],
$revision_phid,
// viewPolicy, editPolicy, commentPHID, commentVersion
'public',
(string)$row['authorPHID'],
$comment_phid,
$comment_version,
// transactionType, oldValue, newValue, contentSource, metadata
$xaction['type'],
json_encode($old),
json_encode($new),
$content_source,
json_encode($meta),
// dates
$row['dateCreated'],
$row['dateModified']);
}
}
echo pht('Done.')."\n";