From 37f1f555577c0be287aeb71f0d4887fc9729763f Mon Sep 17 00:00:00 2001 From: epriestley Date: Thu, 24 Dec 2015 06:37:33 -0800 Subject: [PATCH] Fix a possible deadlock in unit tests after an error Summary: After certain types of errors, we may deadlock when trying to destroy test databases. Specifically, we still have connections open to, say, `phabricator_unittest_abasonaknlbaklnasb_herald` (or whatever) and MySQL sometimes (not sure exactly when?) waits for them before destorying the database. Test Plan: - Added `$m = null; $m->method()` to a fixture test to force a fatal. - Saw consistent deadlock, with `storage destroy` never exiting. - Added `--trace` to the `storage destroy` command and made it use `phutil_passthru()` so I could see what was happening. - Saw it hang on some arbitrary database. - Conneced to MySQL, used `show full processlist;` to see what was wrong. - Saw the `DROP DATABASE ...` command waiting for locks to release on the database, and other connections still open. - Applied patch. - Saw consistent success. - Used `storage destroy --unittest-fixtures` to clean up extra databases. Reviewers: chad Reviewed By: chad Differential Revision: https://secure.phabricator.com/D14875 --- .../testing/fixture/PhabricatorStorageFixtureScopeGuard.php | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/infrastructure/testing/fixture/PhabricatorStorageFixtureScopeGuard.php b/src/infrastructure/testing/fixture/PhabricatorStorageFixtureScopeGuard.php index 50a7f9d105..2624427dc8 100644 --- a/src/infrastructure/testing/fixture/PhabricatorStorageFixtureScopeGuard.php +++ b/src/infrastructure/testing/fixture/PhabricatorStorageFixtureScopeGuard.php @@ -24,6 +24,11 @@ final class PhabricatorStorageFixtureScopeGuard extends Phobject { public function destroy() { PhabricatorLiskDAO::popStorageNamespace(); + // NOTE: We need to close all connections before destroying the databases. + // If we do not, the "DROP DATABASE ..." statements may hang, waiting for + // our connections to close. + PhabricatorLiskDAO::closeAllConnections(); + execx( 'php %s destroy --force --namespace %s', $this->getStorageBinPath(),