1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-28 17:52:43 +01:00
phorge-phorge/src/applications/config/controller
epriestley bc4187d709 When we "discover" new fatal setup issues, stop serving traffic
Summary:
Ref T10759. We may "discover" the presence of a fatal setup error later, after starting Phabricator.

This can happen in a few ways, but most are unlikely. The one I'm immediately concerned about is:

  - Phabricator starts up during a disaster with some databases unreachable.
  - We start with warnings (unreachable databases are generally not fatal, since it's OK for some subset of hosts to be down in replicated/partitioned setups).
  - The unreachable databases later recover and become accessible again.
  - When we run checks against them, we discover that they are misconfigured.

Currently, "fatal" setup issues are not truly fatal if we're "in flight" -- we've survived setup checks at least once in the past. This is bad in the scenario above.

Especially with partitioning, it could lead to mangled data in a disaster scenario where operations staff makes a small configuration mistake while trying to get things running again.

Instead, if we "discover" a fatal error while already "in flight", reset the whole setup process as though the webserver had just restarted. Don't serve requests again until we can make it through setup without hitting fatals.

Test Plan:
  - Started Phabricator with multiple masters, one of which was down and broken.
  - Got a warning about the bad master.
  - Revived the master.
  - Before: Phabricator detects the fatal, but keeps serving requests.
  - After: Phabricator detects the fatal, resets the webserver, and stops serving requests until the fatal is resolved.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10759

Differential Revision: https://secure.phabricator.com/D16903
2016-11-21 15:54:40 -08:00
..
PhabricatorConfigAllController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigApplicationController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigCacheController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigClusterDatabasesController.php Support application partitioning across multiple masters 2016-11-19 14:14:39 -08:00
PhabricatorConfigClusterNotificationsController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigClusterRepositoriesController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigController.php Use new Guide layout in Config->Welcome 2016-09-03 07:32:22 -07:00
PhabricatorConfigDatabaseController.php Update SchemaQuery and the web UI to accommodate multiple master databases 2016-11-12 16:36:52 -08:00
PhabricatorConfigDatabaseIssueController.php Update SchemaQuery and the web UI to accommodate multiple master databases 2016-11-12 16:36:52 -08:00
PhabricatorConfigDatabaseStatusController.php Update SchemaQuery and the web UI to accommodate multiple master databases 2016-11-12 16:36:52 -08:00
PhabricatorConfigEditController.php Make new logo and wordmark more reasonably configurable by human users 2016-08-07 12:00:21 -07:00
PhabricatorConfigGroupController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigHistoryController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigIgnoreController.php Update Config to new UI 2016-04-03 00:27:39 +00:00
PhabricatorConfigIssueListController.php When we "discover" new fatal setup issues, stop serving traffic 2016-11-21 15:54:40 -08:00
PhabricatorConfigIssuePanelController.php When we "discover" new fatal setup issues, stop serving traffic 2016-11-21 15:54:40 -08:00
PhabricatorConfigIssueViewController.php When we "discover" new fatal setup issues, stop serving traffic 2016-11-21 15:54:40 -08:00
PhabricatorConfigListController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigModuleController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00
PhabricatorConfigPurgeCacheController.php Add dialog to purge opcode/data caches 2015-09-10 14:19:02 -07:00
PhabricatorConfigVersionController.php Revert "Clean up more Quicksand" 2016-10-01 12:58:30 -07:00