From 5162f8109525a78514a9e5f3f5270cbdaf510d1a Mon Sep 17 00:00:00 2001 From: epriestley Date: Thu, 12 May 2016 06:54:43 -0700 Subject: [PATCH] Provide SSH host documentation, tweak/supplement cluster documentation Summary: Ref T10751. I think this mostly brings us up to date with the state of the world. Test Plan: Read documentation. Reviewers: chad Reviewed By: chad Maniphest Tasks: T10751 Differential Revision: https://secure.phabricator.com/D15897 --- src/docs/user/cluster/cluster.diviner | 53 ++++++++++++++++--- src/docs/user/cluster/cluster_daemons.diviner | 6 +-- src/docs/user/cluster/cluster_devices.diviner | 2 +- .../cluster/cluster_notifications.diviner | 3 -- .../user/cluster/cluster_repositories.diviner | 5 -- src/docs/user/cluster/cluster_ssh.diviner | 47 ++++++++++++++++ .../user/cluster/cluster_webservers.diviner | 15 +++--- 7 files changed, 104 insertions(+), 27 deletions(-) create mode 100644 src/docs/user/cluster/cluster_ssh.diviner diff --git a/src/docs/user/cluster/cluster.diviner b/src/docs/user/cluster/cluster.diviner index 0180c7ff42..d78c445c4a 100644 --- a/src/docs/user/cluster/cluster.diviner +++ b/src/docs/user/cluster/cluster.diviner @@ -4,11 +4,13 @@ Guide to configuring Phabricator across multiple hosts for availability and performance. + Overview ======== -WARNING: This feature is a very early prototype; the features this document -describes are mostly speculative fantasy. +WARNING: This feature is a prototype. Installs should expect a challening +adventure when deploying clusters. In the best of times, configuring a +cluster is complex and requires significant operations experience. Phabricator can be configured to run on multiple hosts with redundant services to improve its availability and scalability, and make disaster recovery much @@ -19,9 +21,9 @@ single host, but greatly reduces the cost of recovering from hardware and network failures. Each Phabricator service has an array of clustering options that can be -configured independently. Configuring a cluster is inherently complex, and this -is an advanced feature aimed at installs with large userbases and experienced -operations personnel who need this high degree of flexibility. +configured somewhat independently. Configuring a cluster is inherently complex, +and this is an advanced feature aimed at installs with large userbases and +experienced operations personnel who need this high degree of flexibility. The remainder of this document summarizes how to add redundancy to each service and where your efforts are likely to have the greatest impact. @@ -30,6 +32,25 @@ For additional guidance on setting up a cluster, see "Overlaying Services" and "Cluster Recipes" at the bottom of this document. +Clusterable Services +==================== + +This table provides an overview of clusterable services, their setup +complexity, and the rough impact that converting them to run on multiple hosts +will have on availability, resistance to data loss, and scalability. + +| Service | Setup | Availability | Loss Resistance | Scalability +|---------|-------|--------------|-----------|------------ +| **Databases** | Moderate | **High** | **High** | Low +| **Repositories** | Complex | Moderate | **High** | Moderate +| **Daemons** | Minimal | Low | No Risk | Low +| **SSH Servers** | Minimal | Low | No Risk | Low +| **Web Servers** | Minimal | **High** | No Risk | Moderate +| **Notifications** | Minimal | Low | No Risk | Low + +See below for a walkthrough of these services in greater detail. + + Preparing for Clustering ======================== @@ -146,7 +167,7 @@ Cluster: Daemons Configuring multiple daemon hosts is straightforward, but you must configure repositories first. -With daemons running on multiple hosts, you can transparently survive the loss +With daemons running on multiple hosts you can transparently survive the loss of any subset of hosts without an interruption to daemon services, as long as at least one host remains alive. Daemons are stateless, so spreading daemons across multiple hosts provides no resistance to data loss. @@ -161,14 +182,30 @@ in capacity. For details, see @{article:Cluster: Daemons}. +Cluster: SSH Servers +==================== + +Configuring multiple SSH hosts is straightforward, but you must configure +repositories first. + +With multiple SSH hosts you can transparently survive the loss of any subset +of hosts without interruption to repository services, as long as at last one +host remains alive. SSH services are stateless, so putting multiple hosts in +service provides no resistance to data loss because no data is at risk. + +SSH hosts are very rarely a scalability bottleneck. + +For details, see @{article:Cluster: SSH Servers}. + + Cluster: Web Servers ==================== Configuring multiple web hosts is straightforward, but you must configure repositories first. -With multiple web hosts, you can transparently survive the loss of any subset -of hosts as long as at least one host remains alive. Web hosts are stateless, +With multiple web hosts you can transparently survive the loss of any subset +of hosts as long as at least one host remains alive. Web services are stateless, so putting multiple hosts in service provides no resistance to data loss because no data is at risk. diff --git a/src/docs/user/cluster/cluster_daemons.diviner b/src/docs/user/cluster/cluster_daemons.diviner index 61cedd464d..8cde3e7b7d 100644 --- a/src/docs/user/cluster/cluster_daemons.diviner +++ b/src/docs/user/cluster/cluster_daemons.diviner @@ -6,9 +6,6 @@ Configuring Phabricator to use multiple daemon hosts. Overview ======== -WARNING: This feature is a very early prototype; the features this document -describes are mostly speculative fantasy. - You can run daemons on multiple hosts. The advantages of doing this are: - you can completely survive the loss of multiple daemon hosts; and @@ -18,7 +15,8 @@ This configuration is simple, but you must configure repositories first. For details, see @{article:Cluster: Repositories}. Since repository hosts must run daemons anyway, you usually do not need to do -any additional work and can skip this entirely. +any additional work and can skip this entirely if you have already configured +multiple repository hosts. Adding Daemon Hosts diff --git a/src/docs/user/cluster/cluster_devices.diviner b/src/docs/user/cluster/cluster_devices.diviner index 197620e461..57729fdb86 100644 --- a/src/docs/user/cluster/cluster_devices.diviner +++ b/src/docs/user/cluster/cluster_devices.diviner @@ -101,7 +101,7 @@ up with records that look like these: - Device: `repo001.mycompany.net` - Interface: `123.0.0.1:2222` - Interface: `123.0.0.1:80` - - Device: `repo002.mycopmany.net` + - Device: `repo002.mycompany.net` - Interface: `123.0.0.2:2222` - Interface: `123.0.0.2:80` diff --git a/src/docs/user/cluster/cluster_notifications.diviner b/src/docs/user/cluster/cluster_notifications.diviner index 3dd4e9d903..3cdeec3c39 100644 --- a/src/docs/user/cluster/cluster_notifications.diviner +++ b/src/docs/user/cluster/cluster_notifications.diviner @@ -6,9 +6,6 @@ Configuring Phabricator to use multiple notification servers. Overview ======== -WARNING: This feature is a very early prototype; the features this document -describes are mostly speculative fantasy. - You can run multiple notification servers. The advantages of doing this are: diff --git a/src/docs/user/cluster/cluster_repositories.diviner b/src/docs/user/cluster/cluster_repositories.diviner index 35508b0813..07203f03eb 100644 --- a/src/docs/user/cluster/cluster_repositories.diviner +++ b/src/docs/user/cluster/cluster_repositories.diviner @@ -6,9 +6,6 @@ Configuring Phabricator to use multiple repository hosts. Overview ======== -WARNING: This feature is a very early prototype; the features this document -describes are mostly speculative fantasy. - If you use Git, you can deploy Phabricator with multiple repository hosts, configured so that each host is readable and writable. The advantages of doing this are: @@ -296,8 +293,6 @@ the user whose change is holding the lock. currently held, this shows when the lock was acquired. - - Cluster Failure Modes ===================== diff --git a/src/docs/user/cluster/cluster_ssh.diviner b/src/docs/user/cluster/cluster_ssh.diviner new file mode 100644 index 0000000000..940ef98b05 --- /dev/null +++ b/src/docs/user/cluster/cluster_ssh.diviner @@ -0,0 +1,47 @@ +@title Cluster: SSH Servers +@group cluster + +Configuring Phabricator to use multiple SSH servers. + +Overview +======== + +You can run Phabricator on multiple SSH servers. The advantages of doing this +are: + + - you can completely survive the loss of multiple SSH hosts. + +This configuration is simple, but you must configure repositories first. For +details, see @{article:Cluster: Repositories}. + +SSH servers accept SSH requests from commands like `git clone` and relay them +to hosts that can serve the requests. + + +Adding SSH Hosts +================ + +After configuring repositories in cluster mode, you can add more web hosts +at any time. + +First, deploy the Phabricator software and configuration to a host, then +register the host as a cluster device if it is not already registered (for +help, see @{article:Cluster: Devices}. + +Once the host is registered, start the SSH server, and then add the host to the +SSH load balancer pool. + +Phabricator SSH servers are stateless, so you can pull them in and out of +production freely. + +You may also want to run web services on these hosts, since the service is very +similar to SSH, also stateless, and it may be simpler to load balance the +services together. For details, see @{cluster: Web Servers}. + + +Next Steps +========== + +Continue by: + + - returning to @{article:Clustering Introduction}. diff --git a/src/docs/user/cluster/cluster_webservers.diviner b/src/docs/user/cluster/cluster_webservers.diviner index 95699a8291..744696af66 100644 --- a/src/docs/user/cluster/cluster_webservers.diviner +++ b/src/docs/user/cluster/cluster_webservers.diviner @@ -6,9 +6,6 @@ Configuring Phabricator to use multiple web servers. Overview ======== -WARNING: This feature is a very early prototype; the features this document -describes are mostly speculative fantasy. - You can run Phabricator on multiple web servers. The advantages of doing this are: @@ -23,15 +20,21 @@ Adding Web Hosts ================ After configuring repositories in cluster mode, you can add more web hosts -at any time: simply deploy the Phabricator software and configuration to a -host, start the web server, and then add the host to the load balancer pool. +at any time. + +First, deploy the Phabricator software and configuration to a host, then +register the host as a cluster device if it is not already registered (for +help, see @{article:Cluster: Devices}. + +Once the host is registered, start the web server, and then add the host to the +load balancer pool. Phabricator web servers are stateless, so you can pull them in and out of production freely. You may also want to run SSH services on these hosts, since the service is very similar to HTTP, also stateless, and it may be simpler to load balance the -services together. +services together. For details, see @{cluster:SSH Servers}. Next Steps