2 commits

Author	SHA1	Message	Date
epriestley	68b597ff75	SQL Patch Management: SQL Changes ... Summary: Splits out the SQL changes. These are most of the changes, but primarily mechanical: - Moved "initialize.sql" to "0000.legacy.sql" and partially reverted to an older version, such that patches 0000 + 000 + 001 + ... + 137 put us in the right state when applied sequentially. - Removed "create database" commands from all SQL. These are handled by separate DB patches now, so we have the data to do operations like "storage databases" (list databases) and "storage destroy" (drop databases). - Removed "phabricator_" namespace from all SQL, and replaced with "{$NAMESPACE}_" token so we can namespace databases. - Shortened some column lengths so patches apply correctly if originally created as InnoDB; also a few similar tweaks elsewhere. Test Plan: See D2323 for discussion and test plan. Reviewers: edward, vrana, btrahan, jungejason Reviewed By: btrahan CC: aran Maniphest Tasks: T140, T345 Differential Revision: https://secure.phabricator.com/D2329	2012-04-30 07:53:53 -07:00
epriestley	d1ee08b2df	Drydock Rough Cut ... Summary: Rough cut of Drydock. This is very basic and doesn't do much of use yet (it //does// allocate EC2 machines as host resources and expose interfaces to them), but I think the overall structure is more or less reasonable. == Interfaces Vision: Applications interact with Drydock resources through DrydockInterfaces, like **command**, **filesystem** and **httpd** interfaces. Each interface allows applications to perform some kind of operation on the resource, like executing commands, reading/writing files, or configuring a web server. Interfaces have a concrete, specific API: // Filesystem Interface $fs = $lease->getInterface('filesystem'); // Constants, some day? $fs->writeFile('index.html', 'hello world!'); // Command Interface $cmd = $lease->getInterface('command'); echo $cmd->execx('uptime'); // HTTPD Interface $httpd = $lease->getInterface('httpd'); $httpd->restart(); Interfaces are mostly just stock, although installs might add new interfaces if they expose different ways to interact with resources (for instance, a resource might want to expose a new 'MongoDB' interface or whatever). Currently: We have like part of a command interface. == Leases Vision: Leases keep track of which resources are in use, and what they're being used for. They allow us to know when we need to allocate more resources (too many sandcastles on the existing hosts, e.g.) and when we can release resources (because they are no longer being used). They also give applications something to hold while resources are being allocated. // EXAMPLE: How this should work some day. $allocator = new DrydockAllocator(); $allocator->setResourceType('sandcastle'); $allocator->setAttributes( array( 'diffID' => $diff->getID(), )); $lease = $allocator->allocate(); $diff->setSandcastleLeaseID($lease->getID()); // ... if ($lease->getStatus() == DrydockLeaseStatus::STATUS_ACTIVE) { $sandcastle_link = $lease->getInterface('httpd')->getURI('/'); } else { $sandcastle_link = 'Still building your sandcastle...'; } echo "Sandcastle for this diff: ".$sandcastle_link; // EXAMPLE: How this actually works now. $allocator = new DrydockAllocator(); $allocator->setResourceType('host'); // NOTE: Allocation is currently synchronous but will be task-driven soon. $lease = $allocator->allocate(); Leases are completely stock, installs will not define new lease types. Currently: Leases exist and work but are very very basic. == Resources Vision: Resources represent some actual thing we've put somewhere, whether it's a host, a block of storage, a webroot, or whatever else. Applications interact through resources by acquiring leases to them, and then getting interfaces through these leases. The lease acquisition process has a side effect of allocating new resources if a lease can't be acquired on existing resources (e.g., the application wants storage but all storage resources are full) and things are configured to autoscale. Resources may themselves acquire leases in order to allocate. For instance, a storage resource might first acquire a lease to a host resource. A 'test scaffold' resource might lease a storage resource and a mysql resource. Not all resources are auto-allocate: the entry-level version of Drydock is that you manually allocate a couple boxes and configure them through the web console. Then, e.g., 'storage' / 'webroot' resources allocate on top of them, but the host pool itself does not autoscale. Resources are completely stock, they are abstract shells representing any arbitrary thing. Currently: Resource exist ('host' only) but are very very basic. == Blueprints Vision: Blueprints contain instructions for building interfaces to, (possibly) allocating, updating, managing, and destroying a specific type of resource in a specific location. One way to think of them is that they are scripts for creating and deleting resources. For example, the LocalHost, RemoteHost and EC2Host blueprints can all manage 'host' resources. Eventually, we will support more types of resources (storage, webroot, sandcastle, test scaffold, phacility deployment) and more providers for resource types, some of which will be in the Phabricator mainline and some of which will be custom. Blueprints are very custom and specific to application types, so installs will define new blueprints if they are making significant use of Drydock. Currently: They exist but have few capabilities. The stock blueprints do nearly nothing useful. There is a technically functional blueprint for host allocation in EC2. == Allocator This is just the actual code to execute the lease acquisition process. Test Plan: Ran "drydock_control.php" script, it allocated a machine in EC2, acquired a lease on it, interfaced with it, and then released the lease. Ran it again, got a fresh lease on the existing resource. Reviewers: btrahan, jungejason Reviewed By: btrahan CC: aran Differential Revision: https://secure.phabricator.com/D1454	2012-01-19 21:12:57 -08:00