2012-06-08 03:23:57 +02:00
|
|
|
<?php
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Client for an @{class:ArcanistHgProxyServer}. This client talks to a PHP
|
|
|
|
* process which serves as a proxy in front of a Mercurial server process.
|
|
|
|
* The PHP proxy allows multiple clients to use the same Mercurial server.
|
|
|
|
*
|
|
|
|
* This class presents an API which is similar to the hg command-line API.
|
|
|
|
*
|
|
|
|
* Each client is bound to a specific working copy:
|
|
|
|
*
|
|
|
|
* $working_copy = '/path/to/some/hg/working/copy/';
|
|
|
|
* $client = new ArcanistHgProxyClient($working_copy);
|
|
|
|
*
|
|
|
|
* For example, to run `hg log -l 5` via a client:
|
|
|
|
*
|
|
|
|
* $command = array('log', '-l', '5');
|
|
|
|
* list($err, $stdout, $stderr) = $client->executeCommand($command);
|
|
|
|
*
|
|
|
|
* The advantage of using this complex mechanism is that commands run in this
|
|
|
|
* way do not need to pay the startup overhead for hg and the Python runtime,
|
|
|
|
* which is often on the order of 100ms or more per command.
|
|
|
|
*
|
|
|
|
* @task construct Construction
|
Add various flags to the HgProxy daemons
Summary:
- Add flags to exit after an idle time or client count.
- Add flags to control daemonization.
- Add flags to control output.
- Add flags to skip the "hello" frame of the protocol.
- Make the client launch a server if one does not exist.
The one-time overhead to launch a server and run a command through it looks to be ~130% of the overhead to run the command directly with "hg", so even if we never run a second command we're not paying too much.
The incremental overhead to run subsequent command appears to be less than 3% of the overhead to run the command directly with "hg" (and maybe less than 1%, I'm not sure how long the computation part of a command like 'hg log' "actually" takes).
The overhead to launch a PHP client, connect to an existing server, run a command, and then print it and exit is roughly 50% of the overhead to run the command directly with "hg". So theoretically a user can achieve an amortized 2x performance increase for all 'hg' commands by aliasing 'hg' to the PHP client in their shell.
Test Plan:
- Ran servers with idle and client count limits, let them idle and/or hit their connection limits, saw them exit.
- Ran foreground and background servers.
- Ran a daemon server with redirected stdout/stderr. Verified logs appeared.
- Ran with --quiet.
- Ran clients and servers with and without --skip-hello, things work if they agree and break if they disagree. The throughput gain on this is fairly small (maybe 5%?) but it seems simple enough to keep for the moment.
- Ran serverless clients and verified that servers launched the first time, were available subsequently, and relaunched after 15 seconds idle.
Reviewers: csilvers, vrana, btrahan
Reviewed By: csilvers
CC: aran
Differential Revision: https://secure.phabricator.com/D2680
2012-06-26 20:00:26 +02:00
|
|
|
* @task config Configuration
|
2012-06-08 03:23:57 +02:00
|
|
|
* @task exec Executing Mercurial Commands
|
|
|
|
* @task internal Internals
|
|
|
|
*/
|
|
|
|
final class ArcanistHgProxyClient {
|
|
|
|
|
|
|
|
private $workingCopy;
|
|
|
|
private $server;
|
|
|
|
|
Add various flags to the HgProxy daemons
Summary:
- Add flags to exit after an idle time or client count.
- Add flags to control daemonization.
- Add flags to control output.
- Add flags to skip the "hello" frame of the protocol.
- Make the client launch a server if one does not exist.
The one-time overhead to launch a server and run a command through it looks to be ~130% of the overhead to run the command directly with "hg", so even if we never run a second command we're not paying too much.
The incremental overhead to run subsequent command appears to be less than 3% of the overhead to run the command directly with "hg" (and maybe less than 1%, I'm not sure how long the computation part of a command like 'hg log' "actually" takes).
The overhead to launch a PHP client, connect to an existing server, run a command, and then print it and exit is roughly 50% of the overhead to run the command directly with "hg". So theoretically a user can achieve an amortized 2x performance increase for all 'hg' commands by aliasing 'hg' to the PHP client in their shell.
Test Plan:
- Ran servers with idle and client count limits, let them idle and/or hit their connection limits, saw them exit.
- Ran foreground and background servers.
- Ran a daemon server with redirected stdout/stderr. Verified logs appeared.
- Ran with --quiet.
- Ran clients and servers with and without --skip-hello, things work if they agree and break if they disagree. The throughput gain on this is fairly small (maybe 5%?) but it seems simple enough to keep for the moment.
- Ran serverless clients and verified that servers launched the first time, were available subsequently, and relaunched after 15 seconds idle.
Reviewers: csilvers, vrana, btrahan
Reviewed By: csilvers
CC: aran
Differential Revision: https://secure.phabricator.com/D2680
2012-06-26 20:00:26 +02:00
|
|
|
private $skipHello;
|
|
|
|
|
2012-06-08 03:23:57 +02:00
|
|
|
|
|
|
|
/* -( Construction )------------------------------------------------------- */
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Build a new client. This client is bound to a working copy. A server
|
|
|
|
* must already be running on this working copy for the client to work.
|
|
|
|
*
|
|
|
|
* @param string Path to a Mercurial working copy.
|
|
|
|
*
|
|
|
|
* @task construct
|
|
|
|
*/
|
|
|
|
public function __construct($working_copy) {
|
|
|
|
$this->workingCopy = Filesystem::resolvePath($working_copy);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
Add various flags to the HgProxy daemons
Summary:
- Add flags to exit after an idle time or client count.
- Add flags to control daemonization.
- Add flags to control output.
- Add flags to skip the "hello" frame of the protocol.
- Make the client launch a server if one does not exist.
The one-time overhead to launch a server and run a command through it looks to be ~130% of the overhead to run the command directly with "hg", so even if we never run a second command we're not paying too much.
The incremental overhead to run subsequent command appears to be less than 3% of the overhead to run the command directly with "hg" (and maybe less than 1%, I'm not sure how long the computation part of a command like 'hg log' "actually" takes).
The overhead to launch a PHP client, connect to an existing server, run a command, and then print it and exit is roughly 50% of the overhead to run the command directly with "hg". So theoretically a user can achieve an amortized 2x performance increase for all 'hg' commands by aliasing 'hg' to the PHP client in their shell.
Test Plan:
- Ran servers with idle and client count limits, let them idle and/or hit their connection limits, saw them exit.
- Ran foreground and background servers.
- Ran a daemon server with redirected stdout/stderr. Verified logs appeared.
- Ran with --quiet.
- Ran clients and servers with and without --skip-hello, things work if they agree and break if they disagree. The throughput gain on this is fairly small (maybe 5%?) but it seems simple enough to keep for the moment.
- Ran serverless clients and verified that servers launched the first time, were available subsequently, and relaunched after 15 seconds idle.
Reviewers: csilvers, vrana, btrahan
Reviewed By: csilvers
CC: aran
Differential Revision: https://secure.phabricator.com/D2680
2012-06-26 20:00:26 +02:00
|
|
|
/* -( Configuration )------------------------------------------------------ */
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* When connecting, do not expect the "capabilities" message.
|
|
|
|
*
|
|
|
|
* @param bool True to skip the "capabilities" message.
|
|
|
|
* @return this
|
|
|
|
*
|
|
|
|
* @task config
|
|
|
|
*/
|
|
|
|
public function setSkipHello($skip) {
|
|
|
|
$this->skipHello = $skip;
|
|
|
|
return $this;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2012-06-08 03:23:57 +02:00
|
|
|
/* -( Executing Merucurial Commands )-------------------------------------- */
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Execute a command (given as a list of arguments) via the command server.
|
|
|
|
*
|
|
|
|
* @param list<string> A list of command arguments, like "log", "-l", "5".
|
|
|
|
* @return tuple<int, string, string> Return code, stdout and stderr.
|
|
|
|
*
|
|
|
|
* @task exec
|
|
|
|
*/
|
|
|
|
public function executeCommand(array $argv) {
|
|
|
|
if (!$this->server) {
|
Add various flags to the HgProxy daemons
Summary:
- Add flags to exit after an idle time or client count.
- Add flags to control daemonization.
- Add flags to control output.
- Add flags to skip the "hello" frame of the protocol.
- Make the client launch a server if one does not exist.
The one-time overhead to launch a server and run a command through it looks to be ~130% of the overhead to run the command directly with "hg", so even if we never run a second command we're not paying too much.
The incremental overhead to run subsequent command appears to be less than 3% of the overhead to run the command directly with "hg" (and maybe less than 1%, I'm not sure how long the computation part of a command like 'hg log' "actually" takes).
The overhead to launch a PHP client, connect to an existing server, run a command, and then print it and exit is roughly 50% of the overhead to run the command directly with "hg". So theoretically a user can achieve an amortized 2x performance increase for all 'hg' commands by aliasing 'hg' to the PHP client in their shell.
Test Plan:
- Ran servers with idle and client count limits, let them idle and/or hit their connection limits, saw them exit.
- Ran foreground and background servers.
- Ran a daemon server with redirected stdout/stderr. Verified logs appeared.
- Ran with --quiet.
- Ran clients and servers with and without --skip-hello, things work if they agree and break if they disagree. The throughput gain on this is fairly small (maybe 5%?) but it seems simple enough to keep for the moment.
- Ran serverless clients and verified that servers launched the first time, were available subsequently, and relaunched after 15 seconds idle.
Reviewers: csilvers, vrana, btrahan
Reviewed By: csilvers
CC: aran
Differential Revision: https://secure.phabricator.com/D2680
2012-06-26 20:00:26 +02:00
|
|
|
|
|
|
|
try {
|
|
|
|
$server = $this->connectToDaemon();
|
|
|
|
} catch (Exception $ex) {
|
|
|
|
$this->launchDaemon();
|
|
|
|
$server = $this->connectToDaemon();
|
|
|
|
}
|
|
|
|
|
|
|
|
$this->server = $server;
|
2012-06-08 03:23:57 +02:00
|
|
|
}
|
|
|
|
$server = $this->server;
|
|
|
|
|
|
|
|
// Note that we're adding "runcommand" to make the server run the command.
|
|
|
|
// Theoretically the server supports other capabilities, but in practice
|
|
|
|
// we are only concerend with "runcommand".
|
|
|
|
|
|
|
|
$server->write(array_merge(array('runcommand'), $argv));
|
|
|
|
|
|
|
|
// We'll get back one or more blocks of response data, ending with an 'r'
|
|
|
|
// block which indicates the return code. Reconstitute these into stdout,
|
|
|
|
// stderr and a return code.
|
|
|
|
|
|
|
|
$stdout = '';
|
|
|
|
$stderr = '';
|
|
|
|
$err = 0;
|
|
|
|
|
|
|
|
$done = false;
|
|
|
|
while ($message = $server->waitForMessage()) {
|
|
|
|
|
|
|
|
// The $server channel handles decoding of the wire format and gives us
|
|
|
|
// messages which look like this:
|
|
|
|
//
|
|
|
|
// array('o', '<data...>');
|
|
|
|
|
|
|
|
list($channel, $data) = $message;
|
|
|
|
switch ($channel) {
|
|
|
|
case 'o':
|
|
|
|
$stdout .= $data;
|
|
|
|
break;
|
|
|
|
case 'e':
|
|
|
|
$stderr .= $data;
|
|
|
|
break;
|
|
|
|
case 'd':
|
|
|
|
// TODO: Do something with this? This is the 'debug' channel.
|
|
|
|
break;
|
|
|
|
case 'r':
|
|
|
|
// NOTE: This little dance is because the value is emitted as a
|
|
|
|
// big-endian signed 32-bit long. PHP has no flag to unpack() that
|
|
|
|
// can unpack these, so we unpack a big-endian unsigned long, then
|
|
|
|
// repack it as a machine-order unsigned long, then unpack it as
|
|
|
|
// a machine-order signed long. This appears to produce the desired
|
|
|
|
// result.
|
|
|
|
$err = head(unpack('N', $data));
|
|
|
|
$err = pack('L', $err);
|
|
|
|
$err = head(unpack('l', $err));
|
|
|
|
$done = true;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
|
|
|
if ($done) {
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return array($err, $stdout, $stderr);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/* -( Internals )---------------------------------------------------------- */
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* @task internal
|
|
|
|
*/
|
|
|
|
private function connectToDaemon() {
|
|
|
|
$errno = null;
|
|
|
|
$errstr = null;
|
|
|
|
|
|
|
|
$socket_path = ArcanistHgProxyServer::getPathToSocket($this->workingCopy);
|
Add various flags to the HgProxy daemons
Summary:
- Add flags to exit after an idle time or client count.
- Add flags to control daemonization.
- Add flags to control output.
- Add flags to skip the "hello" frame of the protocol.
- Make the client launch a server if one does not exist.
The one-time overhead to launch a server and run a command through it looks to be ~130% of the overhead to run the command directly with "hg", so even if we never run a second command we're not paying too much.
The incremental overhead to run subsequent command appears to be less than 3% of the overhead to run the command directly with "hg" (and maybe less than 1%, I'm not sure how long the computation part of a command like 'hg log' "actually" takes).
The overhead to launch a PHP client, connect to an existing server, run a command, and then print it and exit is roughly 50% of the overhead to run the command directly with "hg". So theoretically a user can achieve an amortized 2x performance increase for all 'hg' commands by aliasing 'hg' to the PHP client in their shell.
Test Plan:
- Ran servers with idle and client count limits, let them idle and/or hit their connection limits, saw them exit.
- Ran foreground and background servers.
- Ran a daemon server with redirected stdout/stderr. Verified logs appeared.
- Ran with --quiet.
- Ran clients and servers with and without --skip-hello, things work if they agree and break if they disagree. The throughput gain on this is fairly small (maybe 5%?) but it seems simple enough to keep for the moment.
- Ran serverless clients and verified that servers launched the first time, were available subsequently, and relaunched after 15 seconds idle.
Reviewers: csilvers, vrana, btrahan
Reviewed By: csilvers
CC: aran
Differential Revision: https://secure.phabricator.com/D2680
2012-06-26 20:00:26 +02:00
|
|
|
$socket = @stream_socket_client('unix://'.$socket_path, $errno, $errstr);
|
2012-06-08 03:23:57 +02:00
|
|
|
|
|
|
|
if ($errno || !$socket) {
|
|
|
|
throw new Exception(
|
|
|
|
"Unable to connect socket! Error #{$errno}: {$errstr}");
|
|
|
|
}
|
|
|
|
|
|
|
|
$channel = new PhutilSocketChannel($socket);
|
|
|
|
$server = new ArcanistHgServerChannel($channel);
|
|
|
|
|
Add various flags to the HgProxy daemons
Summary:
- Add flags to exit after an idle time or client count.
- Add flags to control daemonization.
- Add flags to control output.
- Add flags to skip the "hello" frame of the protocol.
- Make the client launch a server if one does not exist.
The one-time overhead to launch a server and run a command through it looks to be ~130% of the overhead to run the command directly with "hg", so even if we never run a second command we're not paying too much.
The incremental overhead to run subsequent command appears to be less than 3% of the overhead to run the command directly with "hg" (and maybe less than 1%, I'm not sure how long the computation part of a command like 'hg log' "actually" takes).
The overhead to launch a PHP client, connect to an existing server, run a command, and then print it and exit is roughly 50% of the overhead to run the command directly with "hg". So theoretically a user can achieve an amortized 2x performance increase for all 'hg' commands by aliasing 'hg' to the PHP client in their shell.
Test Plan:
- Ran servers with idle and client count limits, let them idle and/or hit their connection limits, saw them exit.
- Ran foreground and background servers.
- Ran a daemon server with redirected stdout/stderr. Verified logs appeared.
- Ran with --quiet.
- Ran clients and servers with and without --skip-hello, things work if they agree and break if they disagree. The throughput gain on this is fairly small (maybe 5%?) but it seems simple enough to keep for the moment.
- Ran serverless clients and verified that servers launched the first time, were available subsequently, and relaunched after 15 seconds idle.
Reviewers: csilvers, vrana, btrahan
Reviewed By: csilvers
CC: aran
Differential Revision: https://secure.phabricator.com/D2680
2012-06-26 20:00:26 +02:00
|
|
|
if (!$this->skipHello) {
|
|
|
|
// The protocol includes a "hello" message with capability and encoding
|
|
|
|
// information. Read and discard it, we use only the "runcommand"
|
|
|
|
// capability which is guaranteed to be available.
|
|
|
|
$hello = $server->waitForMessage();
|
|
|
|
}
|
2012-06-08 03:23:57 +02:00
|
|
|
|
|
|
|
return $server;
|
|
|
|
}
|
|
|
|
|
Add various flags to the HgProxy daemons
Summary:
- Add flags to exit after an idle time or client count.
- Add flags to control daemonization.
- Add flags to control output.
- Add flags to skip the "hello" frame of the protocol.
- Make the client launch a server if one does not exist.
The one-time overhead to launch a server and run a command through it looks to be ~130% of the overhead to run the command directly with "hg", so even if we never run a second command we're not paying too much.
The incremental overhead to run subsequent command appears to be less than 3% of the overhead to run the command directly with "hg" (and maybe less than 1%, I'm not sure how long the computation part of a command like 'hg log' "actually" takes).
The overhead to launch a PHP client, connect to an existing server, run a command, and then print it and exit is roughly 50% of the overhead to run the command directly with "hg". So theoretically a user can achieve an amortized 2x performance increase for all 'hg' commands by aliasing 'hg' to the PHP client in their shell.
Test Plan:
- Ran servers with idle and client count limits, let them idle and/or hit their connection limits, saw them exit.
- Ran foreground and background servers.
- Ran a daemon server with redirected stdout/stderr. Verified logs appeared.
- Ran with --quiet.
- Ran clients and servers with and without --skip-hello, things work if they agree and break if they disagree. The throughput gain on this is fairly small (maybe 5%?) but it seems simple enough to keep for the moment.
- Ran serverless clients and verified that servers launched the first time, were available subsequently, and relaunched after 15 seconds idle.
Reviewers: csilvers, vrana, btrahan
Reviewed By: csilvers
CC: aran
Differential Revision: https://secure.phabricator.com/D2680
2012-06-26 20:00:26 +02:00
|
|
|
|
|
|
|
/**
|
|
|
|
* @task internal
|
|
|
|
*/
|
|
|
|
private function launchDaemon() {
|
|
|
|
$root = dirname(phutil_get_library_root('arcanist'));
|
|
|
|
$bin = $root.'/scripts/hgdaemon/hgdaemon_server.php';
|
|
|
|
$proxy = new ExecFuture(
|
|
|
|
'%s %s --idle-limit 15 --quiet %C',
|
|
|
|
$bin,
|
|
|
|
$this->workingCopy,
|
|
|
|
$this->skipHello ? '--skip-hello' : null);
|
|
|
|
$proxy->resolvex();
|
|
|
|
}
|
|
|
|
|
2012-06-08 03:23:57 +02:00
|
|
|
}
|