1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-04 12:42:43 +01:00
phorge-phorge/src/applications/diffusion/ssh/DiffusionSubversionServeSSHWorkflow.php

425 lines
13 KiB
PHP
Raw Normal View History

<?php
/**
* This protocol has a good spec here:
*
* http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_ra_svn/protocol
*
*/
final class DiffusionSubversionServeSSHWorkflow
extends DiffusionSubversionSSHWorkflow {
private $didSeeWrite;
private $inProtocol;
private $outProtocol;
private $inSeenGreeting;
private $outPhaseCount = 0;
private $internalBaseURI;
private $externalBaseURI;
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
private $peekBuffer;
private $command;
private function getCommand() {
return $this->command;
}
protected function didConstruct() {
$this->setName('svnserve');
$this->setArguments(
array(
array(
'name' => 'tunnel',
'short' => 't',
),
));
}
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
protected function identifyRepository() {
// NOTE: In SVN, we need to read the first few protocol frames before we
// can determine which repository the user is trying to access. We're
// going to peek at the data on the wire to identify the repository.
$io_channel = $this->getIOChannel();
// Before the client will send us the first protocol frame, we need to send
// it a connection frame with server capabilities. To figure out the
// correct frame we're going to start `svnserve`, read the frame from it,
// send it to the client, then kill the subprocess.
// TODO: This is pretty inelegant and the protocol frame will change very
// rarely. We could cache it if we can find a reasonable way to dirty the
// cache.
$command = csprintf('svnserve -t');
$command = PhabricatorDaemon::sudoCommandAsDaemonUser($command);
$future = new ExecFuture('%C', $command);
$exec_channel = new PhutilExecChannel($future);
$exec_protocol = new DiffusionSubversionWireProtocol();
while (true) {
PhutilChannel::waitForAny(array($exec_channel));
$exec_channel->update();
$exec_message = $exec_channel->read();
if ($exec_message !== null) {
$messages = $exec_protocol->writeData($exec_message);
if ($messages) {
$message = head($messages);
$raw = $message['raw'];
// Write the greeting frame to the client.
$io_channel->write($raw);
// Kill the subprocess.
$future->resolveKill();
break;
}
}
if (!$exec_channel->isOpenForReading()) {
throw new Exception(
pht(
'svnserve subprocess exited before emitting a protocol frame.'));
}
}
$io_protocol = new DiffusionSubversionWireProtocol();
while (true) {
PhutilChannel::waitForAny(array($io_channel));
$io_channel->update();
$in_message = $io_channel->read();
if ($in_message !== null) {
$this->peekBuffer .= $in_message;
if (strlen($this->peekBuffer) > (1024 * 1024)) {
throw new Exception(
pht(
'Client transmitted more than 1MB of data without transmitting '.
'a recognizable protocol frame.'));
}
$messages = $io_protocol->writeData($in_message);
if ($messages) {
$message = head($messages);
$struct = $message['structure'];
// This is the:
//
// ( version ( cap1 ... ) url ... )
//
// The `url` allows us to identify the repository.
$uri = $struct[2]['value'];
$path = $this->getPathFromSubversionURI($uri);
return $this->loadRepositoryWithPath($path);
}
}
if (!$io_channel->isOpenForReading()) {
throw new Exception(
pht(
'Client closed connection before sending a complete protocol '.
'frame.'));
}
// If the client has disconnected, kill the subprocess and bail.
if (!$io_channel->isOpenForWriting()) {
throw new Exception(
pht(
'Client closed connection before receiving response.'));
}
}
}
protected function executeRepositoryOperations() {
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
$repository = $this->getRepository();
$args = $this->getArgs();
if (!$args->getArg('tunnel')) {
throw new Exception('Expected `svnserve -t`!');
}
$command = csprintf(
'svnserve -t --tunnel-user=%s',
$this->getUser()->getUsername());
Add "phd.user" with `sudo` hooks for SSH/HTTP writes Summary: Ref T2230. When fully set up, we have up to three users who all need to write into the repositories: - The webserver needs to write for HTTP receives. - The SSH user needs to write for SSH receives. - The daemons need to write for "git fetch", "git clone", etc. These three users don't need to be different, but in practice they are often not likely to all be the same user. If for no other reason, making them all the same user requires you to "git clone httpd@host.com", and installs are likely to prefer "git clone git@host.com". Using three different users also allows better privilege separation. Particularly, the daemon user can be the //only// user with write access to the repositories. The webserver and SSH user can accomplish their writes through `sudo`, with a whitelisted set of commands. This means that even if you compromise the `ssh` user, you need to find a way to escallate from there to the daemon user in order to, e.g., write arbitrary stuff into the repository or bypass commit hooks. This lays some of the groundwork for a highly-separated configuration where the SSH and HTTP users have the fewest privileges possible and use `sudo` to interact with repositories. Some future work which might make sense: - Make `bin/phd` respect this (require start as the right user, or as root and drop privileges, if this configuration is set). - Execute all `git/hg/svn` commands via sudo? Users aren't expected to configure this yet so I haven't written any documentation. Test Plan: Added an SSH user ("dweller") and gave it sudo by adding this to `/etc/sudoers`: dweller ALL=(epriestley) SETENV: NOPASSWD: /usr/bin/git-upload-pack, /usr/bin/git-receive-pack Then I ran git pushes and pulls over SSH via "dweller@localhost". They successfully interacted with the repository on disk as the "epriestley" user. Reviewers: btrahan Reviewed By: btrahan CC: aran Maniphest Tasks: T2230 Differential Revision: https://secure.phabricator.com/D7589
2013-11-18 17:58:35 +01:00
$command = PhabricatorDaemon::sudoCommandAsDaemonUser($command);
$future = new ExecFuture('%C', $command);
$this->inProtocol = new DiffusionSubversionWireProtocol();
$this->outProtocol = new DiffusionSubversionWireProtocol();
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
$this->command = id($this->newPassthruCommand())
->setIOChannel($this->getIOChannel())
->setCommandChannelFromExecFuture($future)
->setWillWriteCallback(array($this, 'willWriteMessageCallback'))
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
->setWillReadCallback(array($this, 'willReadMessageCallback'));
$this->command->setPauseIOReads(true);
$err = $this->command->execute();
if (!$err && $this->didSeeWrite) {
$this->getRepository()->writeStatusMessage(
PhabricatorRepositoryStatusMessage::TYPE_NEEDS_UPDATE,
PhabricatorRepositoryStatusMessage::CODE_OKAY);
}
return $err;
}
public function willWriteMessageCallback(
PhabricatorSSHPassthruCommand $command,
$message) {
$proto = $this->inProtocol;
$messages = $proto->writeData($message);
$result = array();
foreach ($messages as $message) {
$message_raw = $message['raw'];
$struct = $message['structure'];
if (!$this->inSeenGreeting) {
$this->inSeenGreeting = true;
// The first message the client sends looks like:
//
// ( version ( cap1 ... ) url ... )
//
// We want to grab the URL, load the repository, make sure it exists and
// is accessible, and then replace it with the location of the
// repository on disk.
$uri = $struct[2]['value'];
$struct[2]['value'] = $this->makeInternalURI($uri);
$message_raw = $proto->serializeStruct($struct);
} else if (isset($struct[0]) && $struct[0]['type'] == 'word') {
if (!$proto->isReadOnlyCommand($struct)) {
$this->didSeeWrite = true;
$this->requireWriteAccess($struct[0]['value']);
}
// Several other commands also pass in URLs. We need to translate
// all of these into the internal representation; this also makes sure
// they're valid and accessible.
switch ($struct[0]['value']) {
case 'reparent':
// ( reparent ( url ) )
$struct[1]['value'][0]['value'] = $this->makeInternalURI(
$struct[1]['value'][0]['value']);
$message_raw = $proto->serializeStruct($struct);
break;
case 'switch':
// ( switch ( ( rev ) target recurse url ... ) )
$struct[1]['value'][3]['value'] = $this->makeInternalURI(
$struct[1]['value'][3]['value']);
$message_raw = $proto->serializeStruct($struct);
break;
case 'diff':
// ( diff ( ( rev ) target recurse ignore-ancestry url ... ) )
$struct[1]['value'][4]['value'] = $this->makeInternalURI(
$struct[1]['value'][4]['value']);
$message_raw = $proto->serializeStruct($struct);
break;
case 'add-file':
// ( add-file ( path dir-token file-token [ copy-path copy-rev ] ) )
if (isset($struct[1]['value'][3]['value'][0]['value'])) {
$copy_from = $struct[1]['value'][3]['value'][0]['value'];
$copy_from = $this->makeInternalURI($copy_from);
$struct[1]['value'][3]['value'][0]['value'] = $copy_from;
}
$message_raw = $proto->serializeStruct($struct);
break;
}
}
$result[] = $message_raw;
}
if (!$result) {
return null;
}
return implode('', $result);
}
public function willReadMessageCallback(
PhabricatorSSHPassthruCommand $command,
$message) {
$proto = $this->outProtocol;
$messages = $proto->writeData($message);
$result = array();
foreach ($messages as $message) {
$message_raw = $message['raw'];
$struct = $message['structure'];
if (isset($struct[0]) && ($struct[0]['type'] == 'word')) {
if ($struct[0]['value'] == 'success') {
switch ($this->outPhaseCount) {
case 0:
// This is the "greeting", which announces capabilities.
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
// We already sent this when we were figuring out which
// repository this request is for, so we aren't going to send
// it again.
// Instead, we're going to replay the client's response (which
// we also already read).
$command = $this->getCommand();
$command->writeIORead($this->peekBuffer);
$command->setPauseIOReads(false);
$message_raw = null;
break;
case 1:
// This responds to the client greeting, and announces auth.
break;
case 2:
// This responds to auth, which should be trivial over SSH.
break;
case 3:
// This contains the URI of the repository. We need to edit it;
// if it does not match what the client requested it will reject
// the response.
$struct[1]['value'][1]['value'] = $this->makeExternalURI(
$struct[1]['value'][1]['value']);
$message_raw = $proto->serializeStruct($struct);
break;
default:
// We don't care about other protocol frames.
break;
}
$this->outPhaseCount++;
} else if ($struct[0]['value'] == 'failure') {
// Find any error messages which include the internal URI, and
// replace the text with the external URI.
foreach ($struct[1]['value'] as $key => $error) {
$code = $error['value'][0]['value'];
$message = $error['value'][1]['value'];
$message = str_replace(
$this->internalBaseURI,
$this->externalBaseURI,
$message);
// Derp derp derp derp derp. The structure looks like this:
// ( failure ( ( code message ... ) ... ) )
$struct[1]['value'][$key]['value'][1]['value'] = $message;
}
$message_raw = $proto->serializeStruct($struct);
}
}
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
if ($message_raw !== null) {
$result[] = $message_raw;
}
}
if (!$result) {
return null;
}
return implode('', $result);
}
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
private function getPathFromSubversionURI($uri_string) {
$uri = new PhutilURI($uri_string);
$proto = $uri->getProtocol();
if ($proto !== 'svn+ssh') {
throw new Exception(
pht(
'Protocol for URI "%s" MUST be "svn+ssh".',
$uri_string));
}
$path = $uri->getPath();
// Subversion presumably deals with this, but make sure there's nothing
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
// sketchy going on with the URI.
if (preg_match('(/\\.\\./)', $path)) {
throw new Exception(
pht(
'String "/../" is invalid in path specification "%s".',
$uri_string));
}
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
$path = $this->normalizeSVNPath($path);
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
return $path;
}
private function makeInternalURI($uri_string) {
$uri = new PhutilURI($uri_string);
$repository = $this->getRepository();
$path = $this->getPathFromSubversionURI($uri_string);
$path = preg_replace(
'(^/diffusion/[A-Z]+)',
rtrim($repository->getLocalPath(), '/'),
$path);
if (preg_match('(^/diffusion/[A-Z]+/\z)', $path)) {
$path = rtrim($path, '/');
}
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
// NOTE: We are intentionally NOT removing username information from the
// URI. Subversion retains it over the course of the request and considers
// two repositories with different username identifiers to be distinct and
// incompatible.
$uri->setPath($path);
// If this is happening during the handshake, these are the base URIs for
// the request.
if ($this->externalBaseURI === null) {
$pre = (string)id(clone $uri)->setPath('');
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
$external_path = '/diffusion/'.$repository->getCallsign();
$external_path = $this->normalizeSVNPath($external_path);
$this->externalBaseURI = $pre.$external_path;
$internal_path = rtrim($repository->getLocalPath(), '/');
$internal_path = $this->normalizeSVNPath($internal_path);
$this->internalBaseURI = $pre.$internal_path;
}
return (string)$uri;
}
private function makeExternalURI($uri) {
$internal = $this->internalBaseURI;
$external = $this->externalBaseURI;
if (strncmp($uri, $internal, strlen($internal)) === 0) {
$uri = $external.substr($uri, strlen($internal));
}
return $uri;
}
Prepare SSH connections for proxying Summary: Ref T7034. In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request. In order to proxy the request, we need to know which repository the user is interested in accessing. Split the SSH workflow into two steps: # First, identify the repository. # Then, execute the operation. In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit. This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line. This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically: - The client doesn't tell us what it's after. - To get it to tell us, we have to send it a server capabilities string //first//. - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository. - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame. - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do. The approach here is straightforward, but the implementation is not trivial. Roughly: - Start `svnserve`, read the "hello" frame from it. - Kill `svnserve`. - Send the "hello" to the client. - Wait for the client to send us "I want repository X". - Save the message it sent us in the "peekBuffer". - Return "this is a request for repository X", so we can proxy it. Then, to continue the request: - Start the real `svnserve`. - Read the "hello" frame from it and throw it away. - Write the data in the "peekBuffer" to it, as though we'd just received it from the client. - State of the world is normal again, so we can continue. Also fixed some other issues: - SVN could choke if `repository.default-local-path` contained extra slashes. - PHP might emit some complaints when executing the commit hook; silence those. Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git. Reviewers: btrahan Reviewed By: btrahan Subscribers: epriestley Maniphest Tasks: T7034 Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 19:18:07 +01:00
private function normalizeSVNPath($path) {
// Subversion normalizes redundant slashes internally, so normalize them
// here as well to make sure things match up.
$path = preg_replace('(/+)', '/', $path);
return $path;
}
}