Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.
Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.
A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.
Test Plan:
- Ran `hg clone` over SSH.
- Ran `hg fetch` over SSH.
- Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).
Reviewers: btrahan, asherkin
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 21:18:27 +01:00
|
|
|
<?php
|
|
|
|
|
|
|
|
final class DiffusionSSHMercurialServeWorkflow
|
|
|
|
extends DiffusionSSHMercurialWorkflow {
|
|
|
|
|
|
|
|
protected $didSeeWrite;
|
|
|
|
|
|
|
|
public function didConstruct() {
|
|
|
|
$this->setName('hg');
|
|
|
|
$this->setArguments(
|
|
|
|
array(
|
|
|
|
array(
|
|
|
|
'name' => 'repository',
|
|
|
|
'short' => 'R',
|
|
|
|
'param' => 'repo',
|
|
|
|
),
|
|
|
|
array(
|
|
|
|
'name' => 'stdio',
|
|
|
|
),
|
|
|
|
array(
|
|
|
|
'name' => 'command',
|
|
|
|
'wildcard' => true,
|
|
|
|
),
|
|
|
|
));
|
|
|
|
}
|
|
|
|
|
2013-11-11 21:19:06 +01:00
|
|
|
protected function executeRepositoryOperations() {
|
|
|
|
$args = $this->getArgs();
|
|
|
|
$path = $args->getArg('repository');
|
|
|
|
$repository = $this->loadRepository($path);
|
Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.
Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.
A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.
Test Plan:
- Ran `hg clone` over SSH.
- Ran `hg fetch` over SSH.
- Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).
Reviewers: btrahan, asherkin
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 21:18:27 +01:00
|
|
|
|
|
|
|
$args = $this->getArgs();
|
|
|
|
|
|
|
|
if (!$args->getArg('stdio')) {
|
2014-06-09 20:36:49 +02:00
|
|
|
throw new Exception('Expected `hg ... --stdio`!');
|
Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.
Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.
A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.
Test Plan:
- Ran `hg clone` over SSH.
- Ran `hg fetch` over SSH.
- Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).
Reviewers: btrahan, asherkin
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 21:18:27 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if ($args->getArg('command') !== array('serve')) {
|
2014-06-09 20:36:49 +02:00
|
|
|
throw new Exception('Expected `hg ... serve`!');
|
Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.
Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.
A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.
Test Plan:
- Ran `hg clone` over SSH.
- Ran `hg fetch` over SSH.
- Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).
Reviewers: btrahan, asherkin
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 21:18:27 +01:00
|
|
|
}
|
|
|
|
|
Add "phd.user" with `sudo` hooks for SSH/HTTP writes
Summary:
Ref T2230. When fully set up, we have up to three users who all need to write into the repositories:
- The webserver needs to write for HTTP receives.
- The SSH user needs to write for SSH receives.
- The daemons need to write for "git fetch", "git clone", etc.
These three users don't need to be different, but in practice they are often not likely to all be the same user. If for no other reason, making them all the same user requires you to "git clone httpd@host.com", and installs are likely to prefer "git clone git@host.com".
Using three different users also allows better privilege separation. Particularly, the daemon user can be the //only// user with write access to the repositories. The webserver and SSH user can accomplish their writes through `sudo`, with a whitelisted set of commands. This means that even if you compromise the `ssh` user, you need to find a way to escallate from there to the daemon user in order to, e.g., write arbitrary stuff into the repository or bypass commit hooks.
This lays some of the groundwork for a highly-separated configuration where the SSH and HTTP users have the fewest privileges possible and use `sudo` to interact with repositories. Some future work which might make sense:
- Make `bin/phd` respect this (require start as the right user, or as root and drop privileges, if this configuration is set).
- Execute all `git/hg/svn` commands via sudo?
Users aren't expected to configure this yet so I haven't written any documentation.
Test Plan:
Added an SSH user ("dweller") and gave it sudo by adding this to `/etc/sudoers`:
dweller ALL=(epriestley) SETENV: NOPASSWD: /usr/bin/git-upload-pack, /usr/bin/git-receive-pack
Then I ran git pushes and pulls over SSH via "dweller@localhost". They successfully interacted with the repository on disk as the "epriestley" user.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7589
2013-11-18 17:58:35 +01:00
|
|
|
$command = csprintf('hg -R %s serve --stdio', $repository->getLocalPath());
|
|
|
|
$command = PhabricatorDaemon::sudoCommandAsDaemonUser($command);
|
Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.
Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.
A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.
Test Plan:
- Ran `hg clone` over SSH.
- Ran `hg fetch` over SSH.
- Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).
Reviewers: btrahan, asherkin
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 21:18:27 +01:00
|
|
|
|
2013-12-03 00:46:03 +01:00
|
|
|
$future = id(new ExecFuture('%C', $command))
|
|
|
|
->setEnv($this->getEnvironment());
|
Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.
Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.
A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.
Test Plan:
- Ran `hg clone` over SSH.
- Ran `hg fetch` over SSH.
- Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).
Reviewers: btrahan, asherkin
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 21:18:27 +01:00
|
|
|
|
Add "phd.user" with `sudo` hooks for SSH/HTTP writes
Summary:
Ref T2230. When fully set up, we have up to three users who all need to write into the repositories:
- The webserver needs to write for HTTP receives.
- The SSH user needs to write for SSH receives.
- The daemons need to write for "git fetch", "git clone", etc.
These three users don't need to be different, but in practice they are often not likely to all be the same user. If for no other reason, making them all the same user requires you to "git clone httpd@host.com", and installs are likely to prefer "git clone git@host.com".
Using three different users also allows better privilege separation. Particularly, the daemon user can be the //only// user with write access to the repositories. The webserver and SSH user can accomplish their writes through `sudo`, with a whitelisted set of commands. This means that even if you compromise the `ssh` user, you need to find a way to escallate from there to the daemon user in order to, e.g., write arbitrary stuff into the repository or bypass commit hooks.
This lays some of the groundwork for a highly-separated configuration where the SSH and HTTP users have the fewest privileges possible and use `sudo` to interact with repositories. Some future work which might make sense:
- Make `bin/phd` respect this (require start as the right user, or as root and drop privileges, if this configuration is set).
- Execute all `git/hg/svn` commands via sudo?
Users aren't expected to configure this yet so I haven't written any documentation.
Test Plan:
Added an SSH user ("dweller") and gave it sudo by adding this to `/etc/sudoers`:
dweller ALL=(epriestley) SETENV: NOPASSWD: /usr/bin/git-upload-pack, /usr/bin/git-receive-pack
Then I ran git pushes and pulls over SSH via "dweller@localhost". They successfully interacted with the repository on disk as the "epriestley" user.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7589
2013-11-18 17:58:35 +01:00
|
|
|
$io_channel = $this->getIOChannel();
|
Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.
Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.
A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.
Test Plan:
- Ran `hg clone` over SSH.
- Ran `hg fetch` over SSH.
- Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).
Reviewers: btrahan, asherkin
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2230
Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 21:18:27 +01:00
|
|
|
$protocol_channel = new DiffusionSSHMercurialWireClientProtocolChannel(
|
|
|
|
$io_channel);
|
|
|
|
|
|
|
|
$err = id($this->newPassthruCommand())
|
|
|
|
->setIOChannel($protocol_channel)
|
|
|
|
->setCommandChannelFromExecFuture($future)
|
|
|
|
->setWillWriteCallback(array($this, 'willWriteMessageCallback'))
|
|
|
|
->execute();
|
|
|
|
|
|
|
|
// TODO: It's apparently technically possible to communicate errors to
|
|
|
|
// Mercurial over SSH by writing a special "\n<error>\n-\n" string. However,
|
|
|
|
// my attempt to implement that resulted in Mercurial closing the socket and
|
|
|
|
// then hanging, without showing the error. This might be an issue on our
|
|
|
|
// side (we need to close our half of the socket?), or maybe the code
|
|
|
|
// for this in Mercurial doesn't actually work, or maybe something else
|
|
|
|
// is afoot. At some point, we should look into doing this more cleanly.
|
|
|
|
// For now, when we, e.g., reject writes for policy reasons, the user will
|
|
|
|
// see "abort: unexpected response: empty string" after the diagnostically
|
|
|
|
// useful, e.g., "remote: This repository is read-only over SSH." message.
|
|
|
|
|
|
|
|
if (!$err && $this->didSeeWrite) {
|
|
|
|
$repository->writeStatusMessage(
|
|
|
|
PhabricatorRepositoryStatusMessage::TYPE_NEEDS_UPDATE,
|
|
|
|
PhabricatorRepositoryStatusMessage::CODE_OKAY);
|
|
|
|
}
|
|
|
|
|
|
|
|
return $err;
|
|
|
|
}
|
|
|
|
|
|
|
|
public function willWriteMessageCallback(
|
|
|
|
PhabricatorSSHPassthruCommand $command,
|
|
|
|
$message) {
|
|
|
|
|
|
|
|
$command = $message['command'];
|
|
|
|
|
|
|
|
// Check if this is a readonly command.
|
|
|
|
|
|
|
|
$is_readonly = false;
|
|
|
|
if ($command == 'batch') {
|
|
|
|
$cmds = idx($message['arguments'], 'cmds');
|
|
|
|
if (DiffusionMercurialWireProtocol::isReadOnlyBatchCommand($cmds)) {
|
|
|
|
$is_readonly = true;
|
|
|
|
}
|
|
|
|
} else if (DiffusionMercurialWireProtocol::isReadOnlyCommand($command)) {
|
|
|
|
$is_readonly = true;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!$is_readonly) {
|
|
|
|
$this->requireWriteAccess();
|
|
|
|
$this->didSeeWrite = true;
|
|
|
|
}
|
|
|
|
|
|
|
|
// If we're good, return the raw message data.
|
|
|
|
return $message['raw'];
|
|
|
|
}
|
|
|
|
|
|
|
|
}
|