Implement SSHD glue and Conduit SSH endpoint

Summary:
  - Build "sshd-auth" (for authentication) and "sshd-exec" (for command execution) binaries. These are callable by "sshd-vcs", located [[https://github.com/epriestley/sshd-vcs | in my account on GitHub]]. They are based on precursors [[https://github.com/epriestley/sshd-vcs-glue | here on GitHub]] which I deployed for TenXer about a year ago, so I have some confidence they at least basically work.
    - The problem this solves is that normally every user would need an account on a machine to connect to it, and/or their public keys would all need to be listed in `~/.authorized_keys`. This is a big pain in most installs. Software like Gitosis/Gitolite solve this problem by giving you an easy way to add public keys to `~/.authorized_keys`, but this is pretty gross.
    - Roughly, instead of looking in `~/.authorized_keys` when a user connects, the patched sshd instead runs `echo <public key> | sshd-auth`. The `sshd-auth` script looks up the public key and authorizes the matching user, if they exist. It also forces sshd to run `sshd-exec` instead of a normal shell.
    - `sshd-exec` receives the authenticated user and any command which was passed to ssh (like `git receive-pack`) and can route them appropriately.
    - Overall, this permits a single account to be set up on a server which all Phabricator users can connect to without any extra work, and which can safely execute commands and apply appropriate permissions, and disable users when they are disabled in Phabricator and all that stuff.
  - Build out "sshd-exec" to do more thorough checks and setup, and delegate command execution to Workflows (they now exist, and did not when I originally built this stuff).
  - Convert @btrahan's conduit API script into a workflow and slightly simplify it (ConduitCall did not exist at the time it was written).

The next steps here on the Repository side are to implement Workflows for Git, SVN and HG wire protocols. These will mostly just proxy the protocols, but also need to enforce permissions. So the approach will basically be:

  - Implement workflows for stuff like `git receive-pack`.
  - These workflows will implement enough of the underlying protocol to determine what resource the user is trying to access, and whether they want to read or write it.
  - They'll then do a permissons check, and kick the user out if they don't have permission to do whatever they are trying to do.
  - If the user does have permission, we just proxy the rest of the transaction.

Next steps on the Conduit side are more simple:

  - Make ConduitClient understand "ssh://" URLs.

Test Plan: Ran `sshd-exec --phabricator-ssh-user epriestley conduit differential.query`, etc. This will get a more comprehensive test once I set up sshd-vcs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603, T550

Differential Revision: https://secure.phabricator.com/D4229

bin/ssh-auth

@@ -0,0 +1 @@
+../scripts/ssh/ssh-auth.php

bin/ssh-exec

@@ -0,0 +1 @@
+../scripts/ssh/ssh-exec.php

scripts/conduit/api.php

@@ -1,79 +0,0 @@
-#!/usr/bin/env php
-<?php
-
-$root = dirname(dirname(dirname(__FILE__)));
-require_once $root.'/scripts/__init_script__.php';
-
-$time_start = microtime(true);
-
-if ($argc !== 3) {
-  echo "usage: api.php <user_phid> <method>\n";
-  exit(1);
-}
-
-$user     = null;
-$user_str = $argv[1];
-try {
-  $user = id(new PhabricatorUser())
-    ->loadOneWhere('phid = %s', $user_str);
-} catch (Exception $e) {
-  // no op; we'll error in a line or two
-}
-if (empty($user)) {
-  echo "usage: api.php <user_phid> <method>\n" .
-       "user {$user_str} does not exist or failed to load\n";
-  exit(1);
-}
-
-$method           = $argv[2];
-$method_class_str = ConduitAPIMethod::getClassNameFromAPIMethodName($method);
-try {
-  $method_class = newv($method_class_str, array());
-} catch (Exception $e) {
-  echo "usage: api.php <user_phid> <method>\n" .
-       "method {$method_class_str} does not exist\n";
-  exit(1);
-}
-$log = new PhabricatorConduitMethodCallLog();
-$log->setMethod($method);
-
-$params = @file_get_contents('php://stdin');
-$params = json_decode($params, true);
-if (!is_array($params)) {
-  echo "provide method parameters on stdin as a JSON blob";
-  exit(1);
-}
-
-// build a quick ConduitAPIRequest from stdin PLUS the authenticated user
-$conduit_request = new ConduitAPIRequest($params);
-$conduit_request->setUser($user);
-
-try {
-  $result = $method_class->executeMethod($conduit_request);
-  $error_code = null;
-  $error_info = null;
-} catch (ConduitException $ex) {
-  $result = null;
-  $error_code = $ex->getMessage();
-  if ($ex->getErrorDescription()) {
-    $error_info = $ex->getErrorDescription();
-  } else {
-    $error_info = $method_handler->getErrorDescription($error_code);
-  }
-}
-$time_end = microtime(true);
-
-$response = id(new ConduitAPIResponse())
-  ->setResult($result)
-  ->setErrorCode($error_code)
-  ->setErrorInfo($error_info);
-echo json_encode($response->toDictionary()), "\n";
-
-// TODO -- how get $connection_id from SSH?
-$connection_id = null;
-$log->setConnectionID($connection_id);
-$log->setError((string)$error_code);
-$log->setDuration(1000000 * ($time_end - $time_start));
-$log->save();
-
-exit();

scripts/ssh/ssh-auth.php

@@ -0,0 +1,54 @@
+#!/usr/bin/env php
+<?php
+
+$root = dirname(dirname(dirname(__FILE__)));
+require_once $root.'/scripts/__init_script__.php';
+
+$cert = file_get_contents('php://stdin');
+
+$user = null;
+if ($cert) {
+  $user_dao = new PhabricatorUser();
+  $ssh_dao = new PhabricatorUserSSHKey();
+  $conn = $user_dao->establishConnection('r');
+
+  list($type, $body) = array_merge(
+    explode(' ', $cert),
+    array('', ''));
+
+  $row = queryfx_one(
+    $conn,
+    'SELECT userName FROM %T u JOIN %T ssh ON u.phid = ssh.userPHID
+      WHERE ssh.keyBody = %s AND ssh.keyType = %s',
+    $user_dao->getTableName(),
+    $ssh_dao->getTableName(),
+    $body,
+    $type);
+  if ($row) {
+    $user = idx($row, 'userName');
+  }
+}
+
+if (!$user) {
+  exit(1);
+}
+
+if (!PhabricatorUser::validateUsername($user)) {
+  exit(1);
+}
+
+$bin = $root.'/bin/ssh-exec';
+$cmd = csprintf('%s --phabricator-ssh-user %s', $bin, $user);
+// This is additional escaping for the SSH 'command="..."' string.
+$cmd = str_replace('"', '\\"', $cmd);
+
+$options = array(
+  'command="'.$cmd.'"',
+  'no-port-forwarding',
+  'no-X11-forwarding',
+  'no-agent-forwarding',
+  'no-pty',
+);
+
+echo implode(',', $options);
+exit(0);

scripts/ssh/ssh-exec.php

@@ -0,0 +1,87 @@
+#!/usr/bin/env php
+<?php
+
+$root = dirname(dirname(dirname(__FILE__)));
+require_once $root.'/scripts/__init_script__.php';
+
+$args = new PhutilArgumentParser($argv);
+$args->setTagline('receive SSH requests');
+$args->setSynopsis(<<<EOSYNOPSIS
+**ssh-exec** --phabricator-ssh-user __user__ __commmand__ [__options__]
+    Receive SSH requests.
+
+EOSYNOPSIS
+);
+
+// NOTE: Do NOT parse standard arguments. Arguments are coming from a remote
+// client over SSH, and they should not be able to execute "--xprofile",
+// "--recon", etc.
+
+$args->parsePartial(
+  array(
+    array(
+      'name'  => 'phabricator-ssh-user',
+      'param' => 'username',
+    ),
+  ));
+
+try {
+  $user_name = $args->getArg('phabricator-ssh-user');
+  if (!strlen($user_name)) {
+    throw new Exception("No username.");
+  }
+
+  $user = id(new PhabricatorUser())->loadOneWhere(
+    'userName = %s',
+    $user_name);
+  if (!$user) {
+    throw new Exception("Invalid username.");
+  }
+
+  if ($user->getIsDisabled()) {
+    throw new Exception("You have been exiled.");
+  }
+
+  $workflows = array(
+    new ConduitSSHWorkflow(),
+  );
+
+  // This duplicates logic in parseWorkflows(), but allows us to raise more
+  // concise/relevant exceptions when the client is a remote SSH.
+  $remain = $args->getUnconsumedArgumentVector();
+  if (empty($remain)) {
+    throw new Exception("No command.");
+  } else {
+    $command = head($remain);
+    $workflow_names = mpull($workflows, 'getName', 'getName');
+    if (empty($workflow_names[$command])) {
+      throw new Exception("Invalid command.");
+    }
+  }
+
+  $workflow = $args->parseWorkflows($workflows);
+  $workflow->setUser($user);
+
+  $sock_stdin = fopen('php://stdin', 'r');
+  if (!$sock_stdin) {
+    throw new Exception("Unable to open stdin.");
+  }
+
+  $sock_stdout = fopen('php://stdout', 'w');
+  if (!$sock_stdout) {
+    throw new Exception("Unable to open stdout.");
+  }
+
+  $socket_channel = new PhutilSocketChannel(
+    $sock_stdin,
+    $sock_stdout);
+  $metrics_channel = new PhutilMetricsChannel($socket_channel);
+  $workflow->setIOChannel($metrics_channel);
+
+  $err = $workflow->execute($args);
+
+  $metrics_channel->flush();
+} catch (Exception $ex) {
+  echo "phabricator-ssh-exec: ".$ex->getMessage()."\n";
+  exit(1);
+}

src/__phutil_library_map__.php

@@ -193,6 +193,7 @@ phutil_register_library_map(array(
     'ConduitCall' => 'applications/conduit/call/ConduitCall.php',
     'ConduitCallTestCase' => 'applications/conduit/call/__tests__/ConduitCallTestCase.php',
     'ConduitException' => 'applications/conduit/protocol/ConduitException.php',
+    'ConduitSSHWorkflow' => 'applications/conduit/ssh/ConduitSSHWorkflow.php',
     'DarkConsoleConfigPlugin' => 'aphront/console/plugin/DarkConsoleConfigPlugin.php',
     'DarkConsoleController' => 'aphront/console/DarkConsoleController.php',
     'DarkConsoleCore' => 'aphront/console/DarkConsoleCore.php',
@@ -1074,6 +1075,7 @@ phutil_register_library_map(array(
     'PhabricatorRequestOverseer' => 'infrastructure/PhabricatorRequestOverseer.php',
     'PhabricatorS3FileStorageEngine' => 'applications/files/engine/PhabricatorS3FileStorageEngine.php',
     'PhabricatorSQLPatchList' => 'infrastructure/storage/patch/PhabricatorSQLPatchList.php',
+    'PhabricatorSSHWorkflow' => 'infrastructure/ssh/PhabricatorSSHWorkflow.php',
     'PhabricatorScopedEnv' => 'infrastructure/PhabricatorScopedEnv.php',
     'PhabricatorSearchAbstractDocument' => 'applications/search/index/PhabricatorSearchAbstractDocument.php',
     'PhabricatorSearchAttachController' => 'applications/search/controller/PhabricatorSearchAttachController.php',
@@ -1521,6 +1523,7 @@ phutil_register_library_map(array(
     'ConduitAPI_user_whoami_Method' => 'ConduitAPI_user_Method',
     'ConduitCallTestCase' => 'PhabricatorTestCase',
     'ConduitException' => 'Exception',
+    'ConduitSSHWorkflow' => 'PhabricatorSSHWorkflow',
     'DarkConsoleConfigPlugin' => 'DarkConsolePlugin',
     'DarkConsoleController' => 'PhabricatorController',
     'DarkConsoleErrorLogPlugin' => 'DarkConsolePlugin',
@@ -2337,6 +2340,7 @@ phutil_register_library_map(array(
     'PhabricatorRepositorySymbol' => 'PhabricatorRepositoryDAO',
     'PhabricatorRepositoryTestCase' => 'PhabricatorTestCase',
     'PhabricatorS3FileStorageEngine' => 'PhabricatorFileStorageEngine',
+    'PhabricatorSSHWorkflow' => 'PhutilArgumentWorkflow',
     'PhabricatorSearchAttachController' => 'PhabricatorSearchBaseController',
     'PhabricatorSearchBaseController' => 'PhabricatorController',
     'PhabricatorSearchCommitIndexer' => 'PhabricatorSearchDocumentIndexer',

src/applications/conduit/ssh/ConduitSSHWorkflow.php

@@ -0,0 +1,81 @@
+<?php
+
+final class ConduitSSHWorkflow extends PhabricatorSSHWorkflow {
+
+  public function didConstruct() {
+    $this->setName('conduit');
+    $this->setArguments(
+      array(
+        array(
+          'name'      => 'method',
+          'wildcard'  => true,
+        ),
+      ));
+  }
+
+  public function execute(PhutilArgumentParser $args) {
+    $time_start = microtime(true);
+
+    $methodv = $args->getArg('method');
+    if (!$methodv) {
+      throw new Exception("No Conduit method provided.");
+    } else if (count($methodv) > 1) {
+      throw new Exception("Too many Conduit methods provided.");
+    }
+
+    $method = head($methodv);
+
+    $json = $this->readAllInput();
+    $raw_params = json_decode($json, true);
+    if (!is_array($raw_params)) {
+      throw new Exception("Invalid JSON input.");
+    }
+
+    $params = $raw_params;
+    unset($params['__conduit__']);
+    $metadata = idx($raw_params, '__conduit__', array());
+
+    $call = null;
+    $error_code = null;
+    $error_info = null;
+
+    try {
+      $call = new ConduitCall($method, $params);
+      $call->setUser($this->getUser());
+
+      $result = $call->execute();
+    } catch (ConduitException $ex) {
+      $result = null;
+      $error_code = $ex->getMessage();
+      if ($ex->getErrorDescription()) {
+        $error_info = $ex->getErrorDescription();
+      } else if ($call) {
+        $error_info = $call->getErrorDescription($error_code);
+      }
+    }
+
+    $response = id(new ConduitAPIResponse())
+      ->setResult($result)
+      ->setErrorCode($error_code)
+      ->setErrorInfo($error_info);
+
+    $json_out = json_encode($response->toDictionary());
+    $json_out = $json_out."\n";
+
+    $this->getIOChannel()->write($json_out);
+
+    // NOTE: Flush here so we can get an accurate result for the duration,
+    // if the response is large and the receiver is slow to read it.
+    $this->getIOChannel()->flush();
+
+    $time_end = microtime(true);
+
+    $connection_id = idx($metadata, 'connectionID');
+    $log = new PhabricatorConduitMethodCallLog();
+    $log->setConnectionID($connection_id);
+    $log->setMethod($method);
+    $log->setError((string)$error_code);
+    $log->setDuration(1000000 * ($time_end - $time_start));
+    $log->save();
+  }
+}

src/infrastructure/ssh/PhabricatorSSHWorkflow.php

@@ -0,0 +1,41 @@
+<?php
+
+abstract class PhabricatorSSHWorkflow extends PhutilArgumentWorkflow {
+
+  private $user;
+  private $iochannel;
+
+  public function setUser(PhabricatorUser $user) {
+    $this->user = $user;
+    return $this;
+  }
+
+  public function getUser() {
+    return $this->user;
+  }
+
+  final public function isExecutable() {
+    return false;
+  }
+
+  public function setIOChannel(PhutilChannel $channel) {
+    $this->iochannel = $channel;
+    return $this;
+  }
+
+  public function getIOChannel() {
+    return $this->iochannel;
+  }
+
+  public function readAllInput() {
+    $channel = $this->getIOChannel();
+    while ($channel->update()) {
+      PhutilChannel::waitForAny(array($channel));
+      if (!$channel->isOpenForReading()) {
+        break;
+      }
+    }
+    return $channel->read();
+  }
+
+}