1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-02 11:42:42 +01:00
phorge-phorge/src/infrastructure/daemon/PhabricatorDaemon.php

72 lines
2.2 KiB
PHP
Raw Normal View History

<?php
abstract class PhabricatorDaemon extends PhutilDaemon {
protected function willRun() {
parent::willRun();
$phabricator = phutil_get_library_root('phabricator');
$root = dirname($phabricator);
require_once $root.'/scripts/__init_script__.php';
}
protected function willSleep($duration) {
In taskmaster daemons, only close connections which were not used recently Summary: Ref T11458. Depends on D16388. Currently, we're very aggressive about closing connections in the taskmaster daemons. This can end up taking up a lot of resources. In particular, because the outgoing port for outbound connections normally can not be reused for 60 seconds after a connection closes, we may exhaust outbound ports on the host if there's a big queue full of stuff that's being processed very quickly. At a minimum, we //always// are holding open a `worker` connection, which we always need again right away. So even in the best case we end up opening/closing this about once per second and each daemon takes up about ~60 outbound ports when it should take up ~1. So, make two adjustments: - First, only close connections which we haven't issued a query on in the last 60 seconds. This should prevent us from closing connections that we'll need again immediately in most cases. In the worst case, we shouldn't be eating up any extra ports under default TCP behavior. - Second, explicitly close connections. We were relying on implicit/GC behavior (maybe as a holdover from very long ago, before we got connection wrappers in place?), which probably did about the same thing but isn't as predictable and can't be profiled or instrumented. Test Plan: This is somewhat difficult to test completely convincingly in isolation since the problem behavior depends on production scales and the workload, and to some degree on configuration. I tested that this stuff baiscally works by adding logging to connect/close and running the daemons, verifying that they churned connections a lot before this change (e.g., ~1/s even at no load) and churn rarely afterward (e.g., almost never at no load). I ran some workload through them to make sure I didn't completely break anything. The best real test is just seeing how production responds. Current inbound/outbound connections on `secure001` are 1,200: ``` secure001 $ netstat -t | grep :mysql | wc -l 1164 ``` Current outbound from `repo001` are 18,600: ``` repo001 $ netstat -t | grep :mysql | wc -l 18663 ``` Reviewers: chad Reviewed By: chad Maniphest Tasks: T11458 Differential Revision: https://secure.phabricator.com/D16389
2016-08-11 17:47:47 +02:00
LiskDAO::closeInactiveConnections(60);
return;
}
public function getViewer() {
return PhabricatorUser::getOmnipotentUser();
}
Add "phd.user" with `sudo` hooks for SSH/HTTP writes Summary: Ref T2230. When fully set up, we have up to three users who all need to write into the repositories: - The webserver needs to write for HTTP receives. - The SSH user needs to write for SSH receives. - The daemons need to write for "git fetch", "git clone", etc. These three users don't need to be different, but in practice they are often not likely to all be the same user. If for no other reason, making them all the same user requires you to "git clone httpd@host.com", and installs are likely to prefer "git clone git@host.com". Using three different users also allows better privilege separation. Particularly, the daemon user can be the //only// user with write access to the repositories. The webserver and SSH user can accomplish their writes through `sudo`, with a whitelisted set of commands. This means that even if you compromise the `ssh` user, you need to find a way to escallate from there to the daemon user in order to, e.g., write arbitrary stuff into the repository or bypass commit hooks. This lays some of the groundwork for a highly-separated configuration where the SSH and HTTP users have the fewest privileges possible and use `sudo` to interact with repositories. Some future work which might make sense: - Make `bin/phd` respect this (require start as the right user, or as root and drop privileges, if this configuration is set). - Execute all `git/hg/svn` commands via sudo? Users aren't expected to configure this yet so I haven't written any documentation. Test Plan: Added an SSH user ("dweller") and gave it sudo by adding this to `/etc/sudoers`: dweller ALL=(epriestley) SETENV: NOPASSWD: /usr/bin/git-upload-pack, /usr/bin/git-receive-pack Then I ran git pushes and pulls over SSH via "dweller@localhost". They successfully interacted with the repository on disk as the "epriestley" user. Reviewers: btrahan Reviewed By: btrahan CC: aran Maniphest Tasks: T2230 Differential Revision: https://secure.phabricator.com/D7589
2013-11-18 17:58:35 +01:00
/**
* Format a command so it executes as the daemon user, if a daemon user is
* defined. This wraps the provided command in `sudo -u ...`, roughly.
*
* @param PhutilCommandString Command to execute.
* @return PhutilCommandString `sudo` version of the command.
*/
public static function sudoCommandAsDaemonUser($command) {
$user = PhabricatorEnv::getEnvConfig('phd.user');
if (!$user) {
// No daemon user is set, so just run this as ourselves.
return $command;
}
// We may reach this method while already running as the daemon user: for
// example, active and passive synchronization of clustered repositories
// run the same commands through the same code, but as different users.
// By default, `sudo` won't let you sudo to yourself, so we can get into
// trouble if we're already running as the daemon user unless the host has
// been configured to let the daemon user run commands as itself.
// Since this is silly and more complicated than doing this check, don't
// use `sudo` if we're already running as the correct user.
if (function_exists('posix_getuid')) {
$uid = posix_getuid();
$info = posix_getpwuid($uid);
if ($info && $info['name'] == $user) {
return $command;
}
}
Add "phd.user" with `sudo` hooks for SSH/HTTP writes Summary: Ref T2230. When fully set up, we have up to three users who all need to write into the repositories: - The webserver needs to write for HTTP receives. - The SSH user needs to write for SSH receives. - The daemons need to write for "git fetch", "git clone", etc. These three users don't need to be different, but in practice they are often not likely to all be the same user. If for no other reason, making them all the same user requires you to "git clone httpd@host.com", and installs are likely to prefer "git clone git@host.com". Using three different users also allows better privilege separation. Particularly, the daemon user can be the //only// user with write access to the repositories. The webserver and SSH user can accomplish their writes through `sudo`, with a whitelisted set of commands. This means that even if you compromise the `ssh` user, you need to find a way to escallate from there to the daemon user in order to, e.g., write arbitrary stuff into the repository or bypass commit hooks. This lays some of the groundwork for a highly-separated configuration where the SSH and HTTP users have the fewest privileges possible and use `sudo` to interact with repositories. Some future work which might make sense: - Make `bin/phd` respect this (require start as the right user, or as root and drop privileges, if this configuration is set). - Execute all `git/hg/svn` commands via sudo? Users aren't expected to configure this yet so I haven't written any documentation. Test Plan: Added an SSH user ("dweller") and gave it sudo by adding this to `/etc/sudoers`: dweller ALL=(epriestley) SETENV: NOPASSWD: /usr/bin/git-upload-pack, /usr/bin/git-receive-pack Then I ran git pushes and pulls over SSH via "dweller@localhost". They successfully interacted with the repository on disk as the "epriestley" user. Reviewers: btrahan Reviewed By: btrahan CC: aran Maniphest Tasks: T2230 Differential Revision: https://secure.phabricator.com/D7589
2013-11-18 17:58:35 +01:00
// Get the absolute path so we're safe against the caller wiping out
// PATH.
$sudo = Filesystem::resolveBinary('sudo');
if (!$sudo) {
throw new Exception(pht("Unable to find 'sudo'!"));
}
// Flags here are:
//
// -E: Preserve the environment.
// -n: Non-interactive. Exit with an error instead of prompting.
// -u: Which user to sudo to.
return csprintf('%s -E -n -u %s -- %C', $sudo, $user, $command);
}
}