mirror of
https://we.phorge.it/source/phorge.git
synced 2024-12-18 19:40:55 +01:00
No description
5e3efca08a
Summary: Ref T11458. Depends on D16388. Currently, we're very aggressive about closing connections in the taskmaster daemons. This can end up taking up a lot of resources. In particular, because the outgoing port for outbound connections normally can not be reused for 60 seconds after a connection closes, we may exhaust outbound ports on the host if there's a big queue full of stuff that's being processed very quickly. At a minimum, we //always// are holding open a `worker` connection, which we always need again right away. So even in the best case we end up opening/closing this about once per second and each daemon takes up about ~60 outbound ports when it should take up ~1. So, make two adjustments: - First, only close connections which we haven't issued a query on in the last 60 seconds. This should prevent us from closing connections that we'll need again immediately in most cases. In the worst case, we shouldn't be eating up any extra ports under default TCP behavior. - Second, explicitly close connections. We were relying on implicit/GC behavior (maybe as a holdover from very long ago, before we got connection wrappers in place?), which probably did about the same thing but isn't as predictable and can't be profiled or instrumented. Test Plan: This is somewhat difficult to test completely convincingly in isolation since the problem behavior depends on production scales and the workload, and to some degree on configuration. I tested that this stuff baiscally works by adding logging to connect/close and running the daemons, verifying that they churned connections a lot before this change (e.g., ~1/s even at no load) and churn rarely afterward (e.g., almost never at no load). I ran some workload through them to make sure I didn't completely break anything. The best real test is just seeing how production responds. Current inbound/outbound connections on `secure001` are 1,200: ``` secure001 $ netstat -t | grep :mysql | wc -l 1164 ``` Current outbound from `repo001` are 18,600: ``` repo001 $ netstat -t | grep :mysql | wc -l 18663 ``` Reviewers: chad Reviewed By: chad Maniphest Tasks: T11458 Differential Revision: https://secure.phabricator.com/D16389 |
||
|---|---|---|
| bin | ||
| conf | ||
| externals | ||
| resources | ||
| scripts | ||
| src | ||
| support | ||
| webroot | ||
| .arcconfig | ||
| .arclint | ||
| .arcunit | ||
| .editorconfig | ||
| .gitignore | ||
| LICENSE | ||
| NOTICE | ||
| README.md | ||
Phabricator is a collection of web applications which help software companies build better software.
Phabricator includes applications for:
- reviewing and auditing source code;
- hosting and browsing repositories;
- tracking bugs;
- managing projects;
- conversing with team members;
- assembling a party to venture forth;
- writing stuff down and reading it later;
- hiding stuff from coworkers; and
- also some other things.
You can learn more about the project (and find links to documentation and resources) at Phabricator.org
Phabricator is developed and maintained by Phacility.
SUPPORT RESOURCES
For resources on filing bugs, requesting features, reporting security issues, and getting other kinds of support, see Support Resources.
NO PULL REQUESTS!
We do not accept pull requests through GitHub. If you would like to contribute code, please read our Contributor's Guide.
LICENSE
Phabricator is released under the Apache 2.0 license except as otherwise noted.