mirror of
https://we.phorge.it/source/arcanist.git
synced 2025-01-09 06:11:01 +01:00
f4aadb9604
Summary: Fixes T5097. When linting a large list of paths (e.g., with `--everything`), do this internally: $chunks = array_chunk($paths, 32); foreach ($chunks as $chunk) { $this->lintChunk($chunk); } This keeps the advantages of parallelism and artifact sharing for future-based linters, without having memory usage grow in an unbounded way. These callbacks changed: - `willLintPath()`: Useless, no meaningful implementations. Internalized the required side effect and broke the hook. - `didRunLinters()`: Now useless, with no meaningful implementations. Broke the hook. - `didLintPaths()`: New hook which executes opposite `willLintPaths()`. - `lintPath()`: Linters no longer need to implement this method. XHPAST now has an explicit way to release shared futures. These minor changes also happened: - Formalized the "linter ID", which is a semi-durable identifier for the cache. - Removed linter -> exception explicit mapping, which was unused. We now just collect exceptions. - We do the `canRun()` checks first (and separately) now. - Share more service call profiling code. - Fix an issue where the test harness would use the path on disk, even if configuration set a different path. Test Plan: - Ran `arc lint --everything` in `arcanist/`. - With no chunking, saw **unstable** memory usage with a peak at 941 MB. - With chunk size 32, saw **stable** memory usage with a peak at 269 MB. - With chunk size 8, saw **stable** memory usage with a peak at 180 MB. - Ran with `--trace` and saw profiling information. - Created this diff. Reviewers: joshuaspence Reviewed By: joshuaspence Subscribers: epriestley Maniphest Tasks: T5097 Differential Revision: https://secure.phabricator.com/D12501
56 lines
1.3 KiB
PHP
56 lines
1.3 KiB
PHP
<?php
|
|
|
|
abstract class ArcanistFutureLinter extends ArcanistLinter {
|
|
|
|
private $futures;
|
|
|
|
abstract protected function buildFutures(array $paths);
|
|
abstract protected function resolveFuture($path, Future $future);
|
|
|
|
final protected function getFuturesLimit() {
|
|
return 8;
|
|
}
|
|
|
|
final public function willLintPaths(array $paths) {
|
|
$limit = $this->getFuturesLimit();
|
|
$this->futures = id(new FutureIterator(array()))->limit($limit);
|
|
foreach ($this->buildFutures($paths) as $path => $future) {
|
|
$this->futures->addFuture($future, $path);
|
|
}
|
|
}
|
|
|
|
final public function lintPath($path) {
|
|
return;
|
|
}
|
|
|
|
final public function didLintPaths(array $paths) {
|
|
if (!$this->futures) {
|
|
return;
|
|
}
|
|
|
|
$map = array();
|
|
foreach ($this->futures as $path => $future) {
|
|
$this->setActivePath($path);
|
|
$this->resolveFuture($path, $future);
|
|
$map[$path] = $future;
|
|
}
|
|
$this->futures = array();
|
|
|
|
$this->didResolveLinterFutures($map);
|
|
}
|
|
|
|
|
|
/**
|
|
* Hook for cleaning up resources.
|
|
*
|
|
* This is invoked after a block of futures resolve, and allows linters to
|
|
* discard or clean up any shared resources they no longer need.
|
|
*
|
|
* @param map<string, Future> Map of paths to resolved futures.
|
|
* @return void
|
|
*/
|
|
protected function didResolveLinterFutures(array $futures) {
|
|
return;
|
|
}
|
|
|
|
}
|