Summary:
Fixes T5097. When linting a large list of paths (e.g., with `--everything`), do this internally:
$chunks = array_chunk($paths, 32);
foreach ($chunks as $chunk) {
$this->lintChunk($chunk);
}
This keeps the advantages of parallelism and artifact sharing for future-based linters, without having memory usage grow in an unbounded way.
These callbacks changed:
- `willLintPath()`: Useless, no meaningful implementations. Internalized the required side effect and broke the hook.
- `didRunLinters()`: Now useless, with no meaningful implementations. Broke the hook.
- `didLintPaths()`: New hook which executes opposite `willLintPaths()`.
- `lintPath()`: Linters no longer need to implement this method.
XHPAST now has an explicit way to release shared futures.
These minor changes also happened:
- Formalized the "linter ID", which is a semi-durable identifier for the cache.
- Removed linter -> exception explicit mapping, which was unused. We now just collect exceptions.
- We do the `canRun()` checks first (and separately) now.
- Share more service call profiling code.
- Fix an issue where the test harness would use the path on disk, even if configuration set a different path.
Test Plan:
- Ran `arc lint --everything` in `arcanist/`.
- With no chunking, saw **unstable** memory usage with a peak at 941 MB.
- With chunk size 32, saw **stable** memory usage with a peak at 269 MB.
- With chunk size 8, saw **stable** memory usage with a peak at 180 MB.
- Ran with `--trace` and saw profiling information.
- Created this diff.
Reviewers: joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T5097
Differential Revision: https://secure.phabricator.com/D12501
Summary: Ref T6822.
Test Plan: Visual inspection. These methods are only called from within the `ArcanistExternalLinter` and `ArcanistLinter` subclasses.
Reviewers: chad, #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T6822
Differential Revision: https://secure.phabricator.com/D11237
Summary: I'm pretty sure that `@group` annotations are useless now... I believe that they were originally used by Diviner?
Test Plan: Eye-balled it.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin, aurelijus
Differential Revision: https://secure.phabricator.com/D9855
Summary: Various tidying up of linting code.
Test Plan: `arc lint` and `arc unit` still pass.
Reviewers: chad, epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D9625
Summary: Ran `arc lint --apply-patches --everything` over rARC, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.
Test Plan: Eyeballed //most// of the diff.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin, aurelijus
Differential Revision: https://secure.phabricator.com/D9269
Summary: This fixes an issue with the C# linter where a message could be returned from cslint that wasn't intended for use with parameters. This just ensures there's enough parameters so that it won't crash (and consequently ignore lint messages).
Test Plan: Ran the linter, it didn't crash.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D9116
Summary:
Ref T2039. I'll update the corresponding documentation.
It feels a little awkward that this is disconnected from `getLinterConfigurationOptions()`, but I dislike returning weird ad-hoc structures more than I dislike having two methods. Most linters don't implement either of these anyway.
Test Plan: Ran `arc linters` and `arc linters --verbose`.
Reviewers: btrahan, joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T2039
Differential Revision: https://secure.phabricator.com/D9062
Summary:
- Tidied up `ArcanistCSSLintLinter::getDefaultBinary`.
- Tidied up `CSSLintLinter::getDefaultFlags` function.
- Tidied up `ArcanistPhpcsLinter::getDefaultBinary` function.
- Tidied up `ArcanistPEP8Linter::getDefaultFlags` function
- Tidied up `ArcanistFlake8Linter::getDefaultFlags`.
- Tidied up `ArcanistCppcheckLinter::getLintOptions`.
- Tidied up `ArcanistCppcheckLinter::getLintPath`.
- Tidied up `ArcanistCpplintLinter::getLintOptions`.
- Tidied up `ArcanistCpplintLinter::getLintPath`.
- Removed child functions which are identical to the corresponding parent functions.
Test Plan: `arc lint` and `arc unit`.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: epriestley
CC: chad, Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D8385
Summary: This sets the limit for future execution in the C# linter to 8. See D7606 for more information.
Test Plan:
Ran
```
arc lint --everything --trace --never-apply-patches --output json
```
and saw it only run 8 commands at once.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran
Differential Revision: https://secure.phabricator.com/D7607
Summary:
This upgrades `cslint` to support linting multiple files at a time. As this required a backwards-incompatible to `cslint`, I've added a SUPPORTED_VERSION constant which can be used to detect these kinds of breaking changes in the future (and prompt users to upgrade the `cslint` they have in their repository).
The reason for this upgrade is mainly around running `arc lint --everything`, where there are significant performance benefits gained when bulk linting lots of files per command execution.
Test Plan: Upgraded `cslint` in the Tychaia repository and ran `arc lint --everything --trace`. Saw a substantially less number of executions happening for linting and all of the results came through as expected.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley
CC: Korvin, epriestley, aran, waynea
Differential Revision: https://secure.phabricator.com/D7599
Summary:
Completes T3859. This implements a C# linter for Arcanist, which in turn uses `cslint` from `cstools` to actually perform the linting. `cslint` internally uses StyleCop in addition to it's own lint rules.
Unlike other linters, C# is a compiled language, which means that the StyleCop integration must be aware of the full project. To this end, there is the `discovery` setting in `.arclint`. This allows users to define mappings between C# files and the projects they belong to. Here is an configuration for `.arclint` (and is the one we use):
```
{
"linters": {
"csharp": {
"type": "csharp",
"include": "(\\.cs$)",
"binary": "cstools/cslint/bin/Debug/cslint.exe",
"discovery": {
"([^/]+)/(.*?)\\.cs": [
"$1/$1.Linux.csproj"
],
"([^\\\\]+)\\\\(.*?)\\.cs": [
"$1\\$1.Windows.csproj"
]
}
}
}
}
```
Test Plan: Tested under both Linux and Windows. Changed some files, ran `arc lint` and it all worked correctly.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran, jamesr
Maniphest Tasks: T3859
Differential Revision: https://secure.phabricator.com/D7170
