Summary:
Ref T3186. We have about 50 linters which run programs and read the results, all of which have ad-hoc one-off custom config that isn't formalized anywhere.
Consolidate all this stuff into `ArcanistExternalLinter`, which is configurable through `.arclint` (although nothing supports this quite yet).
Extend CSSLint and Pep8Lint from `ArcanistExternalLinter`.
Add unit tests for both.
There are still some rough edges here, but it mostly seems to work pretty well.
Test Plan: Ran unit tests, hit some (most?) of the error cases.
Reviewers: btrahan
Reviewed By: btrahan
CC: chad, aran, Firehed
Maniphest Tasks: T3186
Differential Revision: https://secure.phabricator.com/D6800
Summary:
Ref T3186.
- Every linter builds a WorkingCopyIdentity in the same way, with no specialized data. Don't do that.
- Linters get passed a goofy hardcoded ".php" path. Don't do that.
- Linters generally run on an imaginary path, which might not work. Just give them a real path by building a tiny working copy in `/tmp`.
- Fix a TODO now that we have better typechecking.
Test Plan: `arc unit --everything`, intentionally broke a test to make sure that still works.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3186
Differential Revision: https://secure.phabricator.com/D6798
Summary:
Ref T2039. That task has a bunch of discussion, but basically we do a poor job of serving the midrange of lint configuration right now.
If you have something simple, the default linters work.
If you have something complex, building your own engine lets you do whatever you want.
But many users want something in between, which isn't really well accommodated. The idea is to let you write a `.arclint` file, which looks something like this:
{
"linters" : {
"css" : {
"type" : "csslint",
"include" : "(\.css$)",
"exclude" : "(^externals/)",
"bin" : "/usr/local/bin/csslint"
},
"js" : {
"type" : "jshint",
"include" : "(\.js$)",
"exclude" : "(^externals/)",
"bin" : "support/bin/jshint",
"interpreter" : "/usr/local/bin/node"
}
}
}
...which will provide a bunch of common options around lint severity, interpreter and binary locaitons, included and excluded files, etc.
This implements some basics, and very rough support in the Filename linter.
Test Plan:
Generated a `.arclint` file and saw it apply filename lint correctly. Used `debug` mode and tried invalid regexps.
{
"debug" : true,
"linters" : {
"filename" : {
"type" : "filename",
"exclude" : ["@^externals/@"]
}
}
}
Next steps include:
- Provide an external linter archetype (T3186) and expose a common set of configuration here ("bin", "interpreter", "flags", "severity").
- Provide a `.arcunit` file which works similarly (it can probably be simpler).
Reviewers: btrahan, Firehed
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2039
Differential Revision: https://secure.phabricator.com/D6797
Summary: Found an issue where if arc unit runs phpunit and the datasets included a dataprovider which spanned multiple lines, it wasn't filtered out correctly in the result parser, this fixes it
Test Plan: accidentally ran tests against phpunit itself which exhibits this problem. no longer a problem after this fix, nothing else breaks.
Reviewers: epriestley
Reviewed By: epriestley
CC: aurelijus, epriestley, aran
Differential Revision: https://secure.phabricator.com/D6773
Summary:
arc for mercurial on windows was broken in several way.
Executing a command via passthru failed because passthru on windows
skips the shell so 'set HGPLAIN=1 & ...' was an invalid command. The
fix was to just not set HGPLAIN for passthru commands on windows.
Also removed hardcoded '' quotes in mercurial commands since windows
doesn't support single quots.
Test Plan: arc land --hold on a windows machine
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6763
Summary: Corrects relative vs absolute branch name when using 'arc commit'
Test Plan: 'arc commit' on a release branch gave an error before making the change (change was generated from 'branches/yyy' but working copy root is 'https://xxx/branches/yyy', now it does not.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6743
Summary: My recent change adding --everything to arc lint could sometimes cause a "diff is empty" error, this patch fixes it.
Test Plan: Ran "arc lint --everything" before and after patch. No longer errors out. Only appeared to originally happen when there were uncommited changes in an svn repo.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6732
Summary:
When trying to find an unused bookmark we append hyphened suffixes to the end of the bookmark name. In Mercurial, these are treated the same as the bookmark name without the suffix, here's the output from hg log -r:
```
[mehdi] HGPLAIN=1 hg log -r "arcpatch-1"
abort: unknown revision 'arcpatch'!
[mehdi] HGPLAIN=1 hg log -r "arcpatch_1"
abort: unknown revision 'arcpatch_1'!
```
Test Plan: None.
Reviewers: epriestley, dschleimer
Reviewed By: dschleimer
CC: aran, Korvin, DurhamGoode, dschleimer
Differential Revision: https://secure.phabricator.com/D6698
Summary:
If you are trying to commit someone else's diff, arc commit gives warnings about path mismatch. This changes the path comparison to be based on the repo url rather than the local working directory. E.g. if both the author and committer are working in branches/release/2013_08_07 despite being checked out in ~/dev/2013_08_07 (system user being different, of course) it no longer warns that the WC path is different
Original behavior:
eric@Eric-MBP ~/dev/2013_07_31: arc commit --revision 21
You are not the author of 'D21: WeMerge Automatic Request'. Commit this
revision anyway? [y/N] y
Revision 'D21: WeMerge Automatic Request' was generated from '', but
current working copy root is '/Users/eric/dev/2013_07_31/'. Commit this
revision anyway? [y/N] y
Committing 'D21: WeMerge Automatic Request'...
Adding test
Transmitting file data .
Committed revision 52676.
Closing revision D21 'WeMerge Automatic Request'...
Exception
ERR-CONDUIT-CORE: You can not mark a revision you don't own as closed.
(Run with --trace for a full exception trace.)
New behavior:
eric@Eric-MBP ~/dev/2013_07_31: arc commit --revision 24
You are not the author of 'D24: WeMerge Automatic Request'. Commit this
revision anyway? [y/N] y
Committing 'D24: WeMerge Automatic Request'...
Adding test
Transmitting file data .
Committed revision 52679.
Closing revision D24 'WeMerge Automatic Request'...
Exception
ERR-CONDUIT-CORE: You can not mark a revision you don't own as closed.
(Run with --trace for a full exception trace.)
Test Plan: 'arc diff' changes with one user. 'arc patch Dxx' on a different working copy by a different user to review and test changes. accept review. 'arc commit --revision xx' as reviewer to land the patch. complaint goes away.
Reviewers: epriestley, ghostwriter78
Reviewed By: epriestley
CC: aran, epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D6665
Summary: Update and clarify precedence of CA bundles
Test Plan:
tested with "arc call-conduit user.whoami" from project root
Same from other subdirectory in project
Repeated tests with both https.cabundle and https.cacert settings, both in
.arcconfig and ~/.arcrc
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T3668
Differential Revision: https://secure.phabricator.com/D6647
Summary:
Support for no output from arc unit (For scripts, etc).
Also include --output param, analogous to arc lint --output.
Test Plan: run all 6 variants + `--output bad-value`
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6642
Summary:
Single quotes aren't valid in the windows cmd prompt, so arc feature
didn't work in mercurial when it got to this line.
I have no idea why %C was used before. Nothing in that string should be
broken by the escaping.
Test Plan:
Ran arc feature --trace on my mac. Verified the command was escaped correctly
and the correct feature results were printed.
I don't have a windows machine to try it on, but the builtin escaping should
now account for windows machines.
Reviewers: epriestley
Reviewed By: epriestley
CC: sid0, aran, Korvin
Differential Revision: https://secure.phabricator.com/D6637
Summary:
When adding arcanist support to a new project or adding a new linter,
it's helpful to be able to run new linters against the entire codebase. This
patch adds support for this with an '--everything' option, similar to 'arc unit
--everything'
Test Plan:
Run 'arc lint --everything' and check out the code. Optionally dump
the paths to test in the current lint engine's buildLinters() function to
demonstrate that it's receiving all files in the project rather than just the
changed and/or specified ones
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D6592
Summary: The test result parser in PhpunitTestEngine was receiving $test_path from the previous loop instead of $path from the current one. The variable isn't actually used in the PhpunitResultParser object (it exists for strict compatibility with the parent class) so it didn't cause any problems, but who knows if that could change in the future
Test Plan: Review diff. No changes to the output of running 'arc unit' when using the Phpunit engine, as expected
Reviewers: epriestley
CC: aran, epriestley, aurelijus, chad
Differential Revision: https://secure.phabricator.com/D6587
Summary: We use custom `assert*` functions here and there. Remove them from backtrace.
Test Plan: Ran `XHPASTTreeTestCase`.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6411
Summary:
I suspect that when people use 'algorithmical', they mean it as an adjective and not an adverb.
'algorithmic' = adjective
'algorithmically' = adverb
Test Plan: Add the word 'algorithmical' to a file. Run `arc lint` on the file. See suggestion to correct it to 'algorithmic'.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6373
Summary:
- Replace `maniphest.find` with `maniphest.query`. These calls are nearly identical, it was just a rename for consistency.
- Replace `differential.find` with `differential.query`.
Test Plan:
- Ran `arc tasks`.
- Ran `arc close-revision` on valid, nonexistent, and existent-but-invalid revisions.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6336
Summary:
I wanted to write a couple of workflows that shared some code in an abstract superclass, but ##arc## would fail trying to instantiate the abstract class.
As it turns out, we can modernize the ##buildAllWorkflows## function a bit, and it will only load concrete objects.
Test Plan:
1. Define an abstract subclass of ##ArcanistBaseWorkflow##.
2. ##arc liberate## the source directory that contains it.
3. Try any ##arc## operation without this diff, and see it fail.
4. Patch this diff and see that ##arc## operations work now.
Reviewers: epriestley, vrana
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6324
Summary: Makes sense now
Test Plan: Get someone else to test it
Reviewers: epriestley
CC: aran, Korvin
Maniphest Tasks: T3342
Differential Revision: https://secure.phabricator.com/D6307
Summary: Enables the process to pass the user input for library name. Fixes T3342
Test Plan: Little help... `arc liberate` doesn't run on windows. Though I saw the flag :)
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T3342
Differential Revision: https://secure.phabricator.com/D6306
Summary:
When `arc diff` runs unit tests it uses all of the affected files as the base array of paths. These files may have been deleted. If the deleted file fits the test criteria `unit` will try and run the test and in some cases fail.
An example of this fataling is with PHPUnit;
> Fatal error: Uncaught exception 'Exception' with message 'JSON report file is em
pty, it probably means that phpunit failed to run tests. Try running arc unit wi
th --trace option and then run generated phpunit command yourself, you might get
the answer.' in C:\Websites\facebook\arcanist\src\unit\engine\PhpunitResultPars
er.php on line 156
Test Plan: Re-ran the tests that were causing issues with `arc unit --rev HEAD^ --trace`
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, aurelijus
Differential Revision: https://secure.phabricator.com/D6246
Summary:
Previously, updating a commit via arc diff in mercurial would
prepopulate the update message with part of the commit message. In an
amend workflow this doesn't make sense, so I disabled it. Git already
does this, so now mercurial matches git in this scenario.
We had users complain that new users would often submit diffs with the
default update message, and it wasn't useful since they were using a
amend flow.
Test Plan:
arc diff on a commit that already had a diff
Verified the editor did not have a update message
arc diff on a stack of commits where the bottom one had a diff
Verified the editor provided a default update message
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6215
Summary: We currently swallow the exception message, but this isn't useful. Fixes T3354.
Test Plan:
$ arc diff HEAD^ --conduit-uri=http://local.aphront.com:8080/ --only
Uploading 1 files...
Failed to upload new binary 'large.png'.
[HTTP/500] Internal Server Error
As received by the server, this request had a nonzero content length but no POST data.
Normally, this indicates that it exceeds the 'post_max_size' setting in the PHP configuration on the server. Increase the 'post_max_size' setting or reduce the size of the request.
Request size according to 'Content-Length' was '2093052', 'post_max_size' is set to '100K'.
Continue? [Y/n]
Reviewers: jamesr, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T3354
Differential Revision: https://secure.phabricator.com/D6186
Summary: Now that we examine these in lint, this came up. There is no `ConduitException`; the class is `ConduitClientException`.
Test Plan: Lint.
Reviewers: vrana, chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D6158
Summary: If one wishes to implement a linter which finds unused resources or variables the current scheme does not work as the lint engine filters all changes from lines which were not introduced in a diff. To solve that, I've added an "always show" configuration for a lint message allowing creation of such linters.
Test Plan:
1. Created a custom linter for finding unused Android resources in a project
2. Ran arc lint with linter added and received warnings as expected
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6119
Summary:
Piping data around on Windows doesn't work well when it contains zany characters like "null" and "newline". Fixes T3266.
Instead of piping data into `git apply`, write to a temporary file.
Test Plan:
Ran `arc patch`, got good results.
>>> [17] <exec> $ git apply --index --reject -- '/private/var/folders/8k/c3vkmjy5335gcxdzxkhwq82w0000gn/T/7z9iea6srikoo0sc/4266-ZEyvz9'
Reviewers: btrahan, hach-que
Reviewed By: hach-que
CC: aran
Maniphest Tasks: T3266
Differential Revision: https://secure.phabricator.com/D6070
Summary: On Windows consoles, the star on the unit tests (I'm assuming it's a Unicode character of some kind), doesn't render correctly and you just get garbage. This fixes it so that on Windows, it falls back to just using an ASCII asterisk.
Test Plan: On Windows run some unit tests, the star should now be a plain asterisk.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6087
Summary:
@Afaque_Hussain has done a bunch of utf8 work here; combined with PhutilEditDistanceMatrix we can now do utf8 diffs correctly, in a general way, without a significant performance impact.
Use PhutilEditDistanceMatrix and `phutil_utf8v_combined()` to compute accurate diffs for all (or, at least, most) UTF8 text.
The only thing this doesn't handle completely correctly is lines beginning with combining characters. This is messy/expensive to handle and will probably never actually happen, so I'm punting for now. Nothing should actually break.
The utf8 stuff will be slow, but we only pay for it when we need it.
Test Plan:
Ran unit tests. I changed a few unit tests to use a non-combining character (snowman) for clarity, and some results are different now (since we get combining characters right).
{F44064}
Reviewers: btrahan, Afaque_Hussain
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2379
Differential Revision: https://secure.phabricator.com/D6019
Summary: Replace this old hard-coded implementation with the new vector-based, unicode-capable one.
Test Plan: Ran unit tests. Looked at revisions in Differential, using whitespace modes to bypass cache.
Reviewers: btrahan, Afaque_Hussain
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2379
Differential Revision: https://secure.phabricator.com/D6016
Summary: Also warn against functions not available on Windows at all.
Test Plan: Compared old and new file.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D5975
Summary: These were in my sandbox, but I forgot about them. Without this things break post D5896. Ref T2784
Test Plan: my sandbox works and soon so shall others
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2784
Differential Revision: https://secure.phabricator.com/D5929
