Summary:
This implements Arcanist support for the xUnit testing framework. It also supports code coverage by way of `cstools` (which is something I've written).
The unit test support works under both Linux and Windows, while the code coverage support has only been tested under Linux (one would assume it would also work under Windows given that Windows has a super-set of functionality in the C# world).
The Arcanist support assumes that the directory layout will be something like:
* MyProject
* MyProject.Tests
When files are changed in either MyProject or MyProject.Tests, it causes MyProject.Tests to be built and the xUnit runner to be executed on the resulting binary.
Test Plan: I guess if really wanted to, you could create a C# project in MonoDevelop, set up `.arcconfig` to point to this unit test engine, and download and build xUnit and cstools. Run `arc unit --coverage` to see the results of your unit test coverage.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3859
Differential Revision: https://secure.phabricator.com/D7058
Summary:
Fixes T3856.
Detailed code coverage iterates over all of the files with changes in the current repository, however the code coverage tool might not generate a report for all changed files in the repository, and this would result in an undefined index error.
As an additional bonus, since changes to binary files won't be reported by code coverage tools, this also prevents binary data from being outputted to the console.
Test Plan:
Change a binary file in a repository and run a code coverage tool. The binary file should be reported as 0% code coverage, but the file contents should not be rendered to the console.
Change a code file that is not covered by a code coverage tool and run code coverage. The file should be reported as 0% code coverage, and the file contents should not be displayed.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3856
Differential Revision: https://secure.phabricator.com/D7051
Summary: PhpunitTestEngine incorrectly included some source files as tests when run on a case-insensive fiesystem. This fixes that behavior. It could introduce problems for case-sensive users, but it's extremely unlikely to be an issue in practice (who would put a file in both Tests/ and tests/ and expect different results for the two?)
Test Plan: arc unit before and after change on OS X. Stubs and autoloaders in .../tests/... directories are no longer picked up.
Reviewers: epriestley
Reviewed By: epriestley
CC: aurelijus, Korvin, aran
Differential Revision: https://secure.phabricator.com/D7048
Test Plan: Saw 'Do you want to add these files to the commit?'
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6991
Summary: Automatically correct `New` to `new` in lint.
Test Plan: Ran unit tests.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6948
Summary: See discussion in D6893. Different versions of `svn` do different stuff, just ignore any possible error here.
Test Plan: Ran unit tests.
Reviewers: andrewjcg, btrahan
Reviewed By: andrewjcg
CC: aran
Differential Revision: https://secure.phabricator.com/D6946
Summary:
git ls-tree gives type 'commit' for submodules.
This code-path is only used when a binary file is present in the diff.
Test Plan: arc diff with a binary file
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6943
Summary:
See <https://github.com/facebook/arcanist/issues/102>. PHPCS changed its output format sometime between 1.4.6 (stable) and 1.5.0RC3.
Add a "no errors" test and make the linter work on both versions.
Test Plan: Ran `arc unit` on PHPCS 1.5.0RC3.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6926
Summary:
I got bit by this one earlier (I swear I thought it was delimeter),
so I figured I would add it to the spelling data lint rule.
Test Plan:
changed an instance of delimiter to delimeter, ran `arc lint` fixed the
typo.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6936
Summary: See D6895. We currenty whitelist one of the DOM classes, but should whitelist them all. Classes not in this whitelist raise warnings when users who don't have the corresponding extensions installed lint code. We don't want to require you to install every extension just to send diffs, so we whitelist stuff that we know would exist if you did install everything.
Test Plan: bwahaha
Reviewers: btrahan, Firehed
Reviewed By: Firehed
CC: aran
Differential Revision: https://secure.phabricator.com/D6896
Summary:
See IRC. This fixes an issue when deleting an SVN file that ends with one of the extensions in the regexp, which may only affect newer versions of SVN.
Possibly we shouldn't have this heuristic, or should move it elsewhere or make it more explicit, but at least stop it from being broken for now.
Test Plan: Ran `arc diff --only` in a working copy with a deleted binary file ending in ".jpg".
Reviewers: btrahan, nh
Reviewed By: nh
CC: aran, mbishopim3
Differential Revision: https://secure.phabricator.com/D6893
Summary:
Make sure on failure (restoreBranch()) we call `git submodule update --init --recursive` to handle all those purdy submodules. For the pushing step, wrap the push commands in the try / catch block so everything gets cleaned up nice if there's failure. BONUS - add --recursive to arc patch workflow to so nested submodules work correctly. (Crazy git users)
Fixes T3407, T2945.
Test Plan: I wasn't sure how to simulate a good "push" failure but I think this should work.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T2945, T3407
Differential Revision: https://secure.phabricator.com/D6885
Summary:
When running arc diff in a repository that supports commit ranges, it is
possible that the setting for the default relative commit hasn't been set.
If this is the case, the user will be prompted. This change makes sure that
the prompt happens (and thus the setting is set) before we run the
background lint and unit runs.
Test Plan:
```
rm .git/arc/default-relative-commit
arc diff
```
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T2351
Differential Revision: https://secure.phabricator.com/D6854
Summary:
We have some linters that trigger based on the path name
in the tree (some rules apply in some dirs and not others).
The changes in D6798 caused all the paths to appear to be outside
the tree, so allow for passing a fake through from those test cases
that are sensitive to this.
We also have a test for the copyright linter, and that needs to read
settings from the .arcconfig file. The change to faking a working
copy meant that this config option was effectively unset, so add a way
to pass the entire arcconfig through from the tests that need it.
Lastly, the logic to skip deleted files needs to be special cased
when we're faking paths like this: if we've added data for a file
in the testable engine, we should also consider that file as existing.
Test Plan:
`arc unit --everything` here, and passing our tests in
our repo over there.
Reviewers: epriestley, mareksapota
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6841
Summary:
Ref T3186. Ref T2039. Ref T3771. A few effects here:
# Expose PHPCS as a `.arclint` linter.
# Turn PHPCS into an ArcanistExternalLinter linter.
# Add test coverage for PHPCS.
# Add a `severity.rules` option to `.arclint`. Some linters have very explicit builtin severities ("error", "warning") but their meanings are different from how arc interprets these terms. For example, PHPCS raises "wrong indentation level" as an "error". You can already use the "severity" map to adjust individual rules, but if you want to adjust an entire linter it's currently difficult. This rule map makes it easy. There's substantial precedent for this in other linters, notably all the Python linters.
For `severity.rules`, for example, this will turn all PHPCS "errors" into warnings, and all of its warnings into advice:
"severity.rules" : {
"(^PHPCS\\.E\\.)" : "warning",
"(^PHPCS\\.W\\.)" : "advice"
}
The user can use `severity` (or more rules) to get additional granularity adjustments if they desire.
Test Plan: 5bb919bc3a
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, ajtrichards
Maniphest Tasks: T2039, T3186, T3771
Differential Revision: https://secure.phabricator.com/D6830
Summary:
Ref T3776, Ref T479. Say you have some DN, with a submodule X@Y. Later, X@Z in your working copy / repo. If you run arc patch DN, you'd end up with a dirty working copy claiming that X@Z was wrong and it should be X@Y.
To fix, basically run 'submodule init' and 'submodule update'. This makes it so after "arc patch" if you run "git status" it looks clean.
Gross part though now is if you then "git checkout master" you'll have a dirty checkout the other way. I think this is better though.
Test Plan: made a new repository where I added libphutil @ X, did some work (DX), then made libphutil @ y. When I arc patch'd DX, things looked good!
Reviewers: epriestley
Reviewed By: epriestley
CC: csilvers, Korvin, aran
Maniphest Tasks: T479, T3776
Differential Revision: https://secure.phabricator.com/D6837
Summary:
Currently, we prompt the user to mark non-UTF8 files as binary, but don't actually attach the data to the change when they do. This means we don't upload the data, and can't patch it later.
A simple reproduction case is to build a test file (I used one with bytes from 1..255):
$ # Don't include \0, since Git treats that specially.
$ ./echo_every_byte_from_1_to_255_inclusive.erl > example.txt
Then add it:
$ git add example.txt
$ git commit -a -m derp
$ arc diff --only HEAD^
You'll be prompted to convert the file to binary:
Do you want to mark this file as binary and continue? [Y/n] y
Before this patch, that would be followed by:
Uploading 0 files...
...which is incorrect; we need to upload the new data. After this patch, this shows:
Uploading 1 files...
...which is also incorrect, but only grammatically. Diffs created after this patch apply back cleanly with `arc patch` and restore the file properly.
Test Plan: Followed instructions above, restoring a textual binary conversion by using `arc patch`.
Reviewers: zeeg, btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6815
Summary:
If present, this will override the default phpunit path. Allows easy
integration of Composer-provided installs of phpunit (ex. set phpunit_binary to
vendor/bin/phpunit). If the path provided doesn't resolve to an executable it
will assume the path is relative to the project root.
fix line length issue
Test Plan:
Added phpunit_binary to .arcconfig in a simple project using Composer with
phpunit/phpunit package as part of require-dev (installed to
$ROOT/vendor/bin/phpunit). Phpunit not otherwise installed on the system. Set
unit.engine to PhpunitTestEngine. Confirmed that 'arc unit' used the specified
binary, both at project root and from subdirectories.
Reviewers: epriestley
CC: aurelijus, Korvin, aran
Differential Revision: https://secure.phabricator.com/D6791
Summary: Fixes T3696. Currently, we abort. If stdin is not a TTY, we should just continue. A script which cares could conceivably run `arc lint` and `arc unit` separately, but it seems unlikely that any script would ever want to fail here.
Test Plan: Ran `echo -n '' | arc diff --create --verbatim` with a lint error and got a revision (D6720).
Reviewers: Firehed, btrahan, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3696
Differential Revision: https://secure.phabricator.com/D6721
Summary:
Ref T3186. Ref T2039. Continues work on readying linters for `.arclint`.
- **Ruby**: Make this an ExternalLinter.
- **Priority**: Currently, linters have an implicit "correct" order (notably, the "NoLint" linter needs to run before other linters). Make this explicit by introducing `getLinterPriority()`.
- **Binaries**: Currently, linters manually reject binary files. Instead, reject binary files by default (linters can override this if they do want to lint binary files).
- **Deleted Files**: Currently, linters manually reject deleted files (usually in engines). Instead, reject deleted files by default (linters can override this).
- **Severity**: Move this `.arclint` config option up to top level.
- **willLintPaths()**: This method is abstract, but almost all linters provide a trivial implementation. Provide a trivial implementation in the base class.
- **getLintSeverityMap()/getLintNameMap()**: A bunch of linters have empty implementations; these are redundant. Remove them.
- **Spelling**: clean up some dead / test-only / unconventional code.
- **`.arclint`**: Allow the filename, generated, nolint, text, spelling and ruby linters to be configured via `.arclint`.
Test Plan:
458beca3d6
Ran unit tests.
Reviewers: btrahan
Reviewed By: btrahan
CC: Firehed, aran
Maniphest Tasks: T2039, T3186
Differential Revision: https://secure.phabricator.com/D6805
Summary:
The existing heuristic checks for the existence of .hg/svn, but it
turns out that this directory can exist in a non-svn hg repo if the user or a
script ever runs a 'hg svn' command.
Now we check for a file inside .hg/svn. The hgsubversion maintainer said this
particular file will always be present in hgsubversion repos.
Test Plan:
arc land --trace
Verified it used 'hg push' and not 'hg push -r ...'. This indicates it
considered the repo to be an hgsubversion repo.
Reviewers: epriestley
Reviewed By: epriestley
CC: dschleimer, sid0, Korvin, aran
Differential Revision: https://secure.phabricator.com/D6807
Summary:
Ref T3186. Brings another linter onboard. This one uses the stdin stuff.
The unit test was ostensibly broken so I fixed it, but that might just be some kind of version issue.
Test Plan: Unit tests.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3186
Differential Revision: https://secure.phabricator.com/D6802
Summary:
Ref T3186. We have about 50 linters which run programs and read the results, all of which have ad-hoc one-off custom config that isn't formalized anywhere.
Consolidate all this stuff into `ArcanistExternalLinter`, which is configurable through `.arclint` (although nothing supports this quite yet).
Extend CSSLint and Pep8Lint from `ArcanistExternalLinter`.
Add unit tests for both.
There are still some rough edges here, but it mostly seems to work pretty well.
Test Plan: Ran unit tests, hit some (most?) of the error cases.
Reviewers: btrahan
Reviewed By: btrahan
CC: chad, aran, Firehed
Maniphest Tasks: T3186
Differential Revision: https://secure.phabricator.com/D6800
Summary:
Ref T3186.
- Every linter builds a WorkingCopyIdentity in the same way, with no specialized data. Don't do that.
- Linters get passed a goofy hardcoded ".php" path. Don't do that.
- Linters generally run on an imaginary path, which might not work. Just give them a real path by building a tiny working copy in `/tmp`.
- Fix a TODO now that we have better typechecking.
Test Plan: `arc unit --everything`, intentionally broke a test to make sure that still works.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3186
Differential Revision: https://secure.phabricator.com/D6798
Summary:
Ref T2039. That task has a bunch of discussion, but basically we do a poor job of serving the midrange of lint configuration right now.
If you have something simple, the default linters work.
If you have something complex, building your own engine lets you do whatever you want.
But many users want something in between, which isn't really well accommodated. The idea is to let you write a `.arclint` file, which looks something like this:
{
"linters" : {
"css" : {
"type" : "csslint",
"include" : "(\.css$)",
"exclude" : "(^externals/)",
"bin" : "/usr/local/bin/csslint"
},
"js" : {
"type" : "jshint",
"include" : "(\.js$)",
"exclude" : "(^externals/)",
"bin" : "support/bin/jshint",
"interpreter" : "/usr/local/bin/node"
}
}
}
...which will provide a bunch of common options around lint severity, interpreter and binary locaitons, included and excluded files, etc.
This implements some basics, and very rough support in the Filename linter.
Test Plan:
Generated a `.arclint` file and saw it apply filename lint correctly. Used `debug` mode and tried invalid regexps.
{
"debug" : true,
"linters" : {
"filename" : {
"type" : "filename",
"exclude" : ["@^externals/@"]
}
}
}
Next steps include:
- Provide an external linter archetype (T3186) and expose a common set of configuration here ("bin", "interpreter", "flags", "severity").
- Provide a `.arcunit` file which works similarly (it can probably be simpler).
Reviewers: btrahan, Firehed
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2039
Differential Revision: https://secure.phabricator.com/D6797
Summary: Found an issue where if arc unit runs phpunit and the datasets included a dataprovider which spanned multiple lines, it wasn't filtered out correctly in the result parser, this fixes it
Test Plan: accidentally ran tests against phpunit itself which exhibits this problem. no longer a problem after this fix, nothing else breaks.
Reviewers: epriestley
Reviewed By: epriestley
CC: aurelijus, epriestley, aran
Differential Revision: https://secure.phabricator.com/D6773
Summary:
arc for mercurial on windows was broken in several way.
Executing a command via passthru failed because passthru on windows
skips the shell so 'set HGPLAIN=1 & ...' was an invalid command. The
fix was to just not set HGPLAIN for passthru commands on windows.
Also removed hardcoded '' quotes in mercurial commands since windows
doesn't support single quots.
Test Plan: arc land --hold on a windows machine
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6763
Summary: Corrects relative vs absolute branch name when using 'arc commit'
Test Plan: 'arc commit' on a release branch gave an error before making the change (change was generated from 'branches/yyy' but working copy root is 'https://xxx/branches/yyy', now it does not.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6743
Summary: My recent change adding --everything to arc lint could sometimes cause a "diff is empty" error, this patch fixes it.
Test Plan: Ran "arc lint --everything" before and after patch. No longer errors out. Only appeared to originally happen when there were uncommited changes in an svn repo.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6732
Summary:
When trying to find an unused bookmark we append hyphened suffixes to the end of the bookmark name. In Mercurial, these are treated the same as the bookmark name without the suffix, here's the output from hg log -r:
```
[mehdi] HGPLAIN=1 hg log -r "arcpatch-1"
abort: unknown revision 'arcpatch'!
[mehdi] HGPLAIN=1 hg log -r "arcpatch_1"
abort: unknown revision 'arcpatch_1'!
```
Test Plan: None.
Reviewers: epriestley, dschleimer
Reviewed By: dschleimer
CC: aran, Korvin, DurhamGoode, dschleimer
Differential Revision: https://secure.phabricator.com/D6698
Summary:
If you are trying to commit someone else's diff, arc commit gives warnings about path mismatch. This changes the path comparison to be based on the repo url rather than the local working directory. E.g. if both the author and committer are working in branches/release/2013_08_07 despite being checked out in ~/dev/2013_08_07 (system user being different, of course) it no longer warns that the WC path is different
Original behavior:
eric@Eric-MBP ~/dev/2013_07_31: arc commit --revision 21
You are not the author of 'D21: WeMerge Automatic Request'. Commit this
revision anyway? [y/N] y
Revision 'D21: WeMerge Automatic Request' was generated from '', but
current working copy root is '/Users/eric/dev/2013_07_31/'. Commit this
revision anyway? [y/N] y
Committing 'D21: WeMerge Automatic Request'...
Adding test
Transmitting file data .
Committed revision 52676.
Closing revision D21 'WeMerge Automatic Request'...
Exception
ERR-CONDUIT-CORE: You can not mark a revision you don't own as closed.
(Run with --trace for a full exception trace.)
New behavior:
eric@Eric-MBP ~/dev/2013_07_31: arc commit --revision 24
You are not the author of 'D24: WeMerge Automatic Request'. Commit this
revision anyway? [y/N] y
Committing 'D24: WeMerge Automatic Request'...
Adding test
Transmitting file data .
Committed revision 52679.
Closing revision D24 'WeMerge Automatic Request'...
Exception
ERR-CONDUIT-CORE: You can not mark a revision you don't own as closed.
(Run with --trace for a full exception trace.)
Test Plan: 'arc diff' changes with one user. 'arc patch Dxx' on a different working copy by a different user to review and test changes. accept review. 'arc commit --revision xx' as reviewer to land the patch. complaint goes away.
Reviewers: epriestley, ghostwriter78
Reviewed By: epriestley
CC: aran, epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D6665
Summary: Update and clarify precedence of CA bundles
Test Plan:
tested with "arc call-conduit user.whoami" from project root
Same from other subdirectory in project
Repeated tests with both https.cabundle and https.cacert settings, both in
.arcconfig and ~/.arcrc
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T3668
Differential Revision: https://secure.phabricator.com/D6647
Summary:
Support for no output from arc unit (For scripts, etc).
Also include --output param, analogous to arc lint --output.
Test Plan: run all 6 variants + `--output bad-value`
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6642
Summary:
Single quotes aren't valid in the windows cmd prompt, so arc feature
didn't work in mercurial when it got to this line.
I have no idea why %C was used before. Nothing in that string should be
broken by the escaping.
Test Plan:
Ran arc feature --trace on my mac. Verified the command was escaped correctly
and the correct feature results were printed.
I don't have a windows machine to try it on, but the builtin escaping should
now account for windows machines.
Reviewers: epriestley
Reviewed By: epriestley
CC: sid0, aran, Korvin
Differential Revision: https://secure.phabricator.com/D6637
Summary:
When adding arcanist support to a new project or adding a new linter,
it's helpful to be able to run new linters against the entire codebase. This
patch adds support for this with an '--everything' option, similar to 'arc unit
--everything'
Test Plan:
Run 'arc lint --everything' and check out the code. Optionally dump
the paths to test in the current lint engine's buildLinters() function to
demonstrate that it's receiving all files in the project rather than just the
changed and/or specified ones
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D6592
Summary: The test result parser in PhpunitTestEngine was receiving $test_path from the previous loop instead of $path from the current one. The variable isn't actually used in the PhpunitResultParser object (it exists for strict compatibility with the parent class) so it didn't cause any problems, but who knows if that could change in the future
Test Plan: Review diff. No changes to the output of running 'arc unit' when using the Phpunit engine, as expected
Reviewers: epriestley
CC: aran, epriestley, aurelijus, chad
Differential Revision: https://secure.phabricator.com/D6587
Summary: We use custom `assert*` functions here and there. Remove them from backtrace.
Test Plan: Ran `XHPASTTreeTestCase`.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6411
Summary:
I suspect that when people use 'algorithmical', they mean it as an adjective and not an adverb.
'algorithmic' = adjective
'algorithmically' = adverb
Test Plan: Add the word 'algorithmical' to a file. Run `arc lint` on the file. See suggestion to correct it to 'algorithmic'.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D6373
