Summary:
Make all the broad-spectrum text linters use the new binary check.
I didn't touch `ComprehensiveLintEngine` because it's a nest of vipers and no one has complained; see T2039.
Test Plan: Ran `arc lint` on text files and binaries in a libphutil project (arcanist).
Reviewers: lisp
Reviewed By: lisp
CC: aran
Differential Revision: https://secure.phabricator.com/D5040
Summary:
Add linting capability for detecting files which contain
syntax introduced by unresolved merge conflicts. The detection is
file-type-agnostic (the only requirement is that the file is a text
file).
Test Plan:
Tested in three ways.
The first way is to add all three forms of syntax to a file to
indicate a merge conflict. HPHP will pick this up as a syntax error
before this linter reaches it.
The second way is to add the syntax in a comment. In that case, this
linter will show three warnings. For example:
$ arc lint ArcanistMergeConflictLinter.php
>>> Lint for arcanist/src/lint/linter/ArcanistMergeConflictLinter.php:
Warning (MERGECONFLICT1) Unresolved merge conflict
This syntax indicates there is still an unresolved merge conflict.
20
21 foreach ($lines as $lineno => $line) {
22 /*
>>> 23 >>>>>>>
24
25 =======
Warning (MERGECONFLICT1) Unresolved merge conflict
This syntax indicates there is still an unresolved merge conflict.
22 /*
23 >>>>>>>
24
>>> 25 =======
26
27 <<<<<<<
Warning (MERGECONFLICT1) Unresolved merge conflict
This syntax indicates there is still an unresolved merge conflict.
24
25 =======
26
>>> 27 <<<<<<<
28
29 */
The last test was to test on various different file types, including
JavaScript, PHP, an animated GIF, a PNG, and a Bash file to make sure
the file type detection worked. Each of the aforementioned tests
passed.
Reviewers: vrana, epriestley
Reviewed By: epriestley
CC: aran, epriestley, Korvin
Maniphest Tasks: T2547
Differential Revision: https://secure.phabricator.com/D4966
Summary: We were incorrectly matching `$` in the regexp against a possible `\r\n`. I missed this earlier when trying to catch all of these.
Test Plan:
- Added unit test and made it pass.
- Did another search for `getLine()` to see if I could spot any more of these, but failed to identify any via inspection.
Reviewers: vrana, mbishopim3
Reviewed By: mbishopim3
CC: aran
Differential Revision: https://secure.phabricator.com/D5038
Summary:
Following @epriestley suggestion to use PhutilServiceProfiler to log lint as a service call
Test Plan: arc lint
Reviewers: vrana
CC: phunt, aran, epriestley
Differential Revision: https://secure.phabricator.com/D4820
Summary: This is technically documented, but not currently enforced and we aren't consistent about it in the codebase.
Test Plan: See D5002.
Reviewers: chad, vrana
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D5003
Summary:
Mercurial 'arc land --hold' was taking 90+ seconds on our large
repository. Since most of arc land doesn't require any particular working
directory, I've changed the mercurial logic to avoid all updates except for
two: the one prior to finding the revision (only applies if the user specified
--branch), and the one at the end to leave the user in a good state.
Also got rid of a 'hg outgoing' call when phases are supported. Also changed
the hg-subversion detection to just look for .hg/svn instead of running 'hg
svn info', which was taking 4 seconds.
Now arc land takes about 50 seconds. Still much worse than git's 25 seconds.
One big hot spot is in the two 'hg rebase' calls, which account for 25 seconds
(versus 11 seconds of git).
Test Plan:
Tested arc land with mercurial and git. Tested with and without the --branch
options.
Reviewers: epriestley, bos, sid0, dschleimer
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D5014
Summary: This chunk of code is kind of iffy and not really correct, but make it not fail, at least.
Test Plan:
- Added a file named `swamp@2x.jpg` to a working copy.
- Used `svn propedit svn:mime-type swamp@2x.jpg@` to incorrectly set its mime type to `text/plain`.
- Ran `arc diff`.
- Saw `arc diff` "correctly" change its mime-type to a binary mime type. This isn't really correct, but doing it successfully is better than throwing an exception.
Reviewers: mbishopim3, chad
Reviewed By: mbishopim3
CC: aran
Differential Revision: https://secure.phabricator.com/D4998
Summary:
Arc diff will hash file data and try to upload the file using upload by hash rather than transferring data. If it is unable, it defaults to its normal behavior
Attempts to upload file by hash, use regular upload method otherwise
Test Plan: Figure out how to arc diff to my local install and look at the behavior
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin, mehrapulkit
Differential Revision: https://secure.phabricator.com/D4968
Summary: The `svn diff --xml --summarize` command reports a bunch of item statuses, which may include "none" if you make property changes to a directory (this is fairly rare).
Test Plan: Created property changes, saw "none" status.
Reviewers: chad, mbishopim3
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D4978
Summary: Followup to D4703. When we give paths to `svn`, we need to escape them if they contain an `@`.
Test Plan:
Created a patch full of modifications to files with `@` in their names, and applied it:
$ arc patch --diff 192
A A@2xcopy2
A A@2xcopy
D A@2x
OKAY Successfully applied patch to the working copy.
Reviewers: chad, mbishopim3
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D4977
Summary: D4963 for other linters.
Test Plan: Saw time 0.001 instead of 0.113 in spellcheck linter.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4965
Summary:
We always generate all messages and then filter them out based on minimum severity.
It's lots of useless work, especially in commit hook mode where we are interested only in errors.
Test Plan:
$ arc lint --cache 0 --severity error ArcanistXHPASTLinter.php
0.406 s before, 0.074 after
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4963
Summary:
After D4383, we escape the base commit when constructing a command like this:
hg log --rev (base::. - base)
However, if the base commit is a revset like ".^", we now escape it and Mercurial looks for a commit named ".^" (a valid mercurial branch name) instead.
Fix this by returning nodes for these rules instead of revsets. The "arc:this" rule is automatically used in some operations, like "arc amend", so users can hit this during normal workflows, not just with weird `--base` rules.
Test Plan: Ran "arc amend" in a Mercurial repository, didn't fatal out.
Reviewers: DurhamGoode, sid0
Reviewed By: DurhamGoode
CC: tido, aran
Differential Revision: https://secure.phabricator.com/D4949
Test Plan:
$ arc diff -a
$ arc diff -a # saw amend instead of a new commit
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4947
Summary:
We save repository version to lint cache but ignore it when reading the cache.
Fix it.
Test Plan: Made an error for linter with repo granularity, deleted the error from the cache. Relinted, didn't see the error. Changed another file and relinted, saw the error.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4841
Summary:
Mercurial 'arc land' uses the 'hg strip' command to clean up after
itself, but this command isn't available unless the mq extension is enabled.
The fix is to enable it for that particular command only.
Test Plan: Ran 'arc land' with the mq extension disabled. It worked.
Reviewers: epriestley, bos, sid0, dschleimer
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4955
Test Plan: Used it in Phabricator.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2432
Differential Revision: https://secure.phabricator.com/D4928
Summary: We only caught half of this.
Test Plan: Unit test.
Reviewers: edward
Reviewed By: edward
CC: aran
Maniphest Tasks: T1261
Differential Revision: https://secure.phabricator.com/D4920
Summary: Git spits these out with \n at the end.
Test Plan:
```
>>> orbital ~/devtools/arcanist $ git branch --set-upstream trim master
Branch trim set up to track local branch master.
>>> orbital ~/devtools/arcanist $ arc which --base arc:upstream
RELATIVE COMMIT
If you run 'arc diff', changes between the commit:
acf7600e6e Temporarily restore apache/license linters
...and the current working copy state will be sent to Differential, because
it is the merge-base of the upstream of the current branch and HEAD, and
matched the rule 'arc:upstream' in your args 'base' configuration.
You can see the exact changes that will be sent by running this command:
$ git diff acf7600e6e728395..HEAD
These commits will be included in the diff:
3580555e4b30598f WIP
MATCHING REVISIONS
These Differential revisions match the changes in this working copy:
(No revisions match.)
Since there are no revisions in Differential which match this working copy, a
new revision will be created if you run 'arc diff'.
```
Reviewers: brennantaylor
Reviewed By: brennantaylor
CC: aran
Differential Revision: https://secure.phabricator.com/D4913
Summary:
Restores linters only, without unit tests or entry in ComprehensiveLinter. Marks them deprecated.
If use at Facebook isn't widespread I'd prefer to simply delete them.
Test Plan: none
Reviewers: vrana
Reviewed By: vrana
CC: aran
Maniphest Tasks: T2274
Differential Revision: https://secure.phabricator.com/D4906
Summary: Remove all references to ArcanistLicenseLinter and ArcanistApacheLicenseLinter.
Test Plan: Rerun the linter and ensure nothing is broken.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4901
Summary:
My original idea was to return safe HTML from this function.
But we are down to 20 occurrences in Phabricator and you shouldn't need this function in safe HTML world at all.
Test Plan:
$ arc lint src/applications/audit/controller/PhabricatorAuditListController.php
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4890
Summary: Raise deprecation warnings for these methods. I won't commit this until the phutil_tag branch merges.
Test Plan: Unit tests.
Reviewers: vrana
Reviewed By: vrana
CC: aran
Maniphest Tasks: T2432
Differential Revision: https://secure.phabricator.com/D4770
Summary: It's not trivial to find them inside 700+ lines long functions.
Test Plan:
Linted `reused-iterators.lint-test` renamed to `_.php`, saw other locations.
Repeated for `reused-local.lint-test`.
Repeated for `duplicate-key-in-array.lint-test`.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4871
Summary:
Previously 'arc diff X' with mercurial meant to use X as the base
to diff against. Now it means use gca(X,working directory) as the base to
diff against. This matches the git behavior.
Test Plan:
Ran 'arc diff master' on a repo where master was ahead of the feature branch.
Verified that the diff result included only the diffs in the feature branch.
Reviewers: epriestley, sid0, bos, dschleimer
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4865
Summary:
arc diff on large mercurial repos was taking 14 seconds just to get
to the commit message prompt. With these optimizations it takes 4.
- "ancestor(.) - ancestor(XYZ)" is expensive because it has to build the
entire 400000+ revision history for both. "XYZ::. - XYZ" is much cheaper
because it only looks at the revisions between XYZ and the working directory.
- "hg outgoing" has to talk to the server, which is slow. "hg log -r draft()"
gives us the same information and is much cheaper. We fall back to 'outgoing'
on older versions of mercurial.
Of the remaining 4 seconds, 2.5 are spent in 'hg status', which is a bit harder
improve.
Test Plan: Ran arc diff on our hg repo. Verified it ran faster and the diff was created.
Reviewers: epriestley, sid0, bos, dschleimer
Reviewed By: epriestley
CC: aran, Korvin, tido
Differential Revision: https://secure.phabricator.com/D4838
Summary: Makes sense after D2471.
Test Plan:
Swapped two binary files, ran `arc diff --only`.
Saw time 3.797 s instead of 4.361 s.
Changed `file.upload` to `file.uploa`, saw proper error.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4847
Summary: This is fairly confusing. Make the error message suggest the common remedy (update libphutil).
Test Plan: Eyeballed it.
Reviewers: Afaque_Hussain, btrahan, vrana
Reviewed By: Afaque_Hussain
CC: aran
Differential Revision: https://secure.phabricator.com/D4834
Summary: If provided, have `arc patch` use `authorName` / `authorEmail`. This simplifies handling and makes patches more portable between version control systems (previously, information was generated in the diff's VCS, regardless of which VCS it was being applied to).
Test Plan: Created a diff with author `derp <derp@derp.com>`, ran `arc patch --diff x`, got a local commit with that author.
Reviewers: btrahan, edward, vrana
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D4827
Summary:
Record author email information in `arc diff`, so we can recreate it in `arc patch` and elsewhere without creating any kind of email exposure issues.
In Mercurial, we currently store the whole string ("username <email@domain.com>"). Make this consistent with Git.
Test Plan: Created git and hg diffs, saw authorEmail populated.
Reviewers: btrahan, edward, vrana
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D4825
Summary:
We want to use it for `yield` and `invariant_violation()` which throws.
Having node instead of token would be better but this would be enough.
Test Plan: Implemented a hook in FB repo and added a test case there.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4821
Summary:
We assume `git` is available now, but should not. Specifically, if a user runs a working copy operation like `arc list` in an SVN working copy without `git` available, they get this error: P707
We interpret git errors very narrowly; be more liberal in how we interpret them. This assumes users working with `git` will have a functional `git`, but this seems like a reasonable assumption and lets us remove some error text matching code.
Test Plan: Changed `git` to `girt`, ran `arc list`, saw a reasonable exception. Changed back to `git`, saw git detected.
Reviewers: btrahan, vrana
Reviewed By: vrana
CC: svemir, aran
Differential Revision: https://secure.phabricator.com/D4804
Test Plan: Copied the code in a script, changed `phutil_passthru()` to `echo csprintf()` and ran it.
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4805
Summary: I somehow missed it.
Test Plan: Sent this diff by different copy of Arcanist.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4800
Summary:
This disallows code like this:
$cmd = 'ls';
execx($cmd);
But I guess it's not that big deal?
Test Plan: Linted whole Arcanist and Phabricator codebases, most parts looks fixable.
Reviewers: epriestley
CC: nh, aran, Korvin
Differential Revision: https://secure.phabricator.com/D4794
Summary:
FB currently starts Sandcastle push before starting the diff workflow.
If `arc diff` commits something then we need to restart the push.
I want to avoid this by starting the push after commit.
Test Plan: Will test after implementing the listener.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4785
