1
0
Fork 0
mirror of https://we.phorge.it/source/arcanist.git synced 2024-11-29 10:12:41 +01:00
Commit graph

302 commits

Author SHA1 Message Date
epriestley
89f9eb66a7 Support inspection of remote refs with "arc inspect remote(...)"
Summary: Ref T13546. Expose remote refs for inspection via "arc inspect". For now, this only works in Mercurial.

Test Plan: Ran "arc inspect remote(default)" in Mercurial, got a ref out.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21374
2020-06-30 13:07:25 -07:00
epriestley
b19985a4bd Copy repository URI normalization code from Phabricator to Arcanist
Summary: Ref T13546. Move toward smarter remote repository lookup by providing URI normalization code in Arcanist. This diff duplicates code from Phabricator; the next change will collapse it.

Test Plan: Ran unit tests.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21372
2020-06-30 13:07:12 -07:00
epriestley
b0a9ef8351 In "arc land" under Git, confirm branch creation
Summary: Ref T13546. If "arc land" would create a branch, warn the user before it does.

Test Plan: Ran "arc land --onto mtarse", a typo of "master".

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21354
2020-06-30 06:29:51 -07:00
epriestley
33bb0acf97 Collect scattered implementations of "getDisplayHash()" into RepositoryAPI
Summary: Ref T13546. All of LandEngine, LocalState, and RepositoryAPI implement "getDisplayHash()". Always use the RepositoryAPI implementation.

Test Plan: Grepped for symbols.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21353
2020-06-30 06:28:38 -07:00
epriestley
63f2e667b9 Update "arc land" display of build failures, and rename "DisplayRef" to "RefView"
Summary:
Ref T13546. Show ongoing and failed builds more clearly in "arc land" output.

Also rename "DisplayRef" (which is not a "Ref") to "RefView" with the goal of improving clarity, and let callers "build...()" it so they can add more status, etc., information.

Get rid of "[DisplayRef|RefView]Interface". In theory, future refs (say, in Phabricator) might not do anything here, but every Ref just ends up implementing it. This could perhaps be subclassed more narrowly in the future if necessary.

Test Plan: Ran "arc land", grepped for various symbols.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21352
2020-06-30 06:27:56 -07:00
epriestley
92f860ae9b Improve "--hold", save/restore state, bookmark creation, and some warnings for "arc land" in Mercurial
Summary:
Ref T13546. Ref T9948.

  - Make "--hold" show the same set of commands to manually push that the normal workflow would use.
  - Make save/restore state work.
  - Make bookmark creation prompt for confirmation.
  - Improve / provide some additional warnings and help text.

Test Plan: Ran various increasingly complex "arc land" workflows, e.g. "arc land --hold --onto fauxmark1 --onto fauxmark2 --into default . --revision 118 --trace"

Maniphest Tasks: T13546, T9948

Differential Revision: https://secure.phabricator.com/D21351
2020-06-10 17:31:51 -07:00
epriestley
50c534b591 Correct some minor "arc land" workflow issues in Mercurial
Summary: Ref T9948. Ref T13546. Clean up some minor behaviors to allow "arc land" to function in the simplest cases again. Also, do a capability test for "prune" rather than just falling back.

Test Plan: Ran "arc land <mark>" in Mercurial, got changes pushed.

Maniphest Tasks: T13546, T9948

Differential Revision: https://secure.phabricator.com/D21350
2020-06-10 17:31:50 -07:00
epriestley
488a24c40a In "arc land" in Mercurial, inch closer to making complex branch/bookmark workflows function
Summary:
Ref T9948. Ref T13546. This change moves toward a functional "arc land" in Mercurial.

Because of how "bundlerepo.getremotechanges()" works, "hg arc-ls-markers" does not actually list markers in the remote that aren't different from local markers so it's hard to get anywhere with this.

Test Plan: Got somewhat-encouraging output from "arc land" and "hg arc-ls-markers", but too many things are still broken for this to really work yet.

Maniphest Tasks: T13546, T9948

Differential Revision: https://secure.phabricator.com/D21348
2020-06-10 17:31:50 -07:00
epriestley
b1f807f7ca Disambiguate various types of Mercurial remote markers with "hg arc-ls-remote"
Summary: Ref T13546. Ref T9948. It seems challenging to examine a remote in vanilla Mercurial. Provide an "hg arc-ls-remote" command which functions like "git ls-remote" so we can figure out if "--into X" is a bookmark, branch, both, neither, or a branch with multiple heads without mutating the working copy as a side effect.

Test Plan: Ran various "arc land --into ..." commands in a Mercurial working copy, saw apparently-sensible resolution of remote marker names.

Maniphest Tasks: T13546, T9948

Differential Revision: https://secure.phabricator.com/D21343
2020-06-10 10:27:17 -07:00
epriestley
1bb054ef47 Verify remotes ("paths") in Mercurial during "arc land"
Summary: Ref T13546. Parse "hg paths" and validate that the remotes "arc land" plans to interact with actually exist.

Test Plan: Ran "arc land" with good and bad "--into-remote" and "--onto-remote" arguments, got sensible validation behavior.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21342
2020-06-10 10:27:17 -07:00
epriestley
b003cf9310 Remove "arc feature", "arc branch", "arc bookmark", and significant chunks of obsolete marker code
Summary: Ref T13546. Moves away from the older workflows in favor of "arc branches", "arc bookmarks", and "arc work".

Test Plan: Grepped for affected symbols, didn't find any callers.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21337
2020-06-08 16:27:31 -07:00
epriestley
3d64140ff3 Implement "arc work", to replace "arc feature"
Summary: Ref T13546. Fixes T2928. Adds a new "arc work" workflow which functions like the older "arc feature" workflow, but with modern infrastructure.

Test Plan: Used "arc work" to begin work on branches, bookmarks, and revisions in Git and Mercurial.

Maniphest Tasks: T13546, T2928

Differential Revision: https://secure.phabricator.com/D21336
2020-06-08 16:27:27 -07:00
epriestley
5abf0b96c8 Use MarkerRefs to resolve landing symbols in Mercurial
Summary: Ref T13546. Update the Mercurial code which finds default targets and maps symbols to targets under "arc land" to use the new MarkerRef workflow.

Test Plan: Ran "arc land" with (and without) various arguments in Mercurial, saw them resolve in a seemingly sensible way.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21335
2020-06-08 16:27:24 -07:00
epriestley
599ba0f999 Provide a more powerful query mechanism for "markers" (branches/bookmarks)
Summary:
Ref T13546. Various Arcanist workflows, and particularly the MercurialAPI, currently repeat quite a lot of code around parsing branches and bookmarks.

In modern Mercurial, we can generally use the "head()" and "bookmark()" revsets to do this fairly sensibly.

This change mostly adds //more// code (and introduces "arc bookmarks" and "arc branches" as replacements for "arc bookmark" and "arc branch") but followups should be able to mostly delete code.

Test Plan: Ran "arc branches" and "arc bookmarks" in Git and Mercurial.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21333
2020-06-08 16:27:20 -07:00
epriestley
31d08f9a8f Remove old Mercurial code testing for rebase and phase support
Summary: Ref T13546. The minimum required Mercurial version should now always have these features; if not, they should move to more modern feature tests.

Test Plan: Grepped for affected symbols.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21334
2020-06-08 16:26:39 -07:00
epriestley
f3f31155b7 Format "arc land" passthru commands more nicely, and execute them from CWD
Summary:
Fixes T13380. Ref T13546. Use slightly nicer command formatting for passthru commands, to make it more obvious what's going on and which pieces of output are from "arc" vs various subcommands.

Also, execute repository API passthru commands from the working copy root. All other commands already did this, the older API just didn't support it.

Test Plan: Ran "arc land" in Git and Mercurial repositories, saw nicer output formatting.

Maniphest Tasks: T13546, T13380

Differential Revision: https://secure.phabricator.com/D21330
2020-06-08 16:26:18 -07:00
epriestley
0bf4da60f6 Make Mercurial use "hg shelve" and "hg unshelve" in dirty working copies in "arc land"
Summary: Ref T13546. Implement the equivalents of "git stash" in Mercurial.

Test Plan: Dirtied a working copy in Mercurial, ran "arc land", saw dirty changes survive the process.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21329
2020-06-08 16:22:44 -07:00
epriestley
4d61c00531 Improve final messages under "arc land --hold"
Summary: Ref T13546. Update some of the "arc land --hold" behavior to be more functional/consistent with the updated workflow.

Test Plan: Ran "arc land --hold" under various conditions, got sensible forward/restore instructions.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21328
2020-06-08 16:22:44 -07:00
epriestley
8a53b5a451 When landing changes in an empty repository, merge cleanly in Git
Summary:
Fixes T12876. Ref T13546. When you make the first change in a new Git repository, "arc land" currently can not merge it because there's nothing to merge into.

Support merging into the empty state formally, reachable by using "--into-empty" (which should be uncommon) or "arc land" in an empty repository.

Test Plan:
  - Used "arc land --into-empty --hold ..." to generate merges against the empty state under "squash" and "merge" strategies in Git.
    - Got sensible result commits with appropriate parents and content.

Maniphest Tasks: T13546, T12876

Differential Revision: https://secure.phabricator.com/D21324
2020-06-08 16:19:55 -07:00
epriestley
68f28a1718 Substantially modernize the "arc land" workflow
Summary: Ref T13546. This has a lot of dangerously rough edges, but has managed to land at least one commit in each Git and Mercurial.

Test Plan:
  - Landed one commit under ideal conditions in Git and Mercurial.
  - See followups.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21315
2020-06-08 16:17:19 -07:00
epriestley
0da395ffe4 Introduce "RepositoryLocalState", a modern version of "requireCleanWorkingCopy()"
Summary:
Ref T13546. Introduces a more structured construct for saving and restoring local repository state.

This is similar to old behavior, except that:

  - "arc.autostash" is no longer respected;
  - untracked changes are stashed; and
  - we do not offer to amend.

Test Plan: In future changes, saved and restored various permutations of local state.

Maniphest Tasks: T13546

Differential Revision: https://secure.phabricator.com/D21314
2020-06-08 15:40:01 -07:00
epriestley
284139a24e Restore the ":(attr:filter=lfs)" test for LFS
Summary:
See D21190. The ".gitattributes" approach fails when ".gitattributes" is in a subdirectory (or global). These are probably unusual cases, but at least one is known in the wild.

Instead:

  - Restore the ":(attr:filter=lfs)" test, which seems to be the fastest accurate test available in modern Git.
  - If the test fails, assume the repository is not LFS. This only impacts users running very old versions of Git.

Test Plan:
  - In LFS and non-LFS repositories, created diffs. Saw correct detection again.
  - Broke the command on purpose, saw LFS detection conclude "no LFS", but not fail disastrously.

Subscribers: ptarjan

Differential Revision: https://secure.phabricator.com/D21192
2020-04-29 21:04:44 -07:00
epriestley
ade9b51a1f Detect LFS by looking for tracks in ".gitattributes" instead of using "ls-tree"
Summary:
See PHI1718. See also <https://discourse.phabricator-community.org/t/arc-diff-fails-due-to-git-cmd-fails/3680/>.

Currently, `arc diff` detects Git LFS with `git ls-files -z -- ':(attr:filter=lfs)'` magic. This is an accurate test, but does not work on older Git.

Try a simpler, dumber test and see if that will work. If this also has issues, we can try this stuff:

  - do version detection;
  - pipe the whole tree to `git check-attr`;
  - try a command like `git lfs ls-files` instead, which is probably a wrapper on one of these other commands.

Test Plan:
  - In a non-LFS repository, ran "arc diff" and saw the repository detect as non-LFS.
  - In an LFS repository, ran "arc diff" and saw the repository detect as LFS.

Differential Revision: https://secure.phabricator.com/D21190
2020-04-29 16:23:44 -07:00
epriestley
196f8f54ce Remove "backout", "close", "flag", "start", "stop", "time", and "revert" workflows
Summary: Ref T13490. These workflows don't seem worth the maintenance cost, see T13488 for discussion.

Test Plan: Grepped for methods which looked like they might only be called by these flows, only dug up the backout stuff.

Maniphest Tasks: T13490

Differential Revision: https://secure.phabricator.com/D21100
2020-04-13 07:09:49 -07:00
epriestley
0f2e277cd9 Update "arc amend" for Toolsets
Summary: Ref T13490. Moves "arc amend" to Toolsets with modern ref/hardpoint code.

Test Plan: Ran "arc amend --show", "--revision", etc. Hit all the prompts and errors, probably?

Maniphest Tasks: T13490

Differential Revision: https://secure.phabricator.com/D21095
2020-04-12 13:48:26 -07:00
epriestley
4cc05c377d Add a "CommitSymbolRef" for resolving symbolic commits into stable commit hashes
Summary:
Ref T13490. Frequently, we know the symbolic name of a commit (like "master") but need the immutable identifier for it (the commit hash).

Provide a Ref and Query for doing this lookup.

Test Plan: Ran `arc inspect symbol(...)` with various symbols, saw appropriate resolutions, nulls, or errors.

Maniphest Tasks: T13490

Differential Revision: https://secure.phabricator.com/D21090
2020-04-12 13:19:53 -07:00
epriestley
92be6df0eb Add a mode to "ExecFuture" that makes "resolvex()" semantics the default
Summary:
Ref T13490. "ExecFuture" can raise a "CommandException" when the subprocess fails. In most cases, this is desirable, since we usually do not expect subprocesses to fail.

Currently, "execx()" (which raises these exceptions) is the synchronous default (with the more verbose "exec_manual()" as an alternative if you expect the command may return an error code), but the Future variation has no way to hint that external resolution should raise an exception if the process exits with an error.

Since the "HardpointEngine" yield construct can resolve external futures, add a mode for this to simplify implementing "HardpointQuery" classes a bit. Without this, they either need to retain "$futures" and manually call "resolvex()", or check the "$err" result and throw some other exception, both of which are low-value boilerplate (queries that want to do this still can, of course).

This is basically like providing a hint that the futures should be resolved with "resolvex()" instead of "resolve()".

Also, make this the default behavior of a new "$api->newFuture()" wrapper.

Test Plan: Intentionally broke a command in a HardpointQuery, got a sensible exception automatically during external future resolution.

Maniphest Tasks: T13490

Differential Revision: https://secure.phabricator.com/D21089
2020-04-12 13:18:11 -07:00
epriestley
adea2550f5 Introduce "arc inspect" and some of the new ref/hardpoint classes
Summary:
Ref T11968. Inches toward the new ref/hardpoint code by introducing the modern refs as "RefPro" objects and supporting an "arc inspect <object>" to load objects and hardpoints.

This doesn't impact any existing runtime behavior.

Test Plan: Ran "arc inspect [--all] commit(...)", got hardpoint queries and yield-based data fetching.

Maniphest Tasks: T11968

Differential Revision: https://secure.phabricator.com/D21078
2020-04-10 05:01:25 -07:00
epriestley
0b3cd39230 Update the "WorkingCopy" API and create a fallback "Filesystem" working copy
Summary:
Ref T11968.

  - Allow "WorkingCopy" objects to maintain an API object and update callers ("get...()" instead of "new...()").
  - Always generate a WorkingCopy object and a RepositoryAPI object.

Currently, code has to look like this:

```
$working_copy = ...
if ($working_copy) {
  $repository_api = ...
  if ($repository_api [instanceof ... ]) {
```

This is clunky. There's also no reason some "arc" commands can't run outside a VCS working directory without special-casing how they interact with the filesystem.

Conceptually, model the filesystem as a trivial VCS (which stores exactly one commit, always amends onto it, and discards history). Provide a trivial WorkingCopy and API for it.

(This change isn't terribly interesting on its own, but chips away at landing the new Hardpoint infrastructure.)

Test Plan: Ran `arc version`, `arc upgrade`.

Maniphest Tasks: T11968

Differential Revision: https://secure.phabricator.com/D21070
2020-04-08 09:22:43 -07:00
epriestley
368aec16a1 Update some ancient "set X=Y" environment code for new Windows execution without a shell
Summary:
Depends on D21051. Ref T13504. Ref T13209. This very old Mercurial code uses "X=Y hg ..." on Linux and "set X=Y & hg ..." on Windows.

The latter construct no longer works because we bypass the shell. The former construct is obsolete.

Additionaly, delete some ancient "branch merge" code which has no callers.

Test Plan: Created a diff in a Mercurial repository on Linux. I minimally vetted this on Windows since I don't have a "hg + Windows" environment at the moment.

Maniphest Tasks: T13504, T13209

Differential Revision: https://secure.phabricator.com/D21052
2020-04-02 13:50:57 -07:00
epriestley
8c4f6ce161 Merge the remainder of the "experimental" branch
Summary:
Depends on D20986. Ref T13395. This //mostly// collapses the entire "experimental" branch into "master".

I plan to change the "Ref/Hardpoint" pattern to become future oriented, but this is more steps forward than sideways.

Test Plan: Ran various `arc` workflows.

Maniphest Tasks: T13395

Differential Revision: https://secure.phabricator.com/D20987
2020-02-13 06:05:08 -08:00
epriestley
21a1828ea0 Omit "--" in older fallback commands for Git remote URIs
Summary: Ref T13481. Some older versions of Git appear to not support "--" in these commands. Just drop it. This can lead to ambiguous results with certain obviously-silly remote names, but doesn't appear to lead to anything dangerous.

Test Plan: Will followup with user on ancient Git.

Maniphest Tasks: T13481

Differential Revision: https://secure.phabricator.com/D20952
2020-01-23 16:50:54 -08:00
epriestley
70c0fd3f22 In Git, fall back across versions more cleanly when trying to get the URI for a remote
Summary: Fixes T13481. When identifying the URI for a remote, fall back from "git remote get-url" to "git ls-remote --get-url" to "git config remote.<name>.url" based on command output and version tests.

Test Plan: Ran `arc land --hold`, rigged the subcommands to fail to try all three fallbacks, ran `arc land --hold --remote asdfasdf` to get an explicit failure.

Maniphest Tasks: T13481

Differential Revision: https://secure.phabricator.com/D20950
2020-01-23 15:19:28 -08:00
epriestley
cc1ff38843 When generating diffs in "arc diff", disable Git config option "diff.suppressBlankEmpty"
Summary:
Ref T13432. Git has a "diff.suppressBlankEmpty" config option which makes it emit nonstandard diffs with trimmed trailing whitespace on unchanged blank lines.

Currently, we don't parse these diffs correctly. Even if we do in the future, emitting a more standard diff is desirable.

Explicitly disable this option when executing "git diff" so we build more standard diffs.

Test Plan:
  - Configured this option.
  - Modified a file with a blank line in it without changing the blank line, got this goofy display diff:

{F6985234}

  - Applied patch, rediffed the same change, saw "-c diff.suppressBlankEmpty" in "--trace" output and got this sensible diff:

{F6985235}

Maniphest Tasks: T13432

Differential Revision: https://secure.phabricator.com/D20877
2019-10-29 10:14:40 -07:00
epriestley
ca66743905 Support Perforce/Git repositories in "arc land"
Summary: Ref T13434. Detect perforce remotes and use "git p4" commands in place of "git" commands when operating in Perforce mode.

Test Plan:
  - Landed "master" onto itself, saw master update.
  - Landed "feature1" onto clean "master", saw master update.
  - Landed "feature2" onto dirty "master", saw master stay dirty.
  - Landed with "--hold", got sensible submit instructions.

Maniphest Tasks: T13434

Differential Revision: https://secure.phabricator.com/D20868
2019-10-28 11:25:00 -07:00
epriestley
83661809e5 Work around a Windows escaping issue and security conecern in "hg cat --output ..."
Summary:
See PHI904. Ref T13210. Ref T13209. Currently, we have an `hg cat` construction which attempts to pass a literal `%p` to Mercurial. This fails because you can't pass `%` through `%s` outside of `wilds`.

It also uses `%C` to pass a list of file paths. This is broadly unsafe and can cause command execution if you modify a file named, e.g., `; rm -rf xyz` or similar. I think it would be difficult to turn this into an attack but it's fairly bad. This dates from D5144 in 2013.

Test Plan: With this patch, created D19757 which has valid binary data (see F5962134).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13210, T13209

Differential Revision: https://secure.phabricator.com/D19758
2018-10-26 07:28:50 -07:00
Aviv Eyal
875d018360 Fix arc diff when adding large new file with new git
Summary:
See https://discourse.phabricator-community.org/t/arc-not-supporting-git-2-17-1/.
When treating it a large file  binary, we try to get the "old" and "new" content using `git ls-tree` and `cat-file`.
If the file is new or deleted, there is no old file, so we try to work with filename `null`.

Under git < 2.17.1, that gets treated as `git ls-tree -- .`, which falls in the next condition under "no such path".
In git 2.18, etc, this is an error.

Explicitly bail out if there is no filename.

Test Plan: Add a new, large (>4Mb) file, arc diff.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19513
2018-07-09 17:59:15 +00:00
epriestley
222800a86e Parse Mercurial changeset evolution "instability" log field
Summary: See PHI718. Modern Mercurial with the "evolve" extension enabled may emit this field.

Test Plan: As D19262.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19498
2018-06-19 16:15:30 -07:00
Alex Vandiver
ad3087e5e1 Correctly parse git status --porcelain=2 output with filenames with spaces
Summary:
Filenames are last in `git status --porcelain=2` lines; they
are not escaped in any way, despite the fields being
whitespace-delimited.  `explode` thus happily chops apart filenames
with spaces in them, causing later git operations to operate only on
the filename up to the first space.

Split the lines into the right number of elements -- in all cases,
this is one more than the index we're using, since filenames come last.

Test Plan:
Altering a file with a space in its path, and running `arc diff -a`.

Added tests.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D19389
2018-04-19 19:17:16 -07:00
epriestley
b8c9c385a7 Survive extra "obsolete:" log output from the Mercurial evolve extension
Summary: See PHI502; see <https://bugzilla.mozilla.org/show_bug.cgi?id=1448137>.

Test Plan: I spent all of three minutes trying to install the `evolve` extension without success and gave up, but this probably does the right thing based on the example output in the Bugzilla issue.

Differential Revision: https://secure.phabricator.com/D19262
2018-03-26 14:12:15 -07:00
Alex Vandiver
9144658e69 Accelerate working tree operations in git
Summary:
Currently, `arc` on `git` uses the following commands to examine the
state of the working tree and history; example times for a no-op diff in a
165k-file working tree are also shown:

```
1)  git diff --no-ext-diff --no-textconv --submodule=short --raw 'fb062d4ecce5d9c1786b7bfc8a0dedf6b11fdd96' --
  = 1,722,514 us

2a) git diff --no-ext-diff --no-textconv --submodule=short --raw 'HEAD' --
  = 1,715,507 us

2b) git ls-files --others --exclude-standard
  = 2,359,202 us

3)  git diff-files --name-only
  = 1,333,274 us
```

Steps (2a) and (2b) are run concurrently; this results in a total elapsed
wallclock time of approximately 5.4 seconds.  This is inefficient -- all four of
the above steps must both load the index and examine the working copy, which may
be slow operations when large repositories are used.  Additionally, none of the
effort of those stat calls on the working tree, or load time of the index, is
shared across the processes.

Step (1) is called from `getCommitRangeStatus`, which was split out in D4095; it
is currently never called on its own, only ever from `getWorkingCopyStatus`,
where it it combined with `getUncommittedStatus`.  The current behavior of the
method is to return the set of changes //either// in local commits //or//
uncommitted in the working tree, which duplicates work that
`getUncommittedStatus` is intended to do.  Changing the behavior of this method
(in Git, and other VCSes) to only examine _committed_ status seems both inline
with the name of the method and the original description of it in D4095 -- and
also serves to make it much faster, as it is an operation that need not inspect
the working tree at all.

Steps (2a), (2b), and (3) attempt to gather the state of the working copy, and
as such are all I/O bound but must examine nearly identical data.  For git
2.11.0 and higher, we can instead rely on the machine-parseable `git status
--porcelain=2` format, which provides the information from all of these commands
at once.  It also allows additional performance improvements, as `git status`
has been the focus of several optimizations in the latest versions of git (the
untracked cache and fsmonitor services, for instance), which are not available
in the lower-level `diff`, `ls-files`, and `diff-files` commands.

This has the added benefit of fixing a bug noticed in T9455, in that uncommitted
or unstaged changes in modules can now be detected, regardless of if they also
have changed their base commit.  It further resolves a bug where `.gitmodules`
appeared to have unstaged changes, when in reality the unstaged changes were in
submodules elsewhere in the tree.

For backwards compatibility with versions of git < 2.11.0, the old code is left
in place.  It is possible that the simpler output from v1 of `git status
--porcelain` would also suffice for some of the above benefits, but the payoff
of parsing yet another format is deemed insufficient; users wishing improved
performance should simply upgrade `git`.

Alltogether, these result in the following, for a no-op diff in a
165k-working-file tree:

```
1) git diff --no-ext-diff --no-textconv --submodule=short --raw 'fb062d4ecce5d9c1786b7bfc8a0dedf6b11fdd96' HEAD --
 = 9,227 us
2) git status --porcelain=2 -z
 = 739,964 us
```

...for a total of 749ms, an improvement of 4.7s.

Depends on D18841.

Test Plan: Existing tests.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D18842
2017-12-23 20:12:50 -08:00
Alex Vandiver
f34467914c Add tests for git submodules, based on current behavior
Summary:
This adds tests that detail the current behavior of `arc` in
the presence of `git` submodules.

Test Plan: No behavior change; wrote the tests such that they pass.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D18841
2017-12-23 20:01:20 -08:00
epriestley
76b54ce0a9 Fix parsing of Git branches with common and useful name "0"
Summary: See <https://discourse.phabricator-community.org/t/arc-has-a-spurious-failure-on-arc-diff/521>. Oh, PHP!

Test Plan: Created a branch named "0", ran `arc diff`. Before: fatal. After: this beautiful revision.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D18678
2017-10-04 10:20:48 -07:00
Jakub Vrana
3b6b523c2b Fix errors found by PHPStan
Test Plan: None.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D17376
2017-02-18 09:24:19 +00:00
Alex Vandiver
f3037bf216 [git] Override diff.submodule so git diff output is always parseable
Test Plan:
Removed a submodule with `diff.submodule` set to `log`, saw
`arc diff` error; with this change, it no longer does.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Maniphest Tasks: T10881

Differential Revision: https://secure.phabricator.com/D17327
2017-02-13 17:40:26 -08:00
Chris Burroughs
c243cbbd9f tighten remote URI error handling with idiosyncratic remote names
Summary:
`git ls-remote` has an unusual way to indicate a URL was not
found: echoing back user input
```
$ git ls-remote --get-url does_not_exist
does_not_exist
$ echo $?
0

```

`getRemoteURI` handles checking for remotes other than 'origin', but
the error handling always matched against the string 'origin'
regardless of remote name.

Test Plan:
With a git config along the lines of:
```
[remote "my_special_name"]
        url = ssh://secure@secure.phabricator.com/diffusion/ARC/arcanist.git
        fetch = +refs/heads/*:refs/remotes/my_special_name/*
[branch "master"]
        remote = github
        merge = refs/heads/master
[remote "github"]
         # url = git@github.com:phacility/arcanist.git
         fetch = +refs/heads/*:refs/remotes/github/*
```

and running in a branch tracking `master` (github).  `arc which` would
(without this diff) show:
```
The remote URI for this working copy is "github".
```
With this diff, `arc which` correctly shows:
```
Unable to determine the remote URI for this repository.
```

When diffing against a tracking branch with a propertly configured
remote (the happy path), `arc which` still correctly identifies the
remote URI:
```
The remote URI for this working copy is
"ssh://secure@secure.phabricator.com/diffusion/ARC/arcanist.git".
```

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, chad, epriestley

Differential Revision: https://secure.phabricator.com/D17110
2016-12-29 10:35:59 -05:00
Alex Vandiver
ee6357386d Correctly parse file renames and copies from git diff --raw
Summary:
`parseGitRawDiff` dealt with the `A`, `M`, and `D` status flags from
`git diff --raw`, for file additions, modifications, and deletions
respectively.  However, it failed to cope with `C` and `R` flags, for
copies and renames.  Git version 2.9 and above default to resolving
renames, even in `git diff --raw` output, making this lack of support
only salient now (though users with Git's `diff.rename` set
encountered it previously).

Those two flags differ from the other three in that they offer both the
source and destination filename, separated by a tab.  As
`parseGitRawDiff` was not aware of this property, it returned a
"filename" of `"oldfile\tnewfile"`.  This is surfaced in several
places, including as passed to linters as a filename to check.
Needless to say, this file is nearly guaranteed to never exist on
disk.

Detect both the `C` and `R` flag types, and generate either a file
addition, or a pair of addition/deletion entries.

Test Plan:
Renamed a file, with a linter that printed each file it was
called with.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: jboning, Korvin

Differential Revision: https://secure.phabricator.com/D16387
2016-08-16 17:27:20 -07:00
Allan Jude
19608cffe6 SVN buildSyntheticAdditionDiff: exit sooner if path is a directory
Summary:
When doing svn copy, or svn mv, a SynthenticAdditionDiff is generated.

If the path is a directory, an error will occur when checking the
mime-type of the directory. Immediately after the properties check,
the function returns null if the path is a directory. Move this
check to before the properties check to avoid exiting with an error.

```
Command failed with error #1!
COMMAND
svn propget 'svn:mime-type' '/home/trasz/svn/ports/cad/py-pycam'@

STDOUT
(empty)

STDERR
svn: warning: W200017: Property 'svn:mime-type' not found on '/home/trasz/svn/ports/cad/py-pycam@'
svn: E200000: A problem occurred; see other errors for details

(Run with `--trace` for a full exception trace.)
```

Test Plan: Created differentials of changes with `svn copy` and `svn mv`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Tags: #subversion

Differential Revision: https://secure.phabricator.com/D15985
2016-06-05 15:12:46 -07:00
epriestley
2234c8cacc Fix arc which for svn repos
Summary:
`arc which` is currently broken for svn repos.

Fixes T6635

(I think I independently wrote an identical change to yours)

Test Plan: `∴../../p/arcanist/bin/arc which` on a svn repo.

Reviewers: aik099, epriestley, #blessed_reviewers

Reviewed By: aik099, epriestley, #blessed_reviewers

Subscribers: aik099, Korvin

Maniphest Tasks: T6635

Differential Revision: https://secure.phabricator.com/D15922
2016-05-16 12:54:37 +00:00
epriestley
a2ab38df78 Improve performance of arc branch in Git with many branches
Summary:
This is mostly just a personal quality-of-life fix. I run this command fairly often and having it return a little faster is nice.

This replaces a `git show` for each individual branch with a big `git for-each-ref` which we were already running anyway. This is quite a bit faster.

This command also occasionally hangs or segfaults for me while executing the huge pile of subprocesses. This is unreliable to reproduce, probably some bug in some PHP extension I have, and likely hard to narrow down, and this approach is better in every way anyway.

Test Plan:
  - Ran `arc branch` in Git, observed faster output (in my `phabricator/`, about 2000ms -> 1200ms).
  - Ran `arc feature` in Mercurial.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15735
2016-04-16 16:39:32 -07:00