Summary:
When non-ascii characters appear in revision titles/summaries the `patch` and `diff` (to update) commands will fail on Windows systems. This often occurs due to “smart quotes” or "em—dash" characters being inserted into commit messages by editors on "user-friendly" operating systems like macOS.
This can be worked around by forcing all mercurial commands to use the global option `--encoding utf-8` which applies for any mercurial command. This option was [[ https://www.mercurial-scm.org/repo/hg/rev/a88e02081a88 | added in ~2006 ]] so this should work across all supported versions of mercurial.
Refs T13649
Test Plan:
I created a diff on a mercurial repository using smart quotes in the "Title" and "Summary" fields as well as in the content of a file being changed. Then on macOS, Windows (PowerShell), and Windows (cmd.exe) I was able to `patch` down the revision, make a modification, and `diff` the change back up to Phabricator, as well as `land` the change. I verified the commit and content looked correct on macOS as well as on Windows by using `nvim` which seems to properly detect and render the encoding, whereas mercurial displays the smart quotes and em-dashes with odd characters instead.
I did a grep through Arcanist codebase to find other places where `--encoding` might be specified for mercurial commands and could not find any. In the event that somehow this argument is added elsewhere I verified that multiple specifications of `--encoding utf-8` does not cause any issues and the later specification of `--encoding` appears to "win".
```lang=console
$ hg --encoding utf-8 --encoding utf-8 log -r tip
# prints out results in UTF-8 without issue
$ hg --encoding utf-8 log --encoding latin-1 -r tip
# prints out results in latin-1 without issue
```
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T13649
Differential Revision: https://secure.phabricator.com/D21676
Summary: See PHI2085. Python 3 is stricter about integers and floats than Python 2 was, and we can end up passing a float where an integer was expected if the player wins "arc anoid" using a terminal with certain (most?) character dimensions.
Test Plan:
- Modified "arcanoid.py" to win instantly.
- Adjusted terminal window to 80x24, ran "arc anoid", reproduced crash.
- Ran "python2 arcanoid.py" and observed old victory animation behavior.
- Applied patch.
- Ran "arc anoid" and observed identical victory animation behavior.
Differential Revision: https://secure.phabricator.com/D21667
Summary:
Ref T13649. Currently, "arc" may leave stdin nonblocking after showing a prompt. This can cause various odd behaviors down the line.
I can't immediately reproduce this behavior on macOS in "zsh" or "bash" (I'm unable to get stdin to remain nonblocking beyond the process lifespan), and also don't have pcntl locally so there's a fair amount of handwaving here.
Test Plan: This is somewhat speculative since I can't immediately reproduce the behavior. I tested the locally-reachable paths (no pcntl) but they're not interesting.
Maniphest Tasks: T13649
Differential Revision: https://secure.phabricator.com/D21666
Summary:
Ref T13562. Currently, "Filesystem::copyFile()" uses "copy", which doesn't work now that we no longer invoke "cmd.exe" by default.
Use "copy()" instead.
Note that this whole function is probably nonsense, but I'll follow up on T13562.
Test Plan:
- Created a standalone script which runs "Filesystem::copyFile()".
- Before: failed to copy any file.
- After: succesfully copied normal files.
- After: failed to copy a file over an existing directory with a reasonable error.
- After: failed to copy a file over itself with a reasonable error.
Maniphest Tasks: T13562
Differential Revision: https://secure.phabricator.com/D21643
Summary: See PHI2027. This example command is missing "--", but it's required.
Test Plan: Ran the new command, no longer got an error about "--".
Differential Revision: https://secure.phabricator.com/D21623
Summary: See PHI2022. This link is missing the `/u/` part and currently 404's.
Test Plan: Followed the new link, got documentation.
Differential Revision: https://secure.phabricator.com/D21611
Summary:
Ref T13635. PHP native JSON functions sometimes represent JSON objects as PHP "stdClass" objects.
Accept this representation and emit it correctly in "PhutilJSON".
Test Plan: Added a test, made it pass.
Maniphest Tasks: T13635
Differential Revision: https://secure.phabricator.com/D21604
Summary: The ".git/arc" directory may need to be created when writing to working copy configuration.
Test Plan:
- Tried to store an answer to a prompt in a new working copy.
- Before: error that ".git/arc" does not exist.
- After: prompt saved to working copy configuration.
Differential Revision: https://secure.phabricator.com/D21588
Summary:
Ref T13576. See that task for discussion.
When a user runs `arc land --pick A`, they may be selecting a range of commits ("X..Y") which have ancestors ("V..W") that should NOT land.
We must slice "X..Y" out of history before we can merge it, to avoid landing changes from "V..W".
When "X..Y" is simple and linear, we can rebase the range to pick our desired slice out of history.
When "X..Y" includes merge commits, we frequently can not, and I could not identify any simple alternative. The best alternative I came up with is this "reduce" operation:
- squash "into" onto Y, producing S, to guarantee there are no natural conflicts;
- squash S onto X^, producing T, to get rid of the merge commits;
- rebase T onto "into", producing R, to slice "X..Y" out of history;
- squash R onto "into", producing Q. (R and Q will be the same, but this simplies the code.)
This feels flimsy and fragile, but I can't immediately find a way to break it. See T13576 for more discussion.
Test Plan:
- Applied conflicting changes to `example.txt` in `master` and `feature1`.
- Ran `arc land`, got a merge conflict.
- Resolved the conflict with `git merge master`.
- Ran `arc land`.
- Before: merge conflict.
- After: `arc land` resolves the merge correctly.
- Stacked `feature2` on `feature1`, and made various mutations to `feature1` and `feature2`, then ran `arc land --pick feature2`. Changes made in `feature1` should not land, and they mostly do not. See T13576.
Maniphest Tasks: T13576
Differential Revision: https://secure.phabricator.com/D21590
Summary:
Ref T13576. The "implicit commits" prompt in "arc land" shows a list of implicit and non-implicit commits.
The implicit commits are marked with a background color, but this doesn't survive if you copy/paste the output into a support ticket.
Make my life easier by also marking commits so the marker survives copy/paste.
Test Plan: Ran "arc land" with implicit commits, saw a copy-pastable indicator.
Maniphest Tasks: T13576
Differential Revision: https://secure.phabricator.com/D21589
Summary:
Ref T13624. If we want to send PHP errors to a log, using the "error_log" configuration option catches the broadest set of errors across versions of PHP.
Configuring this disables errors on `stderr`, since they're sent to the log instead. We'd like them to go to both places; provide a simple wrapper for this. Also do a bit of writability testing.
Test Plan: Wrote errors to a new log, see followup changes.
Maniphest Tasks: T13624
Differential Revision: https://secure.phabricator.com/D21578
Summary:
Ref T13100. Ref T13586. These options have been in production for a while and seem to consistently improve behavior, downgrading segfaults to runtime errors.
Make them global options for all CLI environments. See next change for web enviornments.
Test Plan: Ran `arc`.
Maniphest Tasks: T13586, T13100
Differential Revision: https://secure.phabricator.com/D21566
Summary: Ref T13608. Ref T13100. Ref T13586. Properly checking "preg_match()" and similar calls for failure and raising useful exceptions is complicated and error-prone. Provide wrapper functions with an API that's more consistent with the rest of the codebase: matches are returned; and errors raise detailed exceptions.
Test Plan: See next change.
Maniphest Tasks: T13608, T13586, T13100
Differential Revision: https://secure.phabricator.com/D21561
Summary:
These are the instructions from https://github.com/golang/lint.
The fetch location changed to `golang.org/x/lint/golint` from `github.com/golang/lint/golint`.
`-u` tells go to update the package and its deps if they exist.
`-u` Shouldn't strictly be necessary, but figured we might as well follow the instructions from `golint`.
Test Plan:
Enable golint without having it installed.
Ensure that the install instructions now show the new location.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21552
Summary:
Ref T13588. For consistency of behavior between versions on either side of PHP7, any "catch (Exception)" block should generally have a "catch (Throwable)" block in the same catch list.
Raise a lint warning when "Exception" is caught without also catching "Throwable", since this is almost certainly not desired.
Test Plan: Added tests.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21542
Summary: Ref T13588. See D21500. This syntax is unusual and there are some hidden complexities involved; annotate them. See D21500 for more discussion.
Test Plan: Read text, reviewed D21500.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21541
Summary:
Ref T13588. Marking a method "final private" has never been meaningful, and is an error in PHP8.
Add static analysis to detect (and correct) this issue.
Test Plan: Added unit tests, will lint "phabricator/".
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21539
Summary: Ref T13598. This builds on D21537 and adds support for correcting the capitalization of builtin systems.
Test Plan:
- Linted a file that uses "ExCePtIoN", got a lint correction.
Maniphest Tasks: T13598
Differential Revision: https://secure.phabricator.com/D21538
Summary:
Ref T13598. If you spell a symbol like "Polygon" as "PoLyGoN", you currently get an "unknown symbol" lint message. However, provided "Polygon" is a valid symbol, we can unambiguously correct the spelling of the symbol.
Note that this patch can only correct the spelling of application symbols, not builtin symbols (since none of the library maps contain builtin symbols).
Test Plan: {F8374599}
Maniphest Tasks: T13598
Differential Revision: https://secure.phabricator.com/D21537
Summary: Ref T13589. See that task for discussion.
Test Plan:
- Created this diff, ran most commands in isolation.
- This change is difficult to test extensively.
Maniphest Tasks: T13589
Differential Revision: https://secure.phabricator.com/D21509
Summary:
Since 2.1.0 (commit 75bc0c03c145), pyflakes has included the Python
version and platform in its version output, so ignore it if present.
Since 2.2.0 (commit 6ba3f8e0b59b), pyflakes has included the column
number in its messages, so update the parser to include it and drop the
column number from the (only) test in order to work with both old and
new versions. Whilst here, assign names to the capture groups to make
the code clearer.
Test Plan: Ran arc unit
Reviewers: epriestley, joshuaspence, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21504
Summary:
Recent JSHint improves the warning and attributes it to the equals sign
rather than the end of the expression (changed in 897e0359ce19, first
released in 2.11.0-rc1).
Test Plan: Ran arc unit with JSHint 2.12.0
Reviewers: epriestley, joshuaspence, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21503
Summary:
PHP 8 deprecates openssl_free_key as the key is automatically freed, so
silence the warning in PhutilOAuth1Future::signString.
Test Plan: Ran arc lint --everything
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21502
Summary:
In PHP 8 passing an invalid encoding to mb_convert_encoding raises a
ValueError (which extends Error not Exception), so fix the test to also
catch Throwable (but leave the explicit Exception case for PHP 5, which
lacks Throwable).
Test Plan: Ran arc unit --everything
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21501
Summary:
Calling 'Foo::bar' is only supported since PHP 7, whereas the array form
is supported since PHP 5.4, which is below our PHP 5.5 baseline.
Test Plan: No regressions under PHP 8 and snippet tested on 3v4l
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21505
Summary:
PHP 8's sprintf raises a ValueError when encountering unknown format
specifiers (previously it would eat the argument and print nothing), so
linting format strings like %Ls dies with an uncaught ValueError.
Fix this by using a custom callback during linting to turn all format
specifiers into %s and replace the dummy null argument with the original
format specifier, ensuring we always end up providing valid input to the
sprintf at the end. This has the nice property that the output of the
call to xsprintf is the original format string, though any
transformation into valid input would do.
Test Plan: Ran arc lint
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21500
Summary:
In previous versions, passing the wrong type to preg_match would give a
warning that could be suppressed by @ and caught by set_error_handler,
but as of PHP 8 this raises a TypeError and so remains uncaught. Thus
check up-front whether the provided value is a string.
This fixes linting arc itself when run with PHP 8, as includes and
excludes use "optional regex | list<regex>", so would previously try to
pass an array to preg_match for the first alternative and die.
Test Plan: Ran arc lint
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21499
Summary:
PHP 7.2.0 deprecated the 5th parameter and PHP 8 removed it, so stop
using it and provide a default value to avoid erroring with:
```
Too few arguments to function PhutilErrorHandler::handleError(), 4 passed and exactly 5 expected
```
Test Plan: Used to create this revision with PHP 8 on macOS
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21498
Summary:
As of PHP 8, the XML entity loader is disabled by default and the
libxml_disable_entity_loader function is deprecated. Thus suppress the
deprecation warning for now; we could skip the function call, but this
is safer.
Test Plan: Used to create this revision with PHP 8 on macOS
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21497
Summary:
This combination does not make sense and PHP 8 errors with:
```
Private methods cannot be final as they are never overridden by other classes
```
Thus remove the redundant final from all such functions.
Test Plan: Used to create this revision with PHP 8 on macOS
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21496
Summary:
See <https://discourse.phabricator-community.org/t/search-by-name-in-files-doesnt-support-number/4300>.
I can't exactly reproduce the original issue, but when a query like "quack 1234" is tokenized, we end up calling "phutil_utf8v(1234)", where the argument is an integer.
At least in recent versions of PHP, this fatals ("trying to access an offset of an integer"). Cast the argument first.
Test Plan: Searched for "quack 1234" in Files. Before: fatal accessing offset of integer; after: correct results.
Differential Revision: https://secure.phabricator.com/D21477
Summary: Fix tests to work with rubocop 0.92.0 released on September 25, 2020
Test Plan: Unit tests pass
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D21474
Summary:
See PHI1894. PHP 7.4 introduced a new runtime configuration option, "zend.exception_ignore_args", which removes the "args" from exception backtraces.
The "PhutilOpaqueEnvelopeTestCase" relies on this behavior (since it explicitly inspects stack frames). Although the test isn't critical and could be restructured, it seems like there is little value to ever enabling this option in the context of Phabricator.
Disable it at startup so environments are more consistent across different PHP versions and configurations.
Test Plan:
- Enabled "zend.exception_ignore_args" under PHP 7.4.
- Ran "PhutilOpaqueEnvelopeTestCase".
- Before: failure, expected signpost value not present in stack trace (because no "args" are present on the exception).
- After: test passes.
Differential Revision: https://secure.phabricator.com/D21473
Summary:
We are having issues where people run out of file descriptors and the first `git push` will succeed, but the
second one will not. We'd like the diff to not be created in this case as it leads to weird behavior like our tests
running against 0 changed files.
Test Plan: none
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21471
Summary:
See PHI1862. This code calls "execute()" on the future directly, but that skips some steps -- notably, ServiceProfiler hooks.
Call "resolve()", which has the same effect but includes desirable/expected side effects.
Test Plan: Changed a workflow to run "phutil_passthru('ls')", ran it with "--trace". Before: no execution in trace; after: execution in trace.
Differential Revision: https://secure.phabricator.com/D21470
Summary: Ref T13582. When this code is reached with a raw HTTP exception, it currently fatals.
Test Plan:
- Ran `arc branches --conduit-uri=http://example.org` (a bad Conduit URI).
- Before: hard fatal with a bad method call.
- After: non-Conduit exception raised to user. Not ideal, but a step forward.
Maniphest Tasks: T13582
Differential Revision: https://secure.phabricator.com/D21467
Summary:
See T13572. FutureIterator does not release futures, so long-lived iterators (like the one that FuturePool may build) can end up leaking memory.
This affects the FuturePool used by the daemon overseer.
See T13572 for more discussion.
Test Plan:
- Ran the simple FutureIterator script from T13572. Before: memory held during iteration, script grows without bound. After: memory released, script uses stable memory.
- Ran the overseer with memory logging and an immediate wakeup from hibernation. Before: saw memory usage grow without bound at a rate of ~300MB/day. After: saw memory usage stable.
Differential Revision: https://secure.phabricator.com/D21466
Summary: Ref T13581. This message can be slightly more helpful in some cases by showing which method call failed.
Test Plan: Ran `arc branches` under the error condition in D21462, got a more useful error.
Maniphest Tasks: T13581
Differential Revision: https://secure.phabricator.com/D21463
Summary:
Ref T13577. After the lint rule fix in D21453, it can identify more errors. Fix the errors it identifies in "arcanist/".
These all seem fairly obscure/benign.
Test Plan: Ran `arc lint` on the files before and after these changes. Did not specifically re-test these particular messages, but they mostly very obscure.
Maniphest Tasks: T13577
Differential Revision: https://secure.phabricator.com/D21456
Summary:
Ref T13577. I'd like to `arc lint --everything` to find other bad calls to `pht()` and similar functions, but `n_HEREDOC` nodes currently can not generate a response to "getStringLiteralValue()".
Support literal extraction from heredocs.
Test Plan: Added a test, made it pass. Will lint everything.
Maniphest Tasks: T13577
Differential Revision: https://secure.phabricator.com/D21455
Summary: Ref T13577. This call is missing a parameter. After D21453, this is detected properly by lint. Provide the parameter.
Test Plan: Ran `arc lint` on HTTPSFuture before and after the change.
Maniphest Tasks: T13577
Differential Revision: https://secure.phabricator.com/D21454
Summary:
Ref T13577. This lint rule correctly detects the error in `pht('x %s y')` but the narrow test for `n_STRING_SCALAR` prevents it from detecting the error in `pht('x %s y'.'z')`.
Make the test broader.
Test Plan:
- Ran `arc lint` on `HTTPSFuture.php`, got a detection of the issue in T13577.
- Added a failing test and made it pass.
Maniphest Tasks: T13577
Differential Revision: https://secure.phabricator.com/D21453
Summary:
Ref T13555. Currently:
- If an exception is raised in "start()", the exception state is not set on the future.
- Futures do not always call "startFuture()" before starting, and do not always call "endFuture()" once they become resolvable.
- If you start an ExecFuture which immediately fails and then call "getPID()" on it, you get an unclear exception.
Simplify these behaviors:
- In FutureIterator, only start futures which have not already started.
- When starting a future on any pathway, run start code.
- When a future becomes resolvable on any pathway, run end code.
- Raise a more clear exception when calling "getPID()" on a future with no subprocess.
Test Plan: Faked a failing subprocess with "$proc = null", ran "bin/phd debug taskmaster" etc. Got clearer errors and more consistent future lifecycle workflows.
Maniphest Tasks: T13555
Differential Revision: https://secure.phabricator.com/D21423
Summary:
See PHI1808. Currently, "arc land <some bookmark>" does not destroy the bookmark in Mercurial. There are three issues here:
- "hg rebase --keep --collapse" moves bookmarks to the rewritten commits;
- "hg strip" moves bookmarks backwards;
- "hg prune" moves bookmarks backwards.
To get around "hg rebase", save and restore bookmark state.
To get around "hg strip" and "hg prune", explicitly destroy bookmarks pointing at commits before we strip/prune those commits.
Test Plan:
- Ran "arc land <some bookmark> --trace". Saw arc reset the bookmark position after rebasing, and destroy the bookmark explicitly before stripping.
- When the workflow exited, saw no more bookmark (previously: bookmark existed and pointed at a possibly-intermediate state).
Differential Revision: https://secure.phabricator.com/D21397
Summary:
Ref PHI1808. In Mercurial, we must save and restore bookmark state explicitly.
- Save and restore bookmarks.
- Clean up concepts in "arc-ls-markers" slightly, so we don't need separate "isCurrent" and "isActive" flags, hopefully.
I believe the totality of Mercurial state is:
- A (non-bare) working copy points at exactly one commit (which might be the empty/null commit, in an empty repository).
- A working copy has exactly one active branch.
- Each branch has zero or more heads.
- Each head may be closed.
- Each (non-null) commit belongs to exactly one branch.
- Note that the active branch may have zero heads and zero commits which belong to it!
- A working copy has zero or one active bookmark.
To capture this, we now emit:
- A list of branch heads. If a branch head is a working copy commit, that head is flagged as active.
- A list of bookmarks. If a bookmark is the current bookmark, that bookmark is flagged as active.
- A single "branch-state" virtual marker. This covers the case where you have run "hg branch X" to create X, but no objects in the working copy actually correspond to X yet. It also covers the case where you are on a concrete branch, but not any head of that branch.
- A single "commit-state" virtual marker. This always shows the current commit in the working copy.
Test Plan:
- Useful states to test are:
- Empty repository (not all commands currently work here).
- Normal repository, on a bookmark.
- Normal repository, no bookmark.
- "hg up 123" to update to somewhere in history.
- "hg branch X", to start a new branch with no commits.
- Ran "arc branches" and "arc bookmarks" in various states. Saw generally sensible output.
- Ran "arc land --hold ..." in various states against a failing remote. Saw generally sensible output, and saw working properly restored to the original state.
Differential Revision: https://secure.phabricator.com/D21396
Summary: Ref PHI1808. Currently, push failures are messaged awkwardly. Make this exception handling more selective and the user-facing behavior more readable.
Test Plan: Ran "arc land" against a failing remote, saw a human-readable message instead of a stack trace.
Differential Revision: https://secure.phabricator.com/D21395
Summary: See PHI1808. Some refactoring of the "passthru" API resulted in error conditinos here being dropped. Instead, raise them as exceptions.
Test Plan: Forced "hg push" to fail, used "arc land" against a failed push, saw error behavior instead of "success" feedback.
Differential Revision: https://secure.phabricator.com/D21394
