Summary:
Keeping unit tests speedy keeps them useful since people actually won't mind
running them. This diff records the time taken by each test and displays it nice
and colorized. Really, I just want to discourage non-unit tests from making
their way into ##__tests__##.
Some thoughts:
- The "acceptableness" times are subjective but if dependencies are properly
mocked the times seem to be ok. Integration tests that make network requests to
third-party endpoints and pull in megabytes of data will not survive. This is a
good thing.
- Fast tests get a gold star, encouraging small tests. I am sorry that the star
does not sparkle.
- There is no way for a programmer to admit that their test is going to be slow
in some cases. They will be shamed with red text for the life of their test.
- It might be confusing that fast but failing tests get green text and maybe a
gold star.
Test Plan: Ran some of the unit tests within Arcanist and libphutil. See
https://secure.phabricator.com/file/view/PHID-FILE-cdd3c94c219e0fd7470b/ for
sample output.
Reviewed By: epriestley
Reviewers: epriestley
CC: aran, epriestley
Differential Revision: 588
unit'
Summary:
We want to handle 'arc unit' and 'arc diff' differently in our test
framework.
Test Plan:
Tested that with 'arc unit' the value was set to 'unit' and with 'arc diff'
the value was set to 'diff'.
Reviewed By: epriestley
Reviewers: slawekbiel, epriestley
CC: jungejason, grglr, aran, tuomaspelkonen, epriestley
Differential Revision: 430
Summary:
Differential showed 'okay' as the arc unit status even when there were
postponed tests.
Test Plan:
Tested that test results were pushed to differential when there were
postponed tests.
Reviewed By: jungejason
Reviewers: epriestley, jungejason
CC: slawekbiel, aran, jungejason
Differential Revision: 417
Summary:
We changed our Facebook implementation to echo test results while the
tests are running. We do not want to echo the test results twice.
Test Plan:
Tested that implementing the function in PhutilUnitTestEngine and
returning true showed the results and returning false didn't show the
results.
Reviewed By: epriestley
Reviewers: jungejason, epriestley, grglr, slawekbiel
Commenters: slawekbiel, aran
CC: sgrimm, slawekbiel, aran, tuomaspelkonen, epriestley
Differential Revision: 400
Summary:
I typed the wrong number of spaces into some of these.
Test Plan:
Visual inspection, ran 'arc unit'
Reviewed By: tuomaspelkonen
Reviewers: tuomaspelkonen
CC: aran, tuomaspelkonen
Differential Revision: 402
Summary:
Test Engine classes might need the differential Diff ID to be
able to attach postponed test results to diffs.
Test Plan:
Added setDifferentialDiff function to PhutilUnitTestEngine class
and made sure it was called with the correct diff ID when running
'arc diff --preview'
Reviewed By: jungejason
Reviewers: jungejason, grglr
Commenters: sgrimm
CC: epriestley, sgrimm, slawekbiel, aran, tuomaspelkonen, jungejason
Differential Revision: 395
Summary:
Displaying each postponed test separately was spammy.
Now it will only display a single line with the total count
Test Plan:
~/local/arcanist/bin/arc unit --sandcastle-tests
35 tests to run.
Completed 'ParentInfoTestCase.php'
Interrupted 'MentionsUntagTestCase.php'
PASS ParentInfoTestCase
POSTPONED 34 tests
Reviewed By: sgrimm
Reviewers: jungejason, sgrimm
CC: sgrimm
Revert Plan:
sure
Other Notes:
Differential Revision: 173
Summary:
That's for the tests that will be run asynchronously.
Test Plan:
run arc unit with some tests that returned postponed status - they showed up.
Background was yellow.
Reviewed By: epriestley
CC: epriestley
Revert Plan:
sure
Other Notes:
Differential Revision: 77
Summary:
When you add new arguments to the unit command, they need to be
threaded through to the unit engine. This enables you to specify passthru
command which will be shipped through the callstack.
Test Plan:
Ran 'arc diff --maxtests 8 --apply-patches --trace' and verified
resulting behavior was correct.
Differential Revision: 207877
Reviewed By: dschleimer
Reviewers: dschleimer
CC: dschleimer
Revert Plan:
OK
