1
0
Fork 0
mirror of https://we.phorge.it/source/arcanist.git synced 2024-11-25 16:22:42 +01:00
Commit graph

171 commits

Author SHA1 Message Date
epriestley
d8b510596f Don't show "not executable" for files with no coverage information
Summary: We incorrectly merge array() into empty string, which is later interpreted as "this file is entirely not-executable". Instead, show no coverage information in the UI.

Test Plan: Looked at a README diff with no coverage information, got no UI render.

Reviewers: tuomaspelkonen, btrahan, zeeg

Reviewed By: tuomaspelkonen

CC: aran, epriestley

Maniphest Tasks: T965

Differential Revision: https://secure.phabricator.com/D1863
2012-03-12 17:04:51 -07:00
Edward Speyer
bdfc021d39 [ArcanistPhutilTestCase] add ->assertException() to the base test case
Summary: Simpler assert function for asserting a type of exception was raised.

Test Plan: Wrote this for (and tested it with) D1836.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1837
2012-03-08 15:14:30 -08:00
epriestley
64ada40f03 Raise a better error from PhutilUnitTestEngine when trying to use one copy of libphutil to test another copy
Summary:
We currently raise a very confusing error when we hit this case:

  Exception: The phutil library '' has not been loaded!

Because of the trickiness of init-order stuff, it's difficult to detect this more explicitly earlier -- instead, just raise a more descriptive error.

Test Plan: Ran "arc unit" in a copy of libphutil other than the one arc loads.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T580

Differential Revision: https://secure.phabricator.com/D1760
2012-03-05 10:03:13 -08:00
epriestley
6bb43de4bb Add simple, non-magical exception flow to test cases
Summary: Add a boring, simple construct for handling exception-based test cases.

Test Plan: Ran unit tests.

Reviewers: btrahan, tuomaspelkonen, nh, jungejason, vrana

Reviewed By: vrana

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1562
2012-02-06 09:56:43 -08:00
epriestley
4f07c3c8fd Add coverage support to Arcanist
Summary:
Add "--coverage" and "--no-coverage" flags, mechanisms for reporting
coverage information, xdebug coverage support, and CLI coverage reports.

Test Plan: Ran coverage locally.

Reviewers: tuomaspelkonen, btrahan, jungejason

Reviewed By: btrahan

CC: zeeg, aran, epriestley

Maniphest Tasks: T140

Differential Revision: https://secure.phabricator.com/D1526
2012-01-31 12:07:19 -08:00
epriestley
8fe38f8b6d Finalize Arcanist Classes
Summary:
Mark all applicable Arcanist classes as "final", except PhutilLintEngine, which
needs a little finesse.

@jungejason / @nh, does this break any Facebook stuff?

Test Plan: Linter no longer raises warnings. Ran "testEverythingImplemented" in
Phabricator.

Reviewers: btrahan, jungejason, nh

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T795

Differential Revision: https://secure.phabricator.com/D1519
2012-01-31 12:07:05 -08:00
epriestley
91d273a7dd Slightly improve Arcanist unit test output messages
Summary:
  - Show file/line so you can tell which assertion failed if there's a block
with a zillion of them and they don't have messages.
  - Try to format stuff a little better.

Test Plan:   - Ran some failing unit tests.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, btrahan

Differential Revision: https://secure.phabricator.com/D1304
2012-01-04 14:14:34 -08:00
epriestley
99717090ef Avoid infinite loop in 'arc unit'
Summary: If you modify a file at the root level (like the celerity map) we run
the dir/path checks in the wrong order and fail to detect that we can't possibly
find any modules. This leads to an infinite loop inside the while loop below.

Test Plan: Ran "arc unit" on a change which affects the celerity map.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, btrahan

Differential Revision: 1146
2011-12-01 08:50:12 -08:00
epriestley
994163d9ca When running phutil tests from "arc unit", trigger __tests__ in all parent
modules

Summary:
Right now, if you add a file to "__tests__/data/blah.testcase" it doesn't
trigger those __tests__, but it should. Trigger __tests__ in all parent modules
when a file changes.

(We could be a little more complex about choosing what to run but all the tests
are super fast so it hardly matters if we run "too many" tests.)

Test Plan: Ran "arc unit" on this, got a test trigger. Added a file to a
__tests__/data/ directory and got a test trigger. Added some var_dump() and
spot-checked things for sanity.

Reviewers: btrahan, jungejason, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 1127
2011-11-20 14:21:46 -08:00
epriestley
a8b4231244 Provide more information when unit tests fail with exceptions
Summary:
When a unit test throws an exception, provide more data (type, trace) in the
test failure message.

Previously, we would show only the message itself, which may not be very useful
in debugging test failures.

Test Plan: Ran "arc unit" on a test which throws, got a stack trace.

Reviewers: edward, btrahan, jungejason, nh, tuomaspelkonen, aran

Reviewed By: nh

CC: aran, nh

Differential Revision: 1123
2011-11-17 11:29:48 -08:00
James Ide
2892c6ef1a Have Arcanist record how much of my life was spent running each unit test
Summary:
Keeping unit tests speedy keeps them useful since people actually won't mind
running them. This diff records the time taken by each test and displays it nice
and colorized. Really, I just want to discourage non-unit tests from making
their way into ##__tests__##.

Some thoughts:

 - The "acceptableness" times are subjective but if dependencies are properly
mocked the times seem to be ok. Integration tests that make network requests to
third-party endpoints and pull in megabytes of data will not survive. This is a
good thing.
 - Fast tests get a gold star, encouraging small tests. I am sorry that the star
does not sparkle.
 - There is no way for a programmer to admit that their test is going to be slow
in some cases. They will be shamed with red text for the life of their test.
 - It might be confusing that fast but failing tests get green text and maybe a
gold star.

Test Plan: Ran some of the unit tests within Arcanist and libphutil. See
https://secure.phabricator.com/file/view/PHID-FILE-cdd3c94c219e0fd7470b/ for
sample output.
Reviewed By: epriestley
Reviewers: epriestley
CC: aran, epriestley
Differential Revision: 588
2011-07-03 23:32:27 -07:00
tuomaspelkonen
42c6f00315 Let TestEngine know if the command that triggered tests is 'arc diff' or 'arc
unit'

Summary:
We want to handle 'arc unit' and 'arc diff' differently in our test
framework.

Test Plan:
Tested that with 'arc unit' the value was set to 'unit' and with 'arc diff'
the value was set to 'diff'.

Reviewed By: epriestley
Reviewers: slawekbiel, epriestley
CC: jungejason, grglr, aran, tuomaspelkonen, epriestley
Differential Revision: 430
2011-06-10 13:00:03 -07:00
tuomaspelkonen
9b7ee674eb Child classes can prevent echoing test results.
Summary:
We changed our Facebook implementation to echo test results while the
tests are running. We do not want to echo the test results twice.

Test Plan:
Tested that implementing the function in PhutilUnitTestEngine and
returning true showed the results and returning false didn't show the
results.

Reviewed By: epriestley
Reviewers: jungejason, epriestley, grglr, slawekbiel
Commenters: slawekbiel, aran
CC: sgrimm, slawekbiel, aran, tuomaspelkonen, epriestley
Differential Revision: 400
2011-06-07 11:01:51 -07:00
tuomaspelkonen
fe9daa7ae3 'arc diff' passes the diff ID to Test Engine class.
Summary:
Test Engine classes might need the differential Diff ID to be
able to attach postponed test results to diffs.

Test Plan:
Added setDifferentialDiff function to PhutilUnitTestEngine class
and made sure it was called with the correct diff ID when running
'arc diff --preview'

Reviewed By: jungejason
Reviewers: jungejason, grglr
Commenters: sgrimm
CC: epriestley, sgrimm, slawekbiel, aran, tuomaspelkonen, jungejason
Differential Revision: 395
2011-06-02 12:19:05 -07:00
epriestley
17ac4d9ab1 Provide setup/teardown hooks for ArcanistPhutilTestCase
Summary:
I'm making a move on Lisk stubbability and need these so I can set up
Lisk isolation unconditionally in Phabricator tests. Also document and organize
this class.

Test Plan:
Changed the expected counts in the meta-tests and got failures. Ran
all unit tests.

Reviewed By: aran
Reviewers: aran, jungejason, tuomaspelkonen
CC: aran
Differential Revision: 192
2011-04-30 12:04:11 -07:00
slawekbiel
aa44db2f96 [Adding postpone status for unittests]
Summary:
That's for the tests that will be run asynchronously.

Test Plan:
run arc unit with some tests that returned postponed status - they showed up.
Background was yellow.

Reviewed By: epriestley
CC: epriestley
Revert Plan:
sure

Other Notes:

Differential Revision: 77
2011-03-21 19:11:58 -07:00
epriestley
5099b005cf Some documentation.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-19 11:36:08 -08:00
epriestley
83a0083b3b Improve argument passthru behavior to make added args available to test engines.
Summary:
When you add new arguments to the unit command, they need to be
threaded through to the unit engine. This enables you to specify passthru
command which will be shipped through the callstack.

Test Plan:
Ran 'arc diff --maxtests 8 --apply-patches --trace' and verified
resulting behavior was correct.

Differential Revision: 207877
Reviewed By: dschleimer
Reviewers: dschleimer
CC: dschleimer
Revert Plan:
OK
2011-02-01 20:45:32 -08:00
epriestley
3f13e36182 Update arcanist to use the PhutilSymbolLoader.
Summary: This should also fix the bug with double help for certain commands

Test Plan:

Reviewers:

CC:
2011-01-12 16:02:28 -08:00
epriestley
7d36705215 Ship unresolved unit test information to Differential.
Summary:

Test Plan:

Reviewers:

CC:
2011-01-11 22:37:06 -08:00
epriestley
2e73916fa2 Initial commit. 2011-01-09 15:22:25 -08:00