1
0
Fork 0
mirror of https://we.phorge.it/source/arcanist.git synced 2024-11-15 03:12:40 +01:00
Commit graph

62 commits

Author SHA1 Message Date
David Cramer
81a1c09148 Add PytestTestEngine 2013-06-13 14:23:45 -07:00
epriestley
24d54a5fbd Use PhutilEditDistanceMatrix in ArcanistDiffUtils
Summary: Replace this old hard-coded implementation with the new vector-based, unicode-capable one.

Test Plan: Ran unit tests. Looked at revisions in Differential, using whitespace modes to bypass cache.

Reviewers: btrahan, Afaque_Hussain

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2379

Differential Revision: https://secure.phabricator.com/D6016
2013-05-23 14:36:42 -07:00
Aviv Eyal
0476bf8f4a more reusable nosetest engine
Summary:
The NoseTestEngine class has some usefull code for parsing nosetest, but it assumes a very specific code/test
layout. I've pulled this logic abit apart, which lets me reuse the nose-related parts with my existing
project layout, by having my custom engine just call runTests() with the relevant paths.

Test Plan: Run on a customized project with coverage, see coverage results.

Reviewers: epriestley, roman.barzyczak

Reviewed By: epriestley

CC: seporaitis, aran, Korvin, zeeg

Differential Revision: https://secure.phabricator.com/D5921
2013-05-14 06:38:22 -07:00
epriestley
d31385912f Document parameters
Summary: unlike the hooks I copy/pasted, these hooks have parameters

Test Plan: read it

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5264
2013-03-06 13:47:05 -08:00
epriestley
c827490708 Add willRunTestCases / didRunTestCases hooks
Summary: These hooks allow test cases to build shared resources -- notably, database fixtures.

Test Plan: See next diff.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5258
2013-03-06 13:40:07 -08:00
epriestley
5296bf529f Fix bug with arc unit --everything when coverage is available
Summary: We don't set $paths when running --everything.

Test Plan: Ran `arc unit --everything` with coverage.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5260
2013-03-06 13:39:31 -08:00
vrana
2e87419c7b Render unit results progressively
Summary:
Some tests take longer (fixtures usually around 1 second for me) and also FB runs all tests on deploy.
I want to see all results immediately.

Test Plan:
Added `usleep(200000)` to `resultTest()`, then:

  $ arc unit
  Saw results printed one by one.

Also didn't pass `$renderer` to `ArcanistPhutilTestCase` and saw empty output.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5141
2013-02-28 15:33:21 -08:00
epriestley
7a67173b97 Apply new whitespace lint rules to arcanist/
Summary: Automated changes from `arc lint`.

Test Plan: Looked them over.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5017
2013-02-19 14:09:20 -08:00
indiefan
ce0a491bcd Added base test result parser class. Implemented Go test result parser based on Go's built in test utility output.
Test Plan: Ran PhpunitTestEngine unit test and used both test result parsers to generate test results.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, aurelijus

Differential Revision: https://secure.phabricator.com/D4676
2013-01-27 17:17:09 -08:00
Aurelijus
df04c79a90 Support PHPUnit 3.7 message changes
Summary: PHPUnit 3.7 now includes user message as well.

Test Plan: Ran phpunit tests

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4662
2013-01-25 17:19:41 -08:00
epriestley
44e25495ca Raise a lint warning on array_combine(x, x)
Summary: Suggest array_fuse().

Test Plan:
Unit tests, and:

  $ arc lint --lintall src/unit/engine/phutil/ArcanistPhutilTestCase.php
  >>> Lint for src/unit/engine/phutil/ArcanistPhutilTestCase.php:

     Warning  (XHP37) array_combine() Unreliable
      Prior to PHP 5.4, array_combine() fails when given empty arrays. Prefer
      to write array_combine(x, x) as array_fuse(x).

               252     $callable,
               253     $exception_class = 'Exception') {
               254     return $this->tryTestCases(
      >>>      255       array_combine(array_keys($map), array_keys($map)),
               256       array_values($map),
               257       $callable,
               258       $exception_class);

Reviewers: btrahan, vrana, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D4659
2013-01-25 17:06:49 -08:00
indiefan
9946e23f9e Decoupling phpunit test parsing from running and gathering. Should enable third party test engines to reuse common business logic around phpunit without requiring common logic for the typically application-specific running and gathering.
Test Plan: Ran PhpunitTestEngine unit test. Also used refactored PhpunitTestEngine to run phpunit tests.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, aurelijus

Differential Revision: https://secure.phabricator.com/D4651
2013-01-25 17:06:38 -08:00
vrana
8142aab9d3 Fix whitespace around else 2013-01-16 12:15:46 -08:00
epriestley
716ce2482c Implement --everything, --json and --ugly flags for arc unit
Summary: Adds "arc unit --everything", which runs every available test, provided the test engine supports it. Also add JSON output.

Test Plan: Ran `arc unit --everything` in arcanist/, libphutil/ and phabricator/. Saw all tests run.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2065

Differential Revision: https://secure.phabricator.com/D4214
2012-12-17 12:56:41 -08:00
vrana
ec8c214ddf Change lint_engine to lint.engine
Summary: Also unit_engine.

Test Plan: None.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, aurelijus

Differential Revision: https://secure.phabricator.com/D4001
2012-11-20 15:42:35 -08:00
vrana
66d204be81 Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in unit tests and LICENSE file.

Reviewers: epriestley, btrahan, edward

Reviewed By: epriestley

CC: aran, Korvin, davidrecordon

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3881
2012-11-05 11:16:24 -08:00
epriestley
538115bd0e Use PhutilAggregateException to aggregate unit test exceptions
Summary: Currently, a shutdown exception ("script exited with open transactions!") overwhelms the actual test failure exception, which is the one that needs to be fixed.

Test Plan: Ran a fixture test which opens a transaction and then throws. Got diagnostically useful output after this patch.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3780
2012-10-22 11:41:33 -07:00
Stanislav Vorobyev
9c2348f2f4 Fix parsing of PHPUnit JSON log format for PHPUnit 3.7.7
Summary: See https://github.com/facebook/arcanist/pull/55

Reviewed by: epriestley
2012-10-15 11:07:50 -07:00
epriestley
777c119a0e Search for more test locations in PHPUnitTestEngine
Summary:
See https://github.com/facebook/arcanist/pull/53

  - Work correctly for directories with `%` in their name; this breaks under sprintf().
  - Search for `src/` -> `tests/` style directories.
  - Add coverage for search paths.

Test Plan:
Ran unit tests.

I don't have a working test case for PHPUnit tests, can one of you guys apply this and verify I didn't break your setups?

Reviewers: quard, aurelijus

Reviewed By: aurelijus

CC: aran

Differential Revision: https://secure.phabricator.com/D3558
2012-09-29 18:03:43 -07:00
vrana
fefa9e2d72 Use array_mergev() in Phutil test engine
Test Plan: This diff.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3491
2012-09-13 22:35:41 -07:00
vrana
cdb161a19a Allow Phutil tests setting link
Summary: Also delete some copy pasta.

Test Plan: Next diff.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3455
2012-09-07 15:42:50 -07:00
vrana
af31ee4ed0 Link Arcanist test cases
Summary: See D3455.

Test Plan: This diff (after rebase).

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3460
2012-09-07 15:31:14 -07:00
vrana
df81d25f1e Depend on events when attaching Diff ID to postponed unit tests
Summary:
According to @epriestley, it's nasty and kind of crazy: D2933#1.
It also stands in my way for D2614.

Test Plan: Rewrote our callsite to event listener and verified that it still works.

Reviewers: tuomaspelkonen, epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D3171
2012-08-08 12:04:15 -07:00
vissi
b28efb8f88 Allow running phpunit tests from any directory
Test Plan: Run `arc unit` anywhere.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D3017
2012-07-19 15:00:56 -07:00
vrana
23b3dd7e95 Display number of assertions in unit test details
Test Plan: Show Full Unit Results on this diff.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2777
2012-06-17 13:39:46 -07:00
vrana
08b53c3803 Pass for phutil test that was expected to fail and skip
Summary:
I just hope that we will not create `ArcanistPhutilTestCaseTestCaseTestCase`.

Also fix a copy/paste error (@edward's this time).

Test Plan: `arc unit`

Reviewers: epriestley

Reviewed By: epriestley

CC: edward, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2692
2012-06-08 14:29:51 -07:00
vrana
6bc4da8c4c Remove support for skipping tests
Summary: Almost revert D2673 but leave the support for 'repeat'.

Test Plan: `arc help unit`.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2689
2012-06-08 13:42:12 -07:00
vrana
6433f7ff3d Allow skipping unit tests and support 'repeat' workflow arguments
Summary:
We run all tests before deploy but some of them are expected to fail.
We need to skip them.
It can be useful also for someone else.

I don't propagate this to `arc diff` as that would be much more complicated - we would need a new state 'partially skipped'.

The 'path' argument needs to be a file, skipping whole dirs is not supported.
The 'path' argument is in fact engine specific and engines can support format like 'Class::testMethod' but I didn't bother to document it to not confuse people.

Test Plan:
`arc unit --skip test.php`
`arc unit --skip ../src/test.php`

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2673
2012-06-07 17:42:49 -07:00
vrana
52f8f49aef Use project root instead of library root in unit engine 2012-06-05 21:08:59 -07:00
vrana
0b45ec30be Move files in Arcanist one level up
Summary:
- `kill_init.php`
- Manually change library map.
- Manually rename `/data/` test dirs.
- [src/lint/linter] `git mv base/ArcanistLinterTestCase.php __tests__/`
- `arc liberate`

Test Plan: Browse around to make sure I like it better, especially `repository/api`, and `workflow`.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2637
2012-06-01 11:56:00 -07:00
vrana
29c8ff9ddf Fix arc unit when running from non-root directory
Summary:
`$this->getPaths()` gives us paths relative to repository root.
We resolve them relative to cwd.

Test Plan: [src] `arc unit difference`

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2634
2012-06-01 11:19:52 -07:00
epriestley
e214ea7b18 Run phutil unit tests in v2 libraries
Summary:
Move away from setModule(), to setPathPrefix(). Also simplify test location/selection.

Depends on D2621.

Test Plan: Ran "arc unit".

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2622
2012-05-31 10:40:35 -07:00
vrana
f288ee0e21 Depend on autoloading
Test Plan:
  arc lint

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2609
2012-05-30 16:02:36 -07:00
epriestley
71afde1988 Upgrade arcanist to libphutil v2
Summary: Mechanical changes from D2588. No "Class.php" moves yet because they aren't necessary for libraries to function.

Test Plan: See D2588.

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2589
2012-05-30 14:22:59 -07:00
Aurelijus
2f3f45614b Simplify phpunit test names
Summary:
More & more use cases come up with empty suite names, guessing
and making test names nice is mess. Will leave it as it is, except
data set part, as it could be super long.

Test Plan: - Try running some phpunit tests with & without data sets

Reviewers: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2544
2012-05-23 11:55:01 +02:00
Aurelijus
6e87de7304 PHPUnit test name handling improvements
Summary:
Better test name handling for tests with data sets.
Instead of showing test name with full data set, show it's id/name,
e.g. `testConstructor with data set #1`

Test Plan: - Try running tests with data sets

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2535
2012-05-22 09:13:43 -07:00
Aurelijus
4f739014ae PHPUnit test engine
Summary:
PHPUnit wrapper for arc.
The idea here is simple - find the test case which is related to the updated
class file. Generate tmp files for json & clover reports, run phpunit
with provided arguments.
It supports phpunit configuration file setting in `.arcconfig`: `phpunit_config`.
Path should be relative to project root.

Test Plan:
- Set `unit_engine` to `PhpunitTestEngine`
- Try running tests with & without `phpunit_config` option.

Reviewers: epriestley, davidreuss

Reviewed By: epriestley

CC: aran, Koolvin, jungejason

Differential Revision: https://secure.phabricator.com/D2472
2012-05-22 07:12:03 -07:00
Julius Seporaitis
cb051d8568 Wrapper for 'nose' unittest and code coverage tools.
Summary:
Wrapper for Python 'nose' (http://readthedocs.org/docs/nose/en/latest/)
testing tool.

Test Plan:
Install latest 'nose' v1.1.3. Currently it is available through
Github only (``pep install git+https://github.com/nose-devs/nose.git``).

Create a Python project with following structure:

  /package_name/module_name.py
  /tests/package_name/test_module_name.py

Write some tests

Run ``arc unit``

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin, zeeg

Differential Revision: https://secure.phabricator.com/D2322
2012-04-27 12:12:20 -07:00
epriestley
dd11fee5ef Fix PhutilUnitTest issue with symlinks that point into a libphutil library
Summary: Currently, if you change a symlink outside a libphutil library and the link target is something inside a libphutil library, we may enter an inifite loop in the "do { ... } while(...)" later. Just bail if the loop won't resolve.

Test Plan: Ran arc unit, Airtime reported the issue resolved by a similar fix.

Reviewers: cpiro, btrahan

Reviewed By: cpiro

CC: aran

Differential Revision: https://secure.phabricator.com/D2318
2012-04-25 16:13:04 -07:00
Edward Speyer
946a9e44a3 Allow tests to be skipped
Summary:
Allow tests to be skipped by calling assertSkipped().  It's not really
an assertion of anything tangible; more like "assert that we can't
really assert anything right now".

Test Plan: Added a new test to the PhutilUnitTestEngineTestCase.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2312
2012-04-24 21:45:22 -07:00
Edward Speyer
dd6ffa4a13 [Tests] Only use concrete TestCases
Summary:
Don't use abstract subclasses of ArcanistPhutilTestCase, only use
concrete ones.

This lets you put common functionality in an abstract BaseTestCase
(which itself is a subclass of ArcanistPhutilTestCase), then implement
concrete subclasses of the BaseTestCase.

Test Plan: Tested with a simple Base -> {Case1, Case2} setup.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2300
2012-04-23 10:40:56 -07:00
vrana
8971a91444 Return $this from setters
Summary:
Most setters returns `$this` but some don't.
I guess it's not by purpose.

Test Plan:
  arc lint

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2084
2012-04-02 18:47:59 -07:00
epriestley
9db7f294ef Show a better expected vs actual for tests with large output
Summary: When tests have a lot of output, show a diff of the expected/actual.

Test Plan: Used this when developing D2016 to examine large output usefully.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2017
2012-03-26 09:43:47 -07:00
Edward Speyer
bdfc021d39 [ArcanistPhutilTestCase] add ->assertException() to the base test case
Summary: Simpler assert function for asserting a type of exception was raised.

Test Plan: Wrote this for (and tested it with) D1836.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1837
2012-03-08 15:14:30 -08:00
epriestley
64ada40f03 Raise a better error from PhutilUnitTestEngine when trying to use one copy of libphutil to test another copy
Summary:
We currently raise a very confusing error when we hit this case:

  Exception: The phutil library '' has not been loaded!

Because of the trickiness of init-order stuff, it's difficult to detect this more explicitly earlier -- instead, just raise a more descriptive error.

Test Plan: Ran "arc unit" in a copy of libphutil other than the one arc loads.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T580

Differential Revision: https://secure.phabricator.com/D1760
2012-03-05 10:03:13 -08:00
epriestley
6bb43de4bb Add simple, non-magical exception flow to test cases
Summary: Add a boring, simple construct for handling exception-based test cases.

Test Plan: Ran unit tests.

Reviewers: btrahan, tuomaspelkonen, nh, jungejason, vrana

Reviewed By: vrana

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1562
2012-02-06 09:56:43 -08:00
epriestley
4f07c3c8fd Add coverage support to Arcanist
Summary:
Add "--coverage" and "--no-coverage" flags, mechanisms for reporting
coverage information, xdebug coverage support, and CLI coverage reports.

Test Plan: Ran coverage locally.

Reviewers: tuomaspelkonen, btrahan, jungejason

Reviewed By: btrahan

CC: zeeg, aran, epriestley

Maniphest Tasks: T140

Differential Revision: https://secure.phabricator.com/D1526
2012-01-31 12:07:19 -08:00
epriestley
8fe38f8b6d Finalize Arcanist Classes
Summary:
Mark all applicable Arcanist classes as "final", except PhutilLintEngine, which
needs a little finesse.

@jungejason / @nh, does this break any Facebook stuff?

Test Plan: Linter no longer raises warnings. Ran "testEverythingImplemented" in
Phabricator.

Reviewers: btrahan, jungejason, nh

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T795

Differential Revision: https://secure.phabricator.com/D1519
2012-01-31 12:07:05 -08:00
epriestley
91d273a7dd Slightly improve Arcanist unit test output messages
Summary:
  - Show file/line so you can tell which assertion failed if there's a block
with a zillion of them and they don't have messages.
  - Try to format stuff a little better.

Test Plan:   - Ran some failing unit tests.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, btrahan

Differential Revision: https://secure.phabricator.com/D1304
2012-01-04 14:14:34 -08:00
epriestley
99717090ef Avoid infinite loop in 'arc unit'
Summary: If you modify a file at the root level (like the celerity map) we run
the dir/path checks in the wrong order and fail to detect that we can't possibly
find any modules. This leads to an infinite loop inside the while loop below.

Test Plan: Ran "arc unit" on a change which affects the celerity map.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, btrahan

Differential Revision: 1146
2011-12-01 08:50:12 -08:00