Last modified: 2012-03-16 08:20:02 UTC
In some circumstances (at least: make test and using --filter), the test objects are constructed/destructed more than once. This seems unnecessary and leads to issues like the long-standing BlockTest failures that we finally worked around. Couple of points: 1) This doesn't happen all the time. Running a test directly via ./phpunit.php path/to/test.php doesn't seem to trigger the multiple construction issue. 2) It becomes really cumbersome in setUp() and tearDown() to ensure we're beginning with a sane test environment [and leaving a clean one for any subsequent tests]. 3) This may be related to deeper architectural issues in MediaWiki - I know for example the flipping of $wgDBprefix during setUp() is very finicky and I've broken it a lot of times. We may be making some incorrect assumptions about global states while working from a CLI environment...I know phpunit doesn't like globals very much. Cf: bug 9968, bug 23307 and a dozen others. 4) This may be rooted in "how phpunit does things," which ties into #3 above. We may just be trying to test a setup that phpunit isn't designed to test. 5) This is definitely not DBMS-specific. I've experienced the issue on both Mysql and Sqlite. I've done some profiling and nothing immediate has stood out at me. This may end up being a tracker bug for various other issues as I uncover them, but I just wanted to get my initial observations down in case anyone else was feeling adventurous.
I tried adding a quick hack to force output on stdout every time BlockTest::setUp() gets run; whether I run individually or with the suite.xml and with --filter it gets run the expected eight times, one for each test on the test class (two single-run tests, two that run on test sets of 3 items each): brion@stormcloud:~/pages/trunk/tests/phpunit$ php phpunit.php includes/BlockTest.php PHPUnit 3.5.13 by Sebastian Bergmann. BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp . Time: 11 seconds, Memory: 44.50Mb OK (8 tests, 15 assertions) brion@stormcloud:~/pages/trunk/tests/phpunit$ php phpunit.php includes/BlockTest.php ^C brion@stormcloud:~/pages/trunk/tests/phpunit$ php phpunit.php --configuration=suite.xml --filter=BlockTest PHPUnit 3.5.13 by Sebastian Bergmann. .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp .BlockTest::setUp . Time: 9 seconds, Memory: 84.00Mb OK (9 tests, 19 assertions) Note that in this second way something other than BlockTest is also running, which presumably gives the extra test and extra 4 assertions...
Chad do you still have this issue ? I can not reproduce it. Maybe it was fixed upstream?
I have never been able to reproduce this issue so I am marking this bug as working for me. If we ever have this issue again, we will need some way to reproduce it.
