Last modified: 2012-01-30 17:41:17 UTC
Right now all of the fuzz tests are being skipped due to excess memory usage. They should be given a new @group (was thinking MemoryHungry) and disabled by default. That way we can test them without killing ci.tesla :) (I may get to this in the next few days, just filing a bug so I don't forget or if anyone's feeling bored)
How memory hungry are we talking? Does the limit just need to be more appropriately set, or are they like runaway memory leak breakage territory that need to be fixed?
Not really memory hungry at all. The reason they crashed OOM was because testFuzzTests() went and set the memory limit for no real reason that I could tell. They actually fail for a completely different reason which I clarified in r90483. Repurposing bug based on that. Example output below: 1) ParserTests::testFuzzTests PDOException: You cannot serialize or unserialize PDO instances /www/phase3/tests/phpunit/includes/parser/NewParserTest.php:661 /www/phase3/tests/phpunit/includes/parser/NewParserTest.php:611 /www/phase3/tests/phpunit/MediaWikiTestCase.php:60 /www/phase3/tests/phpunit/MediaWikiPHPUnitCommand.php:20 /www/phase3/tests/phpunit/phpunit.php:60
Heh... looks like getMemoryBreakdown() attempts to estimate size of various objects by running them through serialization. This apparently fails in exciting ways with the PDO objects used by DatabaseSQLite, whereas I guess the MySQL resources we use get "serialized" without error. This gets called apparently between every 100 tests if we see that we've used up 90% of an arbitrary 50MB soft memory limit. So... if it's getting called, it would already have been on the way to throwing an exception anyway (which would come with the approx memory breakdown). Probably needs a fix to getMemoryBreakdown to not die on unserializable things, so it at least sends through the rest of the data, but it's still apparently hitting that limit as reported by memory_get_usage(true).
Doing fuzz tests in an infinite loop amongst other normal tests doesn't seem the right design.
Xdebug does not like this test! $ ./phpunit.php --tap --filter FuzzTests includes/parser/NewParserTest.php Fatal error: Maximum function nesting level of '200' reached, aborting! <snip: Call Stack> There is a faulty recursion somewhere in our code :-(
I've asked Antoine to do what Chad suggested in comment 1, which is to give it a @group and disable by default. There's lots of discussion about actually fixing the test which is a lower priority bug that I don't want to stick him with.
Fuzzing tests are now in the ParserFuzz group with r108547. Jenkins updated to exclude the group.
Closing bug since I have fixed it already :)
