Last modified: 2011-08-23 18:11:37 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T32520, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 30520 - qunit tests for wrong expected results do not fail on certain browsers ( false positive ). Should say "test fails" on all browsers
qunit tests for wrong expected results do not fail on certain browsers ( fals...
Status: RESOLVED INVALID
Product: MediaWiki
Classification: Unclassified
Unit tests (Other open bugs)
unspecified
All All
: High normal (vote)
: ---
Assigned To: Nobody - You can work on this!
http://www.mediawiki.org/wiki/Special...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-08-22 22:23 UTC by T. Gries
Modified: 2011-08-23 18:11 UTC (History)
3 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description T. Gries 2011-08-22 22:23:58 UTC
during tests for http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/tests/qunit/suites/resources/jquery/jquery.highlightText.test.js?view=markup&pathrev=95262 I found by chance that

- a wrong expected result may no always result in "test fails" on Firefox or Chrome

This is documented in the comments on http://www.mediawiki.org/wiki/Special:Code/MediaWiki/95262#c21311 .
Comment 1 Roan Kattouw 2011-08-22 22:42:54 UTC
Isn't that a false positive? As in, the test comes out positive while it's really not?
Comment 2 T. Gries 2011-08-22 22:48:29 UTC
(In reply to comment #1)
> Isn't that a false positive? As in, the test comes out positive while it's
> really not?

When using the Firefox (or Chrome), test cases 100 and 101 are *not* detected, which is the problem. 

Both test cases should fail on all browsers, but do not.
Comment 3 Roan Kattouw 2011-08-22 23:17:04 UTC
Yup, so that's a false positive. Tweaking summary.
Comment 4 Brion Vibber 2011-08-23 17:53:37 UTC
So... it doesn't seem to be a false positive, as far as I can tell it's a true positive.

What makes it look like a false positive exactly, Tom?

Your expected text *is* bogus, but gets parsed the way we'd reasonably expect:

  <nowiki></span this wrong tag is not detected as test failure in Firefox 6.0 or Chrome></nowiki>

gets parsed into the expected actual structure:

  <nowiki></span></nowiki>

which of course matches the

  <nowiki></span></nowiki>

that we'd expect. This doesn't look like a useful test case at all; are you trying to test the browser's HTML fragment parser? This won't be exercised in real usages -- you're not even testing on the output, you're testing on your given sample text.

Recommend removing these test cases for being useless.
Comment 5 Brion Vibber 2011-08-23 17:54:08 UTC
bah pretend those <nowiki> bits are not there. I can't remember where I'm commenting anymore. :P
Comment 6 T. Gries 2011-08-23 17:56:40 UTC
(In reply to comment #5)
> bah pretend those <nowiki> bits are not there. I can't remember where I'm
> commenting anymore. :P

So you want to say: everything is fine, when I remove test "cases" 100 and 101 ?
Comment 7 Brion Vibber 2011-08-23 18:00:09 UTC
Removed them in r95315.

Thomas, if you need to do quick 'supposed to fail' test case, be sure to put in something that won't normalize back to the real expected value. For instance add some new elements, or add some text nodes which will be preserved after HTML parsing.

But of course don't commit such a test case, since you wouldn't want failing test cases for no reason.
Comment 8 T. Gries 2011-08-23 18:11:37 UTC
(In reply to comment #7)
> Removed them in r95315.
> 
> Thomas, if you need to do quick 'supposed to fail' test case, 

(My test cases 100 and 101 were simply added by mistake. resolved-invalid, luckily.)

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links