Last modified: 2014-10-29 23:06:40 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T74722, the corresponding Phabricator task for complete and up-to-date bug report information.
First Last Prev Next    This bug is not in your last search results.
Bug 72722 - Jenkins tests shouldn't go red when it's not its fault
Jenkins tests shouldn't go red when it's not its fault
Status: NEW
Product: Wikimedia
Classification: Unclassified
Quality Assurance (Other open bugs)
wmf-deployment
All All
: Unprioritized normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency tree / graph
 
Reported: 2014-10-29 23:01 UTC by Jon
Modified: 2014-10-29 23:06 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Description Jon 2014-10-29 23:01:24 UTC 
The MobileFrontend tests (in Firefox) have been flakey for some time. Take a look at the list on the left for examples of recurring issues.

Example:
https://integration.wikimedia.org/ci/view/Mobile/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/333/

I propose that when a test fails and the content of the Stacktrace contains one of
*Timeout::Error (Timeout::Error)
* Net::ReadTimeout 
* 500 error (Our servers are currently experiencing a technical...)
the test should be marked as unreliable and should not be considered failure.

Could the test go amber when this happens instead? It would allow for better visual scanning when things break.

If there are 2 timeout errors but one actual error the test should stay red as there is a genuine error there.
Comment 1 Chris McMahon 2014-10-29 23:06:40 UTC 
Sorry Jon, a big part of the problem is https://bugzilla.wikimedia.org/show_bug.cgi?id=72366

which should get fixed soonish
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.

Navigation
Links