Last modified: 2014-07-21 13:13:44 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T66957, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 64957 - Mark flaky tests in Jenkins
Mark flaky tests in Jenkins
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Quality Assurance (Other open bugs)
wmf-deployment
All All
: Normal enhancement (vote)
: ---
Assigned To: Antoine "hashar" Musso (WMF)
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-05-06 16:41 UTC by Greg Grossmeier
Modified: 2014-07-21 13:13 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Greg Grossmeier 2014-05-06 16:41:19 UTC
From the QA mailing list today:

<quote name="Jon Robson" date="2014-05-01" time="10:00:38 -0700">
> When I visit the dashboard at
> https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome/
> and click on this
> https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome/416/
> I can see that there was one test failure. When I review the test I
> notice this was due to an issue with betalabs and shouldn't have
> failed as the test wasn't valid.
>
> Is there anyway we could update the UI so we could mark these tests
> yellow or some other colour to show they are a false positive. It
> would be great to be able to manually review these tests and save them
> being reviewed more than once...
Comment 1 Antoine "hashar" Musso (WMF) 2014-05-07 08:08:23 UTC
This comes from a thread on the qa mailing list http://lists.wikimedia.org/pipermail/qa/2014-May/001412.html

Rob Lanphier found out a Jenkins plugin that seems to fulfill the need (Claim Build plugin): http://lists.wikimedia.org/pipermail/qa/2014-May/001420.html

https://wiki.jenkins-ci.org/display/JENKINS/Claim+plugin
Comment 2 Gerrit Notification Bot 2014-05-07 08:09:52 UTC
Change 131940 had a related patch set uploaded by Hashar:
Allow user to claim failed build

https://gerrit.wikimedia.org/r/131940
Comment 3 Gerrit Notification Bot 2014-05-07 08:12:56 UTC
Change 131940 merged by jenkins-bot:
Allow user to claim failed build

https://gerrit.wikimedia.org/r/131940
Comment 4 Antoine "hashar" Musso (WMF) 2014-05-07 08:13:29 UTC
Jobs updated. Pending verification and we might want to document the feature somewhere.
Comment 5 Jon 2014-05-07 17:00:50 UTC
Thanks guys for such the quick response. I currently can't work out how to claim older failed jobs so I can't test it - is this possible?

What is the workflow?

I went to https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/541/ but I don't see a claim link there.

If I visit https://integration.wikimedia.org/ci/claims/? I see two things that can be claimed - will future failed jobs show up here?
Comment 6 Antoine "hashar" Musso (WMF) 2014-05-07 19:34:10 UTC
The Claim plugin is only allowed on the Wikimedia instances and only on the browser tests jobs listed at https://integration.wikimedia.org/ci/view/BrowserTests/

I haven't noticed the link https://integration.wikimedia.org/ci/claims/ , that it is a nice finding :-)

I lost my access to Cloudbees and we will get all jobs migrated whenever Zeljkof comes back.  So I am not sure whether it is worth the effort to install Claim there (if at all possible).


Once you got some experience with claiming builds, would you like to advertise it to the whole engineering and dev community?  It might have usages beyond browsertests.
Comment 7 Jon 2014-05-07 21:00:53 UTC
Okay.. I see how this works but until we move MobileFrontend away from Cloudbees and generate emails from integration.wikimedia.org it's near impossible for me to provide useful feedback on how this works. I'll wait patiently :)

Thanks for the work so far!
Comment 8 Antoine "hashar" Musso (WMF) 2014-07-21 13:13:44 UTC
The browser tests have been migrated out of Cloudbees to Wikimedia Jenkins.  They should all have the claim plugin enabled now.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links