Last modified: 2013-12-06 13:56:30 UTC
This makes sense for ULS because they have a reasonable number of merges in a day, see https://gerrit.wikimedia.org/r/#/q/status:merged+project:mediawiki/extensions/UniversalLanguageSelector,n,z This would entail: * Put the ULS tests into their own Jenkins build, and possibly into their own code repo the way Mobile is * Have Jenkins kick off the build of ULS tests targeting beta labs upon code merged in the ULS branch * Report the build status after each run This came up because we found Bug 52115 in a timely way.
Is this still something we want to do? ULS tests are now in ULS repo, so this should be doable. The only problem is that ULS tests are not stable enough: https://wmf.ci.cloudbees.com/view/r-uls/
What about running the tests on patch submission?
Have added bug 53691 for the same for the VisualEditor repo.
Moving this under Continuous Integration radar.
Following a pair session with Željko, the tests can be flagged with a tag which we can then exclude when running tests. For ULS, I have introduced the tag @specific-settings which let us skip any tests that are not going to pass on a fresh wiki installation: https://gerrit.wikimedia.org/r/#/c/97487/
The browser tests manage to pass with above change: https://integration.wikimedia.org/ci/job/mwext-browsertests-UniversalLanguageSelector-phantomjs/27/consoleFull With a nice HTML report: https://integration.wikimedia.org/ci/job/mwext-browsertests-UniversalLanguageSelector-phantomjs/27/artifact/report.html Whenever the change in comment #5 is merged in, I will validate with i18n team how to get it triggered by Zuul.
Change 97741 had a related patch set uploaded by Hashar: ULS browsertests on check pipeline https://gerrit.wikimedia.org/r/97741
Change 97741 merged by jenkins-bot: ULS browsertests on check pipeline https://gerrit.wikimedia.org/r/97741
The trigger has been added in Zuul with https://gerrit.wikimedia.org/r/97741 The job will success whenever ULS change https://gerrit.wikimedia.org/r/#/c/97487/ is merged in.
The job got fixed yesterday and browser tests are passing right now. It is not blocking changes yet though.
