Last modified: 2014-08-20 12:12:08 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T53461, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 51461 - JJB: generates duplicates jobs with different parameters
JJB: generates duplicates jobs with different parameters
Status: NEW
Product: Wikimedia
Classification: Unclassified
Continuous integration (Other open bugs)
wmf-deployment
All All
: Low normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-16 19:04 UTC by Antoine "hashar" Musso (WMF)
Modified: 2014-08-20 12:12 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Antoine "hashar" Musso (WMF) 2013-07-16 19:04:37 UTC
Daniel Kinzler wrote:

There seems to be an issue with Jenkins. It appears to use an old version of
other extensions under some circumstances.

It's like this:

If you submit change 33 for extension A, which needs change 44 in extension B
(which isn't merged yet), jenkins will fail correctly fail.

BUT: When change 44 got merged into extension B, and you force Jenkins to re-run
(e.g. by rebasing change 33), it will *still* fail, apparently using an old
version of extension B.

It seems this is only the case for the "testextensions-master" job, not the
standalone "repo" and "client" jobs.

Here are some examples:

https://gerrit.wikimedia.org/r/#/c/72962/ fails for no good reason
https://integration.wikimedia.org/ci/job/mwext-Wikibase-testextensions-master/3099/console

https://gerrit.wikimedia.org/r/#/c/73772/ fails for no good reason
https://integration.wikimedia.org/ci/job/mwext-Wikibase-testextensions-master/3093/console

Please gather more evidence/insights if you come across this issue.

Thanks,
Daniel
Comment 1 Antoine "hashar" Musso (WMF) 2013-07-16 19:05:44 UTC
The issue must be related to the script fetching out the MediaWiki dependencies.  It might not get the laster master version, possibly because it is just cloning the dependency and the workspace is never wiped.
Comment 2 Antoine "hashar" Musso (WMF) 2013-07-16 19:07:18 UTC
will work on that tomorrow morning. I stay out of comp tonight sorry.
Comment 3 Antoine "hashar" Musso (WMF) 2013-07-16 19:32:10 UTC
I found out the issue. The dependencies are no more fetched :(
Comment 4 Antoine "hashar" Musso (WMF) 2013-07-16 19:34:00 UTC
Seems to be an issue in Jenkins Job builder:

$ jenkins-jobs --conf jenkins_jobs.ini update config/ mwext-Wikibase-testextensions-master
INFO:root:Updating jobs in config/ (['mwext-Wikibase-testextensions-master'])
INFO:jenkins_jobs.builder:Reconfiguring jenkins job mwext-Wikibase-testextensions-master
INFO:jenkins_jobs.builder:Reconfiguring jenkins job mwext-Wikibase-testextensions-master
$

It generates two version of the job, one of them must be lacking the dependencies :/
Comment 5 Antoine "hashar" Musso (WMF) 2013-07-16 20:50:02 UTC
Jenkins jobs indeed build the job twice, at first with:

 <command>/var/lib/jenkins/tools/fetch-mw-ext</command>

Then another time with the proper command:

 <command>/var/lib/jenkins/tools/fetch-mw-ext Ask,Serialization,Diff,DataValues,WikibaseDataModel,Validator</command>


Jenkins Jobs always proceed them in that order so the job always ends up with the dependencies. BUT, whenever the job generation is aborted (and I did so on 2013-06-26_14-46-02), we end up with missing dependencies :/

The history link:
https://integration.wikimedia.org/ci/job/mwext-Wikibase-testextensions-master/jobConfigHistory/showDiffFiles?timestamp1=2013-07-16_19-35-57&timestamp2=2013-07-16_19-36-05

All the past configurations for the job:

https://integration.wikimedia.org/ci/job/mwext-Wikibase-testextensions-master/jobConfigHistory/?

You will notice that there are double entries a few seconds apart, that highlight the fact that Jenkins Job Builder regenerate the job twice.


Workaround: make sure both versions are pushed (ie do not abort the run)

Goal: fix up jenkins job builder to handle that and only push the second (last) version.


I have lowered the bug priority since it is fixed but I keep this bug open to fix up Jenkins Job Builder.
Comment 6 Antoine "hashar" Musso (WMF) 2013-07-16 20:52:58 UTC
Sorry the issue apparently started on 2013-07-11_08-53-39
Comment 7 Antoine "hashar" Musso (WMF) 2013-07-26 10:20:40 UTC
This is an issue in how we craft our jobs in Jenkins Job Builder.   The workaround is to make sure the JJB command refresh the duplicate jobs.

Unassigning self for now, will revisit later on.
Comment 8 Antoine "hashar" Musso (WMF) 2014-08-20 12:12:00 UTC
This is still occurring.  But there are now less jobs being duplicated (once with defaults, and the second time with a variable set).  Clean up is https://gerrit.wikimedia.org/r/#/c/155249/

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links