Last modified: 2013-12-17 20:05:59 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T49120, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 47120 - [upstream] Jenkins is slow to start
[upstream] Jenkins is slow to start
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Continuous integration (Other open bugs)
wmf-deployment
All All
: Normal normal (vote)
: ---
Assigned To: Nobody - You can work on this!
incident-report
: upstream
Depends on: 47744
Blocks:
  Show dependency treegraph
 
Reported: 2013-04-11 10:09 UTC by Antoine "hashar" Musso (WMF)
Modified: 2013-12-17 20:05 UTC (History)
4 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
jstack trace taken while Jenkins is busy reading files (67.95 KB, text/plain)
2013-04-11 10:09 UTC, Antoine "hashar" Musso (WMF)
Details

Description Antoine "hashar" Musso (WMF) 2013-04-11 10:09:15 UTC
Created attachment 12076 [details]
jstack trace taken while Jenkins is busy reading files

Whenever restart Jenkins it takes roughly half an hour to start up. Running strace on the process it is reading all the build histories.

I have attached a jstack taken during startup. An interesting thread is 25437.

So I guess whenever Jenkins start a project, it ends up reading all the build history for all projects which is definitively a slow operation given the number of builds logs we keep.
Comment 1 Antoine "hashar" Musso (WMF) 2013-04-17 13:32:13 UTC
Seems like upstream has been working on it recently.

The issue  https://issues.jenkins-ci.org/browse/JENKINS-8754 "Improve Start-up Time" has some information about it.

1.485 seems to implements lazy loading.  So I guess we will get it with the next LTS release.
Comment 2 Antoine "hashar" Musso (WMF) 2013-05-02 14:28:26 UTC
Jenkins 1.509.1 (LTS) still has the issue.
Comment 3 Antoine "hashar" Musso (WMF) 2013-05-02 14:47:12 UTC
Upstream bug is https://issues.jenkins-ci.org/browse/JENKINS-17837
Comment 4 Antoine "hashar" Musso (WMF) 2013-05-02 14:53:33 UTC
Maybe it is related to the Downstream buildview plugin which is reference jstack traces. https://wiki.jenkins-ci.org/display/JENKINS/Downstream+buildview+plugin
Comment 5 Antoine "hashar" Musso (WMF) 2013-05-02 15:13:38 UTC
I have removed the downstream-buildview plugin. That was triggered a full parse of all the build history!

The Jenkins upgraded did add the Lazy Loading system.

We are safe now, Jenkins restarts very fast now.
Comment 6 Antoine "hashar" Musso (WMF) 2013-12-17 20:05:59 UTC
Upstream bug ( https://issues.jenkins-ci.org/browse/JENKINS-17837 ) received some attention today. The issue was in the downstream-buildview plugin that would update the whole history of jobs apparently for no worthwhile reason.

Upstream fixed the issue by removing the faulty class with https://github.com/jenkinsci/downstream-buildview-plugin/commit/47801c278f000b34eae5bd9d674029cf470cd8b2

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links