Last modified: 2014-04-02 13:16:25 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T55978, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 53978 - setup labs project for continuous integration jobs
setup labs project for continuous integration jobs
Status: NEW
Product: Wikimedia Labs
Classification: Unclassified
General (Other open bugs)
unspecified
All All
: Unprioritized normal
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks: 45499
  Show dependency treegraph
 
Reported: 2013-09-10 05:58 UTC by Antoine "hashar" Musso (WMF)
Modified: 2014-04-02 13:16 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Antoine "hashar" Musso (WMF) 2013-09-10 05:58:44 UTC
Following a discussion I had with Ryan Lane, the continuous integration project could use an OpenStack projects in labs that would be dedicated to running disposable Jenkins instances (bug 45499).

What we would need to have:

- have the instances fully isolated from the rest of the network (prod / other labs projects), with the exception of communications with Jenkins master and git fetch from Zuul (matrix to be determined).

- a write access to the OpenStack API to be able to spawn instances programmatically.

- possibly have the project running on dedicated hardware to avoid being influenced by other volunteers projects or to avoid influencing them.

- an instance image that is fast to boot up, aka already proving the CI packages and Jenkns slave (omething like role::ci::slave)
Comment 1 Antoine "hashar" Musso (WMF) 2013-11-13 16:40:32 UTC
A related post I sent to the labs mailing list
http://lists.wikimedia.org/pipermail/labs-l/2013-November/001796.html

-----------------8<---------------------8<--------------------8<--------

I could use a new labs project on the wmflabs project to boot up new instances that would be consumed by the Continuous integration system. The aim is to be able to properly isolate some tests we are running.

The idea is to maintain a pool of instances that would be dynamically registered as Jenkins slaves of the CI Jenkins master.  The later would then be able to send test to run on those slaves and we would destroy the instance once the job is completed.

The requisites would be:

- a way to access the OpenStack API so I could script the creation of instances using https://pypi.python.org/pypi/python-novaclient/

- isolate the instances from the rest of the prod/labs networks using a yet to be written security matrix. The instances would at least need:
 * to be reachable from the Jenkins master by ssh
 * the availability to fetch from npm

- A virtual image using Ubuntu Precise and prepopulated withrole::ci::slave::labs::common and contint::slave-scripts.  The idea is to have the new instances booting up as fast as possible.

- A host profile with 2GB of RAM, single CPU and 5GB of /dev/vdb disk.


Optionally:

- having the project to run on dedicated hardware, but that can happen later on.


Who shall I sync with to make it happen?  :-]

Should I fill in a bunch of bugs in Bugzilla for ease of tracking?

-----------------8<---------------------8<--------------------8<--------


I guess I will fill above tasks as bugs depending on this one.
Comment 2 Tim Landscheidt 2013-11-13 17:30:38 UTC
As this setup sounds *very* similar to the concept of Travis CI (https://travis-ci.org/), have you looked into that?  It might be easier than to hammer Labs and Jenkins into shape, and WMF could just sponsor some hardware (or money :-)).
Comment 3 Addshore 2013-11-13 18:25:12 UTC
That sounds like a great idea!
As far as I am aware the only functionality travis doesn't currently have that we need would be gerrit integration.
Comment 4 Ryan Lane 2013-11-13 19:24:07 UTC
We don't use hosted solutions when we can sanely host them ourselves and we don't use proprietary things unless there's no available alternative.

Setting up a labs project for this is doable, so we should be doing it. We just need to take the time to do it. I think priority wise, this should be after the Labs move to eqiad.
Comment 5 Antoine "hashar" Musso (WMF) 2013-11-13 19:38:27 UTC
What Ryan said, we can't depend on a third party at all.  That doesn't prevent us from reusing Travis open source code and integrate it in our build system, that would nicely scale out CI to more people.
Comment 6 Addshore 2013-11-13 20:28:45 UTC
Re Ryan and Hashar, https://github.com/travis-ci, Sorry that is what I meant!
Set up our own slightly modified travis!
I can see our CI taking a big step forward when this bug is resolved :D
Comment 7 Antoine "hashar" Musso (WMF) 2014-04-02 13:16:25 UTC
We have labs instance in the integration projects: integration-slave1001.eqiad.wmflabs and integration-slave1002.eqiad.wmflabs.

They are fully puppetized and let us fetch packages from nodejs/rubygems/pip...

Still does not provide proper isolation nor the ability to spawn new instance on demand.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links