Last modified: 2014-10-15 22:43:59 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T62164, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 60164 - Jenkins: Use web proxy to let git access repositories on from GitHub (e.g. submodules)
Jenkins: Use web proxy to let git access repositories on from GitHub (e.g. su...
Status: NEW
Product: Wikimedia
Classification: Unclassified
Continuous integration (Other open bugs)
wmf-deployment
All All
: Low normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-01-17 10:20 UTC by Antoine "hashar" Musso (WMF)
Modified: 2014-10-15 22:43 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Antoine "hashar" Musso (WMF) 2014-01-17 10:20:25 UTC
The translatewiki repository relies on git submodules hosted on github using either http or the git protocol:

 https://github.com/puppetlabs/puppetlabs-apt.git
 git://github.com/tPl0ch/puppet-composer.git

Since lanthanum does not have direct access to internet, whenever the translatewiki-puppet-validate job is run on that host, the clone of submodules fails:

 https://integration.wikimedia.org/ci/job/translatewiki-puppet-validate/1765/console

I have added a http_proxy parameter which still fail the build:

 https://integration.wikimedia.org/ci/job/translatewiki-puppet-validate/1768/console

We need an addition https_proxy parameter:

 https://integration.wikimedia.org/ci/job/translatewiki-puppet-validate/1769/console

But then that fails fetching git://github.com..


In man git-config, there seems to be possibility to set proxy for both HTTP and GIT protocols and we can even make them on a per host/URL basis!

Example:

           ; Proxy settings
           [core]
                   gitproxy=proxy-command for kernel.org
                   gitproxy=default-proxy ; for all the rest

           ; HTTP
           [http]
                   sslVerify
           [http "https://weak.example.com"]
                   sslVerify = false
                   cookieFile = /tmp/cookie.txt

The jobs are running as jenkins-slave user.  That user has a .gitconfig which is generated by puppet. So potentially we can figure out the proper parameters and have them populated by puppet.
Comment 1 Gerrit Notification Bot 2014-01-17 10:24:38 UTC
Change 108021 had a related patch set uploaded by Hashar:
tie translatewiki-puppet-validate to gallium

https://gerrit.wikimedia.org/r/108021
Comment 2 Gerrit Notification Bot 2014-01-17 10:25:15 UTC
Change 108021 merged by jenkins-bot:
tie translatewiki-puppet-validate to gallium

https://gerrit.wikimedia.org/r/108021
Comment 3 Antoine "hashar" Musso (WMF) 2014-01-17 10:26:00 UTC
As a workaround, I have forced translatewiki-puppet-validate to run only on gallium which has direct access to internet.
Comment 4 Antoine "hashar" Musso (WMF) 2014-01-17 10:34:15 UTC
Pinged QA list about it: http://lists.wikimedia.org/pipermail/qa/2014-January/000921.html
Comment 5 Antoine "hashar" Musso (WMF) 2014-05-19 15:22:34 UTC
Rephrased summary.
Comment 6 Krinkle 2014-10-15 22:43:30 UTC
Do these still run on the production slaves? If so, what would it take to have them run on the labs slaves?

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links