Last modified: 2014-10-24 13:59:07 UTC
At least bin/mw-core-get.sh and tools/fetch-mw-ext from integration/jenkins.git use git.wikimedia.org to get an archive of the source from a git repository. Sadly git.w.o regularly has downtime. So for our jenkins jobs not to fail in that case we should remove that dependency. To make things difficult gerrit does not support downloading an archive either from the UI nor from the SSH endpoint: $ git archive --remote="ssh://jzerebecki@gerrit.wikimedia.org:29418/mediawiki/core" "eab45e37b66582a94e5a7700021e41a59560e665" >test.tar Gerrit Code Review: git-upload-archive: not found fatal: The remote end hung up unexpectedly That leaves us with the following possibilities: 1) Add a gerrit mirror to every jenkins slave that uses those scripts. 2) Add an on demand mirror (git clone/fetch) of the repos needed. 3) Use a gerrit mirror that has git archive --remote support. (Does that work with the intelligent git http protocol? If not this is complicated by the fact that each jenkins slave then needs ssh access to that mirror.) Any other ways? Any preferences for a solution?
Why not just clone with --depth 1? Seems reasonable fast to me: hoo@wikidata-hhvm:/tmp/git-test$ time git clone https://gerrit.wikimedia.org/r/mediawiki/core --depth 1 Cloning into 'core'... remote: Counting objects: 4146, done remote: Finding sources: 100% (4146/4146) remote: Getting sizes: 100% (3955/3955) remote: Compressing objects: 99% (54822/54823) remote: Total 4146 (delta 533), reused 1846 (delta 189) Receiving objects: 100% (4146/4146), 19.18 MiB | 5.05 MiB/s, done. Resolving deltas: 100% (568/568), done. Checking connectivity... done. real 0m9.855s user 0m2.675s sys 0m1.122s
We have two slave scripts (integration/jenkins.git) which still depends on git.wikimedia.org bin/mw-core-get.sh tools/fetch-mw-ext Most jobs have been migrated to Zuul cloner which does a full clone. The left over jobs are misty: mwext-browertests-* jobs which were meant to trigger browser tests on patch proposal. We abandoned that for now mwext-Wikibase* and mwext-Wikidata* jobs which I have yet to migrate to Zuul cloner. There is a few other jobs, will need to investigate.
