Last modified: 2013-03-12 21:25:57 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T48041, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 46041 - Gerrit exhibits poor performance in response to certain client requests
Gerrit exhibits poor performance in response to certain client requests
Status: RESOLVED INVALID
Product: Wikimedia
Classification: Unclassified
Git/Gerrit (Other open bugs)
wmf-deployment
All All
: Unprioritized normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-03-12 20:31 UTC by Ori Livneh
Modified: 2013-03-12 21:25 UTC (History)
10 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
Output of "GIT_CURL_VERBOSE=1 GIT_TRACE=true git clone --verbose --depth 1 https://gerrit.wikimedia.org/r/p/mediawiki/core.git" (20.42 KB, application/octet-stream)
2013-03-12 20:31 UTC, Ori Livneh
Details

Description Ori Livneh 2013-03-12 20:31:47 UTC
Created attachment 11920 [details]
Output of "GIT_CURL_VERBOSE=1 GIT_TRACE=true git clone --verbose --depth 1 https://gerrit.wikimedia.org/r/p/mediawiki/core.git"

When RobLa runs a shallow git-clone of core over HTTPS, the object count is 3,456. When I run it, it's 46,297.

Me (git version 1.7.9.5 on Ubuntu Precise):

vagrant@precise32:~$ time git clone --depth 1 https://gerrit.wikimedia.org/r/p/mediawiki/core.git
Cloning into 'core'...
remote: Counting objects: 46297, done
remote: Finding sources: 100% (46297/46297)
remote: Getting sizes: 100% (25843/25843)
remote: Compressing objects: 76% (19864/25833)
remote: Total 46297 (delta 33063), reused 26399 (delta 20010)
Receiving objects: 100% (46297/46297), 102.66 MiB | 194 KiB/s, done.
Resolving deltas: 100% (37898/37898), done.

real 15m14.500s
user 0m27.562s
sys 0m13.421s


$ git --version
git version 1.7.10.4

$ time git clone --depth 1
https://gerrit.wikimedia.org/r/p/mediawiki/core.git core-shallow
Cloning into 'core-shallow'...
remote: Counting objects: 3456, done
remote: Finding sources: 100% (3456/3456)
remote: Getting sizes: 100% (3074/3074)
remote: Compressing objects: 63% (1958/3069)
remote: Total 3456 (delta 690), reused 1496 (delta 379)
Receiving objects: 100% (3456/3456), 18.71 MiB | 1.49 MiB/s, done.
Resolving deltas: 100% (816/816), done.

real 0m34.507s
user 0m4.252s
sys 0m0.940s

RobLa (git version 1.7.10.4 on Ubuntu Quantal):

$ time git clone --depth 1
https://gerrit.wikimedia.org/r/p/mediawiki/core.git core-shallow
Cloning into 'core-shallow'...
remote: Counting objects: 3456, done
remote: Finding sources: 100% (3456/3456)
remote: Getting sizes: 100% (3074/3074)
remote: Compressing objects: 63% (1958/3069)
remote: Total 3456 (delta 690), reused 1496 (delta 379)
Receiving objects: 100% (3456/3456), 18.71 MiB | 1.49 MiB/s, done.
Resolving deltas: 100% (816/816), done.

real 0m34.507s
user 0m4.252s
sys 0m0.940s

I am attaching (most of) the output of

GIT_CURL_VERBOSE=1 GIT_TRACE=true git clone --verbose --depth 1 https://gerrit.wikimedia.org/r/p/mediawiki/core.git
Comment 1 Ori Livneh 2013-03-12 20:38:57 UTC
(Please review the output before attaching it -- I'm not sure, but it's possible that it would contain your credentials, if you have any configured in git or .netrc.)
Comment 2 Platonides 2013-03-12 20:49:58 UTC
Is it possible that you have the equivalent of  --no-single-branch in your configuration ?

«When creating a shallow clone with the --depth option, this [--single-branch] is the default, unless --no-single-branch is given to fetch the histories near the tips of all branches.»

Or it may be a git bug fixed in 1.7.10

PS: I get 2791 objects.
Comment 3 Chad H. 2013-03-12 20:56:50 UTC
chad@sexy /www/git$ time git clone https://gerrit.wikimedia.org/r/p/mediawiki/core.git --depth 1
Cloning into 'core'...
remote: Counting objects: 2791, done
remote: Finding sources: 100% (2791/2791)
remote: Getting sizes: 100% (2665/2665)
remote: Compressing objects:  67% (1804/2660)
remote: Total 2791 (delta 294), reused 986 (delta 124)
Receiving objects: 100% (2791/2791), 17.75 MiB | 2.88 MiB/s, done.
Resolving deltas: 100% (315/315), done.

real	0m25.874s
user	0m1.353s
sys	0m0.856s
chad@sexy /www/git$ git --version
git version 1.8.1.5
Comment 4 Platonides 2013-03-12 21:07:36 UTC
To make it even funnier, git 1.8.2 will contain more shallow fixes:

commit 2532d891a4aab003a5ce19f04509fd8549754485
Merge: 9a6c84e cfb70e1
Author: Junio C Hamano <gitster@pobox.com>
Date:   Fri Feb 1 12:39:24 2013 -0800

    Merge branch 'nd/fetch-depth-is-broken'
    
    "git fetch --depth" was broken in at least three ways.  The
    resulting history was deeper than specified by one commit, it was
    unclear how to wipe the shallowness of the repository with the
    command, and documentation was misleading.
    
    * nd/fetch-depth-is-broken:
      fetch: elaborate --depth action
      upload-pack: fix off-by-one depth calculation in shallow clone
      fetch: add --unshallow for turning shallow repo into complete one
Comment 5 Platonides 2013-03-12 21:12:49 UTC
--single-branch was added in v1.7.10, which explains the difference of 46297 vs 3456.


It seems still worth upgrading to v1.7.11.2, though:

commit 4ac01b0cbc27b314333362e77323850e78a7448c
Merge: c8382c1 0ec4b16
Author: Junio C Hamano <gitster@pobox.com>
Date:   Wed Jul 11 12:48:29 2012 -0700

    Merge branch 'nd/clone-single-fix' into maint
    
    "git clone --single-branch" to clone a single branch did not limit
    the cloning to the specified branch.
    
    * nd/clone-single-fix:
      clone: fix ref selection in --single-branch --branch=xxx
Comment 6 Platonides 2013-03-12 21:13:19 UTC
CLOSED UPDATE-YOUR-CLIENT
Comment 7 Ori Livneh 2013-03-12 21:25:57 UTC
(In reply to comment #5)
> --single-branch was added in v1.7.10, which explains the difference of 46297
> vs
> 3456.

Yes! This is correct. Wooot! Thanks Platonides.

The following workaround works for 1.7.9.5 and earlier:

  vagrant@precise32:~$ mkdir core ; cd core
  vagrant@precise32:~/core$ git init
  Initialized empty Git repository in /home/vagrant/core/.git/
  vagrant@precise32:~/core$ time git fetch --depth=1 https://gerrit.wikimedia.org/r/p/mediawiki/core.git master:refs/remotes/origin/master
  remote: Counting objects: 2796, done
  remote: Finding sources: 100% (2796/2796)
  remote: Getting sizes: 100% (2666/2666)
  remote: Compressing objects:  67% (1806/2661)
  remote: Total 2796 (delta 298), reused 990 (delta 128)
  Receiving objects: 100% (2796/2796), 17.76 MiB | 373 KiB/s, done.
  Resolving deltas: 100% (319/319), done.
  From https://gerrit.wikimedia.org/r/p/mediawiki/core
   * [new branch]      master     -> origin/master

  real	0m42.048s
  user	0m3.104s
  sys	0m2.120s

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links