Last modified: 2011-11-29 03:21:01 UTC
Some but not all of the squid caches seem to be serving stale versions of both thumbnails and original images. Purging or reuploading does not seem to help. I have personally only encountered this problem with SVG images so far, but I expect it probably affects all images. See for example [[Wikipedia talk:WikiProject Mathematics/Graphics#Blank thumb in svg?]]: the image http://upload.wikimedia.org/wikipedia/en/thumb/0/05/Paley13.svg/200px-Paley13.svg.png shows up for me as an old, blank version, while the 201-pixel thumb shows up just fine. The HTTP response headers I get for the affected thumbnail are: Connection: close Date: Sat, 04 Nov 2006 20:12:44 GMT Via: 1.0 sq6.pmtpa.wmnet:80 (squid/2.6.STABLE3), 1.0 sq12.wikimedia.org:80 (squid/2.6.STABLE4), 1.0 ragweed.knams.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 vandale.knams.wikimedia.org:80 (squid/2.6.STABLE3) Age: 2618 Server: lighttpd/1.4.13 Content-Length: 271 Content-Type: image/png Client-Date: Sat, 04 Nov 2006 20:55:41 GMT Client-Peer: 145.97.39.156:80 Client-Response-Num: 1 X-Cache: HIT from sq6.pmtpa.wmnet X-Cache: MISS from sq12.wikimedia.org X-Cache: HIT from ragweed.knams.wikimedia.org X-Cache: MISS from vandale.knams.wikimedia.org X-Cache-Lookup: HIT from sq6.pmtpa.wmnet:80 X-Cache-Lookup: MISS from sq12.wikimedia.org:80 X-Cache-Lookup: HIT from ragweed.knams.wikimedia.org:80 X-Cache-Lookup: MISS from vandale.knams.wikimedia.org:80 X-Powered-By: PHP/5.1.4 whereas the headers for the seemingly unaffected 201-pixel thumbnail are: Connection: close Date: Sat, 04 Nov 2006 19:59:07 GMT Via: 1.0 sq7.pmtpa.wmnet:80 (squid/2.6.STABLE3), 1.0 sq12.wikimedia.org:80 (squid/2.6.STABLE4), 1.0 ragweed.knams.wikimedia.org:80 (squid/2.6.STABLE5) Age: 3452 Server: lighttpd/1.4.13 Content-Type: image/png Client-Date: Sat, 04 Nov 2006 20:55:57 GMT Client-Peer: 145.97.39.156:80 Client-Response-Num: 1 X-Cache: HIT from sq7.pmtpa.wmnet X-Cache: MISS from sq12.wikimedia.org X-Cache: HIT from ragweed.knams.wikimedia.org X-Cache-Lookup: HIT from sq7.pmtpa.wmnet:80 X-Cache-Lookup: MISS from sq12.wikimedia.org:80 X-Cache-Lookup: HIT from ragweed.knams.wikimedia.org:80 X-Powered-By: PHP/5.1.4 Of course, it's possible the both thumbs are affected, but the latter one just happens to have been generated after the original image was fixed, and thus does not appear blank.
http://upload.wikimedia.org/wikipedia/en/thumb/8/8a/RobertJohson.jpg/250px-RobertJohson.jpg is also affected, serving a thumbnail of a version that's been deleted since 15 October! See original complaint at http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Distorted_image I've tried purging, reverting, deleting, undeleting and various combinations thereof without results. Manually issuing HTTP "Cache-Control: no-cache" request headers doesn't help either. For what it's worth, the response headers I'm getting include: GET http://upload.wikimedia.org/wikipedia/en/thumb/8/8a/RobertJohson.jpg/250px-RobertJohson.jpg --> 200 OK Connection: close Date: Sun, 12 Nov 2006 20:53:06 GMT Via: 1.0 sq14.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 sq13.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 mayflower.knams.wikimedia.org:80 (squid/2.6.STABLE3), 1.0 ragweed.knams.wikimedia.org:80 (squid/2.6.STABLE5) Accept-Ranges: bytes Age: 909 ETag: "-299518909" Server: lighttpd/1.4.13 Content-Length: 31440 Content-Type: image/jpeg Last-Modified: Wed, 28 Jun 2006 23:25:07 GMT Client-Date: Sun, 12 Nov 2006 21:08:16 GMT Client-Peer: 145.97.39.156:80 Client-Response-Num: 1 X-Cache: HIT from sq14.wikimedia.org X-Cache: MISS from sq13.wikimedia.org X-Cache: HIT from mayflower.knams.wikimedia.org X-Cache: MISS from ragweed.knams.wikimedia.org X-Cache-Lookup: HIT from sq14.wikimedia.org:80 X-Cache-Lookup: MISS from sq13.wikimedia.org:80 X-Cache-Lookup: HIT from mayflower.knams.wikimedia.org:80 X-Cache-Lookup: MISS from ragweed.knams.wikimedia.org:80 with some variants including GET http://upload.wikimedia.org/wikipedia/en/thumb/8/8a/RobertJohson.jpg/250px-RobertJohson.jpg --> 200 OK Connection: close Date: Sun, 12 Nov 2006 20:53:06 GMT Via: 1.0 sq14.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 sq13.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 mayflower.knams.wikimedia.org:80 (squid/2.6.STABLE3), 1.0 vandale.knams.wikimedia.org:80 (squid/2.6.STABLE3) Accept-Ranges: bytes Age: 906 ETag: "-299518909" Server: lighttpd/1.4.13 Content-Length: 31440 Content-Type: image/jpeg Last-Modified: Wed, 28 Jun 2006 23:25:07 GMT Client-Date: Sun, 12 Nov 2006 21:08:12 GMT Client-Peer: 145.97.39.156:80 Client-Response-Num: 1 X-Cache: HIT from sq14.wikimedia.org X-Cache: MISS from sq13.wikimedia.org X-Cache: HIT from mayflower.knams.wikimedia.org X-Cache: MISS from vandale.knams.wikimedia.org X-Cache-Lookup: HIT from sq14.wikimedia.org:80 X-Cache-Lookup: MISS from sq13.wikimedia.org:80 X-Cache-Lookup: HIT from mayflower.knams.wikimedia.org:80 X-Cache-Lookup: MISS from vandale.knams.wikimedia.org:80 and GET http://upload.wikimedia.org/wikipedia/en/thumb/8/8a/RobertJohson.jpg/250px-RobertJohson.jpg --> 200 OK Connection: close Date: Sun, 12 Nov 2006 20:53:06 GMT Via: 1.0 sq14.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 sq13.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 mayflower.knams.wikimedia.org:80 (squid/2.6.STABLE3), 1.0 sage.knams.wikimedia.org:80 (squid/2.6.STABLE3) Accept-Ranges: bytes Age: 895 ETag: "-299518909" Server: lighttpd/1.4.13 Content-Length: 31440 Content-Type: image/jpeg Last-Modified: Wed, 28 Jun 2006 23:25:07 GMT Client-Date: Sun, 12 Nov 2006 21:08:02 GMT Client-Peer: 145.97.39.156:80 Client-Response-Num: 1 X-Cache: HIT from sq14.wikimedia.org X-Cache: MISS from sq13.wikimedia.org X-Cache: HIT from mayflower.knams.wikimedia.org X-Cache: MISS from sage.knams.wikimedia.org X-Cache-Lookup: HIT from sq14.wikimedia.org:80 X-Cache-Lookup: MISS from sq13.wikimedia.org:80 X-Cache-Lookup: HIT from mayflower.knams.wikimedia.org:80 X-Cache-Lookup: MISS from sage.knams.wikimedia.org:80 and GET http://upload.wikimedia.org/wikipedia/en/thumb/8/8a/RobertJohson.jpg/250px-RobertJohson.jpg --> 200 OK Connection: close Date: Sun, 12 Nov 2006 20:53:06 GMT Via: 1.0 sq14.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 sq13.wikimedia.org:80 (squid/2.6.STABLE5), 1.0 mayflower.knams.wikimedia.org:80 (squid/2.6.STABLE3) Accept-Ranges: bytes Age: 518 ETag: "-299518909" Server: lighttpd/1.4.13 Content-Length: 31440 Content-Type: image/jpeg Last-Modified: Wed, 28 Jun 2006 23:25:07 GMT Client-Date: Sun, 12 Nov 2006 21:01:44 GMT Client-Peer: 145.97.39.156:80 Client-Response-Num: 1 X-Cache: HIT from sq14.wikimedia.org X-Cache: MISS from sq13.wikimedia.org X-Cache: HIT from mayflower.knams.wikimedia.org X-Cache-Lookup: HIT from sq14.wikimedia.org:80 X-Cache-Lookup: MISS from sq13.wikimedia.org:80 X-Cache-Lookup: HIT from mayflower.knams.wikimedia.org:80
Hey there, i'm not sure if I should've opened a new report, but I think your observations only prove mine: I was having a (deeper) look at your includes/SquidUpdate.php to get a clue of how to deal with my own squids. Here's the broken code: http://phpfi.com/267445 (relevant comments prefixed with "globe:") Here's a simple test proving the bug: http://phpfi.com/267447 The programmer's supplying an index, where he definately shouldn't have. $sockets[$so+1] should be $sockets[] and everything is fine :) I've never used mediaWiki, so i've never actually run the code. But from reading (and the little test) i'd say it's not a feature ;) It seems that the bug is in both, the branch and trunk: http://svn.wikimedia.org/viewvc/mediawiki/branches/REL1_11/phase3/includes/SquidUpdate.php?view=markup http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/includes/SquidUpdate.php?view=markup keep it up, cheers, Rod
Created attachment 4977 [details] Patch to includes/SquidUpdate.php (not committing this myself, since I can't test it) Wow, that indeed looks broken. And it's been that way since r13845! Here's a simple patch that should fix it; the only reason I'm not committing it myself is that I can't test it, not having any squids to purge.
Change sounds reasonable -- applied on r36351. This code path hasn't been exercised on our servers in a long long time (we'd switched to HTCP and multicast stuff long ago).
