Wikimedia Bugzilla is closed!

Hmm, when I try getting a cached version in Varnish (with GET), it only takes about 6 seconds for the download to start. When I try for an uncached version, it waits 1 minute 10 seconds and then gives me a 503.

I've confirmed that Swift responds instanteously, so this is probably related to Varnish streaming support -- i.e. Varnish not streaming the file as it comes from the backend, but fetching it all first then serving it from the cache. I'll have a closer look and/or poke Mark.

Fixed in https://gerrit.wikimedia.org/r/#/c/81651/

std.integer returns a signed 32 bit integer, and can wrap and become negative.

is the fix deployed? still fails here now even with a timeout:

time curl -I http://upload.wikimedia.org/wikipedia/commons/2/2a/2012_State_Of_The_Union_Address_%28720p%29.ogv
HTTP/1.1 504 Gateway Time-out
Server: nginx/1.1.19
Content-Type: text/html
Content-Length: 183
Connection: keep-alive


real	1m0.157s
user	0m0.016s
sys	0m0.004s

Yes, also responds instantaneous for me. Could you please post full headers?

ipv4 i get:

curl -4 -I http://upload.wikimedia.org/wikipedia/commons/2/2a/2012_State_Of_The_Union_Address_%28720p%29.ogv
HTTP/1.1 503 Service Unavailable
Server: Varnish
Content-Type: text/html; charset=utf-8
Content-Length: 3051
Accept-Ranges: bytes
Date: Mon, 02 Sep 2013 11:04:53 GMT
X-Varnish: 3458126360
Age: 58
Via: 1.1 varnish
Connection: close
X-Cache: cp3010 frontend miss (0)
Access-Control-Allow-Origin: *


real	0m58.215s
user	0m0.008s
sys	0m0.016s

ipv6:
curl -6 -I http://upload.wikimedia.org/wikipedia/commons/2/2a/2012_State_Of_The_Union_Address_%28720p%29.ogv
HTTP/1.1 504 Gateway Time-out
Server: nginx/1.1.19
Date: Mon, 02 Sep 2013 11:06:10 GMT
Content-Type: text/html
Content-Length: 183
Connection: keep-alive


real	1m0.130s
user	0m0.012s
sys	0m0.012s

host upload.wikimedia.org
upload.wikimedia.org is an alias for upload-lb.esams.wikimedia.org.
upload-lb.esams.wikimedia.org has address 91.198.174.234
upload-lb.esams.wikimedia.org has IPv6 address 2620:0:862:ed1a::b

GET seams to be fast now:
curl -v http://upload.wikimedia.org/wikipedia/commons/2/2a/2012_State_Of_The_Union_Address_%28720p%29.ogv > /dev/null 

while HEAD requests (curl -I) are still slow and end in a 504:
time curl -I -v http://upload.wikimedia.org/wikipedia/commons/2/2a/2012_State_Of_The_Union_Address_%28720p%29.ogv
* About to connect() to upload.wikimedia.org port 80 (#0)
*   Trying 2620:0:862:ed1a::b...
* Connected to upload.wikimedia.org (2620:0:862:ed1a::b) port 80 (#0)
> HEAD /wikipedia/commons/2/2a/2012_State_Of_The_Union_Address_%28720p%29.ogv HTTP/1.1
> User-Agent: curl/7.29.0
> Host: upload.wikimedia.org
> Accept: */*
> 
< HTTP/1.1 504 Gateway Time-out
HTTP/1.1 504 Gateway Time-out
< Server: nginx/1.1.19
Server: nginx/1.1.19
< Date: Mon, 02 Sep 2013 11:21:10 GMT
Date: Mon, 02 Sep 2013 11:21:10 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 183
Content-Length: 183
< Connection: keep-alive
Connection: keep-alive

< 
* Connection #0 to host upload.wikimedia.org left intact

real	1m0.128s
user	0m0.012s
sys	0m0.008s

that said, the video opens fast in the chrome and firefox now since since they only make GET requests with ranges.

marking as fixed since the long loading times are gone and will open bug about HEAD requests if they persist.

Yeah that makes sense. Varnish converts a HEAD into a GET when doing a backend request on a miss, and then doesn't allow streaming. It'll wait until the entire object has been retrieved before generating the response.

In theory, in "streaming mode" it could do this once the response headers (for the GET) have been received, and then just not stream the body.

I'm not sure how much it's worth spending time on now though, given that Varnish 4 (slated to come out in a few months) will have a different implementation of streaming. Is this ever a problem in practice?