Last modified: 2014-01-08 19:05:36 UTC
You know, there is a constantly reappearing issue with files not being purged. There are dozens of reports about this here in Bugzilla. Some thumbnails show an outdated version for hours, sometimes for months. An action=purge does not solve the problem in these cases. Today I have the same problem with JavaScript files. Whenever I edit a file I can not test my change because "action=raw" continues to deliver the old version. Manually calling "action=purge" does nothing. I still get the old version. Steps to reproduce: 1. Call [1]. 2. Edit the file. 3. Try to reload [1]. You get the old version. 4. Add a dummy parameter to the URL, e.g. [2]. You get the new version. 5. Try to purge the file. Nothing changes. 6. Wait ten minutes. Eventually [1] will return the new version. I call this a "blocker" because it blocks me again and again. [1]https://de.wikipedia.org/w/index.php?title=Benutzer:TMg/weblinkChecker.js&action=raw&ctype=text/javascript [2]https://de.wikipedia.org/w/index.php?title=Benutzer:TMg/weblinkChecker.js&action=raw&ctype=text/javascript&dummy=1
(In reply to comment #0) > You know, there is a constantly reappearing issue with files not being > purged. > There are dozens of reports about this here in Bugzilla. Some thumbnails show > an outdated version for hours, sometimes for months. An action=purge does not > solve the problem in these cases. > > Today I have the same problem with JavaScript files. Whenever I edit a file I > can not test my change because "action=raw" continues to deliver the old > version. Manually calling "action=purge" does nothing. I still get the old > version. Javascript and Media files have pruging handled entirely separately. Javascript files like these are not purged on page edits. This is the way its always been. However, javascript files loaded via Resource loader (e.g. Things from gadgets) do get purged on edit (I think)
It might make sense as an enhancement request to send a purge request to ?action=raw&ctype=text/javascript url if the page being edited is in user namespace and ends in .js (ditto for css). However I'm not sure if we've standardized on that form of a url, or if other parameters (smaxage, maxage, usemsgcache, etc) are still common in these js urls. ---- >However, javascript files loaded via Resource loader (e.g. Things from gadgets) >do get purged on edit (I think) To be more clear, I think one could make the gadget require a non-existent right, so its not on Special:preferences, but still can be loaded via RL. (I'm not that familiar with RL, might have things wrong)
(In reply to comment #1) > However, javascript files loaded via Resource loader (e.g. Things from > gadgets) do get purged on edit (I think) bawolff: See bug 56778?
I'm sorry, but it's at least a normal priority bug and not an "enhancement" if the purge feature pretends to do something (there is even an "are you sure" question when calling [1] while not being logged in) but in reality does not what it's expected to do. If the servers still deliver old versions after a purge something is broken. I have to ask why it got worse yesterday if "its always been" that way. I edit my .js files very regularly. I never had to wait ten minutes before. The URL I'm using is the one created by importScript in wikibits.js[2] which still lacks an adequate, equally user-friendly replacement. If that's how it works you should add these (still) official URLs to the purge request. The root cause is the same as for media files not being purged. From what I understood purging is done based on URLs instead of the original page name. Because of that it's impossible to simply "purge a page". There is no "page" on that level. You don't keep track of the URLs used to call a page (different image sizes, dummy parameters and what not) and you have no way to find out. I called this "broken by design" in some previous reports. Do the users simply have to live with this, accepting that it gets worse when the servers have to deal with heavier page load and caching becomes more aggressive in the next years? [1]https://de.wikipedia.org/wiki/Benutzer:TMg/autoFormatter.js/Beta.js?action=purge [2]https://git.wikimedia.org/blob/mediawiki%2Fcore.git/HEAD/skins%2Fcommon%2Fwikibits.js#L210
Ah sorry, wasn't aware of the UI implications here and had "the way its always been" in mind when setting "enhancement". Thanks for correcting!
I should clarify my comment above wasn't to say its less important of an issue, its just important to clarify which bugs are we intend that x does y, but that doesn't happen, and which bugs are it would be nice if x does y but no one has made that happen yet. (In reply to comment #4) > I'm sorry, but it's at least a normal priority bug and not an "enhancement" > if > the purge feature pretends to do something (there is even an "are you sure" > question when calling [1] while not being logged in) but in reality does not > what it's expected to do. If the servers still deliver old versions after a > purge something is broken. Well it does something, just not what you want it to do... It clears cache of things viewed through normal web view. The dialog is primarily to stop people from linking directly to the purge page Instead of via a normal url. > I have to ask why it got worse yesterday if "its always been" that way. I > edit > my .js files very regularly. I never had to wait ten minutes before. I don't think you are having the caching issue you think you are. The squid/varnish cache for non-rl js *only applies to logged out users *would last about 30 days not ten minutes. You description makes it maybe sound like something to do with local browser cache maybe. Would need more investigation. > The URL I'm using is the one created by importScript in wikibits.js[2] which > still lacks an adequate, equally user-friendly replacement. If that's how it > works you should add these (still) official URLs to the purge request. I don't disagree. (In certain cases anyhow) > The root cause is the same as for media files not being purged. From what I > understood purging is done based on URLs instead of the original page name. > Because of that it's impossible to simply "purge a page". There is no "page" > on > that level. You don't keep track of the URLs used to call a page (different > image sizes, dummy parameters and what not) and you have no way to find out. > I > called this "broken by design" in some previous reports. Do the users simply > have to live with this, accepting that it gets worse when the servers have to > deal with heavier page load and caching becomes more aggressive in the next > years? Perhaps an unideal design, but per above I don't think this is related to the issue you are seeing > > [1]https://de.wikipedia.org/wiki/Benutzer:TMg/autoFormatter.js/Beta. > js?action=purge > [2]https://git.wikimedia.org/blob/mediawiki%2Fcore.git/HEAD/ > skins%2Fcommon%2Fwikibits.js#L210
(In reply to comment #3) > (In reply to comment #1) > > However, javascript files loaded via Resource loader (e.g. Things from > > gadgets) do get purged on edit (I think) > > bawolff: See bug 56778? That's fairly unrelated. From what I understand local storage of gadget JS has cache invalidation all worked out, its just some extra things might stick around taking up space in LocalStorage which is unideal but wouldn't cause old versions to show up.
>I don't think you are having the caching issue you think you are. The >squid/varnish cache for non-rl js >*only applies to logged out users >*would last about 30 days not ten minutes. Sorry the thirty days part was a lie. I was thinking about how long things would at most stay in the cache, but ?action=raw sets a header to say max 5 minutes. It also seems like when I tested that logged in users still get things cached (which I didn't think happened for stuff going through index.php)... Moral of the story, I should always double check the things I say, as when I talk out of my hat, I usually end up being wrong...
Change 95095 had a related patch set uploaded by Brian Wolff: Purge user css/js when not for a skin https://gerrit.wikimedia.org/r/95095
+1. Thanks TMg for filing this, and thanks Brian for supplying a patch. This has been bothering me a lot as well, especially because there's still quite a large landscape where ResourceLoader isn't used (e.g. user scripts shared that aren't yet gadgetised and personal user scripts, such as global.js - which is being worked on - and cross-wiki pseudo-gadgets).
(In reply to comment #6) > just not what you want it to do. Don't get me wrong. I see your point. But I still think a request like action=purge&title=... is expected to invalidate everything that's based on the given title. That's what the request says. I find it counter-intuitive if it invalidates only a few selected URLs. Seen from this users point of view I still think this makes it the same issue as with the images. > sound like something to do with local browser cache maybe. I can assure you it's not the browser. I'm a developer for a living, I clear caches, use different browsers, even use different machines in different networks. So I'm not only sure it's not my browser, I'm also sure it's not my ISP. This is the response I get along with the outdated version after I made an edit: HTTP/1.1 200 OK Server: nginx/1.1.19 Date: Fri, 15 Nov 2013 13:47:22 GMT Content-Type: text/javascript; charset=UTF-8 Content-Length: 797 Connection: keep-alive X-Powered-By: PHP/5.3.10-1ubuntu3.8+wmf2 X-Content-Type-Options: nosniff Last-modified: Fri, 11 Oct 2013 18:52:26 GMT Content-Encoding: gzip Vary: Accept-Encoding X-Vary-Options: Accept-Encoding;list-contains=gzip X-Varnish: 2242262276, 917483657 917405329, 1727502691 1727437547 Via: 1.1 varnish, 1.1 varnish, 1.1 varnish Accept-Ranges: bytes Age: 103 X-Cache: cp1067 miss (0), amssq61 hit (7), amssq57 frontend hit (1) Cache-Control: private, s-maxage=0, max-age=0, must-revalidate When I wait it eventually switches to the new version: HTTP/1.1 200 OK Server: nginx/1.1.19 Date: Fri, 15 Nov 2013 13:51:34 GMT Content-Type: text/javascript; charset=UTF-8 Content-Length: 799 Connection: keep-alive X-Powered-By: PHP/5.3.10-1ubuntu3.8+wmf2 X-Content-Type-Options: nosniff Last-modified: Fri, 15 Nov 2013 13:46:36 GMT Content-Encoding: gzip Vary: Accept-Encoding X-Vary-Options: Accept-Encoding;list-contains=gzip X-Varnish: 2242883096, 917751928 917706534, 2731510524 Via: 1.1 varnish, 1.1 varnish, 1.1 varnish Accept-Ranges: bytes Age: 49 X-Cache: cp1067 miss (0), amssq61 hit (1), amssq51 frontend miss (0) Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
I know this is not helpful, but at the moment every change I do goes "live" immediately. Like it was before. Anybody knows what the problem was? Is this really just the page load?
And now it's delayed again. Is this some bug that actually goes away if you don't look at it? O_o? Here are the two relevant responses again for debugging purposes. I find the "miss" and "hit" stuff interesting. HTTP/1.1 200 OK Server: nginx/1.1.19 Date: Wed, 20 Nov 2013 22:18:48 GMT Content-Type: text/javascript; charset=UTF-8 Content-Length: 6798 Connection: keep-alive X-Powered-By: PHP/5.3.10-1ubuntu3.8+wmf2 X-Content-Type-Options: nosniff Last-modified: Wed, 20 Nov 2013 21:58:42 GMT Content-Encoding: gzip Vary: Accept-Encoding X-Vary-Options: Accept-Encoding;list-contains=gzip X-Varnish: 1870685724, 664763206 664710782, 3431397977 3431197706 Via: 1.1 varnish, 1.1 varnish, 1.1 varnish Accept-Ranges: bytes Age: 164 X-Cache: cp1066 miss (0), amssq60 hit (3), amssq51 frontend hit (3) Cache-Control: private, s-maxage=0, max-age=0, must-revalidate HTTP/1.1 200 OK Server: nginx/1.1.19 Date: Wed, 20 Nov 2013 22:21:52 GMT Content-Type: text/javascript; charset=UTF-8 Content-Length: 6820 Connection: keep-alive X-Powered-By: PHP/5.3.10-1ubuntu3.8+wmf2 X-Content-Type-Options: nosniff Last-modified: Wed, 20 Nov 2013 22:18:39 GMT Content-Encoding: gzip Vary: Accept-Encoding X-Vary-Options: Accept-Encoding;list-contains=gzip X-Varnish: 1871226048, 664938905, 2276370343 2276298771 Via: 1.1 varnish, 1.1 varnish, 1.1 varnish Accept-Ranges: bytes Age: 47 X-Cache: cp1066 miss (0), amssq60 miss (0), amssq54 frontend hit (1) Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
I would geuss coincidence (small cache time = high probability you will sometimes edit the page just before the 5 minute ttl is up) Also from what I understand, small js files that are below some threshold aren't squid cached. That doesn't seem to be affecting you. ---- Yes a hit header implies cached response. There are layers of varnish, which is why you get both hit/miss depending on which layer had the cached object. The other header to pay attention to is age, which tells you how old the cached object is in seconds
Change 95095 merged by jenkins-bot: Send cache purges for action=raw after editing user css/js https://gerrit.wikimedia.org/r/95095
Patch merged, I'm assuming this is resolved.
