Wikimedia Bugzilla is closed!

A few more notes (I was the original reporter of this problem, thanks Nimish for double checking):

• Watchlist feeds work fine (but they are generated by calling api.php?action=feedwatchlist )
• Revision feeds also appear to be broken in Safari
• The issue appears to be broader than Safari. If you open the feed in FireFox and try to subscribe to it via Mail.app (the default feed reader on Mac OS), the feed is also broken

FYI my setup is:

Mac 10.6.8
Safari 5.0.5
Mail 4.5

Looks like Domas hacked something into our squid configurations to block it in February 2010, presumably due to some brain-dead automatic polling:


http://wikitech.wikimedia.org/view/Server_admin_log/Archive_15#February_16

03:16 domas: blocked MacOSX Atom syndication on the edge, this will save terabytes of diskspace on unsuspecting Safari user computers :-) 


Of course if this just stops you from being able to read the feed at all even on explicit view, that's a bit annoying. :) Not sure what the current state of this is, but I can confirm being unable to read the Recentchanges Atom & RSS feeds from Safari here.

The bug is on Safari's side. It's downloading each and every RSS feed that it can find and also tries to update it. This is putting a high load on our servers and as well pollutes the disks of Wikipedia editors that are visiting many pages with RSS feeds.

There's nothing we can do from the "shell" side to solve this issue, so that I've removed the "shell" keyword.

Is the exact behavior documented? I can't reproduce "downloading each and every RSS feed that it can find" with Safari 5.0.5; it doesn't seem to fetch autodiscovered feeds until I explicitly request them by clicking on the RSS icon.

There are options to 'fetch updates' automatically for RSS items in bookmarks bar or whole bookmarks menu, but it looks like this will only apply if you've bookmarked the feed, not if you've bookmarked a page that happens to have a feed on it. On the other hand it might. :)  Waiting 30 mins to see what happens...


On the block itself; it looks like when you hit a feed, Safari does two hits: first as itself (regular Safari UA string), then again as Apple-PubSub. If it's the second one we're blocking, then I guess Safari's only using the first download to confirm it's a feed and then passes the URL off to its feed reader submodule to fetch again and process, where it dies.

It'd be nice if we can make sure a sane error message makes it back, but not sure if that's feasible.

To be honest, I just relayed the answer Domas gave me yesterday regarding his motivation to block Safari. I'm sorry that I can't provide you with any details.

So far it looks like Safari will re-fetch the feed in the background if you bookmarked a feed directly, but does not appear to be fetching feeds if you just bookmark an article page (which has autodiscovery markers for the RecentChanges feed, and thus shows the 'rss' icon so you can get *to* the feed).

Since the raw RC feed is nigh-useless on big sites anyway we probably shouldn't actually be offering it as autodiscovery anyway, but it looks like it takes an explicit action (click) to get there, and another explicit action (bookmark) to get it to where it'll get reloaded.

Retooling the feeds so that the default feed URL is sanely squid-cachable would likely also help.

Restoring 'shell' keyword -- if the reason for the block is no longer valid (maybe an old version of Safari was worse?) then it should be removed, and only shell users can do that.

If removing unnecessary and useless RC feed links on the biggest sites would help reduce the frequency of people having silly feeds, then disabling that is probably something that needs to be done from site configuration, which needs a shell user.

If making the feeds more squid-cachable would help, then that'll need some development (should open a new bug and mark it as blocking this).

The problem is that one you _open_ RSS feed, it gets syncidated forever in some local store, and whole RSS then bloats gigabytes of storage locally (and yes, there're useless calls to RSS layer).

As we can't differentiate between proper intended RSS request and unintended one, we are saving our users. 

It isn't link existence that causes the behavior, it is actually opening that link from Safari.

So that is a user issue... we shouldn't be blocking that.

who are you, and on what did you base your opinion? 
in the end, everything is a user issue, we shouldn't do anything.

(In reply to comment #9)
> The problem is that one you _open_ RSS feed, it gets syncidated forever in some
> local store, and whole RSS then bloats gigabytes of storage locally (and yes,
> there're useless calls to RSS layer).
> 
> As we can't differentiate between proper intended RSS request and unintended
> one, we are saving our users. 
> 
> It isn't link existence that causes the behavior, it is actually opening that
> link from Safari.

As far as others have been able to determine from testing, this doesn't actually happen; only bookmarking the feed causes it to be loaded and stored in the future.

Can you provide steps to reproduce?

You can see all subscriptions using 'pubsub list' command, or checking them directly in PubSub database:

sqlite ~/Library/PubSub/Database/Database.sqlite3 

I don't see this behavior on newest Safari yet, but I see quite a few feeds I'd never add myself ;-)

ACLs should be re-enabled and then Tim will then gather some statistics on subscriptions.

(In reply to comment #14)
> ACLs should be re-enabled and then Tim will then gather some statistics on
> subscriptions.

Did this happen?
Is the Safari bug filed somewhere?

Does the problem still happen in recent Safari versions? If yes, which version?

Tim: Do you know if ACLs got reenabled?

Domas: Is this still a problem? If so, in which Safari version?

I don't know! Doesn't this bug say that these rules were removed back in 2011?!!?

If the bug report says so, I've failed to find that comment. :(