Last modified: 2012-02-02 00:17:08 UTC
The first stage of Swift deployment will involve using Swift as a persistent, reliable thumbnail cache, which will reduce the load on the ms5 backend. In order to use Swift as a cache, we need the ability to purge it. This deployment stage is temporary, eventually we will use a SwiftBackend which will be able to purge thumbnails by overriding the relevant method. But for now, we just want a hook into LocalFile::purgeThumbList(). The hook will connect to Swift and delete the cached files.
In function hooked onto purgeThumbList(), is the plan to use the FS thumb path strings as the Swift object names (e.g. "$dir/$file")? Where will the swift thumb caches be generated? thumb.php seems like the best choice, since it already gets the File object and does parameter validation and such. So then another hook will be needed there too.
(In reply to comment #1) > Where will the swift thumb caches be generated? thumb.php seems like the best > choice, since it already gets the File object and does parameter validation and > such. So then another hook will be needed there too. Gah, never mind that. If ms5 load needs to be reduced, that's way too far down the chain (http://wikitech.wikimedia.org/images/1/1c/Thumbnail_request_path.jpg). I guess a new 404-handler layer is needed before ms5/nginx.
It might be better placed in purgeThumbnails() and purgeOldThumbnails(). The hook would just pass in $dir, and the handler would have to transform $dir (via regex or something ugly) to truncate away the ".../thumb/" prefix. purgeThumbList() has no outside callers and we don't want to just purge the files found on the FS but everything under $dir (or the corresponding swift "directory" for the file name). Ideally the file lists would correspond exactly, but we wan't to deal with files stuck in the swift cache because they aren't on the FS and thus don't end up in the purge list.
(In reply to comment #3) > It might be better placed in purgeThumbnails() and purgeOldThumbnails(). The > hook would just pass in $dir, and the handler would have to transform $dir (via > regex or something ugly) to truncate away the ".../thumb/" prefix. > > purgeThumbList() has no outside callers and we don't want to just purge the > files found on the FS but everything under $dir (or the corresponding swift > "directory" for the file name). Ideally the file lists would correspond > exactly, but we wan't to deal with files stuck in the swift cache because they > aren't on the FS and thus don't end up in the purge list. OK, sounds good.
Code committed in r101387.
Deployed.
