Last modified: 2013-10-16 12:07:24 UTC
Reported on #wikimedia-tech by Dragonfly6-7: 6:28 Dragonfly6-7 I just killed an image while trying to rename it 6:29 Dragonfly6-7 the system told me "Fatal error (numbers that I didn't copy in time)", and then I tried to rename it again, and now it's gone gone gone 6:32 Dragonfly6-7 originally it was [[File:Kufuor.JPG]] (on en.wiki) 6:34 Dragonfly6-7 and I tried moving it to [[File:John Agyekum Kufuor (June 28, 2001).jpg]] MaxSem looked in server logs and found: 6:41 MaxSem 2013-09-27 00:27:28 mw1049 enwiki: [770efea1] /w/index.php?title=Special:MovePage&action=submit Exception from line 1053 of /usr/local/apache/common-l 6:41 MaxSem ocal/php-1.22wmf18/includes/filebackend/SwiftFileBackend.php: Got InvalidResponseException exception. This seems to be a semi-expected error condition in SwiftFileBackend. SwiftFileBackend::handleException() looks specifically for InvalidResponseException and forces a re-connect and re-auth. The loss of the image from enwiki is the troubling part. It seems like the move operation should have been rolled back. Instead the move took place but the media file is not found at the new expected location.
(Data loss → critical.)
2013-09-27 00:27:28 mw1049 enwiki: [770efea1] /w/index.php?title=Special:MovePage&action=submit Exception from line 1053 of /usr/local/apache/common-local/php-1.22wmf18/includes/filebackend/SwiftFileBackend.php: Got InvalidResponseException exception. #0 /usr/local/apache/common-local/php-1.22wmf18/includes/filebackend/SwiftFileBackend.php(1701): SwiftFileBackend->getFileListPageInternal('wikipedia-en-lo...', '2/28/Kufuor.JPG', NULL, 9000, Array) #1 /usr/local/apache/common-local/php-1.22wmf18/includes/filebackend/SwiftFileBackend.php(1635): SwiftFileBackendFileList->pageFromList('wikipedia-en-lo...', '2/28/Kufuor.JPG', NULL, 9000, Array) #2 /usr/local/apache/common-local/php-1.22wmf18/includes/filerepo/file/LocalFile.php(792): SwiftFileBackendList->rewind() #3 /usr/local/apache/common-local/php-1.22wmf18/includes/filerepo/file/LocalFile.php(880): LocalFile->getThumbnails() #4 /usr/local/apache/common-local/php-1.22wmf18/includes/filerepo/file/LocalFile.php(837): LocalFile->purgeThumbnails(Array) #5 /usr/local/apache/common-local/php-1.22wmf18/includes/filerepo/file/File.php(1079): LocalFile->purgeCache() #6 /usr/local/apache/common-local/php-1.22wmf18/includes/filerepo/file/LocalFile.php(1508): File->purgeEverything() #7 /usr/local/apache/common-local/php-1.22wmf18/includes/Title.php(3631): LocalFile->move(Object(Title)) #8 /usr/local/apache/common-local/php-1.22wmf18/includes/specials/SpecialMovepage.php(518): Title->moveTo(Object(Title), true, '', true) #9 /usr/local/apache/common-local/php-1.22wmf18/includes/specials/SpecialMovepage.php(100): MovePageForm->doSubmit() #10 /usr/local/apache/common-local/php-1.22wmf18/includes/SpecialPage.php(631): MovePageForm->execute(NULL) #11 /usr/local/apache/common-local/php-1.22wmf18/includes/SpecialPageFactory.php(489): SpecialPage->run(NULL) #12 /usr/local/apache/common-local/php-1.22wmf18/includes/Wiki.php(291): SpecialPageFactory::executePath(Object(Title), Object(RequestContext)) #13 /usr/local/apache/common-local/php-1.22wmf18/includes/Wiki.php(590): MediaWiki->performRequest() #14 /usr/local/apache/common-local/php-1.22wmf18/includes/Wiki.php(459): MediaWiki->main() #15 /usr/local/apache/common-local/php-1.22wmf18/index.php(55): MediaWiki->run() #16 /usr/local/apache/common-local/w/index.php(3): require('/usr/local/apac...') #17 {main}
Aaron: Could you investigate this?
Change 86286 had a related patch set uploaded by Aaron Schulz: Hide filebackend listing exceptions for thumbnail purges https://gerrit.wikimedia.org/r/86286
Change 86286 merged by jenkins-bot: Hide filebackend listing exceptions for thumbnail purges https://gerrit.wikimedia.org/r/86286
Aaron: Can this report be closed as RESOLVED FIXED (now that your patch is merged), or is more work needed?
It looks like that patch just hides the error when it happens? Or does it interact with something in such a way that the root cause is fixed?
Aaron's patch keeps the move process from breaking due to a bad response from swift while listing thumbnails at the prior location. He probably could have added a log message so we could track these events, but meh.
