Last modified: 2012-06-07 00:13:15 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T38440, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 36440 - Images permanently disappear when renaming files on Commons
Images permanently disappear when renaming files on Commons
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
General/Unknown (Other open bugs)
unspecified
All All
: Highest critical with 1 vote (vote)
: ---
Assigned To: Aaron Schulz
: code-update-regression
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-02 13:52 UTC by material.scientist
Modified: 2012-06-07 00:13 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description material.scientist 2012-05-02 13:52:20 UTC
At least three images have disappeared from Commons as a result of file renaming over the last few weeks. All renames were carried out using the 

http://commons.wikimedia.org/wiki/Template:Rename

template. Nothing abnormal was seen during these renames, except for the "usual" general red error box (something like API error - retry/cancel/abort). 

1. http://commons.wikimedia.org/wiki/File:Capuchin_monastery_in_Lubart%C3%B3w,_Poland.jpg

2. http://commons.wikimedia.org/wiki/Commons:Administrators%27_noticeboard/Archive_35#Botched_file_move

3. http://commons.wikimedia.org/wiki/Commons:Deletion_requests/File:Vail_Hotel.jpg
(there were more, but can't find links)
Comment 1 Aaron Schulz 2012-05-04 15:00:03 UTC
You should try moving the page back and seeing if the file looks OK again. This is typically how move failures are handled (or delete/restore for that matter).

Does this only happen via API?
Comment 2 Aaron Schulz 2012-05-04 23:00:00 UTC
(In reply to comment #1)
> You should try moving the page back and seeing if the file looks OK again. This
> is typically how move failures are handled (or delete/restore for that matter).
> 
Looking around via shell it looks like the old name copies were deleted too :(

https://gerrit.wikimedia.org/r/6638 deployed in response. That is a confirmed issue, though I don't no for sure it it caused this.
Comment 3 material.scientist 2012-05-06 22:42:52 UTC
Same thing happened since with 
http://commons.wikimedia.org/wiki/File:Crypte_de_l%27%C3%A9glise_de_Gargilesse-Resurection_des_morts_G-20120429.jpg

Here, there might have been a move conflict: I've tried to move that file and http://commons.wikimedia.org/wiki/File:Crypte_de_l%27%C3%A9glise_de_Gargilesse-R%C3%A9surection_des_morts_D-20120429.jpg, and the final move might be same (uploader requested, and it was hard to see what he/she wants)
Comment 4 Aaron Schulz 2012-05-07 01:19:11 UTC
https://gerrit.wikimedia.org/r/6638 has caught several errors according to the logs, but those were just for uploads. So something else is also going on.

Looking at LocalFileMoveBatch, I can see that the begin() calls break the file locking (which uses FOR UPDATE). Note that the file locking hasn't worked at for anything until the last few months, so I'm not sure that's the cause.
Comment 5 Aaron Schulz 2012-05-07 01:28:41 UTC
(In reply to comment #4)
> Looking at LocalFileMoveBatch, I can see that the begin() calls break the file
> locking (which uses FOR UPDATE). Note that the file locking hasn't worked at
> for anything until the last few months, so I'm not sure that's the cause.

*hasn't worked at all
Comment 6 material.scientist 2012-05-07 03:16:34 UTC
Naive comments: (i) upload mistakes do block file moves - we have to delete failed uploads in a file page to unlock the move (or we get that red API error box, without telling the problem). (ii) I thought maybe move conflicts are a factor? To clarify: in this case I tried to move two files into the same destination from 2 firefox windows (one got "moved" one got API error). Often two editors try to move one file simultaneously, and I see that only indirectly (as a glitch with another mover).
Comment 7 Aaron Schulz 2012-05-07 06:37:41 UTC
See also: https://gerrit.wikimedia.org/r/#/c/6783/
Comment 8 material.scientist 2012-05-07 22:26:53 UTC
One more from minutes ago
http://commons.wikimedia.org/wiki/File:Mrpfpindiraghandi.jpg
This happened before: an image disappears from the file "after clicking on the button in the rename template" without a log (well, at least on my screen).
Comment 9 Aaron Schulz 2012-05-08 21:49:20 UTC
(In reply to comment #7)
> See also: https://gerrit.wikimedia.org/r/#/c/6783/

Deployed now.
Comment 10 Aaron Schulz 2012-05-10 22:57:11 UTC
(In reply to comment #9)
> (In reply to comment #7)
> > See also: https://gerrit.wikimedia.org/r/#/c/6783/
> 
> Deployed now.

Have there been instances since that comment?
Comment 11 Mark A. Hershberger 2012-05-11 13:49:13 UTC
I've asked for input http://commons.wikimedia.org/w/index.php?diff=71016664&oldid=71005678 and we'll see if we get any.
Comment 12 material.scientist 2012-05-12 05:21:47 UTC
API errors are common with me these days, i.e. renaming scrip fails with 50/50 chances, but the files are not lost (just have to manually complete the rename). That said, have a look at
http://commons.wikimedia.org/wiki/File:Felimare_fontandraui.jpg
I haven't touched that file and was only involved with commonsdelinker - looks like a double move (Hypselodoris fontandraui (Pruvot-Fol, 1951) 0 .jpg -> Felimare fontandraui (Pruvot-Fol, 1951).jpg -> Felimare_fontandraui.jpg) with the same image disappearance as when I started this thread. The image thumbnail can be found by ImageGoogling "Hypselodoris fontandraui (Pruvot-Fol, 1951) 0 .jpg" - a violet horizontal slag.
Comment 13 Aaron Schulz 2012-05-19 08:36:07 UTC
Maybe a list of affected files can be kept somewhere?
Comment 14 Aaron Schulz 2012-05-25 19:59:14 UTC
(In reply to comment #12)
> API errors are common with me these days, i.e. renaming scrip fails with 50/50
> chances, but the files are not lost (just have to manually complete the
> rename). That said, have a look at
> http://commons.wikimedia.org/wiki/File:Felimare_fontandraui.jpg

Have you ran into any new instances?
Comment 15 material.scientist 2012-06-02 23:14:58 UTC
(In reply to comment #14)
> (In reply to comment #12)
> > API errors are common with me these days, i.e. renaming scrip fails with 50/50
> > chances, but the files are not lost (just have to manually complete the
> > rename). That said, have a look at
> > http://commons.wikimedia.org/wiki/File:Felimare_fontandraui.jpg
> 
> Have you ran into any new instances?

http://commons.wikimedia.org/wiki/File:Arjuna_and_His_Charioteer_Krishna_Confront_Karna,_crop.jpg 
Images (being renamed) temporarily disappear during database delays (during which the renaming script also fails) but the above seems permanent.
Comment 16 Aaron Schulz 2012-06-03 08:57:42 UTC
See https://gerrit.wikimedia.org/r/9878
Comment 17 material.scientist 2012-06-04 22:20:31 UTC
Two fresh losses (looks like double moves have a high chance of failure) 
http://commons.wikimedia.org/wiki/File:2010-08-04_%2827%29_Purpur-Fruchtwanze,_Carpocoris_purpureipennis.jpg
http://commons.wikimedia.org/wiki/File:BSicon_kSTRq%2Br.svg
Comment 18 Materialscientist 2012-06-05 04:57:49 UTC
(In reply to comment #17)
> Two fresh losses (looks like double moves have a high chance of failure) 
> http://commons.wikimedia.org/wiki/File:2010-08-04_%2827%29_Purpur-Fruchtwanze,_Carpocoris_purpureipennis.jpg
> http://commons.wikimedia.org/wiki/File:BSicon_kSTRq%2Br.svg
:Sorry, the BSicon file (2nd) was not a loss [a file mover managed to manually overwrite a redirect with a page that looked like a lost image image to me :-].
Comment 19 Aaron Schulz 2012-06-05 22:35:18 UTC
(In reply to comment #16)
> See https://gerrit.wikimedia.org/r/9878

Deployed just now.
Comment 20 Dereckson 2012-06-06 08:35:47 UTC
Bug status set to ASSIGNED (previously: NEW).

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links