Last modified: 2014-05-19 20:54:01 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T56776, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 54776 - Do something about a few thousands files with empty oi_archive_name
Do something about a few thousands files with empty oi_archive_name
Status: NEW
Product: Wikimedia
Classification: Unclassified
Media storage (Other open bugs)
unspecified
All All
: Normal major (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks: 35367
  Show dependency treegraph
 
Reported: 2013-09-30 03:48 UTC by Nemo
Modified: 2014-05-19 20:54 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
Filename and timestamp with empty oi_archive_name (687.83 KB, text/plain)
2013-09-30 03:48 UTC, Nemo
Details

Description Nemo 2013-09-30 03:48:11 UTC
Created attachment 13404 [details]
Filename and timestamp with empty oi_archive_name

(Bawolff in bug 35637 comment #8)
> Well the easiest to find would be everything select oi_name,  oi_timestamp
> from
> oldimage where oi_archive_name = ''; this could be done by anyone with labs
> 
> After that one can look in the thumbnail log. From what I've seen of it, its
> full of line about thumbnail failed due to missing src path (this seems to be
> the main cause of failing png thumbnails now that vips has removed the size
> limit on that format)
> 
> As an aside, It'd be nice if we graphed number of missing files somewhere in
> ganglia. Ancedotally it seems like there are more of them then there used to
> be. It would be good to get real stats on this very scary problem.

$ grep -oE "[0-9]{14}$" oi_archive_name-empty.txt | grep -oE "^[0-9]{6}" | sort | uniq -c
      3 200410
     32 200411
     16 200412
     11 200501
      9 200502
     13 200503
     11 200504
      6 200505
     18 200506
     17 200507
      7 200508
      6 200509
     10 200510
     28 200511
     12 200512
      8 200601
     20 200602
     17 200603
     27 200604
     12 200605
     11 200606
     13 200607
     16 200608
      8 200609
      9 200610
     21 200611
     10 200612
     31 200701
     13 200702
     39 200703
      5 200704
     23 200705
     16 200706
     19 200707
     18 200708
     13 200709
     15 200710
      9 200711
     16 200712
     10 200801
      9 200802
     24 200803
     14 200804
     12 200805
      3 200806
      7 200807
      4 200808
      2 200809
      1 200810
      6 200811
      4 200812
      2 200901
      1 200902
      1 200903
      1 200904
     10 200905
     44 200907
      4 200910
      2 200911
      3 200912
      5 201001
      5 201002
      2 201004
      1 201005
      7 201006
      3 201007
      2 201009
      1 201012
      1 201101
      1 201104
      2 201105
      2 201106
      1 201107
     17 201108
    253 201109
     41 201110
     73 201111
     87 201112
     53 201201
     63 201202
    102 201203
    321 201204
    366 201205
    215 201206
    833 201207
    142 201208
   2613 201209
    636 201210
    307 201211
    351 201212
    345 201301
    383 201302
    358 201303
    495 201304
    332 201305
    379 201306
    373 201307
    521 201308
   2374 201309
Comment 1 Bawolff (Brian Wolff) 2013-09-30 06:31:25 UTC
Most recent ones might be due to the upload wizard making it easier for users to hit race condition in file backend (don't have bug number off hand)
Comment 2 Nemo 2013-09-30 06:39:28 UTC
(In reply to comment #1)
> Most recent ones might be due to the upload wizard making it easier for users
> to hit race condition in file backend (don't have bug number off hand)

Yes, that's my suspicion as well. Sometimes uploaded files go in the wrong place (multiple files under the same title) and when you fix it (by reuploading with Special:Upload) old versions sometimes disappear: bug 40304.
There are peaks for every WikiLovesMonuments so it must be related to UploadWizard uploads, not – say – bot uploads.
Comment 3 Bawolff (Brian Wolff) 2013-09-30 11:42:26 UTC
(In reply to comment #1)
> Most recent ones might be due to the upload wizard making it easier for users
> to hit race condition in file backend (don't have bug number off hand)

For reference bug 54750 was what i was thinking of.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links