Last modified: 2014-07-03 13:04:15 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T50205, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 48205 - Server-side upload of large number of files to Commons
Server-side upload of large number of files to Commons
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Site requests (Other open bugs)
wmf-deployment
All All
: Normal enhancement (vote)
: ---
Assigned To: Sam Reed (reedy)
: shell
Depends on: 55541
Blocks:
  Show dependency treegraph
 
Reported: 2013-05-07 10:43 UTC by Lokal_Profil
Modified: 2014-07-03 13:04 UTC (History)
11 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Lokal_Profil 2013-05-07 10:43:49 UTC
I'm currently in contact with a Swedish museum which has decided to release all of it's high resolution images on Commons. In total this is about 15000 images with file-sizes ranging between 50 and 125 MB.

As this ends up being roughly 1 TB of material I wanted to investigate the possibility of simply sending a harddrive with the images (correctly named and with description pages pre-generated) for a server-side upload to be made rather than uploading each one remotely.

This would of course only occur once the test stages of a normal batch uploading process are completed and any issues regarding descriptions/categorisation have been dealt with.
Comment 1 Tomasz W. Kozlowski 2013-05-07 15:09:59 UTC
CC'ing Roan and Sam since they are some of the people responsible for server-side uploads.

(Roan uploaded Wikimania 2011 videos to Commons [lots of data!], and Sam deals with upload requests on a regular basis.)
Comment 2 Andre Klapper 2013-05-14 13:17:41 UTC
Tentatively assigning this to Reedy - feel free to reassign.
Comment 3 Sam Reed (reedy) 2013-05-14 15:00:28 UTC
Yeah, this has been done before, so there is a likelihood of being able to do so again.

If you want to leave a message on this bug when you're in a position to actually send the hard drive, we can loop in the relevant people to move forward, and should be able to accommodate.

If we give you the address for the data centre, then after sending it by an appropriate tracked courier, this should be sufficient. We need to follow procedures to inform the data centre of pending deliveries etc. But this won't be a problem
Comment 4 Rob Lanphier 2013-05-15 16:13:15 UTC
I believe that we can do this.  Sam and I discussed, and this seems like a perfect case for "Resolved/Later" (if we still had that resolution), so I'm reassigning to Andre to figure out the right resolution for this.

I'm cc'ing Faidon so that he's aware of this, and can weigh in if needed.
Comment 5 Lokal_Profil 2013-05-16 12:40:38 UTC
Many thanks. I'll drop a new comment here when we have fixed the descriptions and gotten feedback from the first test uploads.

Would the drive have to be sent to the data centre in the US or does the one in Amsterdam have the same functionality/possibility?
Comment 6 Sam Reed (reedy) 2013-05-16 22:44:39 UTC
(In reply to comment #5)
> Would the drive have to be sent to the data centre in the US or does the one
> in
> Amsterdam have the same functionality/possibility?

It would have to be in the US (EQIAD - Washington DC), the Amsterdam one isn't really setup for working on these sorts of requests, and would be quicker to get processed in EQIAD anyway
Comment 7 Andre Klapper 2013-06-14 20:47:45 UTC
(In reply to comment #4)
> I believe that we can do this.  Sam and I discussed, and this seems like a
> perfect case for "Resolved/Later" (if we still had that resolution), so I'm
> reassigning to Andre to figure out the right resolution for this.

This should probably be assigned to the reporter (Lokal_Profil) until the drive has been sent, and after it has been sent, reassigned to somebody in EQIAD/ops.

Lokal_Profil: Any progress / status update that you could share? :)
Comment 8 Lokal_Profil 2013-06-16 13:35:15 UTC
(In reply to comment #7)
> (In reply to comment #4)
> > I believe that we can do this.  Sam and I discussed, and this seems like a
> > perfect case for "Resolved/Later" (if we still had that resolution), so I'm
> > reassigning to Andre to figure out the right resolution for this.
> 
> This should probably be assigned to the reporter (Lokal_Profil) until the
> drive
> has been sent, and after it has been sent, reassigned to somebody in
> EQIAD/ops.
> 
> Lokal_Profil: Any progress / status update that you could share? :)

Sorry for the silence. Have the drive and have built the scripts for outputting the descriptions. Awaiting some additional mapping between keywords and categories and will then proceed with a few test uploads (200-500 depending on feedback). Due to summer vacations (most people are away in July) the drive probably won't be ready to ship until end of August though.

A few practical questions. Currently the files are placed in several sub-directories, I assume this won't be a problem (all files on the drive are intended for upload). Filename-wise, should each file have the same filename as the target on Commons or is it ok to have the target filename be the first line in the description file?

If either is an issue then I'll rename and shift them all into the same directory prior to sending the drive.
Comment 9 Sam Reed (reedy) 2013-06-27 19:03:43 UTC
(In reply to comment #8)
> A few practical questions. Currently the files are placed in several
> sub-directories, I assume this won't be a problem (all files on the drive are
> intended for upload). Filename-wise, should each file have the same filename
> as
> the target on Commons or is it ok to have the target filename be the first
> line
> in the description file?
> 
> If either is an issue then I'll rename and shift them all into the same
> directory prior to sending the drive.

For automation purposes having ImageFileName.ext and ImageFileName.txt or similar would be the easiest way, with ImageFileName being where you want it to go on Commons.

The usually used import script can search and import recursively. So having them in whatever sort of structure should be fine to make organisation on your end easier, and have no difference on the final upload. That is, as long as the descriptions files are in the same directory as the image file.
Comment 10 Lokal_Profil 2013-09-13 10:39:27 UTC
Ok. As suggested the naming scheme is ImageFileName.ext and ImageFileName.txt, where ImageFileName is the target filename on Commons.

After more holiday related delays the drive is now almost ready to be shipped. Barring any complaints on Commons to the latest file descriptions I have only to fix a few files which weren't converted correctly by Imagemagick and make a backup of everything to a separate drive.
Comment 11 Lokal_Profil 2013-09-20 13:31:07 UTC
Drive is now ready!

If you supply me with address/delivery instructions I'll post it sometime next week.
Comment 12 Rob Halsell 2013-09-20 18:21:10 UTC
So we can have the drive shipped directly to our datacenter, but there are a few restrictions for it:

- Shipment cannot go USPS, has to go via FedEx, UPS, or some other third party vendor.

- We have to put a ticket in with the datacenter, so we'll need the tracking information.

I'm not entirely comfortable just putting our address for the datacenter in the ticket.  So I'll be emailing the requestor directly with the address.  It is not exactly secret, but may as well not paste it here.
Comment 13 Rob Halsell 2013-09-20 18:26:01 UTC
I've emailed Lokal directly, including both office and datacenter addresses.

If the shipment is via USPS, it will come to the office first, so I can send it on via FedEx.  (Datacenter won't receive USPS for customers.)
Comment 14 Lokal_Profil 2013-09-25 15:57:08 UTC
Drive has now been sent (via FedEx)
Comment 15 Rob Halsell 2013-09-27 19:45:20 UTC
Please note that Lokal has requested the following for the image import:

When it is time the uploads should be tied to the "LSHuploadBot" account rather than my private account since I will pass this account on to the museum for future (much smaller) uploads.
Comment 16 Chris Johnson 2013-09-27 19:50:51 UTC
(In reply to comment #14)
> Drive has now been sent (via FedEx)

The drive arrived to the Virginia data center today.
Comment 17 Lokal_Profil 2013-10-07 11:02:10 UTC
Thanks. Is there a rough estimate for when the server side upload will be done? Asking since the museum needs to prepare their press release.
Comment 18 Sam Reed (reedy) 2013-10-07 19:53:34 UTC
(In reply to comment #17)
> Thanks. Is there a rough estimate for when the server side upload will be
> done?
> Asking since the museum needs to prepare their press release.

https://commons.wikimedia.org/w/index.php?title=Special%3AListFiles&limit=600&user=LSHuploadBot

There isn't currently. I've imported the m_o and m_t folders (18G or so) today. Noticed one that didn't thumbnail. Poking it further it seems it might have been one of the uploads that failed. Will go and fix that soonish for you.

Broken: https://commons.wikimedia.org/wiki/File:M%C3%B6beluppsats,_12_delar,_VI:II:A.a.11_-_Hallwylska_museet_-_39426.tif

Relatedly, there's some redlink categories such as https://commons.wikimedia.org/w/index.php?title=Category:Furniture_in_the_Hallwyl_museum&action=edit&redlink=1


Seems it did ~520 images (18G) in around 100 (97, lets round up) minutes or so...

1.3TB of images = 1331.2G

(1331.2/18)*100/60/24 = 5 days at full speed

Obviously, we don't want to overload our servers with work (and cause other problems), though the image scalers would only be creating images when anyone tries to view thumbnails of those images - Special:NewFiles etc.
Comment 19 Lokal_Profil 2013-10-08 10:05:40 UTC
Thanks for the update.

The red-linked categories seem to be the ones at https://commons.wikimedia.org/wiki/Commons:Batch_uploading/LSH/ObjKeywords#class:_HWY-grupp . I've fixed those.

I've encountered the thumbnailing issue before. Sometimes the thumbnail is correctly generated one time only to be gone the next one. Not exactly sure what this depends on.

Thanks also for the time estimate. Based on this I'd guessed that (bar anything unexpected) the images could be ingested in roughly two weeks? Either way I'll get the museum started on writing their press release and blog posts.

Many thanks,
André
Comment 20 Sam Reed (reedy) 2013-10-08 10:29:49 UTC
As a heads up, Rob has pointed the Wikimedia Communications department at this bug and this donation. There is a reasonably chance there could be a blog post with regards to this rather large donation of files! :)
Comment 21 Sam Reed (reedy) 2013-10-08 12:36:38 UTC
Is it possible that numerous files already exist on commons? I noticed there were some other uploads a while ago, but doing the m_a folder I get the following stats:

Found: 2140
Added: 988
Skipped: 1152

Just want to make sure there's no issue here I need to check/resolve
Comment 22 Lokal_Profil 2013-10-08 13:27:22 UTC
I removed all of the photos I uploaded during the testing from the drive before sending it so none of them should have caused any skipping.

Is it possible to bring up one or two of the skipped filenames and the files on commons which caused them to skip?
Comment 23 Sam Reed (reedy) 2013-10-08 13:34:06 UTC
Oh, wait. It's presumably me being stupid. Chances of it running through a whole large folder in one go is quite low; with there being many reasons it may fail, it's going to find one of them.

Which was what happened overnight, and I started it again. Of course a load of them are going to be skipped on the second run
Comment 24 Sam Reed (reedy) 2013-10-14 00:41:52 UTC
Just as an aside for bug 55541, there seems to be a handful of files that upset the PHP EXIF code.

As such, I'm just moving them into a different directory out of the way, so we can continue uploading other files.

The files currently are:

reedy@terbium:/media/external/keepers$ ls -al notuploaded/
total 657060
drwxrwxrwx 1 root root      4096 Oct 14 00:40 .
drwxrwxrwx 1 root root      4096 Oct 14 00:33 ..
-rwxrwxrwx 1 root root  99737948 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_51015.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:58 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_51015.txt
-rwxrwxrwx 1 root root  64282362 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59574.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:57 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59574.txt
-rwxrwxrwx 1 root root  55393232 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59575.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:58 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59575.txt
-rwxrwxrwx 1 root root  87114228 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_68136.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:57 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_68136.txt
-rwxrwxrwx 1 root root 126220272 Nov  6  2012 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87780.tif
-rwxrwxrwx 1 root root      1537 Aug 26 14:58 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87780.txt
-rwxrwxrwx 1 root root 126201190 Oct  9 23:01 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87785.tif
-rwxrwxrwx 1 root root      1093 Oct  9 23:01 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87785.txt
-rwxrwxrwx 1 root root 113828698 Nov  6  2012 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87786.tif
-rwxrwxrwx 1 root root      1124 Aug 26 14:58 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87786.txt
Comment 25 Sam Reed (reedy) 2013-10-14 17:06:23 UTC
All top level folders bar m_b have been uploaded, noting the exceptions below. We would seem to be around a third of the files done - 378.4GB uploaded, 836 to go.


The following files from m_dig haven't been uploaded due to bug 55541

reedy@terbium:/media/external/keepers$ ls -al m_dig-notuploaded/
total 1731476
drwxrwxrwx 1 root root     20480 Oct 14 15:13 .
drwxrwxrwx 1 root root      4096 Oct 14 16:58 ..
-rwxrwxrwx 1 root root  99737948 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_51015.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:58 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_51015.txt
-rwxrwxrwx 1 root root  64282362 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59574.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:57 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59574.txt
-rwxrwxrwx 1 root root  55393232 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59575.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:58 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59575.txt
-rwxrwxrwx 1 root root  87114228 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_68136.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:57 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_68136.txt
-rwxrwxrwx 1 root root  99750768 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33898.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33898.txt
-rwxrwxrwx 1 root root  71847212 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33899.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33899.txt
-rwxrwxrwx 1 root root  85470850 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33900.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33900.txt
-rwxrwxrwx 1 root root  99762232 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42456.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42456.txt
-rwxrwxrwx 1 root root  99748964 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42457.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42457.txt
-rwxrwxrwx 1 root root  99753286 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42458.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42458.txt
-rwxrwxrwx 1 root root  91795152 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42459.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42459.txt
-rwxrwxrwx 1 root root  99763146 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_51016.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_51016.txt
-rwxrwxrwx 1 root root  99754792 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_85258.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_85258.txt
-rwxrwxrwx 1 root root 126227546 Nov  6  2012 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87778.tif
-rwxrwxrwx 1 root root      1815 Aug 26 14:58 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87778.txt
-rwxrwxrwx 1 root root 126244040 Nov  6  2012 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87779.tif
-rwxrwxrwx 1 root root      1699 Aug 26 14:58 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87779.txt
-rwxrwxrwx 1 root root 126220272 Nov  6  2012 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87780.tif
-rwxrwxrwx 1 root root      1537 Aug 26 14:58 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87780.txt
-rwxrwxrwx 1 root root 126201190 Oct  9 23:01 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87785.tif
-rwxrwxrwx 1 root root      1093 Oct  9 23:01 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87785.txt
-rwxrwxrwx 1 root root 113828698 Nov  6  2012 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87786.tif
-rwxrwxrwx 1 root root      1124 Aug 26 14:58 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87786.txt
reedy@terbium:/media/external/keepers$
Comment 26 Lokal_Profil 2013-10-15 10:25:39 UTC
Many Thanks.

Do we know what triggers bug 55541?
Comment 27 Sam Reed (reedy) 2013-10-15 13:01:55 UTC
(In reply to comment #26)
> Many Thanks.
> 
> Do we know what triggers bug 55541?

I don't other than it being some value in the EXIF data that the code doesn't account for correctly, but Tim might be able to elaborate and suggest if we can fix the metadata to then upload it.
Comment 28 Sam Reed (reedy) 2013-10-18 15:36:30 UTC
Ok, so these are all done now, bar 26 that fail due to bug 55541


(In reply to comment #13)
> (In reply to comment #12)
> > Tim, do you know which are the erroneous values that the code is falling
> > over?
> > Just wondering if there's anything we can do to modify those values so we can
> > import the images into Commons
> 
> Maybe a null conversion in ImageMagick would fix it. Or something like
> http://paste.tstarling.com/p/IKYsdr.html would probably work as a temporary
> patch.
Comment 29 Lokal_Profil 2013-10-18 15:58:39 UTC
Many thanks!

I'll tell the museum to push announcements etc. (probably won't get e-mails until Monday).

Are those 26 so problematic that they would fail a manual upload as well? If not then I'll just do them by hand.

Noticed two things during the upload.
First is that the edit comments seemed to get stuck. I.e. everything since "File:Rysk mässkåpa - Livrustkammaren - 35643-negative.tif" (201310172030) got its edit comment. Not an actual problem but looks a bit weird

A few of the files persistently refuse to generate thumbnails, e.g.
* https://commons.wikimedia.org/wiki/File:Vas_med_frostblommor_-_Hallwylska_museet_-_87149.tif
* https://commons.wikimedia.org/wiki/File:%C3%84lghudskyller,_Sverige_1600-talets_b%C3%B6rjan_-_Livrustkammaren_-_43603.tif

Is there an easy way of seeing if this is due to something during the upload or something intrinsic in the files?
Comment 30 Sam Reed (reedy) 2013-10-18 17:07:23 UTC
If something won't ever thumbnail, check and see if you can view the original (click on the big thumbnail)

So for

https://commons.wikimedia.org/wiki/File:Vas_med_frostblommor_-_Hallwylska_museet_-_87149.tif

is

https://upload.wikimedia.org/wikipedia/commons/c/ca/Vas_med_frostblommor_-_Hallwylska_museet_-_87149.tif

It's giving a 404. So it needs reuploading.

Same for

https://commons.wikimedia.org/wiki/File:%C3%84lghudskyller,_Sverige_1600-talets_b%C3%B6rjan_-_Livrustkammaren_-_43603.tif

https://upload.wikimedia.org/wikipedia/commons/b/b9/%C3%84lghudskyller%2C_Sverige_1600-talets_b%C3%B6rjan_-_Livrustkammaren_-_43603.tif

I have now re-uploaded them.


Uploading them manually is going to have the same problem; the issue lies with extracting stuff EXIF data, so no matter how you try, it's always going to fail. The fix being to patch the library as Tim suggested, or push it through something like imagemagick to see if it normalises/fixes the error the PHP EXIF code doesn't like.
Comment 31 Sam Reed (reedy) 2013-10-18 17:10:42 UTC
(In reply to comment #29)
> Noticed two things during the upload.
> First is that the edit comments seemed to get stuck. I.e. everything since
> "File:Rysk mässkåpa - Livrustkammaren - 35643-negative.tif" (201310172030)
> got
> its edit comment. Not an actual problem but looks a bit weird

The first image uploads fine with the correct summary, but all following images use the same edit summary?

(change visibility) 02:51, 18 October 2013 (diff | hist) . . (+926)‎ . . N File:Begravningsvapen Strängnäs domkyrka - E 15871 - Livrustkammaren - 26663.tif ‎ ({{LSH artwork |artist= |title= {{Title|Rysk mässkåpa.}} |description= {{sv|Rysk mässkåpa, i formen rundskuren, klockformad med en öppning för huvudet som är något rundad i nacken och rak i kanterna på sidorna. Av rött turkiskt siden kallat lamp) (current)
(change visibility) 02:51, 18 October 2013 (diff | hist) . . (+772)‎ . . N File:Begravningsvapen Strängnäs domkyrka - E 15871 - Livrustkammaren - 26663-negative.tif ‎ ({{LSH artwork |artist= |title= {{Title|Rysk mässkåpa.}} |description= {{sv|Rysk mässkåpa, i formen rundskuren, klockformad med en öppning för huvudet som är något rundad i nacken och rak i kanterna på sidorna. Av rött turkiskt siden kallat lamp) (current)

Looks like a bug to me.
Comment 32 Lokal_Profil 2013-10-21 08:18:58 UTC
(In reply to comment #30)
> Uploading them manually is going to have the same problem; the issue lies
> with
> extracting stuff EXIF data, so no matter how you try, it's always going to
> fail. The fix being to patch the library as Tim suggested, or push it through
> something like imagemagick to see if it normalises/fixes the error the PHP
> EXIF code doesn't like.

If the list above is complete then I'll run the images through imagemagick and see if I can upload them afterwards.

Once again many thanks for the help.
Comment 33 Sam Reed (reedy) 2013-10-21 09:10:00 UTC
Complete lits:

reedy@terbium:~$
reedy@terbium:~$ screen -r
-rwxrwxrwx 1 root root  66988870 Sep 13 03:21 Värja,_defekt,_Frankrike,_1600-talets_mitt,_klinga_och_fäste_ej_ursprungligen_samhörande_-_Livrustkammaren_-_18215.tif
-rwxrwxrwx 1 root root      2482 Sep 20 07:57 Värja,_defekt,_Frankrike,_1600-talets_mitt,_klinga_och_fäste_ej_ursprungligen_samhörande_-_Livrustkammaren_-_18215.txt

m_dig-notuploaded:
total 1731476
drwxrwxrwx 1 root root     20480 Oct 14 15:13 .
drwxrwxrwx 1 root root      4096 Oct 18 17:00 ..
-rwxrwxrwx 1 root root  99737948 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_51015.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:58 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_51015.txt
-rwxrwxrwx 1 root root  64282362 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59574.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:57 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59574.txt
-rwxrwxrwx 1 root root  55393232 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59575.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:58 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_59575.txt
-rwxrwxrwx 1 root root  87114228 Nov  6  2012 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_68136.tif
-rwxrwxrwx 1 root root       736 Aug 26 14:57 Dokumentation,_utställningen_"Den_oumbärliga_väskan"_-_år_2008_-_Livrustkammaren_-_68136.txt
-rwxrwxrwx 1 root root  99750768 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33898.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33898.txt
-rwxrwxrwx 1 root root  71847212 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33899.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33899.txt
-rwxrwxrwx 1 root root  85470850 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33900.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_33900.txt
-rwxrwxrwx 1 root root  99762232 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42456.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42456.txt
-rwxrwxrwx 1 root root  99748964 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42457.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42457.txt
-rwxrwxrwx 1 root root  99753286 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42458.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42458.txt
-rwxrwxrwx 1 root root  91795152 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42459.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_42459.txt
-rwxrwxrwx 1 root root  99763146 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_51016.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_51016.txt
-rwxrwxrwx 1 root root  99754792 Nov  6  2012 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_85258.tif
-rwxrwxrwx 1 root root       717 Aug 26 14:58 Dokumentation,_utställningen_"Krigsbyte"_2007-2008_-_Livrustkammaren_-_85258.txt
-rwxrwxrwx 1 root root 126227546 Nov  6  2012 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87778.tif
-rwxrwxrwx 1 root root      1815 Aug 26 14:58 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87778.txt
-rwxrwxrwx 1 root root 126244040 Nov  6  2012 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87779.tif
-rwxrwxrwx 1 root root      1699 Aug 26 14:58 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87779.txt
-rwxrwxrwx 1 root root 126220272 Nov  6  2012 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87780.tif
-rwxrwxrwx 1 root root      1537 Aug 26 14:58 Pressbilder_till_Ars_11_-_Hallwylska_museet_-_87780.txt
-rwxrwxrwx 1 root root 126201190 Oct  9 23:01 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87785.tif
-rwxrwxrwx 1 root root      1093 Oct  9 23:01 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87785.txt
-rwxrwxrwx 1 root root 113828698 Nov  6  2012 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87786.tif
-rwxrwxrwx 1 root root      1124 Aug 26 14:58 Pressbilder_till_Ars_11,_reprofotografi_av_bild_från_resealbum_-_Hallwylska_museet_-_87786.txt
reedy@terbium:/media/external/keepers$

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links