Last modified: 2014-08-22 18:11:25 UTC
If we are trying to upload a lot of files and some of them already exist, it would be nice to have an option to automatically skip those uploads (because of resuming the batch upload operation, or because of manual upload of some of them) There's still no option to do a "batch upload", but I handle it with a script that runs a list of pywikibot commands in a batch. And it gets stuck waiting for a user response when it receives a warning, which is annoying.
Change 155560 had a related patch set uploaded by Martineznovo: upload.py: Add option to automatically abort upload on warning https://gerrit.wikimedia.org/r/155560
Well, Gerrit change #155560 is not exactly for warnings about existing files, but for any warning. This usually would do the trick. It's currently not possible to distinguish between different warnings without changing more things in site.py
It shouldnt be too difficult for api.py&site.py to provide upload.py with the exact warning received by the MW API. Is this fix urgent? Do you have a project that needs this checked in promptly, or can we work on trying to ignore only specific warnings?
Well, I did that change to my code to bypass this limitation, and it worked very well, that's why I shared my code pushing the change to gerrit, so for me the issue is already solved :) I'm not really fluent with python, since I've never worked before with it. I just know a bit of its syntax and changed the code to make it work, looking at how other parameters work. It could be improved to get the type of warning in the UploadWarning class, but I don't know if I'll be able to do that, so feel free to create a new patch solving this issue if you like
Change 155560 merged by jenkins-bot: upload.py: Add option to automatically abort upload on warning https://gerrit.wikimedia.org/r/155560
The type of warning is decoded into the exception message as follows: upload_warnings = { # map API warning codes to user error messages # %(msg)s will be replaced by message string from API responsse 'duplicate-archive': "The file is a duplicate of a deleted file %(msg)s.", 'was-deleted': "The file %(msg)s was previously deleted.", 'emptyfile': "File %(msg)s is empty.", 'exists': "File %(msg)s already exists.", 'duplicate': "Uploaded file is a duplicate of %(msg)s.", 'badfilename': "Target filename is invalid.", 'filetype-unwanted-type': "File %(msg)s type is unwanted type.", } So from the message, in a way, it is possile to understand the kind of warning. If this is not clean enough, it can be improved further.
Parsing a text is not very clean IMHO :) I was expecting UploadWarning to have a property that will return the error key from the api (the key values from upload_warnings), that would be a good way to handle this
I would suggest to add a "code" to the UploadWarning which uses the keys of the upload_warnings dictionary. So you could do something like: try: site.upload(…) except UploadWarning as e: if e.code == 'exists': # do stuff here: the file already exists It's still 'parsing a text' but far easier, but maybe somebody has a better idea.
Yes, that's what I was thinking. Maybe if I feel comfortable understanding how to do that I'll do that someday...
No worries, I'm currently working on a patch where you could simply parse the code.
Change 155739 had a related patch set uploaded by XZise: [FEAT] UploadWarning has uses the warning code https://gerrit.wikimedia.org/r/155739
Change 155739 merged by jenkins-bot: [FEAT] UploadWarning uses the warning code https://gerrit.wikimedia.org/r/155739
