Last modified: 2014-08-23 03:58:31 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T69488, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 67488 - Cache broken with non-ascii username/user namespace
Cache broken with non-ascii username/user namespace
Status: RESOLVED FIXED
Product: Pywikibot
Classification: Unclassified
General (Other open bugs)
core-(2.0)
All All
: High critical
: ---
Assigned To: Pywikipedia bugs
:
Depends on:
Blocks: 68215
  Show dependency treegraph
 
Reported: 2014-07-03 20:09 UTC by Merlijn van Deen (test)
Modified: 2014-08-23 03:58 UTC (History)
4 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Merlijn van Deen (test) 2014-07-03 20:09:33 UTC
Reported on IRC.

Password for user UserBot on mywiki:ar (no characters will be shown): 
Logging in to mywiki:ar as UserBot
Could not load cache: UnicodeDecodeError('ascii', 'Site("ar", "mywiki")User(\xd9\x85\xd8\xb3\xd8\xaa\xd8\xae\xd8\xaf\xd9\x85:UserBot)[(\'action\', \'paraminfo\'), (\'format\', \'json\'), (\'maxlag\', \'5\'), (\'querymodules\', \'info|revisions\')]', 31, 32, 'ordinal not in range(128)')
Traceback (most recent call last):
  File "pwb.py", line 153, in <module>
    run_python_file(fn, argv, argvu)
  File "pwb.py", line 67, in run_python_file
    exec(compile(source, filename, "exec"), main_mod.__dict__)
  File "scripts/pagefromfile.py", line 274, in <module>
    main()
  File "scripts/pagefromfile.py", line 271, in main
    bot.run()
  File "scripts/pagefromfile.py", line 85, in run
    self.save(title, contents)
  File "scripts/pagefromfile.py", line 141, in save
    page.save(comment, minor=self.getOption('minor'))
  File "/Users/user/shell/core/pywikibot/__init__.py", line 423, in wrapper
    return method(*__args, **__kw)
  File "/Users/user/shell/core/pywikibot/page.py", line 886, in save
    **kwargs)
  File "/Users/user/shell/core/pywikibot/page.py", line 896, in _save
    watch=watchval, bot=botflag, **kwargs)
  File "/Users/user/shell/core/pywikibot/site.py", line 557, in callee
    return fn(self, *args, **kwargs)
  File "/Users/user/shell/core/pywikibot/site.py", line 2787, in editpage
    lastrev = page.latestRevision()
  File "/Users/user/shell/core/pywikibot/page.py", line 364, in latestRevision
    self.site.loadrevisions(self)
  File "/Users/user/shell/core/pywikibot/site.py", line 1922, in loadrevisions
    step=step, total=total, **rvargs)
  File "/Users/user/shell/core/pywikibot/site.py", line 693, in _generator
    gen = gen_class(type_arg, site=self, **args)
  File "/Users/user/shell/core/pywikibot/data/api.py", line 885, in __init__
    QueryGenerator.__init__(self, prop=prop, **kwargs)
  File "/Users/user/shell/core/pywikibot/data/api.py", line 560, in __init__
    self.update_limit()  # sets self.prefix
  File "/Users/user/shell/core/pywikibot/data/api.py", line 654, in update_limit
    for param in self._modules[mod].get("parameters", []):
  File "/Users/user/shell/core/pywikibot/data/api.py", line 604, in _modules
    data = paramreq.submit()
  File "/Users/user/shell/core/pywikibot/data/api.py", line 510, in submit
    self._write_cache(self._data)
  File "/Users/user/shell/core/pywikibot/data/api.py", line 503, in _write_cache
    with open(self._cachefile_path(), 'wb') as f:
  File "/Users/user/shell/core/pywikibot/data/api.py", line 478, in _cachefile_path
    return os.path.join(self._get_cache_dir(), self._create_file_name())
  File "/Users/user/shell/core/pywikibot/data/api.py", line 474, in _create_file_name
    self._uniquedescriptionstr().encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd9 in position 31: ordinal not in range(128)
<type 'exceptions.UnicodeDecodeError'>
CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort



We should add a test with a non-ascii username/namespace (or we should just test on arwiki instead of enwiki :-p)
Comment 1 John Mark Vandenberg 2014-07-04 04:43:11 UTC
Is the backtrace in this example modified?  i.e. 'private' words replaced?

With the current code, I can log into ar:wikipedia and edit on Linux using 'JVbot' which should be functionally identical to the ascii 'UserBot'.

https://ar.wikipedia.org/wiki/User:John Vandenberg/test?action=history

I will continue investigating.
Comment 2 Philippe Elie 2014-07-18 00:54:02 UTC
This occur since July 13 or 14, the shared pywikibot on tools.wmflabs.org is broken too. Perhaps the shared version on labs should be reverted to a working revision?
Comment 3 Yann Forget 2014-07-18 12:16:38 UTC
It is broken since 5 days, and needs an urgent fix. Thanks, Yann
Comment 4 John Mark Vandenberg 2014-07-18 14:02:20 UTC
The change for this bug was merged three weeks ago.  Maybe labs only updated their code 5 days ago?

I'll try harder to reproduce it tomorrow, and provide a fix when I do.

I had a patch that probably fixed this, and other bugs, but it wasnt being reviewed for a week, so I started splitting it up.
https://gerrit.wikimedia.org/r/#/c/143038/
Comment 5 John Mark Vandenberg 2014-07-29 02:31:41 UTC
https://gerrit.wikimedia.org/r/#/c/149898

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links