Last modified: 2014-07-08 22:36:10 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T64094, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 62094 - Cannot pickle Page objects
Cannot pickle Page objects
Status: RESOLVED FIXED
Product: Pywikibot
Classification: Unclassified
General (Other open bugs)
core-(2.0)
All All
: Unprioritized major
: ---
Assigned To: Pywikipedia bugs
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-03-01 06:44 UTC by xqt
Modified: 2014-07-08 22:36 UTC (History)
4 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description xqt 2014-03-01 06:44:30 UTC
import pickle
import pywikibot

p = pywikibot.Page(pywikibot.Site('de'), 'user:Xqt/Test')
laFile = open('la.data', 'wb')
pickle.dump(p, laFile)
laFile.close()

causes the following error in core whereas it works in compat:


c:\Pywikipedia\core>pwb.py la2
Traceback (most recent call last):
  File "C:\Pywikipedia\core\pwb.py", line 143, in <module>
    run_python_file(fn, argv, argvu)
  File "C:\Pywikipedia\core\pwb.py", line 67, in run_python_file
    exec(compile(source, filename, "exec"), main_mod.__dict__)
  File "C:\Pywikipedia\core\scripts\la2.py", line 9, in <module>
    pickle.dump(p, laFile)
  File "C:\Python27\lib\pickle.py", line 1370, in dump
    Pickler(file, protocol).dump(obj)
  File "C:\Python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 663, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 663, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 663, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "C:\Python27\lib\copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle lock objects
<type 'exceptions.TypeError'>
CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort

c:\Pywikipedia\core>
Comment 1 Kunal Mehta (Legoktm) 2014-05-02 09:03:52 UTC
The reason for this is because Site objects are not pickle-able.

>>> import pywikibot, pickle
>>> s=pywikibot.Site()
>>> f=pickle.dumps(s)
Traceback (most recent call last):
[same traceback here]
  File "/home/km/python/lib64/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle lock objects
Comment 2 Gerrit Notification Bot 2014-05-02 09:09:15 UTC
Change 131032 had a related patch set uploaded by Legoktm:
Remove unused APISite.sitelock

https://gerrit.wikimedia.org/r/131032
Comment 3 Gerrit Notification Bot 2014-05-02 09:30:34 UTC
Change 131033 had a related patch set uploaded by Legoktm:
Make Site objects pickle-able

https://gerrit.wikimedia.org/r/131033
Comment 4 Gerrit Notification Bot 2014-05-02 12:44:42 UTC
Change 131032 merged by jenkins-bot:
Remove unused APISite.sitelock

https://gerrit.wikimedia.org/r/131032
Comment 5 xqt 2014-05-02 12:51:10 UTC
That bug is still there after the patch.
Comment 6 Merlijn van Deen (test) 2014-05-02 12:53:14 UTC
Yes:

"There's still a lock in BaseSite, but this gets us closer to
being able to pickle Pages."
Comment 7 Gerrit Notification Bot 2014-07-07 05:52:48 UTC
Change 131033 merged by jenkins-bot:
Make Site objects pickle-able

https://gerrit.wikimedia.org/r/131033
Comment 8 Gerrit Notification Bot 2014-07-08 22:10:28 UTC
Change 144698 had a related patch set uploaded by Legoktm:
Page pickling test

https://gerrit.wikimedia.org/r/144698
Comment 9 Gerrit Notification Bot 2014-07-08 22:11:56 UTC
Change 144698 merged by jenkins-bot:
Page pickling test

https://gerrit.wikimedia.org/r/144698
Comment 10 John Mark Vandenberg 2014-07-08 22:36:10 UTC
Still waiting for review of the unit test, but the bugs were all solved days ago.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links