Last modified: 2014-01-11 14:08:17 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T60976, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 58976 - PreloadingItemGenerator crashes with NoPage exception
PreloadingItemGenerator crashes with NoPage exception
Status: RESOLVED FIXED
Product: Pywikibot
Classification: Unclassified
Wikidata (Other open bugs)
core-(2.0)
All All
: Unprioritized normal
: ---
Assigned To: xqt
:
Depends on: 54999
Blocks:
  Show dependency treegraph
 
Reported: 2013-12-26 20:10 UTC by Maarten Dammers
Modified: 2014-01-11 14:08 UTC (History)
1 user (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Maarten Dammers 2013-12-26 20:10:06 UTC
The PreloadingItemGenerator does what the name suggests: Preload Wikidata items so that these can be processed quickly.

When it encounters an item that has been deleted it crashes with a "pywikibot.exceptions.NoPage: Page [[wikidata:-1]] doesn't exist." exception. 

It shouldn't crash. It should either:
* Not preload the item and pass the empty item (and let downstream check)
* Skip the item

Probably best to have the same behavior as with the PreloadingPageGenerator (or does that one crash too?)

Full trace:

Traceback (most recent call last):
  File "C:\pywikibot\core\cc_to_sitelinks.py", line 131, in <module>
    main()
  File "C:\pywikibot\core\cc_to_sitelinks.py", line 128, in main
    bot.run()
  File "C:\pywikibot\core\cc_to_sitelinks.py", line 31, in run
    for item in self.generator:
  File "C:\pywikibot\core\pywikibot\pagegenerators.py", line 890, in PreloadingI
temGenerator
    for i in site.preloaditempages(group, step):
  File "C:\pywikibot\core\pywikibot\site.py", line 3625, in preloaditempages
    item.get()  # parses json and preloads the various properties
  File "C:\pywikibot\core\pywikibot\page.py", line 2592, in get
    super(ItemPage, self).get(force=force, *args)
  File "C:\pywikibot\core\pywikibot\page.py", line 2421, in get
    raise pywikibot.NoPage(self)
pywikibot.exceptions.NoPage: Page [[wikidata:-1]] doesn't exist.
Comment 1 xqt 2013-12-27 04:37:51 UTC
Remarks for PreloadingPageenerator in compat which works without exception:
>>> import wikipedia as wp
>>> import pagegenerators
>>> gen = [wp.Page('de', 'user:xqt/NonexistentPage')]
>>> gen = pagegenerators.PreloadingGenerator(gen)
>>> for page in gen:
	print page

	
Getting 1 page from wikipedia:de...
[[de:Benutzer:Xqt/NonexistentPage]]
>>> page.exists()
False
>>>
Comment 2 xqt 2013-12-27 04:41:40 UTC
Same behaviour for core. It works without exception.
Comment 3 Gerrit Notification Bot 2013-12-27 06:09:56 UTC
Change 103900 had a related patch set uploaded by Xqt:
(bug 58976) Do not get() the item straight after preloading it.

https://gerrit.wikimedia.org/r/103900
Comment 4 Gerrit Notification Bot 2014-01-11 13:47:11 UTC
Change 103900 merged by jenkins-bot:
(bug 58976) Do not get() the item straight after preloading it.

https://gerrit.wikimedia.org/r/103900
Comment 5 Maarten Dammers 2014-01-11 14:08:17 UTC
Thank you xqt and Merlijn!

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links