Last modified: 2014-01-03 21:21:43 UTC
This issue was converted from https://jira.toolserver.org/browse/DRTRIGON-90. Summary: sequence of sometimes occuring "KeyError: u'parse'" Issue type: Sub-task - The sub-task of the issue Priority: Minor Status: In Progress Assignee: drtrigon <dr.trigon@surfeu.ch> ------------------------------------------------------------------------------- From: drtrigon <dr.trigon@surfeu.ch> Date: Sat, 09 Jul 2011 11:14:57 ------------------------------------------------------------------------------- The error: > Traceback (most recent call last): File "/home/drtrigon/pywikipedia/bot_control.py", line 225, in run self.bot.main() File "/home/drtrigon/pywikipedia/sum_disc.py", line 1307, in main bot.run() File "/home/drtrigon/pywikipedia/sum_disc.py", line 369, in run self.getLatestRelevantNews() File "/home/drtrigon/pywikipedia/sum_disc.py", line 747, in getLatestRelevantNews entries = PageSections(page, self._param) File "/home/drtrigon/pywikipedia/sum_disc.py", line 1138, in __init__ sections = page.getSections(minLevel=1) File "/home/drtrigon/pywikipedia/dtbext/dtbext_wikipedia.py", line 50, in <lambda> obj.__dict__[\'getSections\'] = lambda *args, **kwds: Page.__dict__[\'getSections\'](obj, *args, **kwds) File "/home/drtrigon/pywikipedia/dtbext/dtbext_wikipedia.py", line 160, in getSections r = result[u\'parse\'][u\'sections\'] KeyError: u'parse' occurs from time to time - look at [1] and older changes. As temporary work-a-round the error raised is converted to > pywikibot.Error('Problem occured during data retrieval for sections in %s!' % self.title(asLink=True)) and the 'result' dict gets printed into the log file to be able to track the origin of this down. In future there should be a **final solution** to this bug: * either keep the current code (without the print) which converts the error such that later the whole page gets processed in one part (fall back) * or add a loop with e.g. maximal 3 re-tries (like for low level url retrieval) and then use e.g. the same fall back like before [1] http://de.wikipedia.org/w/index.php?title=Benutzer:DrTrigonBot/Status&direction=prev&oldid=90413407
------------------------------------------------------------------------------- From: drtrigon <dr.trigon@surfeu.ch> Date: Sat, 06 Aug 2011 13:01:29 ------------------------------------------------------------------------------- With r135 the bot should notify SGE in case of errors and be re-runned by SGE. This could also solve that problem.
------------------------------------------------------------------------------- From: drtrigon <dr.trigon@surfeu.ch> Date: Thu, 25 Aug 2011 14:48:42 ------------------------------------------------------------------------------- Seams to occur rarely. Since creation of this ticket and introduction of work-a-round in code at '09 July' until today '25 Aug.' no such event was logged. You should find a debug output prior to a "Problem resolving section data" message, as you can see by looking into 'dtbext.wikipedia.getSections'.
------------------------------------------------------------------------------- From: drtrigon <dr.trigon@surfeu.ch> Date: Fri, 23 Dec 2011 19:37:28 ------------------------------------------------------------------------------- The code introduced in r158 to track down this issue is somehow useless since it triggers now a "Problem resolving section data, processing the whole page at once..." which is the most often triggered problem. Thus the print in front of the raise was enhanced with the message "Query result:" - this string should be simpler to find in the log files in order to reproduce this bug.
