Last modified: 2014-06-09 13:07:56 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T68256, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 66256 - Category.articles() infinitely loops on 503 error
Category.articles() infinitely loops on 503 error
Status: RESOLVED FIXED
Product: Pywikibot
Classification: Unclassified
General (Other open bugs)
core-(2.0)
All All
: Normal major
: ---
Assigned To: xqt
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-06-06 13:40 UTC by xqt
Modified: 2014-06-09 13:07 UTC (History)
2 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description xqt 2014-06-06 13:40:31 UTC
If total parameter is None, the generator sometimes does not yield any pages but gives an 503 status error:

>>> import pwb
>>> import pywikibot as py
>>> s = py.Site('commons', 'commons')
>>> c = py.Category(s, 'Korea.net_photos_to_check')
>>> for p in c.articles(total=10, content=False, recurse=False, step=None):
	print p.title()

	
File:Kim Yu-na G20 Seoul Summit Ambassador.jpg
File:KOCIS 150m-long kimbap (5911816764).jpg
File:KOCIS 150m-long kimbap (5911817226).jpg
File:KOCIS 2010 UNESCO World Conference of Arts Education (4640759629).jpg
File:KOCIS 2010 UNESCO World Conference of Arts Education (4641367198).jpg
File:KOCIS 2010 UNESCO World Conference of Arts Education (4641367274).jpg
File:KOCIS 2010 UNESCO World Conference of Arts Education (4641367322).jpg
File:KOCIS 2012 Yeosu Expo (7548642068).jpg
File:KOCIS 2012 Yeosu Expo (7548642206).jpg
File:KOCIS 38th Annual Los Angeles Korean Festival (6301325981).jpg

whereas total=None fails into 503 loop

>>> for p in c.articles(total=None, content=False, recurse=False, step=None):
	print p.title()

	
WARNING: Http response status 503
WARNING: Non-JSON response received from server commons:commons; the server may be down.
Set gcmlimit = 2500
WARNING: Waiting 5 seconds before retrying.

Traceback (most recent call last):
  File "<pyshell#20>", line 1, in <module>
    for p in c.articles(total=None, content=False, recurse=False, step=None):
  File "pywikibot\page.py", line 1909, in articles
    endsort=endsort
  File "pywikibot\data\api.py", line 719, in __iter__
    self.data = self.request.submit()
  File "pywikibot\data\api.py", line 306, in submit
    body=paramstring)
  File "pywikibot\comms\http.py", line 144, in request
    time.sleep(0.1)
KeyboardInterrupt
>>>
Comment 1 xqt 2014-06-06 14:24:30 UTC
Hm, when the gcmlimit starts with ~3000 or higher this error occurs. Wenn the limit is ~2500 or lower it doesn't. Unfortunately it doesn't disappear by just downsizing the limit as the QueryGenerator does.
Comment 2 Gerrit Notification Bot 2014-06-06 15:02:24 UTC
Change 137936 had a related patch set uploaded by Xqt:
(bug 66256) decrease api high limit

https://gerrit.wikimedia.org/r/137936
Comment 3 Gerrit Notification Bot 2014-06-09 13:03:48 UTC
Change 137936 merged by jenkins-bot:
(bug 66256) decrease api high limit and update the paramstring

https://gerrit.wikimedia.org/r/137936

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links