Last modified: 2014-11-20 22:31:44 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T51203, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 49203 - SMW Ask query offset has a hardcoded limit
SMW Ask query offset has a hardcoded limit
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
Semantic MediaWiki (Other open bugs)
master
All All
: Unprioritized normal with 3 votes (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-05 18:47 UTC by jayhuggins
Modified: 2014-11-20 22:31 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
SMWQueryProcessor without $params['offset']['upperbound']. (22.77 KB, text/x-wiki)
2013-06-05 18:47 UTC, jayhuggins
Details

Description jayhuggins 2013-06-05 18:47:33 UTC
Created attachment 12471 [details]
SMWQueryProcessor without $params['offset']['upperbound'].

Running an ask query with an offset greater than 5000 returns the results from the query with offset=0 instead.  Because of this, the only way to reliably fetch all results for large queries is to set offset=0, limit=<very_large_number> and fetch all results at once.  This is less desirable for the user as well as the site host than "paging through" large queries by specifying a limit and incrementing the offset by the limit size for every page queried.  But limiting the offset in any way makes this preferred scenario impossible.

The code responsible is the 'upperbound' setting for $params['offset'] in SemanticMediaWiki/includes/SMW_QueryProcessor.php at line 601:

                $params['offset'] = array(
			'type' => 'integer',
			'default' => 0,
			'negatives' => false,
			'upperbound' => 5000 // TODO: make setting
		);

My guess is that the upperbound was intended for $params['limit'], but was added to $params['offset'] by mistake.

The solution is not to make 'upperbound' configurable with a global variable as the comment implies, but simply to remove this line altogether (see attachment).
Comment 1 Jamie Thingelstad 2013-07-27 12:09:13 UTC
I've now hit this issue a few times, but just now did the debugging to figure out what was going on. I've been recently trying Yaron's new Miga [1] project and have been trying to use it with the Wikinosh [2] dataset. Wikinosh has 21,000+ food items and I hit this bug trying to page through results and export everything to Miga [3].

[1] http://migadv.com
[2] http://wikinosh.com/wiki/Main_Page
[3] https://github.com/yaronkoren/miga/issues/4
Comment 2 [[kgh]] 2014-11-19 09:41:35 UTC
This bug is now recorded on GitHub as issue 242 [1]


[1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/242
Comment 3 [[kgh]] 2014-11-20 22:31:44 UTC
This bug was fixed with pull request 637 [1]. Thus closing as RESOLVED FIXED.

[1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/637

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links