Last modified: 2012-08-04 22:58:10 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T39645, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 37645 - Message import does not scale to 200k+ messages
Message import does not scale to 200k+ messages
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
Translate (Other open bugs)
unspecified
All All
: Unprioritized normal (vote)
: ---
Assigned To: Niklas Laxström
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-16 08:14 UTC by Niklas Laxström
Modified: 2012-08-04 22:58 UTC (History)
2 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Niklas Laxström 2012-06-16 08:14:34 UTC
Few problems here:

First of all, most PHP configurations limit the number of post parameters to 1000 (meaning one can only import 999 changes at time). It is possible to increase this number, but lets face it, it is hard to do and people will look grumpy changing settings made for security.

Another thing is you can only display about 10k changes at once, because of the time and memory needed to create the page. This is bug 37167.

One work around is to add limit to processMessageChanges, which stops after N changes, telling the user to import those first and then running the script again.

The first thing to fix would be to device work around for the 1000 post parameters limit, perhaps by adding paging for every 999 changes.
Comment 1 Niklas Laxström 2012-08-04 22:58:10 UTC
https://gerrit.wikimedia.org/r/#/c/17189/ fixes this one too.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links