Last modified: 2014-03-11 17:09:04 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T60433, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 58433 - ClearMessageBlobs loop should check slave lag
ClearMessageBlobs loop should check slave lag
Status: NEW
Product: MediaWiki extensions
Classification: Unclassified
WikimediaMaintenance (Other open bugs)
master
All All
: Normal major (vote)
: ---
Assigned To: Nobody - You can work on this!
: performance
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-12-13 01:56 UTC by Sean Pringle
Modified: 2014-03-11 17:09 UTC (History)
3 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Sean Pringle 2013-12-13 01:56:10 UTC
ClearMessageBlobs uses TRUNCATE TABLE in a tight loop:

foreach ( $wgConf->getLocalDatabases() as $wiki ) {
	$lb = wfGetLB( $wiki );
	$db = $lb->getConnection( DB_MASTER, array(), $wiki );
	$db->query( "TRUNCATE TABLE " . $db->tableName( 'msg_resource' ), __METHOD__ );
	$db->query( "TRUNCATE TABLE " . $db->tableName( 'msg_resource_links' ), __METHOD__ );
	$lb->reuseConnection( $db );
}

This results in a lot of replication slave lag on WMF production shards.

It should use a batched delete if possible, instead of truncate.

It should definitely check slave lag in the loop.
Comment 1 Gerrit Notification Bot 2014-01-21 22:18:27 UTC
Change 108838 had a related patch set uploaded by Reedy:
Add wfWaitForSlaves() call in ClearMessageBlobs loop

https://gerrit.wikimedia.org/r/108838
Comment 2 Sam Reed (reedy) 2014-01-21 22:19:09 UTC
(In reply to comment #1)
> Change 108838 had a related patch set uploaded by Reedy:
> Add wfWaitForSlaves() call in ClearMessageBlobs loop
> 
> https://gerrit.wikimedia.org/r/108838

That's the "basic lazy" improvement

Batched deleting shouldn't be too much work to do it
Comment 3 Gerrit Notification Bot 2014-01-23 17:54:39 UTC
Change 108838 merged by jenkins-bot:
Add wfWaitForSlaves() call in ClearMessageBlobs loop

https://gerrit.wikimedia.org/r/108838
Comment 4 Andre Klapper 2014-03-11 17:09:04 UTC
Patch merged; resetting bug status

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links