Last modified: 2012-03-07 08:00:38 UTC
The minutely clean-up cron is currently down: Normally triggered from cron but can also be triggered from the browser (should only take a few seconds): https://integration.mediawiki.org/testswarm/?state=wipe Response: Invalid query: The total number of locks exceeds the lock table size Query: https://github.com/jquery/testswarm/blob/master/logic/wipe.php Possible related reading: https://rackerhacker.com/2010/02/16/mysql-the-total-number-of-locks-exceeds-the-lock-table-size-2/ This clean up is fairly important to the function of the swarm, without it tests that fail or time out for various reasons (i.e. bad browsers, disconnects, magic, ..) don't get wiped and don't get re-run in another client. Probably the query is to blame, but perhaps the default size can be increased if reasonable still (as a short term fix)
upstream: https://github.com/jquery/testswarm/issues/112 Suggested short term fix: Re-install TestSwarm (that is, drop the database tables and re-install from the sql file). Only thing to remember is to create a 'MediaWiki' user afterwards and update fetcher.ini in production to reflect the new authtoken.
short term fix is to raise innodb_buffer_pool_size to some value bigger than the default 8M. The request is tracked with RT https://rt.wikimedia.org/Ticket/Display.html?id=2574
Lowering priority since the immediate issue has been dealt with.
It might have magically fixed itself, but so far the innodb_buffer_pool_size variable is still at 8MB
Issue fixed by the operation team. A specific mysql configuration snippet has been added in /etc/mysql/conf.d which set the innodb_buffer_pool_size to 256M. RT ticket has been closed.
