Last modified: 2013-09-19 21:30:55 UTC
I got a Blank page with the following message at https://pt.wikipedia.org/w/index.php?title=Vetor_(matem%C3%A1tica)&action=submit when I tried to save this edit: https://pt.wikipedia.org/wiki/Vetor_(matem%C3%A1tica)?diff=36766326 A database error has occurred. Did you forget to run maintenance/update.php after upgrading? See: https://www.mediawiki.org/wiki/Manual:Upgrading#Run_the_update_script Query: REPLACE INTO `math` (math_inputhash,math_outputhash,math_html_conservativeness,math_html,math_mathml) VALUES ('�Y8�5zx&,1�%�#','c8Je�v��\\1m��;','0',NULL,'') Function: MathRenderer::writeToDatabase Error: 1205 Lock wait timeout exceeded; try restarting transaction (10.64.16.23)
Is this reproducible?
I wasn't able to reproduce it by copying the two versions to the sandbox: https://pt.wikipedia.org/w/index.php?diff=36768244 Maybe at this point there is a copy of the formulas in some cache and MW doesn't need to generate them again, and this makes the bug not to happen? Anyway, I found it strange to see things which looks like SQL shown to the user...
I did replaced all occurrences of "<math>" by "<math>bug?" to create new formulas and got another message (after a few seconds): -------------------------------- WIKIMEDIA FOUNDATION Error Our servers are currently experiencing a technical problem. This is probably temporary and should be fixed soon. Please try again in a few minutes. You may be able to get further information in the #wikipedia channel on the Freenode IRC network. The Wikimedia Foundation is a non-profit organisation which hosts some of the most popular sites on the Internet, including Wikipedia. It has a constant need to purchase new hardware. If you would like to help, please donate. If you report this error to the Wikimedia System Administrators, please include the details below. Request: POST http://pt.wikipedia.org/w/index.php?title=Wikip%C3%A9dia:P%C3%A1gina_de_testes/1&action=submit, from 208.80.154.134 via cp1020.eqiad.wmnet (squid/2.7.STABLE9) to 10.64.0.138 (10.64.0.138) Error: ERR_READ_TIMEOUT, errno [No Error] at Tue, 27 Aug 2013 18:26:56 GMT -------------------------------- Then I clicked in the back button from Google Chrome and pressed "Save" again. Bingo: A database error has occurred. Did you forget to run maintenance/update.php after upgrading? See: https://www.mediawiki.org/wiki/Manual:Upgrading#Run_the_update_script Query: REPLACE INTO `math` (math_inputhash,math_outputhash,math_html_conservativeness,math_html,math_mathml) VALUES ('�U^{#<�J���VƐ','w�\niNq ¾#c','0',NULL,'') Function: MathRenderer::writeToDatabase Error: 1205 Lock wait timeout exceeded; try restarting transaction (10.64.16.23)
Given both errors are timeouts (read or write), I'm tagging ops to see if they know anything.
I had the same error on my test wikis several time... But I could never reproduce it.
Ideally the math files would be batch uploaded into swift and the REPLACE query done as a batch in a separate transaction in autocommit mode. A much simpler fix for now would just be to wrap the REPLACE query in $dbw->onTransactionIdle() so the locks are not held during the whole parse (including swift uploading) process.
I implemented that... it's in the dev branch https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FMath.git/9103fe0c451485cf7e492d1099d21188c5fa20d4/MathRenderer.php#L248 for me that reduced the error but caused deadlocks on very heavy "write attacks" on the database However in the last test I had no errors but write delays of 120 seconds and more . (I used 16 cores to write in parallel to my local mysql database) I did not submit that to the master branch because it's hard to get code reviews for something like that.
physikerwelt's patch that wraps the REPLACE queries in onTransactionIdle() has been merged into master. See https://gerrit.wikimedia.org/84868/
Marking as resolved. Will need verification post-deploy.
