Last modified: 2013-12-13 20:17:16 UTC
see sample log at wtp1007:/var/lib/parsoid/nohup.out these entries are 20k lines per exception so I'll just give the start and end of one here: Util.tokensToString, invalid token: undefined tokens: [ '<onlyinclude>', { dataAttribs: { tsr: [Object] } }, undefined, { dataAttribs: { tsr: [Object] } }, undefined, ' ', undefined, undefined, ' kbt ', undefined, undefined, ' ', undefined, 'Abadi', { name: 'a', attribs: [], dataAttribs: {} }, ' ', undefined, undefined, ' ', undefined, undefined, ' Levend ', undefined, undefined, ' 4300', undefined, ... 'Zweeds', { name: 'a', attribs: [], dataAttribs: {} }, ' ', undefined, undefined, ' ', undefined, undefined, ' ', undefined, undefined, undefined, { dataAttribs: { tsr: [Object] } }, undefined, { dataAttribs: { tsr: [Object] } }, undefined, undefined, '</onlyinclude>' ] Trace at Object.Util.tokensToString (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.Util.js:346:13) at stripMetaTags (/srv/deployment/parsoid/Parsoid/js/lib/ext.core.AttributeExpander.js:77:18) at AttributeExpander._returnAttributes (/srv/deployment/parsoid/Parsoid/js/lib/ext.core.AttributeExpander.js:164:17) at AttributeTransformManager.process (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.TokenTransformManager.js:1077:8) at AttributeExpander.onToken (/srv/deployment/parsoid/Parsoid/js/lib/ext.core.AttributeExpander.js:125:6) at AsyncTokenTransformManager.transformTokens (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.TokenTransformManager.js:611:17) at AsyncTokenTransformManager.onChunk (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.TokenTransformManager.js:395:17) at SyncTokenTransformManager.EventEmitter.emit (events.js:88:17) at SyncTokenTransformManager.onChunk (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.TokenTransformManager.js:955:7) at PegTokenizer.EventEmitter.emit (events.js:88:17) The logs were filling with these at about 1 gb in 10 minutes or so, logging is temporarily turned off for now.
Change 99022 had a related patch set uploaded by GWicke: Bug 57707: Hotfix, don't push undefined onto wikitext array https://gerrit.wikimedia.org/r/99022
Change 99022 merged by jenkins-bot: Bug 57707: Hotfix, don't push undefined onto wikitext array https://gerrit.wikimedia.org/r/99022
We still have to investigate the source of the problem. The hotfix only prevents crashers and should prevents logs from filling up.
This doesn't appear to be a regression. The above trace seems to be coming from, https://nl.wikipedia.org/wiki/Lijst_van_talen_van_de_wereld If I checkout a commit from 8 months back, it spits out the same errors. A minimal test case that can be gleamed from that page is, {| |-<onlyinclude> |- </onlyinclude>|}
Oooh .. a test case .. just what was needed! How did you find a failing page?? :-)
google search for zweeds abadi levend ;-) reminder that there are logs in /var/lib/parsoid on wtp1007 which you can look at to see what was being parsed before that error, some of those pages may be candidates.
Ah yes, indeed ... Gabriel and I both missed got distracted by the zillion 'undefined' entries in that log and missed the google keywords there! :-) Gabriel said he looked at the logs and couldn't find the source. In any case, this test case should be sufficient to debug and fix the problem.
Change 99096 had a related patch set uploaded by Subramanya Sastry: (Bug 57707) Clean up dead code (that was the source of exceptions) https://gerrit.wikimedia.org/r/99096
Change 99096 merged by jenkins-bot: (Bug 57707) Clean up dead code (that was the source of exceptions) https://gerrit.wikimedia.org/r/99096
Deployed now.
Change 101279 had a related patch set uploaded by GWicke: Merge "Bug 57707: Hotfix, don't push undefined onto wikitext array" https://gerrit.wikimedia.org/r/101279
Change 101282 had a related patch set uploaded by GWicke: Merge "(Bug 57707) Clean up dead code (that was the source of exceptions)" https://gerrit.wikimedia.org/r/101282
Change 101331 had a related patch set uploaded by GWicke: Bug 57707: Hotfix, don't push undefined onto wikitext array https://gerrit.wikimedia.org/r/101331
Change 101334 had a related patch set uploaded by GWicke: (Bug 57707) Clean up dead code (that was the source of exceptions) https://gerrit.wikimedia.org/r/101334
Change 101282 merged by GWicke: Merge "(Bug 57707) Clean up dead code (that was the source of exceptions)" https://gerrit.wikimedia.org/r/101282
Change 101334 merged by GWicke: (Bug 57707) Clean up dead code (that was the source of exceptions) https://gerrit.wikimedia.org/r/101334
Change 101279 merged by GWicke: Merge "Bug 57707: Hotfix, don't push undefined onto wikitext array" https://gerrit.wikimedia.org/r/101279
Change 101331 merged by GWicke: Bug 57707: Hotfix, don't push undefined onto wikitext array https://gerrit.wikimedia.org/r/101331
