Last modified: 2014-05-30 17:47:52 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T66025, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 64025 - Production log errors: Tokenizer failure
Production log errors: Tokenizer failure
Status: RESOLVED FIXED
Product: Parsoid
Classification: Unclassified
tokenizer (Other open bugs)
unspecified
All All
: Unprioritized normal
: ---
Assigned To: Gabriel Wicke
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-04-16 21:26 UTC by ssastry
Modified: 2014-05-30 17:47 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description ssastry 2014-04-16 21:26:08 UTC
enwiki:Talk:Tony_Clement: Expected ">", [ \t\n\r\f], [ \t] or [^=<>{\n\r&'"\t [\]|{}\/!] but "/" found.
enwiki:Template:Archive_top/doc: Expected "&", "<!--", "<", "</", "=", ">", "{{", "{{{", [ \t] or [^=<>{\n\r&'"\t [\]|{}\/!] but "\n" found.
enwiki:Template:Chembox_DeltaGf: Expected "&", "/", ";", "<!--", "<", "</", ">", "{{", "{{{", [ \t\n\r\f], [#0-9a-zA-Z], [&%\/{}] or [^=<>{}\n\r&'"\t\/ ] but "=" found.
enwiki:Template:Chembox_Entropy/sandbox: Expected "&", "/", ";", "<!--", "<", "</", ">", "{{", "{{{", [ \t\n\r\f], [#0-9a-zA-Z], [&%\/{}] or [^=<>{}\n\r&'"\t\/ ] but "=" found.
enwiki:Template:Chembox_HeatCapacity/sandbox: Expected "&", "/", ";", "<!--", "<", "</", ">", "{{", "{{{", [ \t\n\r\f], [#0-9a-zA-Z], [&%\/{}] or [^=<>{}\n\r&'"\t\/ ] but "=" found.
enwiki:User:Czarkoff/common.css: Expected ">", [ \t\n\r\f], [ \t] or [^=<>{\n\r&'"\t [\]|{}\/!] but "/" found.
fiwiki:Wikipedia:Ohjaus: Expected "&", "/", "<!--", "<", "</", "=", ">", "{{", "{{{", [ \t\n\r\f] or [^=<>{\n\r&'"\t [\]|{}\/!] but "[" found.
Comment 1 ssastry 2014-04-17 23:16:09 UTC
(In reply to ssastry from comment #0)
> enwiki:Template:Chembox_DeltaGf: Expected "&", "/", ";", "<!--", "<", "</",
> ">", "{{", "{{{", [ \t\n\r\f], [#0-9a-zA-Z], [&%\/{}] or [^=<>{}\n\r&'"\t\/
> ] but "=" found.

Found this in the wikitext source of this page which cause the tokenizer to bomb/go into an infinite loop.

<pre <pre style="color:darkgreen">
| ...(more Chembox parameters)
}}
</pre>
Comment 2 ssastry 2014-04-17 23:19:14 UTC
Looks like I clipped the transclusion source in #c1 but, the problem is with the <pre tag. That one causes an infinite loop. This snippet below causes a tokenizer failure:

{{echo|foo
<pre <pre style="color:darkgreen">
| ...(more Chembox parameters)
}}
</pre>
Comment 3 ssastry 2014-05-15 15:47:48 UTC
https://gerrit.wikimedia.org/r/#/c/133483/ is a patch that handles tests in comment 1 and comment 2.

The following 2 pages continue to crash and is probably something else in the tokenizer that needs fixing:

enwiki/Talk:Tony_Clement
enwiki/User:Czarkoff/common.css
Comment 4 Gerrit Notification Bot 2014-05-20 18:12:38 UTC
Change 133483 abandoned by Subramanya Sastry:
(Bug 64025) Fixed tokenizer to accept pre-like strings without crashing.

https://gerrit.wikimedia.org/r/133483
Comment 5 Gerrit Notification Bot 2014-05-22 04:38:38 UTC
Change 134783 had a related patch set uploaded by Arlolra:
Accept pre-like strings without crashing

https://gerrit.wikimedia.org/r/134783
Comment 6 Gerrit Notification Bot 2014-05-30 17:42:51 UTC
Change 134783 merged by jenkins-bot:
Accept pre-like strings without crashing

https://gerrit.wikimedia.org/r/134783

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links