Last modified: 2013-07-30 10:07:32 UTC
Visual Editor sometimes creates nowiki markup around simple square text brackets. Interestingly, it does that only in some cases, and not in others. For the same structure, it can also handle the text with nowiki, but does it in others. Text case: The example below shows that in some cases this is done, in others not: # Remains unchanged: Test text for the VisualEditor: [[Link target|These are ['''just''' normal brackets]]]. # Superflous nowiki-tags added on editing: Test text for the VisualEditor: [[Link target|These are '''just''' [normal brackets]]]. Edit with Visual Editor to see the effect.
This appears to be a Parsoid bug; adjusting to reflect that. If you use the RT form - http://parsoid.wmflabs.org/_rtform/ - on the above string: [[Link target|These are '''just''' [normal brackets]]] -> [[Link target|These are '''just'''<nowiki> [normal brackets]</nowiki>]] However, it generates correct HTML; for the other string: [[Link target|These are ['''just''' normal brackets]]] -> <p data-parsoid="{"dsr":[0,54]}"><a rel="mw:WikiLink" href="./Link_target" data-parsoid="{"tsr":[0,53],"a":{"href":"./Link_target"},"sa":{"href":"Link target"},"stx":"piped","dsr":[0,53]}">These are [<b data-parsoid="{"tsr":[25,28],"dsr":[25,35]}">just</b> normal brackets</a>]</p> Whereas canonical HTML from the PHP parser is: <p><a href="/wiki/Link_target" title="Link target">These are [<b>just</b> normal brackets]</a></p> [i.e., the closing not-used-for-a-link square bracket is inside the link, not outside it - however, this is probably a separate bug; happy to re-file.]
The nowiki-escaping is related to how Parsoid parses the text: [[Link target|These are ['''just''' normal brackets]]] and the fact that the last "]" is outside the link. The nowiki escaping strategy is safe (but conservative) and tries not to accidentally create a wikilink when a "]" precedes a regular link bracket. We (Parsoid team) should fix the parse first, add new tests, and then update the nowiki escaping.
(In reply to comment #2) > and the fact that the last > "]" is outside the link. The first ] is inside the link and the last 2 are the closing link. I agree that there may be more pressing matters, but compare that in one case: [[Link target|These are ['''just''' normal brackets]] the parser already works correct (does not add superfluous nowiki). Only if there is no formatting inside single square brackets it adds nowiki. I suspect this is not very logical. (... and annyoing on the live testing on en.wikipedia.)
One underlying issue is that our tokenizer is relatively liberal in what it accepts as an external link target, while a stricter validation is applied after templates etc are expanded. Our wikitext escaper also makes use of the tokenizer (plus some regexps), but does not (yet) consider external links that turn out to have invalid targets. Current output: echo "[[Link target|These are '''just''' [normal brackets]]]" | nodejs parse --wt2wt [[Link target|These are '''just'''<nowiki> [normal brackets]</nowiki>]]
I believe adding illogical nowiki markup is somewhat irritating to existing editors. Perhaps a simple solution: Is there any justification for the tokenizer to consider an opening [ as a token? I believe only [http://, [https://, [irc://, [ircs://, [ftp://, [news://, [mailto: and [gopher:// should be external link tokens.
The problem is that till templates are expanded, it is hard to know whether the content of a "[..]" is going to a link or not. That is the reason the tokenizer defers decisions for all [..] content till all templates are expanded so it can all be handled in a single place. But, there is another ticket for addressing this a bit more selectively which should address nowiki situations like this: https://bugzilla.wikimedia.org/show_bug.cgi?id=44449 --
https://gerrit.wikimedia.org/r/49190 added URL validation in the tokenizer based on the wiki's configured URL protocols. This now makes the nowiki escaping more precise and fixes the snippet in this bug report. [subbu@earth lib] echo '[[Link target|These are '''just''' [normal brackets]]]' | node parse --wt2wt [[Link target|These are just [normal brackets]]] However, this wont be visible in the VE yet till a new deploy of Parsoid (another couple weeks, I expect).
Note about the lost "'''" in the example above: that is a different bug unrelated to nowiki-escaping -- it should be fixed after the ongoing serializer refactor is complete.
Much of this is now resolved. [foo] is no longer nowiki-escaped, but [''foo''] is due to the apostrophes. There is still some room for (minor) improvement by moving the nowiki inside the brackets, although in some cases it is actually easier to read to wrap an entire line in nowiki instead of several nested nowikis.
[Parsoid component reorg by merging JS/General and General. See bug 50685 for more information. Filter bugmail on this comment. parsoidreorg20130704]
