Wikimedia Bugzilla is closed!

The cited example is
https://de.wikipedia.org/wiki/Delius?veaction=edit

The entry 'Christina Rau' is not editable, as VE believes it is part of the 'template' {{:Christian Delius}}

Possibly because the onlyinclude on
https://de.wikipedia.org/w/index.php?title=Christian_Delius&action=edit

Relevant wikitext excerpt:

| {{:Christian Delius}}
| <!-- Änderungen bitte dort vornehmen, siehe unterhalb des Bearbeitungsfensters -->
| * [[Christina Rau]], geb. Delius (* 1956), deutsche Politologin

Yeah, this is because the HTML comment and <li> in the wikitext below the template insert themselves into the implicit <ul> that the template creates (and so get marked as part of the same "MediaWiki transclusions" generated content block as the template by Parsoid.

I'm not sure what Parsoid could do to "fix" this, really. It's correct parsing of broken-by-design wikitext (which doesn't seem obvious if you're not parsing it).

This is not really a problem that affects enwp.  If my SQL is right, enwp has only 14 dab pages that transclude mainspace - easily a manual fixup task, esp. when some look of them look like this:
   e.g. https://en.wikipedia.org/wiki/Howes?veaction=edit

Some impact analysis for the next set of wikis in line for VE deployment:

dewp: 22226 content pages transcluding mainspace - i.e. 1.4% of all 'content pages', of which 9012 are dab pages - i.e. 0.5% of all content pages
 - not all of those suffer from the second issue raised; 
   sometimes they have whitespace between transclusion and the next item
   e.g. https://de.wikipedia.org/wiki/Kalam?veaction=edit

frwp,hewp,nlwp: between 1&4 pages transcluding mainspace

svwp: 250, all dab pages

ruwp: 1616, all dab pages - i.e. 0.15% of all content pages
  Looking at a Russian example; it doesnt face the second problem raised in comment 1; the last item on this dab page is not consumed by the {{:Lloyd's}} above it.
https://ru.wikipedia.org/wiki/%D0%9B%D0%BB%D0%BE%D0%B9%D0%B4?veaction=edit
https://ru.wikipedia.org/wiki/Lloyd%E2%80%99s

ruwp uses noinclude instead of onlyinclude.

My SLQ on dewp:
select count(*)
from   templatelinks tl inner join page p on (tl.tl_from = p.page_id)
where  tl_namespace = 0
and    page_namespace = 0
and    tl_from in 
  (select cl_from from categorylinks where cl_to =  'Begriffsklärung')
;

As James indicates in https://bugzilla.wikimedia.org/show_bug.cgi?id=51119#c2, there is really not much we can do about this. The implicit <ul> is started by a template, so the entire list needs to be marked as template-affected even if some items in it are not templated.

Since I don't see a good way to improve this situation in Parsoid, I'm going to close this bug as wontfix for now. Please reopen this bug if you have an idea on how this could be handled better.

The bug still exists. VE should handle this differently: mark the whole page as not compatible with VE if you must.

*** Bug 52814 has been marked as a duplicate of this bug. ***

(In reply to comment #5)
> The bug still exists. VE should handle this differently: mark the whole page
> as
> not compatible with VE if you must.

John: can you clarify what could be done here? 

As far as I can tell, there is no broken HTML, and neither is there any corruption on edit and save. The only problem is that top-level page content is sucked into transclusion content and is not editable. In the current scenario, at least some part of the page is editable in VE. How would marking the whole page not editable solve this issue?

That said, there is a possibility that for lists, this problem is solvable by marking a series of adjacent list nodes as template generated -- Parsoid already has the mechanism of marking a forest of adjacent DOM nodes as template-generated instead of marking just a root-node.  I suspect the reason this is not already happening is because of an (un)related list parsing bug.  Bug 49974, now merged into Bug 52762.  Once that is resolved, this problem will be fixed for the specific example pages on this page.  And as long as most of the disambiguation pages use this pattern (list items), it should be okay for them.

Please read the bug report and reproduce. The current behaviour is sub-optimal. Perhaps its not a high priority as it doesnt break the wiki-text, but that doesn't mean it isnt contrary to user expectations.
Text on the page (in the wiki text) should be editable on the page. If the VE cant edit all the wikitext on the page, the page is not yet functional in VE, and only SE should be offered to the user (or VE should tell the user to use SE to edit those bits).

I personally am not going to apply that metric to this page -- for example, table tags cannot be edited in VE, extension source cannot be edited in VE, and wikitext that shows up in between multi-part templates cannot be edited in VE.  I think this particular issue falls in the same category.

I do agree that the experience is sub-optimal.  But given that it does not break wikitext, I mean no disrespect to your opinion, but I disagree with you that we ought to therefore make the entire page uneditable.  As to your suggestion of providing a generic warning for uneditable pieces in VE, I'll James address that part.

But, as I noted above, at least for dab pages using list items, this problem might be solvable if we can figure out why transclusions are splitting existing lists at the transclusion point.

(In reply to comment #8)
> That said, there is a possibility that for lists, this problem is solvable by
> marking a series of adjacent list nodes as template generated -- Parsoid
> already has the mechanism of marking a forest of adjacent DOM nodes as
> template-generated instead of marking just a root-node. 

The problem is that the list is also template-affected, and needs to be marked as such. A change in transclusion output (say, from *foo to #foo) will change the list structure significantly, so it is not safe to mark the list items only.

The example page exposes a different issue though, which you mention as well: The comments between transclusions (which probably end up on their own line after expansion) break up the list in Parsoid, but don't do so in PHP (bug 52762). This is not surprising as the PHP parser strips those comments before doing anything else, but still something we could improve by not breaking up the list.

Sadly, the effect of this will be that the entire list will be template-affected.

This problem would not be there if the bullet for the list item was not templated:

* {{:Christian Delius}}
* [[Christina Rau]], geb. Delius (* 1956), deutsche Politologin

In this case we know that the first list item will always create an unordered list, no matter what the transclusion generates (ignoring unbalanced transclusions for now). Each item will be editable separately without being sucked into a big template-affected block.

A deeper investigation revealed additional issues:

1) Completely templated list items currently always break up lists in Parsoid. Test case: echo -ne '*foo\n{{:User:GWicke/li|bar}}\n*baz' | node parse

2) This covers up another bug in the template encapsulation code. Template-affected lists are only marked as such because the list handler starts a new list for each templated list item. This (correctly) places the meta element we use to track template-affected content outside the <ul> triggered by the template-generated list item. Once 1) is fixed, we will have to fix this up with a special rule in the encapsulation code that marks the parent list as template-affected if one of its wikitext-syntax items is template-generated. The meta element will end up between list items, so a propagation rule should be sufficient.

None of this will do much for the ease of editing. I realize that fixing up a lot of pages to use explicit bullets in the page will create a lot of work for bots, but at the same time I don't see a sane alternative that provides a good editing experience.

Opened Parsoid bug 53368 for the issues described in comment #12 above.