Repository navigation
PI (<? ?>) is escaped in error with md_in_html #1070
Description
In any of the following source documents:
<div markdown=1>
<?php print("foo"); >
</div>
<div markdown=1>
<?php print("foo"); >
</div>
<div markdown=1><?php print("foo"); ></div>
We get the same output:
<div>
<p><?php print("foo"); ></p>
</div>Which is correct except for the fact the the opening and closing brackets of the processing instruction (PI) are escaped. We should be getting:
<div>
<p><?php print("foo"); ></p>
</div>The problem is related to the fact that the extension builds an etree object of raw HTML when markdown=1. However, the PI (<?php print("foo"); >) is simply passed to the etree object as text data (to treebuilder.data). Upon serialization, it is escaped. The obvious fix would be to pass the content of the PI to treebuilder.pi, except that xml.etree.ElementTree.TreeBuilder.pi was only added in PY3.8. What do we do for PY3.6 and PY3.7?
Activity
waylan commentedon Nov 19, 2020
I just remembered that the workaround for no support for
treebuilder.piwas to pass the text of the object tohtmlStash, which avoids escaping by the serializer. See here:markdown/markdown/extensions/md_in_html.py
Lines 200 to 210 in 81cc5b8
However, for some reason,
handle_empty_tagis not getting called in the example cases in this issue.facelessuser commentedon Nov 19, 2020
Yes, this is a little bit of why I had a little bit of trouble jumping into the parser. I assumed that things were always going down the appropriate path, but there appear to be some cases, like this, where things don't seem to get handled properly.
Now, I went down the wrong path and tried to cover for this by injecting code that caused other issues instead of getting to the root. While much easier to debug than the old parser, it is still not without some tricky places.
waylan commentedon Nov 19, 2020
markdown/markdown/htmlparser.py
Lines 34 to 35 in 81cc5b8
It took a while, but I finally remembered that we modified the piclose regex to only match
?>. We don't consider it a valid PI to close with>. The parser then does not consider the relevant bit to be a valid PI and falls back to treating it as random data which is enclosed between angle braces.And, as PHP requires that it be closed with
?>(not just>), that is a perfectly acceptable solution.In other words, all my example cases here are invalid. With the proper closing (
?>), the multi-line examples work just fine. However, we still have an issue with the single line example:[-]PI (<? >) is escaped in error with md_in_html[/-][+]PI (<? ?>) is escaped in error with md_in_html[/+]Properly parse processing instructions in md_in_html
Properly parse processing instructions in md_in_html