Smarty extension causes generation of invalid HTML syntax

[andersk]

This works as expected:

$ echo '![x"x](x)' | python3 -m markdown -x smarty
<p><img alt="x&quot;x" src="x" /></p>

but this unexpectedly generates invalid HTML syntax:

$ echo '![x\"x](x)' | python3 -m markdown -x smarty
<p><img alt="x"x" src="x" /></p>

I would expect the second output to be the same as the first. Backslash should suppress special interpretation by Markdown; it should not also induce special interpretation by HTML.

[waylan]

I could argue that the parser did exactly what you requested: insert a literal double quote character at that location (bad input => bad output). However, I also checked what the reference implementation does (using "both" the Markdown and Smartypants filters) and it renders as alt="x\"x". So this appears to be a bug.

In most instances, all content is inserted into the HTML document via the ElementTree object. When that object gets serialized, any invalid content like this would automatically get escaped. However, in this case, placeholders are replaced by the literal characters in the HTML string after being serialized. Therefore, we have no context for where in the document the character even is to determine if it should be literal or not.

It occurs to me that given the purpose of backslash escapes is to prevent the character which follows it from being parsed as Markdown, we don't need the placeholders after we finish inline parsing. Therefore we could use a treeprocessor to walk the tree and swap any placeholders for the literal characters. That would ensure that the literal characters are properly escaped (if need be) in the serialized HTML.