Bug report

Bug description:

When a DefaultHandler or DefaultHandlerExpand callback raises, pyexpat clears all of the parser's C handlers. For a document that needs encoding conversion (any encoding but UTF-8 or ASCII, including UTF-16), Expat delivers a token longer than 1024 bytes to the default handler in several pieces and re-reads the handler between them, so the next piece is a call through a NULL function pointer.

from xml.parsers.expat import ParserCreate

def default(data):
    if data.startswith("<!--"):
        raise ValueError("comments are not allowed")

p = ParserCreate()
p.DefaultHandler = default
p.Parse(b'<?xml version="1.0" encoding="iso-8859-1"?><a><!--' + b"x" * 2000 + b"--></a>", True)

With encoding="utf-8" this raises ValueError; with iso-8859-1 the interpreter dies with SIGSEGV. Setting p.DefaultHandler = None from inside the callback, or a CharacterDataHandler that raises when buffer_text is flushed before a default-handler call, crashes the same way. The character data handler had the same problem and was fixed in gh-42389 by installing a no-op handler instead of NULL; the default handler was left out.

The default C xml.etree.ElementTree, xml.sax and xml.dom.minidom are not affected; the pure-Python ElementTree parser (used only without _elementtree) is.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-161060: Fix pyexpat crash when the default handler is cleared #161061