gh-115133: Fix tests for XMLPullParser with Expat 2.6.0#115164

serhiy-storchaka · 2024-02-08T12:25:55Z

Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

Issue: test.test_xml_etree*.XMLPullParserTest.test_simple_xml fails with (system) expat 2.6.0 #115133

Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

lazka · 2024-02-10T17:43:38Z

It stills fails here with this patch applied:

FAIL: test_simple_xml_chunk_8 (test.test_xml_etree.XMLPullParserTest.test_simple_xml_chunk_8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\cpython-mingw\cpython-mingw\Lib\test\test_xml_etree.py", line 1438, in test_simple_xml_chunk_8
    self.test_simple_xml(chunk_size=8)
  File "D:\a\cpython-mingw\cpython-mingw\Lib\test\test_xml_etree.py", line 1418, in test_simple_xml
    self.assert_event_tags(parser, [('end', 'element')])
  File "D:\a\cpython-mingw\cpython-mingw\Lib\test\test_xml_etree.py", line 1405, in assert_event_tags
    self.assertEqual([(action, elem.tag) for action, elem in events],
AssertionError: Lists differ: [] != [('end', 'element')]

Second list contains 1 additional elements.
First extra element 0:
('end', 'element')

- []
+ [('end', 'element')]

> python3 -c "import pyexpat; print(pyexpat.version_info)"
(2, 6, 0)

serhiy-storchaka · 2024-02-10T18:06:05Z

What is the smallest value of chunk_size with which the test would pass?

lazka · 2024-02-10T18:10:43Z

chunk_size=22 is the smallest value that works on my machine.

Lib/test/test_xml_etree.py

serhiy-storchaka · 2024-02-10T18:50:55Z

Thank you for testing @lazka.

miss-islington-app · 2024-02-11T10:08:42Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

bedevere-app · 2024-02-11T10:08:51Z

GH-115288 is a backport of this pull request to the 3.12 branch.

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

bedevere-app · 2024-02-11T10:08:57Z

GH-115289 is a backport of this pull request to the 3.11 branch.

…5164) (GH-115288) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

…5164) (GH-115289) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

miss-islington-app · 2024-02-15T15:21:25Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

miss-islington-app · 2024-02-15T15:21:25Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

miss-islington-app · 2024-02-15T15:21:25Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington-app · 2024-02-15T15:21:28Z

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a08e7b3431cd32a0daf22a33421cd3035343dc4 3.8

miss-islington-app · 2024-02-15T15:21:44Z

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a08e7b3431cd32a0daf22a33421cd3035343dc4 3.9

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

bedevere-app · 2024-02-15T15:22:08Z

GH-115525 is a backport of this pull request to the 3.10 branch.

…ythonGH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

sethmlarson · 2024-02-15T20:23:41Z

Created backports for 3.9 and 3.8 manually:

…5164) (#115525) gh-115133: Fix tests for XMLPullParser with Expat 2.6.0 (GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

…ythonGH-115164) (pythonGH-115288) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

) (GH-115536) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

…ythonGH-115164) (pythonGH-115288) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <[email protected]>

pythongh-115133: Fix tests for XMLPullParser with Expat 2.6.0

e27eafb

Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

serhiy-storchaka added tests Tests in the Lib/test dir needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Feb 8, 2024

bedevere-app bot added the awaiting core review label Feb 8, 2024

bedevere-app bot mentioned this pull request Feb 8, 2024

test.test_xml_etree*.XMLPullParserTest.test_simple_xml fails with (system) expat 2.6.0 #115133

Closed

Snild-Sony approved these changes Feb 8, 2024

View reviewed changes

hartwork approved these changes Feb 10, 2024

View reviewed changes

serhiy-storchaka commented Feb 10, 2024

View reviewed changes

Lib/test/test_xml_etree.py Outdated Show resolved Hide resolved

Update Lib/test/test_xml_etree.py

aaffd5d

serhiy-storchaka merged commit 4a08e7b into python:main Feb 11, 2024

bedevere-app bot removed the awaiting core review label Feb 11, 2024

serhiy-storchaka deleted the test-etree-xmlpullparser-expat-2.6.0 branch February 11, 2024 10:08

bedevere-app bot removed the needs backport to 3.12 only security fixes label Feb 11, 2024

bedevere-app bot removed the needs backport to 3.11 only security fixes label Feb 11, 2024

serhiy-storchaka mentioned this pull request Feb 11, 2024

gh-115133: test_xml_etree.py: Fix for Expat >=2.6.0 with reparse deferral (fixes #115133) #115138

Closed

hartwork mentioned this pull request Feb 12, 2024

expat: 2.5.0 -> 2.6.0 NixOS/nixpkgs#287040

Merged

13 tasks

hartwork mentioned this pull request Feb 14, 2024

Release Expat 2.6.0 libexpat/libexpat#775

Closed

28 tasks

hartwork mentioned this pull request Feb 14, 2024

[3.8] Upgrade bundled libexpat to 2.6.0 (GH-115399) #115475

Merged

sethmlarson added needs backport to 3.8 needs backport to 3.10 only security fixes labels Feb 15, 2024

miss-islington-app bot assigned serhiy-storchaka Feb 15, 2024

bedevere-app bot removed the needs backport to 3.10 only security fixes label Feb 15, 2024

Bill-Sommerfeld mentioned this pull request Feb 20, 2024

epoll patch Bill-Sommerfeld/oi-userland#13

Closed

edmorley mentioned this pull request Oct 10, 2024

Python 3.13: test_xml_etree fails on expat 2.2.5 (RHEL8 / Rocky linux 8 / Ubuntu 22.04) #125067

Open

hugovk removed the needs backport to 3.9 label Feb 18, 2025

Uh oh!

Conversation

serhiy-storchaka commented Feb 8, 2024 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lazka commented Feb 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka commented Feb 10, 2024

Uh oh!

lazka commented Feb 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

serhiy-storchaka commented Feb 10, 2024

Uh oh!

miss-islington-app bot commented Feb 11, 2024

Uh oh!

bedevere-app bot commented Feb 11, 2024

Uh oh!

bedevere-app bot commented Feb 11, 2024

Uh oh!

miss-islington-app bot commented Feb 15, 2024

Uh oh!

miss-islington-app bot commented Feb 15, 2024

Uh oh!

miss-islington-app bot commented Feb 15, 2024

Uh oh!

miss-islington-app bot commented Feb 15, 2024

Uh oh!

miss-islington-app bot commented Feb 15, 2024

Uh oh!

bedevere-app bot commented Feb 15, 2024

Uh oh!

sethmlarson commented Feb 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

serhiy-storchaka commented Feb 8, 2024 •

edited by bedevere-app bot

Loading

lazka commented Feb 10, 2024 •

edited

Loading

lazka commented Feb 10, 2024 •

edited

Loading