`test.test_urllib2.MiscTests.test_issue16464` flaky due to external connection

[sobolevn]

BPO	46648
Nosy	@gpshead, @vstinner, @ambv, @corona10, @miss-islington, @sobolevn, @GBeauregard
PRs	bpo-46648: test_urllib2.test_issue16464() uses httpbin.org #31146 bpo-46648: Skip test_urllib2.test_issue16464() #31161 [3.10] bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) #31162 [3.9] bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) #31163 [3.8] bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) #31173 bpo-46648: move test_issue16464 to urllib2_localnet #31186 [3.10] bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) #31189 [3.9] bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) #31190

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/sobolevn'
closed_at = <Date 2022-02-07.22:08:55.169>
created_at = <Date 2022-02-05.11:45:33.833>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.11', 'tests']
title = '`test.test_urllib2.MiscTests.test_issue16464` flaky due to external connection'
updated_at = <Date 2022-02-08.09:52:52.616>
user = 'https://github.com/sobolevn'

bugs.python.org fields:

activity = <Date 2022-02-08.09:52:52.616>
actor = 'lukasz.langa'
assignee = 'sobolevn'
closed = True
closed_date = <Date 2022-02-07.22:08:55.169>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2022-02-05.11:45:33.833>
creator = 'sobolevn'
dependencies = []
files = []
hgrepos = []
issue_num = 46648
keywords = ['patch']
message_count = 11.0
messages = ['412567', '412588', '412635', '412637', '412638', '412674', '412737', '412740', '412790', '412795', '412824']
nosy_count = 7.0
nosy_names = ['gregory.p.smith', 'vstinner', 'lukasz.langa', 'corona10', 'miss-islington', 'sobolevn', 'GBeauregard']
pr_nums = ['31146', '31161', '31162', '31163', '31173', '31186', '31189', '31190']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46648'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

[sobolevn]

Today I've noticed that a lot of CI runs fail because of this test.

Output:

======================================================================
ERROR: test_issue16464 (test.test_urllib2.MiscTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/support/socket_helper.py", line 245, in transient_internet
    yield
    ^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_urllib2.py", line 1799, in test_issue16464
    opener.open(request, "1".encode("us-ascii"))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib.error.HTTPError: HTTP Error 404: Not Found

----------------------------------------------------------------------
Ran 1 test in 0.448s

FAILED (errors=1)
/Users/sobolev/Desktop/cpython/Lib/test/support/__init__.py:705: ResourceWarning: unclosed <socket.socket [closed] fd=7, family=2, type=1, proto=6>
  gc.collect()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test test_urllib2 failed
test_urllib2 failed (1 error)

== Tests result: FAILURE ==

I can also reproduce this failure locally with:

./python.exe -m test -v test_urllib2 -m test_issue16464 -u network

Related https://bugs.python.org/issue36019

[vstinner]

I close my bpo-46651 as a duplicate of this issue. Copy of my message.

test_urllib2.test_issue16464() fails randomly.

It uses http://www.example.com/ server.

Instead, I proposed to use http://httpbin.org/post URL which is written to support HTTP POST.

$ ./python -m test test_urllib2 -u all -v -m test_issue16464

======================================================================
ERROR: test_issue16464 (test.test_urllib2.MiscTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/test/support/socket_helper.py", line 245, in transient_internet
    yield
    ^^^^^
  File "/home/vstinner/python/main/Lib/test/test_urllib2.py", line 1799, in test_issue16464
    opener.open(request, "1".encode("us-ascii"))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib.error.HTTPError: HTTP Error 404: Not Found

[vstinner]

New changeset 1578de2 by Victor Stinner in branch 'main':
bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161)
1578de2

[miss-islington]

New changeset 1b8a34a by Miss Islington (bot) in branch '3.9':
bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161)
1b8a34a

[miss-islington]

New changeset c88407c by Miss Islington (bot) in branch '3.10':
bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161)
c88407c

[gpshead]

The merged PRs disable the test without anything tracking fixing or re-re-enabling it. Reopening & retitling this issue.

This specific test does not need to use an external server. That has nothing to do with what the purposes of the test is which is all around a content-length issue per https://bugs.python.org/issue16464

The appropriate fix is to have the test manage it's own http server on localhost in another thread or process.

We do not need to test it with https.