test_nntplib depends on unreliable external servers

[cjerdonek]

BPO	40735
Nosy	@vstinner, @cjerdonek, @zware, @iritkatriel

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-13.19:06:04.946>
created_at = <Date 2020-05-22.21:07:45.966>
labels = ['type-feature', 'tests', '3.11']
title = 'test_nntplib depends on unreliable external servers'
updated_at = <Date 2022-03-14.15:50:45.562>
user = 'https://github.com/cjerdonek'

bugs.python.org fields:

activity = <Date 2022-03-14.15:50:45.562>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-13.19:06:04.946>
closer = 'iritkatriel'
components = ['Tests']
creation = <Date 2020-05-22.21:07:45.966>
creator = 'chris.jerdonek'
dependencies = []
files = []
hgrepos = []
issue_num = 40735
keywords = []
message_count = 5.0
messages = ['369648', '411529', '411649', '415081', '415150']
nosy_count = 4.0
nosy_names = ['vstinner', 'chris.jerdonek', 'zach.ware', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40735'
versions = ['Python 3.11']

[cjerdonek]

A sporadic failure of test_nntplib.NetworkedNNTP_SSLTests.test_with_statement on the CI for macOS:
https://github.com/python/cpython/pull/20321/checks?check_run_id=700729471#step:6:612

See also:

• https://bugs.python.org/issue19613 (test_article_head_body)
• https://bugs.python.org/issue19756 (test_capabilities)

ERROR: test_with_statement (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/test/test_nntplib.py", line 277, in test_with_statement
    server = self.NNTP_CLASS(self.NNTP_HOST,
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 1025, in __init__
    super().__init__(host, port, user, password, readermode,
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 334, in __init__
    self.sock = self._create_socket(timeout)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 1031, in _create_socket
    sock = _encrypt_on(sock, self.ssl_context, self.host)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 292, in _encrypt_on
    return context.wrap_socket(sock, server_hostname=hostname)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1097)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/test/test_nntplib.py", line 250, in wrapped
    meth(self)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/test/test_nntplib.py", line 293, in test_with_statement
    if re.search(r'(?i)KEY.TOO.SMALL', ssl_err.reason):
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/re.py", line 201, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

[iritkatriel]

Following discussion with Zach and Victor: This is probably an issue with the external servers these tests used. The fix would be to rewrite the tests so that they don't rely on external infrastructure.

[vstinner]

IMO for now it's better to *always* skip test_nnptlib tests which rely on external servers and close related issues.

• https://bugs.python.org/issue19613 (test_article_head_body)
• https://bugs.python.org/issue19756 (test_capabilities)

These issues are open for 9 years. NNTP protocol is getting less and less popular, I don't expect that anyone will come in 2022 and suddenly propose a fix.