support.transient_internet() doesn't catch timeout on FTP tests of test_urllib2net

[vstinner]

BPO	22289
Nosy	@vstinner, @ned-deily, @berkerpeksag
Files	transient_internet_ftp_timeout.patch issue22289_3x.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-03-22.08:40:21.072>
created_at = <Date 2014-08-27.12:27:29.652>
labels = ['tests']
title = "support.transient_internet() doesn't catch timeout on FTP tests of test_urllib2net"
updated_at = <Date 2015-03-22.08:40:21.070>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2015-03-22.08:40:21.070>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2015-03-22.08:40:21.072>
closer = 'ned.deily'
components = ['Tests']
creation = <Date 2014-08-27.12:27:29.652>
creator = 'vstinner'
dependencies = []
files = ['36486', '38631']
hgrepos = []
issue_num = 22289
keywords = ['patch']
message_count = 6.0
messages = ['225979', '229965', '238883', '238884', '238885', '238886']
nosy_count = 4.0
nosy_names = ['vstinner', 'ned.deily', 'python-dev', 'berker.peksag']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue22289'
versions = ['Python 3.4', 'Python 3.5']

[vstinner]

Attached patch should fix the following sporadic error. I wrote the patch for Python 3.5. I will adapt it for Python 2.7 and 3.4 if the review is positive.

http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/4837/steps/test/logs/stdio

======================================================================
ERROR: test_ftp_basic (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 1388, in ftp_open
    fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 1410, in connect_ftp
    persistent=False)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 2232, in __init__
    self.init()
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 2238, in init
    self.ftp.connect(self.host, self.port, self.timeout)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/ftplib.py", line 153, in connect
    source_address=self.source_address)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/socket.py", line 655, in create_connection
    raise err
  File "/opt/python/3.x.langa-ubuntu/build/Lib/socket.py", line 646, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/python/3.x.langa-ubuntu/build/Lib/test/test_urllib2net.py", line 308, in test_ftp_basic
    u = _urlopen_with_retry(self.FTP_HOST)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/test/test_urllib2net.py", line 33, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/test/test_urllib2net.py", line 29, in _retry_thrice
    raise last_exc
  File "/opt/python/3.x.langa-ubuntu/build/Lib/test/test_urllib2net.py", line 23, in _retry_thrice
    return func(*args, **kwargs)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 455, in open
    response = self._open(req, data)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 473, in _open
    '_open', req)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 1406, in ftp_open
    raise exc.with_traceback(sys.exc_info()[2])
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 1388, in ftp_open
    fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 1410, in connect_ftp
    persistent=False)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 2232, in __init__
    self.init()
  File "/opt/python/3.x.langa-ubuntu/build/Lib/urllib/request.py", line 2238, in init
    self.ftp.connect(self.host, self.port, self.timeout)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/ftplib.py", line 153, in connect
    source_address=self.source_address)
  File "/opt/python/3.x.langa-ubuntu/build/Lib/socket.py", line 655, in create_connection
    raise err
  File "/opt/python/3.x.langa-ubuntu/build/Lib/socket.py", line 646, in create_connection
    sock.connect(sa)
urllib.error.URLError: <urlopen error ftp error: TimeoutError(110, 'Connection timed out')>

[ned-deily]

There seems to be a small problem with the patch that can be easily fixed by changing the test for "ftp error: timeout()" to "ftp error: timeout(". Otherwise, it seems OK. Note that there is now also a related patch for bpo-22596 that modifies the same test. Perhaps they can be combined.

[ned-deily]

Here's an updated patch for 3.4 and default. To test it, I had to inject a faulty test server with a timeout error, as the test with the dead URL skips properly on my system. For 2.7, the test doesn't fail for me even with the injected server so I propose not bothering to change anything there.

[vstinner]

The patch looks good to me.

[python-dev]

New changeset 9eb1ce7d8039 by Ned Deily in branch '3.4':
Issue bpo-22289: Prevent test_urllib2net failures due to ftp connection timeout.
https://hg.python.org/cpython/rev/9eb1ce7d8039

New changeset 1aad901e0307 by Ned Deily in branch 'default':
Issue bpo-22289: merge from 3.4
https://hg.python.org/cpython/rev/1aad901e0307

[ned-deily]

Thanks for the review. Fix committed for release in 3.4.4 and 3.5.0.