test_lib2to3 fails since Python 3.9

[felixonmars]

BPO	41970
Nosy	@felixonmars, @pablogsal, @miss-islington, @tirkarthi
PRs	bpo-41970: Avoid test failure in test_lib2to3 if the module is already imported #22595 [3.9] bpo-41970: Avoid test failure in test_lib2to3 if the module is already imported (GH-22595) #22609

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-09.10:09:01.376>
created_at = <Date 2020-10-07.23:05:57.966>
labels = ['tests', '3.9', '3.10']
title = 'test_lib2to3 fails since Python 3.9'
updated_at = <Date 2020-10-09.10:09:01.375>
user = 'https://github.com/felixonmars'

bugs.python.org fields:

activity = <Date 2020-10-09.10:09:01.375>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-09.10:09:01.376>
closer = 'pablogsal'
components = ['Tests']
creation = <Date 2020-10-07.23:05:57.966>
creator = 'felixonmars'
dependencies = []
files = []
hgrepos = []
issue_num = 41970
keywords = ['patch']
message_count = 14.0
messages = ['378190', '378196', '378205', '378208', '378232', '378234', '378238', '378240', '378241', '378242', '378245', '378246', '378249', '378310']
nosy_count = 5.0
nosy_names = ['felixonmars', 'pablogsal', 'miss-islington', 'xtreak', 'federico.granata']
pr_nums = ['22595', '22609']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41970'
versions = ['Python 3.9', 'Python 3.10']

[felixonmars]

I am packaging Python for Arch and the tests suite of Python 3.8.6 pass here without this:

0:09:06 load avg: 0.87 [205/424] test_lib2to3                                                                                                                                                                                                                                                                         
test test_lib2to3 crashed -- Traceback (most recent call last):                                                                                                                                                                                                                                                       
  File "/build/python/src/Python-3.9.0/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner                                                                                                                                                                                                                  
    refleak = _runtest_inner2(ns, test_name)                                                                                                                                                                                                                                                                          
  File "/build/python/src/Python-3.9.0/Lib/test/libregrtest/runtest.py", line 221, in _runtest_inner2                                                                                                                                                                                                                 
    the_module = importlib.import_module(abstest)                                                                                                                                                                                                                                                                     
  File "/build/python/src/Python-3.9.0/Lib/importlib/__init__.py", line 127, in import_module                                                                                                                                                                                                                         
    return _bootstrap._gcd_import(name[level:], package, level)                                                                                                                                                                                                                                                       
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import                                                                                                                                                                                                                                                     
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load                                                                                                                                                                                                                                                  
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked                                                                                                                                                                                                                                          
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked                                                                                                                                                                                                                                                   
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module                                                                                                                                                                                                                                             
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed                                                                                                                                                                                                                                        
  File "/build/python/src/Python-3.9.0/Lib/test/test_lib2to3.py", line 5, in <module>                                                                                                                                                                                                                                 
    from lib2to3.tests import load_tests                                                                                                                                                                                                                                                                              
  File "/build/python/src/Python-3.9.0/Lib/contextlib.py", line 124, in __exit__                                                                                                                                                                                                                                      
    next(self.gen)                                                                                                                                                                                                                                                                                                    
  File "/build/python/src/Python-3.9.0/Lib/test/support/__init__.py", line 1171, in _filterwarnings                                                                                                                                                                                                                   
    raise AssertionError("filter (%r, %s) did not catch any warning" %                                                                                                                                                                                                                                                
AssertionError: filter ('', PendingDeprecationWarning) did not catch any warning                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                      
0:09:06 load avg: 0.87 [206/424/1] test_linecache -- test_lib2to3 failed    

[pablogsal]

Same thing as in https://bugs.python.org/issue41971. I cannot reproduce this with a fresh install of 3.9 and our buildbots are not complaining about this.

ownloads/Python-3.9.0
❯ ./python -m test test_lib2to3
0:00:00 load avg: 3.64 Run tests sequentially
0:00:00 load avg: 3.64 [1/1] test_lib2to3

== Tests result: SUCCESS ==

1 test OK.

Total duration: 7.5 sec
Tests result: SUCCESS

-------

❯ ./lel/bin/python3 -m test test_lib2to3
0:00:00 load avg: 3.87 Run tests sequentially
0:00:00 load avg: 3.87 [1/1] test_lib2to3

== Tests result: SUCCESS ==

1 test OK.

Total duration: 7.4 sec
Tests result: SUCCESS

[tirkarthi]

I have also seen this and linked issue on running whole test suite while building python myself but running individual tests show no error and buildbots also never complained so I thought it's something wrong with my setup. I introduced the commit to silence deprecation warning due to lib2to3 and cannot reproduce the same running it individually.

[pablogsal]

I have also seen this and linked issue on running whole test suite while building python myself but running individual tests show no error and buildbots also never complained so I thought it's something wrong with my setup. I introduced the commit to silence deprecation warning due to lib2to3 and cannot reproduce the same running it individually.

What I don't understand then is how the buildbots or the CI in Github that run all tests sequentially with the extra flags (-uall and friends) have not complained yet about this.

@felix, how consistent is this failure you are experiencing?

[federicogranata]

I just experienced the very same issue.
I use pyenv and doing all tests gives me a failure on test_lib2to3 while running it individually end succesfully.

Tell me if I can provide any useful information

[tirkarthi]

Pablo, I can see the error in Travis. The difference I can see is that the tests are executed sequentially in coverage run but normal run has tests executing in parallel. Felix and Federico, can you please add the command you use to run the tests?

Sample error build : https://travis-ci.com/github/python/cpython/jobs/396688564#L2022

coverage run but has true in the end thus being always successful

xvfb-run ./venv/bin/python -m coverage run --branch --pylib -m test --fail-env-changed -uall,-cpu -x test_multiprocessing_fork -x test_multiprocessing_forkserver -x test_multiprocessing_spawn -x test_concurrent_futures || true

Normal run

/home/travis/build/python/cpython/python -u -W default -bb -E -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=1200 -j4 -uall,-cpu

[felixonmars]

I use this in Arch's packaging:

LC_CTYPE=en_US.UTF-8 xvfb-run -s "-screen 0 1920x1080x16 -ac +extension GLX" -a -n "$servernum" \
    "${srcdir}/Python-${pkgver}/python" -m test.regrtest -v -uall -x test_tk

(test_tk is currently skipped due to https://bugs.python.org/issue41306)

[federicogranata]

I just run "python -m test" for it to fail and "python -m test test_lib2to3" for it to succeded.

[tirkarthi]

I can reproduce this by running test_lib2to3 twice or running test___all__ with test_lib2to3

./python -m test test_lib2to3 test_lib2to3
0:00:00 load avg: 0.03 Run tests sequentially
0:00:00 load avg: 0.03 [1/2] test_lib2to3
0:00:13 load avg: 0.25 [2/2] test_lib2to3
test test_lib2to3 crashed -- Traceback (most recent call last):
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 272, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 223, in _runtest_inner2
    the_module = importlib.import_module(abstest)
  File "/root/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/cpython/Lib/test/test_lib2to3.py", line 5, in <module>
    from lib2to3.tests import load_tests
  File "/root/cpython/Lib/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "/root/cpython/Lib/test/support/warnings_helper.py", line 179, in _filterwarnings
    raise AssertionError("filter (%r, %s) did not catch any warning" %
AssertionError: filter ('', PendingDeprecationWarning) did not catch any warning

test_lib2to3 failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_lib2to3

Total duration: 13.5 sec
Tests result: FAILURE

./python -m test test___all__ test_lib2to3     
0:00:00 load avg: 0.05 Run tests sequentially
0:00:00 load avg: 0.05 [1/2] test___all__
0:00:00 load avg: 0.05 [2/2] test_lib2to3
test test_lib2to3 crashed -- Traceback (most recent call last):
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 272, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 223, in _runtest_inner2
    the_module = importlib.import_module(abstest)
  File "/root/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/cpython/Lib/test/test_lib2to3.py", line 5, in <module>
    from lib2to3.tests import load_tests
  File "/root/cpython/Lib/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "/root/cpython/Lib/test/support/warnings_helper.py", line 179, in _filterwarnings
    raise AssertionError("filter (%r, %s) did not catch any warning" %
AssertionError: filter ('', PendingDeprecationWarning) did not catch any warning

test_lib2to3 failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_lib2to3

Total duration: 1.1 sec
Tests result: FAILURE

[federicogranata]

I can have the test to fails if I run "python -m test test_lib2to3 test_lib2to3" as well

[pablogsal]

Ok, thanks everyone for the help reproducing this. I think I can work on a fix today