Some tests leak temporary files

[vstinner]

BPO	37421
Nosy	@vstinner, @zware, @serhiy-storchaka, @pablogsal, @miss-islington
PRs	[WIP] bpo-37421: regrtest detects if a test leaks temporary files #14415 bpo-37421: Fix test_shutil: don't leak temporary files #14416 [3.8] bpo-37421: Fix test_shutil: don't leak temporary files (GH-14416) #14417 bpo-37421: _test_multiprocessing calls _run_finalizers() #14527 bpo-37421: test_urllib calls urlcleanup() #14529 [3.8] bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527) #14554 bpo-37421: test_winconsoleio doesn't leak temp file anymore #14562 bpo-37421: test_concurrent_futures cleans up multiprocessing #14563 bpo-37421: Fix test_distutils.test_build_ext() #14564 [3.8] bpo-37421: test_winconsoleio doesn't leak temp file anymore (GH-14562) #14569 [3.8] bpo-37421: test_concurrent_futures cleans up multiprocessing (GH-14563) #14570 [3.8] bpo-37421: Fix test_distutils.test_build_ext() (GH-14564) #14571 bpo-37421: Fix multiprocessing get_temp_dir() finalizer #14572 [3.8] bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572) #14585 [3.7] bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572) #14586 bpo-37421: multiprocessing tests now stop ForkServer #14601 [3.8] bpo-37421: multiprocessing tests now stop ForkServer (GH-14601) #14602 bpo-37421: test_concurrent_futures stop ForkServer #14643 [3.8] bpo-37421: test_concurrent_futures stops ForkServer (GH-14643) #14645 bpo-38546: multiprocessing tests stop the resource tracker #17641

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-07-08.22:10:28.983>
created_at = <Date 2019-06-26.22:34:15.944>
labels = ['tests', '3.9']
title = 'Some tests leak temporary files'
updated_at = <Date 2020-04-23.22:44:12.215>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-04-23.22:44:12.215>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-07-08.22:10:28.983>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-06-26.22:34:15.944>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 37421
keywords = ['patch']
message_count = 26.0
messages = ['346692', '346693', '346699', '346700', '346703', '347127', '347135', '347137', '347140', '347191', '347192', '347193', '347199', '347201', '347204', '347265', '347270', '347273', '347343', '347348', '347352', '347355', '347492', '347493', '347513', '367157']
nosy_count = 5.0
nosy_names = ['vstinner', 'zach.ware', 'serhiy.storchaka', 'pablogsal', 'miss-islington']
pr_nums = ['14415', '14416', '14417', '14527', '14529', '14554', '14562', '14563', '14564', '14569', '14570', '14571', '14572', '14585', '14586', '14601', '14602', '14643', '14645', '17641']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue37421'
versions = ['Python 3.9']

[vstinner]

When running tests, the tempoary directory is not left clean: there are temporary files which are not removed.

I wrote a PoC change to detect such bugs and I found that at least test_shutil and test_urllib leak such temporary files/directories.

[vstinner]

Test leaking a temporary file in test_urllib:
test.test_urllib.urlretrieve_HttpTests.test_short_content_raises_ContentTooShortError_without_reporthook

I'm not sure of my fix:

diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
index f6ce9cb6d5..fdddd6e2d8 100644
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -283,6 +283,7 @@ def urlretrieve(url, filename=None, reporthook=None, data=None):
                     reporthook(blocknum, bs, size)
 
     if size >= 0 and read < size:
+        urlcleanup()
         raise ContentTooShortError(
             "retrieval incomplete: got only %i out of %i bytes"
             % (read, size), result)

[vstinner]

Using PR 14415, the following test of test_multiprocessing_spawn emits a false alarm because multiprocessing use "Finalizer" objects which are only finalized "later":
test.test_multiprocessing_spawn.WithManagerTestMyManager.test_mymanager_context_prestarted

Workaround, call explicitly _run_finalizers():

diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py
index eef262d723..bfecbab9ee 100644
--- a/Lib/test/_test_multiprocessing.py
+++ b/Lib/test/_test_multiprocessing.py
@@ -5651,6 +5651,7 @@ def install_tests_in_module_dict(remote_globs, start_method):
         if need_sleep:
             time.sleep(0.5)
         multiprocessing.process._cleanup()
+        multiprocessing.util._run_finalizers()
         test.support.gc_collect()
 
     remote_globs['setUpModule'] = setUpModule

[vstinner]

New changeset 4c26abd by Victor Stinner in branch 'master':
bpo-37421: Fix test_shutil: don't leak temporary files (GH-14416)
4c26abd

[miss-islington]

New changeset 7fe81ce by Miss Islington (bot) in branch '3.8':
bpo-37421: Fix test_shutil: don't leak temporary files (GH-14416)
7fe81ce

[vstinner]

About urllib.request, I created: bpo-37475 "What is urllib.request.urlcleanup() function?".

[vstinner]

New changeset 039fb49 by Victor Stinner in branch 'master':
bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527)
039fb49

[miss-islington]

New changeset 632cb36 by Miss Islington (bot) in branch '3.8':
bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527)
632cb36

[vstinner]

New changeset 7cb9204 by Victor Stinner in branch 'master':
bpo-37421: urllib.request tests call urlcleanup() (GH-14529)
7cb9204

[vstinner]

New changeset b71d8d6 by Victor Stinner in branch 'master':
bpo-37421: test_winconsoleio doesn't leak temp file anymore (GH-14562)
b71d8d6

[vstinner]

New changeset 684cb47 by Victor Stinner in branch 'master':
bpo-37421: test_concurrent_futures cleans up multiprocessing (GH-14563)
684cb47

[vstinner]

New changeset 74c9dd5 by Victor Stinner in branch 'master':
bpo-37421: Fix test_distutils.test_build_ext() (GH-14564)
74c9dd5