test_multiprocessing_fork: dangling threads warning

[vstinner]

BPO	33676
Nosy	@vstinner, @miss-islington
PRs	bpo-33676: Fix dangling thread in _test_multiprocessing #10755 [3.7] bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755) #10756 [3.6] bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755) #10757
Files	mp_debug.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-11-28.00:44:38.987>
created_at = <Date 2018-05-28.23:03:48.911>
labels = ['3.8', '3.7', 'tests']
title = 'test_multiprocessing_fork: dangling threads warning'
updated_at = <Date 2018-11-28.00:44:38.984>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2018-11-28.00:44:38.984>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-28.00:44:38.987>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2018-05-28.23:03:48.911>
creator = 'vstinner'
dependencies = []
files = ['47953']
hgrepos = []
issue_num = 33676
keywords = ['patch']
message_count = 6.0
messages = ['317948', '330560', '330565', '330566', '330567', '330568']
nosy_count = 2.0
nosy_names = ['vstinner', 'miss-islington']
pr_nums = ['10755', '10756', '10757']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue33676'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[vstinner]

AMD64 FreeBSD 10.x Shared 3.7:

http://buildbot.python.org/all/#/builders/124/builds/327

== CPU count: 2
Run tests in parallel using 4 child processes
(...)
0:02:09 load avg: 3.70 [138/415/1] test_multiprocessing_fork failed (env changed) (108 sec) -- running: test_concurrent_futures (124 sec)
(...)
test_starmap (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_starmap_async (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_terminate (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_traceback (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_wrapped_exception (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok

Warning -- Dangling threads: {<DummyProcess(Thread-158, started daemon 34498362368)>}

[vstinner]

The bug is still alive:
https://buildbot.python.org/all/#/builders/168/builds/294

I found a bug using attached mp_debug.patch.

I wrote PR 10755 to try to fix it.

I have access to FreeBSD CURRENT buildbot. I only reproduced the bug once using:

./python -m test -v test_multiprocessing_fork -m WithThreadsTestPool --fail-env-changed -F

I ran this command 4 times in parallel in 4 terminals, but I failed to reliably reproduce the bug. Even if I stress the computer using "./python -m test -j4 -r" run in different shell, or using my "system_load.py 3" script, I fail to trigger the bug.

[vstinner]

New changeset b727873 by Victor Stinner in branch 'master':
bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755)
b727873

[miss-islington]

New changeset 80db40c by Miss Islington (bot) in branch '3.7':
bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755)
80db40c

[miss-islington]

New changeset dd52938 by Miss Islington (bot) in branch '3.6':
bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755)
dd52938

[vstinner]

I *hope* that it will be enough :-/