test_multiprocessing_forkserver, SemLock: test_thousand() logged on AMD64 Ubuntu Shared 3.x: Exception ignored in: <Finalize object, dead>

[vstinner]

BPO	37338
Nosy	@pitrou, @vstinner, @applio, @pablogsal

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-02-07.10:52:13.610>
created_at = <Date 2019-06-19.09:57:40.253>
labels = ['3.8', 'tests']
title = 'test_multiprocessing_forkserver, SemLock: test_thousand() logged on AMD64 Ubuntu Shared 3.x: Exception ignored in: <Finalize object, dead>'
updated_at = <Date 2020-02-07.10:52:13.609>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-02-07.10:52:13.609>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-02-07.10:52:13.610>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-06-19.09:57:40.253>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 37338
keywords = []
message_count = 3.0
messages = ['346033', '346034', '361572']
nosy_count = 4.0
nosy_names = ['pitrou', 'vstinner', 'davin', 'pablogsal']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue37338'
versions = ['Python 3.8']

[vstinner]

Oh, I'm happy! My regrtest change (bpo-37069) to detect unraisable exception detected its first issue :-)

AMD64 Ubuntu Shared 3.x:
https://buildbot.python.org/all/#/builders/141/builds/2029

test_thousand (test.test_multiprocessing_forkserver.WithProcessesTestBarrier) ... Warning -- Unraisable exception
Exception ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/util.py", line 189, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/synchronize.py", line 87, in _cleanup
    sem_unlink(name)
FileNotFoundError: [Errno 2] No such file or directory
Warning -- Unraisable exception
Exception ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/util.py", line 189, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/synchronize.py", line 87, in _cleanup
    sem_unlink(name)
FileNotFoundError: [Errno 2] No such file or directory
Warning -- Unraisable exception
Exception ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/util.py", line 189, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/synchronize.py", line 87, in _cleanup
    sem_unlink(name)
FileNotFoundError: [Errno 2] No such file or directory
Warning -- Unraisable exception
Exception ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/util.py", line 189, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/synchronize.py", line 87, in _cleanup
    sem_unlink(name)
FileNotFoundError: [Errno 2] No such file or directory
Warning -- Unraisable exception
Exception ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/util.py", line 189, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/synchronize.py", line 87, in _cleanup
    sem_unlink(name)
FileNotFoundError: [Errno 2] No such file or directory
ok

--

I'm unable to reproduce the issue, it looks like a race condition.

I ran the following commands 4 times in parallel and I stressed my laptop:

./python -m test --fail-env-changed test_multiprocessing_forkserver -m test_thousand -v -F

[vstinner]

Currently, SemLock does not provide a way to explicitly delete it. Maybe a new close() method should be added, similar to Process.close() and Pool.terminate(), to make resource management more deterministic (more reliable).

[vstinner]

I didn't see the failure recently, I close it.