Feature or enhancement
The multiprocessing SemLock code maintains an internal count field. When operating as a recursive mutex, count is the number of times the thread has acquired the mutex (i.e., 0, 1, .. N). When operating as a semaphore, count is often 0 or 1, but can be negative if the SemLock is initialized with maxvalue > 1
The modification of count is not thread-safe without the GIL within the process. Note that the count field is not shared across processes, unlike the underlying sem_t or HANDLE.
In particular, the modification of count after the semaphore is released is not thread-safe without the GIL:
|
if (sem_post(self->handle) < 0) |
|
return PyErr_SetFromErrno(PyExc_OSError); |
|
|
|
--self->count; |
|
Py_RETURN_NONE; |
I think this is the source of deadlocks when running test_multiprocessing_forkserver.test_processes with the GIL disabled.
Linked PRs
Feature or enhancement
The multiprocessing
SemLockcode maintains an internalcountfield. When operating as a recursive mutex,countis the number of times the thread has acquired the mutex (i.e., 0, 1, .. N). When operating as a semaphore,countis often0or1, but can be negative if theSemLockis initialized withmaxvalue > 1The modification of
countis not thread-safe without the GIL within the process. Note that thecountfield is not shared across processes, unlike the underlyingsem_torHANDLE.In particular, the modification of
countafter the semaphore is released is not thread-safe without the GIL:cpython/Modules/_multiprocessing/semaphore.c
Lines 444 to 448 in 9dae05e
I think this is the source of deadlocks when running
test_multiprocessing_forkserver.test_processeswith the GIL disabled.Linked PRs
SemLockthread-safe in free-threaded build #117436