Skip to content

bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed #3247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
pitrou merged 4 commits into from
Nov 3, 2017
Merged

bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed #3247

pitrou merged 4 commits into from
Nov 3, 2017

Conversation

@pitrou
Copy link
Member

@pitrou pitrou commented Aug 30, 2017 •
edited by bedevere-bot
Loading

pitrou added 3 commits August 30, 2017 18:21
@pitrou
Avoid mucking with process state in test.
459efde 
Add a warning if the semaphore process died, as semaphores may then be leaked.
@pitrou
Add NEWS entry
ee72cba
@pitrou
Merge branch 'master' into semaphore_tracker_death
6eb0607
@pitrou pitrou merged commit cbe1756 into python:master Nov 3, 2017
@pitrou pitrou deleted the semaphore_tracker_death branch November 3, 2017 13:31
pitrou added a commit to pitrou/cpython that referenced this pull request Nov 3, 2017
@pitrou
[3.6] bpo-31310: multiprocessing's semaphore tracker should be launch…
5623e3a 
…ed again if crashed (pythonGH-3247)

* bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed

* Avoid mucking with process state in test.
Add a warning if the semaphore process died, as semaphores may then be leaked.

* Add NEWS entry
(cherry picked from commit cbe1756)
pitrou added a commit that referenced this pull request Nov 3, 2017
@pitrou
[3.6] bpo-31310: multiprocessing's semaphore tracker should be launch…
b5f09ac 
…ed again if crashed (GH-3247) (#4254)

* bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed

* Avoid mucking with process state in test.
Add a warning if the semaphore process died, as semaphores may then be leaked.

* Add NEWS entry
(cherry picked from commit cbe1756)
embray pushed a commit to embray/cpython that referenced this pull request Nov 9, 2017
@pitrou @embray
bpo-31310: multiprocessing's semaphore tracker should be launched aga…
b0ce127 
…in if crashed (python#3247)

* bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed

* Avoid mucking with process state in test.
Add a warning if the semaphore process died, as semaphores may then be leaked.

* Add NEWS entry
@tomMoral
Copy link
Contributor

tomMoral commented Jan 13, 2018

With this PR, the semaphore_tracker is not shared with the children anymore and each process launches its own tracker.

The easy fix would be to pass the _pid to the children but the current mechanism to check if the semaphore_tracker is alive relies on waitpid which cannot be used in child processes (the semaphore_tracker is only a sibling of these processes).
I did some experiements with an extra cmd in the semaphore_tracker in #5172 . The main issue is to have a reliable check that either:

  • The pipe is open. This is what is done here by sending a message. I don't know if there is a more efficient way to check it.
  • Check that a given pid is alive. As we cannot rely on waitpid, I don't see an efficient mechanism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@pitrou @tomMoral @the-knights-who-say-ni @bedevere-bot