Repository navigation
Add multiprocessing.Process.interrupt #131913
Description
Feature or enhancement
Proposal:
We have .terminate() and .kill() but not .interrupt() for terminating multiprocessing.Process. I believe the latter could be a useful addition when we have a python subprocess that we want to terminate in a "normal" way: i.e. triggering finalizers, etc.
This roughly demonstrates the difference when using one or the other (linux):
from signal import SIGINT
from multiprocessing import Process
from time import sleep
def payload():
try:
print("working")
sleep(10)
finally:
print("a very important teardown")
print("> kill or terminate")
p = Process(target=payload)
p.start()
sleep(1)
p.kill() # or terminateoutput
> kill or terminate
working
class MyProcess(Process):
def interrupt(self):
return self._popen._send_signal(SIGINT)
print("> interrupt")
p = MyProcess(target=payload)
p.start()
sleep(1)
p.interrupt()output
> interrupt
working
a very important teardown
Process MyProcess-2:
Traceback (most recent call last):
File "/usr/lib64/python3.13/multiprocessing/process.py", line 313, in _bootstrap
self.run()
~~~~~~~~^^
File "/usr/lib64/python3.13/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "test.py", line 9, in payload
sleep(10)
~~~~~^^^^
KeyboardInterrupt
Motivation for sending SIGINT to subprocesses in general:
- it allows usual python patterns in the payload: we can now define
finally,with, etc. and expect them to trigger normally - SIGINT is a very lean way to cancel heavy payloads. It is widely supported such that if you have non-pure-python stack you still have higher chances to exit gracefully
- I also think it enables a better interaction with nested subprocesses: if a subprocess
Smanages somePoolorconcurrentobject, sendingSIGINTtoSwill let nested subprocesses to exit gracefully as opposed to, for example, leaving orphans on linux. A pure nestedProcesswill still leave orphans but you can write some code in the payload to work with this:
def payload():
try:
p = MyProcess(target=inner)
p.start()
p.join()
finally:
p.interrupt()Motivation for having Process.interrupt() in the standard library:
- it is very easy to misuse
terminateorkillwithout understanding the consequences or particularities of the two..interruptcould become the default or recommended way to interrupt long-running tasks. Even without any payload-specific code, it prints error and python stack trace on termination which is a good starting point to write and debug multiprocessing code, thus, making it a friendlier environment - I feel like an average developer knows more about the difference between
.interruptand.killthan between.terminateand.kill. This is subjective, of course, but, discarding internal reasons to have both.terminateand.kill, I do not understand how.interruptis not in this list.
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Linked PRs
Metadata
Metadata
Assignees
Projects
- Status
Activity
picnixz commentedon Mar 30, 2025
Force shutdown via
kill()andterminate()on ProcessPoolExecutor (not multiprocessing.Process) required a discussion so I think this one also needs a discussion (see discuss.python.org/t/cancel-running-work-in-processpoolexecutor/58605/1 and #128041).I don't have time to expand here so I'll leave it to @gpshead (or I'll do it later)
gpshead commentedon Mar 30, 2025
I don't mind the idea if a
.interruptmethod that plumbs through sending a SIGINT on POSIX similar to how terminate works with SIGTERM.Note that on Windows this concept does not really appear to exist. CTRL_C_EVENT or CTRL_BREAK_EVENT goes to an entire process group.
So decide up front what the Windows behavior should be. Would the .interrupt API not exist there? Would it raise there? Is it another terminate equivalent there (which feels undesirable)?
On POSIX people can already setup SIGTERM handlers; but doing so in the context of a multiprocessing child process is awkward, so it is understandable to want an easier way to use SIGINT.
They could also already send a SIGINT with something similar to:
so this is more of a convenience wrapper around that. literally just this in
multiprocessing/popen_fork.pywith the question being what to put in
multiprocessing/popen_spawn_win32.py.pulkin commentedon Mar 30, 2025
If it can be worked out for individual processes ATM with
GenerateConsoleCtrlEvent(assuming python bindsSIGINThandlers toSetConsoleCtrlHandler) I would go for it. As far as I read your and Claude's comment, we will need to also change how the process is created. I'd love to test the latter but I am not a win user.gpshead commentedon Mar 31, 2025
I would not change how we create windows processes, that'd mess with expectations. if interrupt winds up only being reliably meaningful on POSIX, we just document it as such.
pulkin commentedon Mar 31, 2025
Worth mentioning that
signal(signal.SIGTERM, signal.default_int_handler)and using.terminateis as good for the purposes described in OP. Presumably, this recipe does not have the aforementioned issues on win.pulkin commentedon Apr 11, 2025
Can I assign to myself and raise a MR?
gpshead commentedon Apr 12, 2025
yep, go ahead!
pulkin commentedon Apr 22, 2025
Can you please check the MR?
gh-131913: multiprocessing: add interrupt for POSIX (GH-132453)
gpshead commentedon Apr 23, 2025
merged, thanks!