bpo-31326: ProcessPoolExecutor waits for the call queue thread (#3265)

* bpo-31326: ProcessPoolExecutor waits for the call queue thread

concurrent.futures.ProcessPoolExecutor.shutdown() now explicitly
closes the call queue. Moreover, shutdown(wait=True) now also join
the call queue thread, to prevent leaking a dangling thread.

* Fix for shutdown() being called twice.

Author: vstinner

Lib/concurrent/futures/process.py

@@ -507,7 +507,11 @@ def shutdown(self, wait=True):
         # To reduce the risk of opening too many files, remove references to
         # objects that use file descriptors.
         self._queue_management_thread = None
-        self._call_queue = None
+        if self._call_queue is not None:
+            self._call_queue.close()
+            if wait:
+                self._call_queue.join_thread()
+            self._call_queue = None
         self._result_queue = None
         self._processes = None
     shutdown.__doc__ = _base.Executor.shutdown.__doc__

Misc/NEWS.d/next/Library/2017-09-01-18-48-06.bpo-31326.TB05tV.rst

@@ -0,0 +1,3 @@
+concurrent.futures.ProcessPoolExecutor.shutdown() now explicitly closes the
+call queue. Moreover, shutdown(wait=True) now also join the call queue
+thread, to prevent leaking a dangling thread.