subprocess Popen objects are not thread safe w.r.t. wait() and returncode being set

[gpshead]

BPO	21291
Nosy	@gpshead
Files	subprocess_wait_problem.py issue21291-fix-gps01.diff issue21291-patch-with-test-gps01.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2014-04-23.08:35:30.727>
created_at = <Date 2014-04-17.21:49:43.112>
labels = ['type-bug']
title = 'subprocess Popen objects are not thread safe w.r.t. wait() and returncode being set'
updated_at = <Date 2014-04-23.08:35:30.725>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2014-04-23.08:35:30.725>
actor = 'gregory.p.smith'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2014-04-23.08:35:30.727>
closer = 'gregory.p.smith'
components = []
creation = <Date 2014-04-17.21:49:43.112>
creator = 'gregory.p.smith'
dependencies = []
files = ['34955', '34960', '35007']
hgrepos = []
issue_num = 21291
keywords = ['patch']
message_count = 4.0
messages = ['216757', '216770', '217057', '217059']
nosy_count = 2.0
nosy_names = ['gregory.p.smith', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21291'
versions = ['Python 3.4', 'Python 3.5']

[gpshead]

Executing the supplied test code you either get:

sys.version = 3.4.0+ (3.4:635817da596d, Apr 17 2014, 14:30:34) 
[GCC 4.6.3]

---

Results with <class 'subprocess.Popen'>:
r0 = None, expected None
r1 = None, expected None
ri0 = None, expected None
ri1 = -9, expected -9
ri2 = -9, expected -9
r2 = 0, expected -9 *** MISMATCH ***
r3 = 0, expected -9 *** MISMATCH ***

or

Results with <class 'subprocess.Popen'>:
r0 = None, expected None
r1 = None, expected None
ri0 = None, expected None
ri1 = -9, expected -9
ri2 = -9, expected -9
r2 = 0, expected -9 *** MISMATCH ***
r3 = 0, expected -9 *** MISMATCH ***

At first glance it appears that the .returncode attribute is not safely set after the wait call... This test code is using the Popen object from multiple threads at once.

[gpshead]

Attaching a proposed fix for this issue. It should make the wait() and poll() methods thread safe. I need to turn the reproducer code into an actual test case and add more test cases for coverage of all code paths being touched.

I haven't examined the windows side of this code for the issue, it may also be useful there.

[python-dev]

New changeset 5d745d97b7da by Gregory P. Smith in branch '3.4':
subprocess's Popen.wait() is now thread safe so that multiple threads
http://hg.python.org/cpython/rev/5d745d97b7da

New changeset df45d0336dad by Gregory P. Smith in branch 'default':
subprocess's Popen.wait() is now thread safe so that multiple threads
http://hg.python.org/cpython/rev/df45d0336dad

[gpshead]

This fix is also present in subprocess32 3.2.6 on PyPI for use on Python 2.