race condition in initstdio() (python aborts running under nohup)

[yiding]

BPO	24891
Nosy	@ronaldoussoren, @vstinner, @rbtcollins, @ned-deily, @mpaolini
Files	test.py test.sh issue24891.patch issue24891_2.patch issue24891_3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-09-04.15:35:10.873>
created_at = <Date 2015-08-18.22:18:48.236>
labels = ['interpreter-core', 'type-crash']
title = 'race condition in initstdio() (python aborts running under nohup)'
updated_at = <Date 2015-09-04.20:23:28.036>
user = 'https://bugs.python.org/YiDing'

bugs.python.org fields:

activity = <Date 2015-09-04.20:23:28.036>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2015-09-04.15:35:10.873>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2015-08-18.22:18:48.236>
creator = 'Yi Ding'
dependencies = []
files = ['40209', '40210', '40354', '40355', '40357']
hgrepos = []
issue_num = 24891
keywords = ['patch']
message_count = 19.0
messages = ['248797', '248798', '248800', '248801', '248803', '248804', '248805', '248806', '249514', '249753', '249754', '249756', '249760', '249766', '249770', '249773', '249775', '249823', '249824']
nosy_count = 7.0
nosy_names = ['ronaldoussoren', 'vstinner', 'rbcollins', 'ned.deily', 'python-dev', 'mpaolini', 'Yi Ding']
pr_nums = []
priority = 'low'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue24891'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

[yiding]

Looks like this bug https://bugs.python.org/issue7111 has resurfaced in python3 (python 2.6 works as far as I can tell) at least on Macs.

I've attached a simple test script.

Steps:

1. SSH to remote server.
2. Run nohup ./test.sh &
3. exit SSH.
4. SSH back in and see that there are a bunch of errors in your nohup.out file.

[yiding]

test.sh attached

[rbtcollins]

What sort of errors?

[vstinner]

I'm unable to reproduce the issue on Linux with Python 3.4.2.

[yiding]

Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
./test.sh: line 4: 49632 Abort trap: 6 python3 test.py -o hello.txt

[yiding]

That's from my nohup.out. It might be a Mac OS specific thing.

[vstinner]

I can reproduce the bug with gdb if the file descriptor 0 is closed before calling:
    std = create_stdio(iomod, fd, 0, "<stdin>", encoding, errors);
and after the following lines were called:
    fd = fileno(stdin);
    if (!is_valid_fd(fd)) {

In initstdio () at Python/pylifecycle.c:1156.

create_stdio() fails in FileIO constructor, on the line:
if (_Py_fstat(self->fd, &fdfstat) < 0)
of _io_FileIO___init___impl() at Modules/_io/fileio.c:480

It's a corner case of the issue bpo-7111, I would call it a race condition.

[vstinner]

Python 2 is not affected, PyFile_FromFile() doesn't check if the file descriptor is valid. On Python 3, we call fstat() to check the block size of the file descriptor to optimize buffering.

[yiding]

I'm not sure this is a race condition. There's a crash every single time python is started when running the test.sh script under nohup.

If it's a race condition, it should only happen some of the time, not every single time.

Can you reeevaluate the priority?

[mpaolini]

Attaching a patch. Does it make any sense?