stdout error at interpreter shutdown fails to return OS error status

[mkc]

BPO	5319
Nosy	@loewis, @birkenfeld, @terryjreedy, @pitrou, @rbtcollins, @vadmium, @zooba, @corona10
PRs	bpo-39828: Fix json.tool to catch BrokenPipeError. #18779
Files	devfull.patch finalize-error.patch finalize-error.v2.patch: Py_Finalize() returns -1 Py_FinalizeEx.patch: New API Py_FinalizeEx-120.patch: Exit status 120

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-11-30.04:12:14.400>
created_at = <Date 2009-02-19.17:38:00.077>
labels = ['interpreter-core', 'type-bug']
title = 'stdout error at interpreter shutdown fails to return OS error status'
updated_at = <Date 2020-03-04.22:11:56.278>
user = 'https://bugs.python.org/mkc'

bugs.python.org fields:

activity = <Date 2020-03-04.22:11:56.278>
actor = 'corona10'
assignee = 'none'
closed = True
closed_date = <Date 2015-11-30.04:12:14.400>
closer = 'martin.panter'
components = ['Interpreter Core']
creation = <Date 2009-02-19.17:38:00.077>
creator = 'mkc'
dependencies = []
files = ['18395', '37513', '39592', '39636', '40306']
hgrepos = []
issue_num = 5319
keywords = ['patch']
message_count = 26.0
messages = ['82482', '82484', '112342', '112919', '112923', '113330', '148423', '232967', '244647', '244662', '244666', '244812', '244817', '244891', '248628', '249183', '249187', '249191', '249192', '249356', '249370', '249377', '249378', '255585', '255605', '255606']
nosy_count = 11.0
nosy_names = ['loewis', 'georg.brandl', 'terry.reedy', 'mkc', 'pitrou', 'rbcollins', 'Arfrever', 'python-dev', 'martin.panter', 'steve.dower', 'corona10']
pr_nums = ['18779']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue5319'
versions = ['Python 3.6']

[mkc]

$ python2.5 -c 'print((1, 2, 3))' > /dev/full || echo error status
close failed: [Errno 28] No space left on device
$

The above sequence should also output 'error status' before the next
prompt. That is, python should not be returning EXIT_SUCCESS in this
situation.

[mkc]

Oops, I should have used the old python print syntax. Nonetheless, the
behavior is the same:

$ python2.5 -c 'print 1, 2, 3' > /dev/full || echo error status
close failed: [Errno 28] No space left on device
$

[birkenfeld]

Python 2.6, from bpo-5320:

$ python2.6 -c 'print 1, 2, 3' > /dev/full || echo error status
close failed in file object destructor:
Error in sys.excepthook:

Original exception was:
$

Python 3.1 doesn't output anything and also doesn't set an error status.

[terryjreedy]

Georg, are you saying that there is or is not a problem with 2.6 (which is beyond non-critical bug fixes) and/or 3.1?

[pitrou]

Here is a possible patch for 3.x.
It doesn't change the return status of the process, though.

[pitrou]

I've committed the py3k fix in r83854 (3.1 in r83855).
As for 2.6/2.7, I've finally diagnosed the problem: 2.6 and upwards use PyErr_Print() in the file destructor, but the sys.stderr and sys.excepthook objects are already None at that time (because of interpreter shutdown). Therefore the error can't be printed out.
2.5 and earlier used a direct fprintf() of the errno code to stderr.

I propose to ignore the 2.6/2.7 issue of the error not being printed properly, since it only happens at shutdown with a misbehaving stdout, which is not very common.

As for the issue of the stdout error not being reported in the process return code, this would be possible in 3.x, although it would involve changing the signature of Py_Finalize() to include a return status (it currently returns void).

[python-dev]

New changeset 150e096095e5 by Antoine Pitrou in branch '3.2':
Issue bpo-13444: When stdout has been closed explicitly, we should not attempt to flush it at shutdown and print an error.
http://hg.python.org/cpython/rev/150e096095e5

New changeset 37300a1df7d7 by Antoine Pitrou in branch 'default':
Issue bpo-13444: When stdout has been closed explicitly, we should not attempt to flush it at shutdown and print an error.
http://hg.python.org/cpython/rev/37300a1df7d7

[vadmium]

Here is a patch that changes Py_Finalize() to return -1 on error, and then sets the exit status to 1. It did not introduce any failures in the test suite for me. However I suspect it deserves more consideration about backwards compatibility etc, which is beyond my knowledge.

[vadmium]

Updated patch resolving minor merge conflicts with current code, a coding style fix, and some tweaks and syntax fixes to the documentation.

[pitrou]

I don't know how acceptable it is to add a return value to Py_Finalize(). Can it break the stable ABI?

[zooba]

Going from void to int is fine for when the return value is ignored. However, those who check the return value will see garbage on earlier versions.

As always, safest to add a new function here. Or make the stable version continue to be void - it won't affect the exports.

[vadmium]

I guess this would involve:

• Making a new API called Py_Finalize2() or something that returns the status
• Redefine the existing Py_Finalize() to call Py_Finalize2() and ignore the return value