Py_SetStandardStreamEncoding leads to a memory error in debug mode

[ncoghlan]

BPO	29818
Nosy	@ncoghlan, @vstinner

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-05-15.02:53:12.376>
created_at = <Date 2017-03-15.11:27:39.955>
labels = ['interpreter-core', '3.7', 'type-crash']
title = 'Py_SetStandardStreamEncoding leads to a memory error in debug mode'
updated_at = <Date 2019-05-15.02:53:12.374>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2019-05-15.02:53:12.374>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-05-15.02:53:12.376>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2017-03-15.11:27:39.955>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 29818
keywords = []
message_count = 2.0
messages = ['289668', '342540']
nosy_count = 2.0
nosy_names = ['ncoghlan', 'vstinner']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue29818'
versions = ['Python 3.7']

[ncoghlan]

For PEP-538, setting PYTHONIOENCODING turned out to have undesirable side effects on Python 2 instances in subprocesses, since Python 2 has no 'surrogateescape' error handler.

So I switched to using the "Py_SetStandardStreamEncoding" API defined in http://bugs.python.org/issue16129 instead, but this turns out to have problematic interactions with the dynamic memory allocator management, so it fails with a fatal exception in debug mode. An example of the error can be seen here: https://travis-ci.org/python/cpython/jobs/211293576

The problem appears to be that between the allocation of the memory with _PyMem_RawStrdup in Py_SetStandardStreamEncoding and the release of that memory in initstdio, the active memory manager has changed (at least in a debug build), so the deallocation as part of the interpreter startup fails.

That interpretation is based on this comment in Programs/python.c:

    /* Force again malloc() allocator to release memory blocks allocated
       before Py_Main() */
    (void)_PyMem_SetupAllocators("malloc");

The allocations in Py_SetStandardStreamEncoding happen before the call to Py_Main/Py_Initialize, but the deallocation happens in Py_Initialize.

The "fix" I applied to the PEP branch was to make the default allocator conditional in Programs/python.c as well:

#ifdef Py_DEBUG
    (void)_PyMem_SetupAllocators("malloc_debug");
#  else
    (void)_PyMem_SetupAllocators("malloc");
#  endif

While that works (at least in the absence of a PYTHONMALLOC setting) it seems fragile. It would be nicer if there was a way for Py_SetStandardStreamEncoding to indicate which allocator should be used for the deallocation.

[vstinner]

I fixed this issue in Python 3.7. Py_SetStandardStreamEncoding() now uses:

PyMemAllocatorEx old_alloc;
_PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

... _PyMem_RawStrdup() ...

    PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &old_alloc);