Python3.7 crash in PyCFunction_New due to broken _PyObject_GC_TRACK

[etejedor]

BPO	35408
Nosy	@ncoghlan, @vstinner, @ericsnowcurrently
Files	reproducer.cpp

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-12-04.14:29:58.186>
created_at = <Date 2018-12-04.12:35:18.331>
labels = ['interpreter-core', '3.7', 'invalid', 'type-crash']
title = 'Python3.7 crash in PyCFunction_New due to broken _PyObject_GC_TRACK'
updated_at = <Date 2018-12-05.14:07:25.296>
user = 'https://bugs.python.org/etejedor'

bugs.python.org fields:

activity = <Date 2018-12-05.14:07:25.296>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-12-04.14:29:58.186>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2018-12-04.12:35:18.331>
creator = 'etejedor'
dependencies = []
files = ['47971']
hgrepos = []
issue_num = 35408
keywords = []
message_count = 5.0
messages = ['331040', '331047', '331048', '331111', '331113']
nosy_count = 4.0
nosy_names = ['ncoghlan', 'vstinner', 'eric.snow', 'etejedor']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue35408'
versions = ['Python 3.7']

[etejedor]

Attached is a reproducer that calls PyCFunction_New. The reproducer runs normally with Python 3.6.5, but it crashes with Python 3.7.1. The reason seems to be that the _PyObject_GC_TRACK macro ends up being called and it is broken in Python3.7.

A fix for that macro seems to have been committed to master:

#10507

[vstinner]

Python3.7 crash in PyCFunction_New due to broken _PyObject_GC_TRACK

It's unrelated. Your must not use the Python API before Python is initialized. If you modify your code like that, it works as expected:

int main()
{
Py_Initialize();

PyMethodDef methoddef_ = {
const_cast< char* >( "myfun" ),
(PyCFunction) myfun,
METH_O,
NULL
};

  PyObject* myFunPtr = PyCFunction_New( &methoddef_, NULL );

Py_Finalize();
  return 0;
}

I don't think that it's a regression.

Python initialization is now well documented:
https://docs.python.org/dev/c-api/init.html

The documentation starts with:

"In an application embedding Python, the Py_Initialize() function must be called before using any other Python/C API functions; with the exception of a few functions and the global configuration variables."

PyCFunction_New() is an example of function of the Python C API.

[vstinner]

I close the issue as "not a bug".

Even if the code "worked" in Python 3.6, it worked because of a mistake :-)

[etejedor]

Hi Victor,

Thank you for clarifying.

If the call to PyCFunction_New is done from a C extension module, is it also necessary to call Py_Initialize()?

[vstinner]

If the call to PyCFunction_New is done from a C extension module, is it also necessary to call Py_Initialize()?

Py_Initialize() must always be called first.