[C API] stable_abi.py script must not include PyType_HasFeature() static inline function in the stable ABI

[vstinner]

BPO	43690
Nosy	@vstinner, @encukou, @pablogsal
PRs	bpo-43690: stable_abi.py no longer parses macros #25136

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-04-01.12:16:45.673>
created_at = <Date 2021-04-01.09:37:33.583>
labels = ['expert-C-API', '3.10']
title = '[C API] stable_abi.py script must not include PyType_HasFeature() static inline function in the stable ABI'
updated_at = <Date 2021-04-01.12:20:27.623>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-04-01.12:20:27.623>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-04-01.12:16:45.673>
closer = 'vstinner'
components = ['C API']
creation = <Date 2021-04-01.09:37:33.583>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 43690
keywords = ['patch']
message_count = 4.0
messages = ['389964', '389976', '389977', '389978']
nosy_count = 3.0
nosy_names = ['vstinner', 'petr.viktorin', 'pablogsal']
pr_nums = ['25136']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue43690'
versions = ['Python 3.10']

[vstinner]

I just ran "make regen-limited-abi" and it added PyType_HasFeature():

commit baf10da
Author: Victor Stinner <vstinner@python.org>
Date: Thu Apr 1 11:29:46 2021 +0200

bpo-43688: Run make regen-limited-abi (GH-25134)

diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat
index 3adee103bc..ed20521b7f 100644
--- a/Doc/data/stable_abi.dat
+++ b/Doc/data/stable_abi.dat
@@ -192,6 +192,7 @@ PyExc_ConnectionRefusedError
 PyExc_ConnectionResetError
 PyExc_DeprecationWarning
 PyExc_EOFError
+PyExc_EncodingWarning
 PyExc_EnvironmentError
 PyExc_Exception
 PyExc_FileExistsError
@@ -615,6 +616,7 @@ PyType_GetFlags
 PyType_GetModule
 PyType_GetModuleState
 PyType_GetSlot
+PyType_HasFeature
 PyType_IsSubtype
 PyType_Modified
 PyType_Ready

The problem is that PyType_HasFeature() is currently implemented as a static inline function in the limited C API for best performance.

Issue about PyType_HasFeature() performance in CPython itself:
https://bugs.python.org/issue39542#msg372962

Currently, PyType_HasFeature() is declared in Include/object.h as:

static inline int
PyType_HasFeature(PyTypeObject *type, unsigned long feature)
{
    unsigned long flags;
#ifdef Py_LIMITED_API
    // PyTypeObject is opaque in the limited C API
    flags = PyType_GetFlags(type);
#else
    flags = type->tp_flags;
#endif
    return ((flags & feature) != 0);
}

IMO static inline functions must not be listed in the stable *ABI*. At the ABI level, libpython doesn't export "PyType_HasFeature" symbol:

$ objdump -T /lib64/libpython3.10.so.1.0|grep '\<PyObject_CallFunction\>'
00000000000fedf0 g    DF .text	00000000000000d0  Base        PyObject_CallFunction
$ objdump -T /lib64/libpython3.10.so.1.0|grep '\<PyType_HasFeature\>'
# nothing

"PyObject_CallFunction" symbol is exported, but not "PyType_HasFeature".

--

Maybe for the stable ABI, it would be a good idea to export PyType_HasFeature() as an opaque function. But that's out of the scope of this issue which is about the stable_abi.py script ;-)

[vstinner]

New changeset 61092a9 by Victor Stinner in branch 'master':
bpo-43690: stable_abi.py no longer parses macros (GH-25136)
61092a9

[vstinner]

Pablo: My previous commit added PyType_HasFeature to Doc/data/stable_abi.dat. But it prevented me to fix another bug, PR 25135. So I merged this change (PR 25136) to unblock the CI. Feel free to revert/adjust my change as soon as it doesn't add PyType_HasFeature back into Doc/data/stable_abi.dat :-)

[vstinner]

Oh, I forgot the mention "Tests / Check if generated files are up to date" job error message:

LD_LIBRARY_PATH=/home/runner/work/cpython/cpython:/opt/hostedtoolcache/Python/3.9.2/x64/lib ./python ./Tools/scripts/stable_abi.py check ./Doc/data/stable_abi.dat
Some symbols from the limited API are missing: PyType_HasFeature

This error means that there are some missing symbols among the ones exported
in the Python library ("libpythonx.x.a" or "libpythonx.x.so"). This normally
means that some symbol, function implementation or a prototype, belonging to
a symbol in the limited API has been deleted or is missing.

Check if this was a mistake and if not, update the file containing the limited
API symbols. This file is located at:

./Doc/data/stable_abi.dat

You can read more about the limited API and its contracts at:

https://docs.python.org/3/c-api/stable.html

And in PEP-384:

https://www.python.org/dev/peps/pep-0384/