Repository navigation
Move PyArena C API to the internal C API #87410
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2021-03-24.01:26:54.289>
created_at = <Date 2021-02-17.18:31:30.182>
labels = ['expert-C-API', '3.10']
title = 'Move PyArena C API to the internal C API'
updated_at = <Date 2021-05-26.22:44:01.883>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2021-05-26.22:44:01.883>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-03-24.01:26:54.289>
closer = 'vstinner'
components = ['C API']
creation = <Date 2021-02-17.18:31:30.182>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 43244
keywords = ['patch']
message_count = 19.0
messages = ['387180', '388973', '388976', '388981', '389001', '389016', '389018', '389075', '389408', '389416', '389419', '389423', '389424', '389501', '390416', '390444', '390472', '390473', '394489']
nosy_count = 2.0
nosy_names = ['vstinner', 'shihai1991']
pr_nums = ['24688', '24907', '24908', '24910', '24911', '24912', '24913', '24914', '24933', '25000', '25001', '25007', '25242', '25243', '25252', '25256']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue43244'
versions = ['Python 3.10']
Activity
vstinner commentedon Feb 17, 2021
Include/cpython/pyarena.h declares 4 functions which are implemented in Python/pyarena.c. The header file exists to access these functions from the compiler. IMO there was no intent to expose these functions to the public Python C API.
Moreover, the PEP-384 explicitly excludes this API from the stable ABI (from the limited C API).
I propose to move these functions to the internal C API.
Since these functions are no even documented, I don't think that a deprecation period is needed.
Note: The PR 24550 moved Include/pyarena.h to Include/cpython/pyarena.h.
vstinner commentedon Mar 17, 2021
New changeset b4536e1 by Victor Stinner in branch 'master':
bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (GH-24907)
b4536e1
vstinner commentedon Mar 17, 2021
New changeset 526fdeb by Victor Stinner in branch 'master':
bpo-43244: Add pycore_ast.h header file (GH-24908)
526fdeb
vstinner commentedon Mar 18, 2021
New changeset e0bf70d by Victor Stinner in branch 'master':
bpo-43244: Fix test_peg_generator for PyAST_Validate() (GH-24912)
e0bf70d
vstinner commentedon Mar 18, 2021
New changeset 6af528b by Victor Stinner in branch 'master':
bpo-43244: Fix test_peg_generators on Windows (GH-24913)
6af528b
vstinner commentedon Mar 18, 2021
New changeset eec8e61 by Victor Stinner in branch 'master':
bpo-43244: Remove the PyAST_Validate() function (GH-24911)
eec8e61
vstinner commentedon Mar 18, 2021
The work on this issue started in Python 3.9 with bpo-21120 which excluded Python-ast.h, ast.h and asdl.h from the limited C API: commit 421a72a.
vstinner commentedon Mar 19, 2021
New changeset 28ad12f by Victor Stinner in branch 'master':
bpo-43244: Remove symtable.h header file (GH-24910)
28ad12f
vstinner commentedon Mar 23, 2021
New changeset 94faa07 by Victor Stinner in branch 'master':
bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)
94faa07
vstinner commentedon Mar 23, 2021
New changeset a81fca6 by Victor Stinner in branch 'master':
bpo-43244: Add pycore_compile.h header file (GH-25000)
a81fca6
vstinner commentedon Mar 24, 2021
New changeset 57364ce by Victor Stinner in branch 'master':
bpo-43244: Remove parser_interface.h header file (GH-25001)
57364ce
vstinner commentedon Mar 24, 2021
New changeset 8370e07 by Victor Stinner in branch 'master':
bpo-43244: Remove the pyarena.h header (GH-25007)
8370e07
vstinner commentedon Mar 24, 2021
Ok, the whole C API related to parser, compiler, AST, ASDL and PyArena moved to the internal C API. So we will have more freedom to evolve it!
vstinner commentedon Mar 25, 2021
In short, this issue is a follow-up of bpo-40939.
These C API removal are related to the final step of the PEP-617 "New PEG parser for CPython": removal of the PyParser C API.
The PyParser C API produced "node*" objects, like PyParser_SimpleParseFile(FILE*) => node*. This C API was removed in Python 3.10: bpo-40939 "Remove the old parser".
The AST C API takes "node*" as input: PyAST_FromNode(node*) => mod_ty. I removed these APIs.
symtable takes "mod_ty" as input.
ASDL and PyArena API were only useful for the removed AST C APIs.
I searched for removed functions in the top PyPI 4000 projects, there is a single project on 4000: typed_ast. This typed_ast project is special, it copies directly code from CPython. Even after the C API removed, typed_ast can continue to use the internal C API, it only has to define the Py_BUILD_CORE_MODULE macro.
vstinner commentedon Apr 7, 2021
New changeset d36d6a9 by Victor Stinner in branch 'master':
bpo-43244: Remove Yield macro from pycore_ast.h (GH-25243)
d36d6a9
vstinner commentedon Apr 7, 2021
typed-ast was updated for these changes and to avoid including explicitly pydebug.h (it's included by Python.h: see bpo-35134 change): python/typed_ast#158
vstinner commentedon Apr 7, 2021
New changeset d27f8d2 by Victor Stinner in branch 'master':
bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)
d27f8d2
vstinner commentedon Apr 7, 2021
"Remove Yield macro from pycore_ast.h"
I'm happy that this one is gone, it was a common and annoying compiler warning on Windows!
vstinner commentedon May 26, 2021
I marked bpo-35197 as a duplicate of this issue.