PEP 573: Module State Access from C Extension Methods

[Dormouse759]

BPO	38787
Nosy	@terryjreedy, @ncoghlan, @scoder, @vstinner, @encukou, @Dormouse759, @miss-islington, @shihai1991
PRs	[PEP 573] bpo-38787: Module State Access from C Extension Methods #17145 bpo-38787: C API for module state access from extension methods (PEP 573) #19936 bpo-38787: update docs of structures.rst and code style of _testmultiphase.c #19980 bpo-38787: Add PyCFunction_CheckExact() macro for exact type checks #20024 bpo-38787: Fix Argument Clinic defining_class_converter #20074 bpo-38787: Clarify docs for PyType_GetModule and warn against common mistake #20215 Remove a redundant NULL in descrobject.c #20344 [3.9] bpo-38787: Clarify docs for PyType_GetModule and warn against common mistake (GH-20215) #21984

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-20.08:56:18.304>
created_at = <Date 2019-11-13.14:30:40.496>
labels = ['extension-modules', 'type-feature', '3.9']
title = 'PEP 573: Module State Access from C Extension Methods'
updated_at = <Date 2020-11-25.17:08:25.229>
user = 'https://github.com/Dormouse759'

bugs.python.org fields:

activity = <Date 2020-11-25.17:08:25.229>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-20.08:56:18.304>
closer = 'petr.viktorin'
components = ['Extension Modules']
creation = <Date 2019-11-13.14:30:40.496>
creator = 'Dormouse759'
dependencies = []
files = []
hgrepos = []
issue_num = 38787
keywords = ['patch']
message_count = 18.0
messages = ['356533', '367552', '367569', '367636', '368180', '368337', '368356', '368577', '368580', '368733', '368735', '368768', '368789', '368804', '375991', '376411', '379107', '381853']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'ncoghlan', 'scoder', 'vstinner', 'petr.viktorin', 'Dormouse759', 'miss-islington', 'shihai1991']
pr_nums = ['17145', '19936', '19980', '20024', '20074', '20215', '20344', '21984']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38787'
versions = ['Python 3.9']

[Dormouse759]

Currently, there is not way to access per-module state from the methods of types defined in extension modules.

This PEP provides a fix to this issue.

PEP-573 - https://www.python.org/dev/peps/pep-0573/

Reference implementation - https://github.com/Dormouse759/cpython/tree/pep-c-rebase_newer

[scoder]

What can we do to move this forward? I see that the original PR was closed in January. Is there or will there be a new one? Can I help with anything?

[terryjreedy]

The PEP is listed as accepted, with you as the BDFL-delegate, but it lacks a link to the public acceptance message (usually on pydev list).

The PR was not rejected by a core dev but was closed without explanation by the author on the same day the last commits were added. Perhaps OP just gave up getting tests to pass. But they disappear on closes.

However, the contribution had been made under the CLA and you or Petr or any other core dev should be able to download it to a local branch, try to whip it into shape, and make a new PR before beta 1, in about a month. Or start from scratch.

I cannot test downloading at the moment. Apparently, OP's fork branch has been altered, so that merely reopening is not an option. The OP used force-pushes rather than update merges, and this often creates problems.

[encukou]

I'm working on the implementation as my time allows, here: encukou#3 .

Would it help to create a proper PR for CPython now? That would mean I could no longer do rebases/force-pushes, so the commits would be less readable. But happy to do it if it would help you.

[vstinner]

See also bpo-40137: TODO list when PEP-573 "Module State Access from C Extension Methods" will be implemented.

[vstinner]

New changeset e1becf4 by Petr Viktorin in branch 'master':
bpo-38787: C API for module state access from extension methods (PEP-573) (GH-19936)
e1becf4

[vstinner]

New changeset c068b53 by Hai Shi in branch 'master':
bpo-38787: Update structures.rst docs (PEP-573) (GH-19980)
c068b53

[vstinner]

See also bpo-40574: segfault causing regression from PEP-573 implementation (PyQt5).

[vstinner]

The posix/nt and _abc modules now use the PEP-573 APIs:

• posix/nt: commit 1c2fa78 (bpo-40549)
• _abc: commit 77c6146 (bpo-40566)

[miss-islington]

New changeset 4c9ea09 by scoder in branch 'master':
bpo-38787: Add PyCFunction_CheckExact() macro for exact type checks (GH-20024)
4c9ea09

[shihai1991]

In PR19982, petr have mentioned that PyType_GetModule can not worked if the type is the subtype.

I try to use PEP-573 in functools, I get the error info from testcases:
======================================================================
ERROR: test_arg_combinations (test.test_functools.TestPartialCSubclass)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/temp/shihai/cpython/Lib/test/test_functools.py", line 109, in test_arg_combinations
    p = self.partial(capture)
TypeError: PyType_GetModule: Type 'CPartialSubclass' has no associated module

======================================================================
ERROR: test_attributes (test.test_functools.TestPartialCSubclass)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/temp/shihai/cpython/Lib/test/test_functools.py", line 71, in test_attributes
    p = self.partial(capture, 1, 2, a=10, b=20)
TypeError: PyType_GetModule: Type 'CPartialSubclass' has no associated module

[encukou]

I guess I'll need to clarify the documentation here.
Call PyType_GetModule on the defining class, not on type(self). See the PEP for details.
The associated module is not inherited; each class in the MRO can be defined in a different module.