test__xxsubinterpreters leaked [1, 1, 1] references: test_ids_global()

[vstinner]

BPO	40071
Nosy	@vstinner, @phsilva
PRs	bpo-1635741: Port _functools module to multiphase initialization (PEP-489) #19151 bpo-40071: Fix refleak in _functools module. #19172 bpo-40071: Fix potential crash in _functoolsmodule.c #19273

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-04-01.15:09:56.815>
created_at = <Date 2020-03-26.01:25:25.537>
labels = ['interpreter-core', '3.9']
title = 'test__xxsubinterpreters  leaked [1, 1, 1] references: test_ids_global()'
updated_at = <Date 2020-04-01.15:09:56.814>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-04-01.15:09:56.814>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-01.15:09:56.815>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-03-26.01:25:25.537>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40071
keywords = ['patch']
message_count = 9.0
messages = ['365042', '365044', '365053', '365054', '365066', '365067', '365446', '365481', '365483']
nosy_count = 2.0
nosy_names = ['vstinner', 'phsilva']
pr_nums = ['19151', '19172', '19273']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40071'
versions = ['Python 3.9']

[vstinner]

$ ./python -m test -R 3:3 test__xxsubinterpreters -m test_ids_global
0:00:00 load avg: 0.80 Run tests sequentially
0:00:00 load avg: 0.80 [1/1] test__xxsubinterpreters
beginning 6 repetitions
123456
......
test__xxsubinterpreters leaked [1, 1, 1] references, sum=3
test__xxsubinterpreters leaked [1, 1, 1] memory blocks, sum=3
test__xxsubinterpreters failed

== Tests result: FAILURE ==

1 test failed:
test__xxsubinterpreters

Total duration: 819 ms
Tests result: FAILURE

It started to leak since:

commit 7dd549e
Author: Paulo Henrique Silva <ph.silva@carta.com>
Date: Tue Mar 24 23:19:58 2020 -0300

bpo-1635741: Port _functools module to multiphase initialization (PEP-489) (GH-19151)

[vstinner]

The folowing test is enough to reproduce the leak:

    def test_ids_global(self):
        interp1 = interpreters.create()
        script, rpipe = _captured_script("pass")
        interpreters.run_string(interp1, script)
        rpipe.close()

        interp2 = interpreters.create()
        script, rpipe = _captured_script("pass")
        interpreters.run_string(interp2, script)
        rpipe.close()

[phsilva]

I've got it, will investigate asap.

[phsilva]

The module still uses static state. Fixed the leak and will convert it to use per-module state in a separate issue.

[vstinner]

New changeset b09ae3f by Paulo Henrique Silva in branch 'master':
bpo-40071: Fix refleak in _functools module (GH19172)
b09ae3f

[vstinner]

(...) will convert it to use per-module state in a separate issue.

Great!

[phsilva]

As discussed on PR19172, this module uses a global state in functions that do not receive a PyModule* and right now converting such cases to per-module state is not trivial. I will wait for PEP-573 implementation that will hopefully make this easier.

Pushed PR19273 to avoid the potential crash induced by the original change.

[vstinner]

New changeset eacc074 by Paulo Henrique Silva in branch 'master':
bpo-40071: Fix potential crash in _functoolsmodule.c (GH-19273)
eacc074

[vstinner]

I closed the issue, the leak is now fixed and _functools has been fixed.

I created bpo-40137: TODO list when PEP-573 "Module State Access from C Extension Methods" will be implemented.