Dynamically control debugging output

[serhiy-storchaka]

BPO	23034
Nosy	@pitrou, @vstinner, @ezio-melotti, @serhiy-storchaka
Files	show_alloc_counts.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-07-04.16:29:27.735>
created_at = <Date 2014-12-11.21:09:33.817>
labels = ['interpreter-core', 'type-feature']
title = 'Dynamically control debugging output'
updated_at = <Date 2016-07-04.16:29:27.734>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-07-04.16:29:27.734>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-07-04.16:29:27.735>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2014-12-11.21:09:33.817>
creator = 'serhiy.storchaka'
dependencies = []
files = ['39350']
hgrepos = []
issue_num = 23034
keywords = ['patch']
message_count = 6.0
messages = ['232495', '232538', '232539', '232553', '242968', '269771']
nosy_count = 6.0
nosy_names = ['pitrou', 'vstinner', 'ezio.melotti', 'python-dev', 'serhiy.storchaka', 'bkabrda']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23034'
versions = ['Python 3.6']

[serhiy-storchaka]

When Python is compiled with some defined macros (e.g. COUNT_ALLOCS) it prints additional debugging output at shutdown. This additional output can cause tests failure (see bpo-19527). It would be good if Python will silent by default and produce debugging output only if special command line option or environment variable is set.

Debugging output switched by Py_REF_DEBUG is now enabled only when "-X showrefcount" is specified (bpo-17323).

[pitrou]

Also this debug output should be printed on stderr, not stdout.

[vstinner]

Debugging output switched by Py_REF_DEBUG is now enabled only when "-X showrefcount" is specified (bpo-17323).

Yes, I like the idea of doing that for other debug options.

[serhiy-storchaka]

Here is (conceivably incomplete) list of debugging and tracing output (but not
error reporting) from C code.

Controlled output.

Import and shutdown details -- controlled by the -v flag.
Parser tracing -- controlled by the -d flag.
If Py_REF_DEBUG is defined -- controlled by the -X showrefcount flag.
If PYMALLOC_DEBUG is defined -- controlled by the PYTHONMALLOCSTATS environment
variable.
If Py_TRACE_REFS is defined -- controlled by the PYTHONDUMPREFS environment
variable.
Garbage collecting details -- controlled by gc.set_debug().
If LLTRACE is defined -- controlled by the __ltrace__ global variable.

Uncomtrolled output.

To stderr:

In PyType_ClearCache (Objects/typeobject.c) if MCACHE_STATS is defined.
Lists allocations if SHOW_ALLOC_COUNT is defined.
Tuples allocations if SHOW_TRACK_COUNT is defined.
Interned strings statistics if __INSURE__ is defined.
In Python/ceval.c if WITH_TSC is defined.
In _PyHash_Fini if Py_HASH_STATS is defined.

To stdout:

In Modules/_ctypes/malloc_closure.c if MALLOC_CLOSURE_DEBUG is defined.
Regex matching trace in Modules/_sre.c if VERBOSE is defined.
Statistics in Modules/hashtable.c if Py_DEBUG is defined.
Allocation counts if COUNT_ALLOCS is defined.

There is also dead code in Parser/parsetok.c, Parser/tokenizer.c,
Python/pyarena.c, Python/compile.c, Python/formatter_unicode.c,
Modules/_elementtree.c, Modules/_tkinter.c, Objects/unicodeobject.c.

[serhiy-storchaka]

Proposed patch adds the "-X showalloccount" option, that turn on the output of allocated objects counts if COUNT_ALLOCS, SHOW_ALLOC_COUNT, or SHOW_TRACK_COUNT are defined. The output of COUNT_ALLOCS is now written to stderr.

[python-dev]

New changeset cd911e06bf6c by Serhiy Storchaka in branch 'default':
Issue bpo-23034: The output of a special Python build with defined COUNT_ALLOCS,
https://hg.python.org/cpython/rev/cd911e06bf6c