Add _PyEval_SetTrace(tstate, func, arg) function

[fabioz]

BPO	35370
Nosy	@vstinner, @fabioz
PRs	bpo-35370: Add _PyEval_SetTrace() function #18975 bpo-35370: PyEval_SetTrace() logs unraisable error #18977 [3.8] bpo-35370: PyEval_SetTrace() logs unraisable error (GH-18977) #19029

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-12-01.10:40:37.422>
labels = ['interpreter-core', 'type-feature', '3.9']
title = 'Add _PyEval_SetTrace(tstate, func, arg) function'
updated_at = <Date 2020-03-16.19:01:40.756>
user = 'https://github.com/fabioz'

bugs.python.org fields:

activity = <Date 2020-03-16.19:01:40.756>
actor = 'fabioz'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2018-12-01.10:40:37.422>
creator = 'fabioz'
dependencies = []
files = []
hgrepos = []
issue_num = 35370
keywords = ['patch']
message_count = 21.0
messages = ['330849', '364081', '364082', '364085', '364086', '364087', '364088', '364092', '364093', '364097', '364098', '364100', '364101', '364102', '364109', '364337', '364341', '364342', '364344', '364350', '364353']
nosy_count = 2.0
nosy_names = ['vstinner', 'fabioz']
pr_nums = ['18975', '18977', '19029']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue35370'
versions = ['Python 3.9']

[fabioz]

Right now it's hard for debuggers to set the tracing function to be used for running threads.

This would be really handy for debuggers when attaching to a running program to debug all threads.

-- Note: currently there is a way to achieve that by pausing all the threads then selectively switching to a thread to make it current and setting the tracing function using the C-API (see: https://github.com/fabioz/PyDev.Debugger/blob/master/pydevd_attach_to_process/dll/attach.cpp#L1224), but I believe this is very hacky and not portable to other Python implementations.

[fabioz]

As a note, the workaround is now in https://github.com/fabioz/PyDev.Debugger/blob/pydev_debugger_1_9_0/pydevd_attach_to_process/common/py_settrace_37.hpp#L150

[vstinner]

Can't you use PyEval_SetTrace()?
https://docs.python.org/dev/c-api/init.html#c.PyEval_SetTrace

[vstinner]

Note: currently there is a way to achieve that by pausing all the threads then selectively switching to a thread to make it current and setting the tracing function using the C-API (see: https://github.com/fabioz/PyDev.Debugger/blob/master/pydevd_attach_to_process/dll/attach.cpp#L1224), but I believe this is very hacky and not portable to other Python implementations.

I'm not sure that I understand your need. Do you need a variant of PyEval_SetTrace() which accepts a tstate argument?

[fabioz]

I'd like to, but it only sets the tracing to the currently running thread (the request is for setting the tracing for other threads).

Just for info, the problem I'm solving is that the debugger is multi-threaded, but the user can start the code without any tracing in place, and then, when it gets to an attach to process or a programmatic attach to the debugger, the debugger needs to set the tracing to threads that are already running (and any new thread created afterward) and Python doesn't really have any structure for that in place (so, I'm using the C-API as a workaround to do what PyEval_SetTrace does but targeting any thread).

[fabioz]

> Note: currently there is a way to achieve that by pausing all the threads then selectively switching to a thread to make it current and setting the tracing function using the C-API (see: https://github.com/fabioz/PyDev.Debugger/blob/master/pydevd_attach_to_process/dll/attach.cpp#L1224), but I believe this is very hacky and not portable to other Python implementations.

I'm not sure that I understand your need. Do you need a variant of PyEval_SetTrace() which accepts a tstate argument?

Yes

[fabioz]

Maybe better would be the thread id so that the tstate structure is not needed there.

[vstinner]

Attached PR 18975 adds _PyEval_SetTrace(tstate, func, arg) function. It requires that the caller holds the GIL. The function calls PySys_Audit() in the context of the current thread state (not in "tstate").

[vstinner]

Maybe better would be the thread id so that the tstate structure is not needed there.

PyInterpreterState.tstate_head allows to iterate on all thread states, but I'm not aware of an API to get a PyThreadState from its identifier (tstate->id).

--

See also bpo-1021318: "PyThreadState_Next not thread safe".