inspect.getsourcelines finds wrong lines when lambda used argument to decorator

[ballingt]

BPO	21217
Nosy	@terryjreedy, @ncoghlan, @pitrou, @meadori, @PCManticore, @1st1
Files	issue21217.patch issue21217-v2.patch issue21217-v3.patch issue21217-v4.patch issue21217-v5.patch issue21217-v6.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-04-14.22:42:37.704>
created_at = <Date 2014-04-14.17:31:00.288>
labels = ['type-bug', 'library']
title = 'inspect.getsourcelines finds wrong lines when lambda used argument to decorator'
updated_at = <Date 2015-07-24.04:00:15.970>
user = 'https://bugs.python.org/ballingt'

bugs.python.org fields:

activity = <Date 2015-07-24.04:00:15.970>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2015-04-14.22:42:37.704>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2014-04-14.17:31:00.288>
creator = 'ballingt'
dependencies = []
files = ['34857', '34882', '34885', '38959', '38965', '38970']
hgrepos = []
issue_num = 21217
keywords = ['patch']
message_count = 22.0
messages = ['216127', '216128', '216245', '216344', '216345', '216351', '216352', '216357', '240738', '240749', '240787', '240791', '241050', '241051', '241077', '241079', '245868', '245872', '245876', '245915', '247202', '247245']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'ncoghlan', 'pitrou', 'meador.inge', 'Claudiu.Popa', 'python-dev', 'yselivanov', 'akaptur', 'ballingt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21217'
versions = ['Python 3.5']

[ballingt]

https://gist.github.com/thomasballinger/10666031

"""
inspect.getsourcelines incorrectly guesses what lines correspond
to the function foo

see getblock in inspect.py
once it finds a lambda, def or class it finishes it then stops
so get getsourcelines returns only the first two noop decorator
lines of bar, while normal behavior is to return all decorators
as it does for foo
"""
import inspect
from pprint import pprint

def noop(arg):
def inner(func):
return func
return inner

@noop(1)
@noop(2)
def foo():
return 1

@noop(1)
@noop(lambda: None)
@noop(1)
def bar():
return 1

pprint(inspect.getsourcelines(foo))
pprint(inspect.getsourcelines(bar))

[ballingt]

"The code object's co_lnotab is how inspect should be getting the sourcelines of the code, instead of looking for the first codeblock."

I'm looking at this now, thanks to Yhg1s for the above.

[akaptur]

This patch adds tests demonstrating broken behavior inspect.getsource and inspect.getsourcelines of decorators containing lambda functions, and modifies inspect.getsourcelines to behave correctly.

We use co_lnotab to extract line numbers on all objects with a code object. inspect.getsourcelines can also take a class, which cannot use co_lnotab as there is no associated code object.

@ballingt and I paired on this patch.

Some open questions about inspect.getsource not created or addressed by this patch:

• Is this a bug that should be patched in previous versions as well?
• the docs for say it can take a traceback. What is the correct behavior here? There aren't any tests at the moment. We suggest the line of code that caused the traceback, i.e. the line at tb.tb_lineno
• We added tests of decorated classes. The source of decorated classes does not include the decorators, which is different than the usual behavior of decorated functions. What is the correct behavior here?
• inspect.getblock and inspect.BlockFinder use the term "block" in a way that is inconsistent with its typical use in the interpreter (that is, in ceval.c). Should this be renamed? If so, to what? ("chunk"?)

[akaptur]

v2 of the patch incorporating the comments at http://bugs.python.org/review/21217/

[1st1]

Apart from one nit, the patch is looking good.

Also, could you please sign the contributor agreement, as described here: https://docs.python.org/devguide/coredev.html#sign-a-contributor-agreement

[PCManticore]

"- We added tests of decorated classes. The source of decorated classes does not include the decorators, which is different than the usual behavior of decorated functions. What is the correct behavior here?"

There is an open issue for this, http://bugs.python.org/issue1764286. It has a patch which uses inspect.unwrap in order to unwrap the decorated functions.

[1st1]

Claudiu: I'll take a look at your patch, thanks!

[akaptur]

v3 of patch, including misc/news update, docstring for function, and removing class decorator tests, since it sounds like those are better handled in http://bugs.python.org/issue1764286.

[ballingt]

v4 of patch, with tests updated for changed lines in inspect fodder file

[ballingt]

Patch reformatted to be non-git style, NEWS item removed

[ballingt]

Use dis.findlinestarts() to find lines of function instead of grubbing with co_lnotab manually, making dis module dependency non-optional.

[ncoghlan]

It sounds like at least a somewhat functional dis module is a pragmatic requirement for a Python implementation to support introspection, so +1 for reverting to the mandatory dependency on dis rather than duplicating its logic.

[python-dev]

New changeset ac86e5b2d45b by Antoine Pitrou in branch 'default':
Issue bpo-21217: inspect.getsourcelines() now tries to compute the start and
https://hg.python.org/cpython/rev/ac86e5b2d45b