Idle: Use inspect.signature for calltips

[terryjreedy]

BPO	19903
Nosy	@terryjreedy, @serhiy-storchaka, @vedgar, @mlouielu
PRs	bpo-19903: IDLE: Change to inspect.signature for calltips #1382 bpo-19903: IDLE: Calltips changed to use inspect.signature #2822 [3.6] bpo-19903: IDLE: Calltips changed to use inspect.signature (GH-2822) #3053
Dependencies	bpo-20122: Move CallTips tests to idle_tests bpo-20223: inspect.signature does not support new functools.partialmethod bpo-20401: inspect.signature removes initial starred method params (bug)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2017-08-10.04:59:32.044>
created_at = <Date 2013-12-06.00:21:46.168>
labels = ['expert-IDLE', 'type-bug', '3.7']
title = 'Idle: Use inspect.signature for calltips'
updated_at = <Date 2017-08-10.06:15:00.611>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2017-08-10.06:15:00.611>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2017-08-10.04:59:32.044>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2013-12-06.00:21:46.168>
creator = 'terry.reedy'
dependencies = ['20122', '20223', '20401']
files = []
hgrepos = []
issue_num = 19903
keywords = []
message_count = 37.0
messages = ['205339', '208757', '213878', '219203', '235154', '235156', '235157', '288586', '292636', '292712', '292719', '292721', '292741', '292787', '292864', '292964', '292966', '292969', '292985', '293008', '293010', '293011', '293012', '293013', '293019', '293020', '293024', '293042', '293052', '293055', '293083', '300041', '300042', '300045', '300047', '300048', '300049']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'serhiy.storchaka', 'veky', 'louielu']
pr_nums = ['1382', '2822', '3053']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19903'
versions = ['Python 3.6', 'Python 3.7']

[terryjreedy]

Change idlelib.CallTips.get_argspec to use inspect.signature, new in 3.3, instead of inspect.getfullargspec and inspect.formatargspec. One thing it handles better is a namedtuple class, which has a C-coded __init__ inherited from object a Python-coded __new__ written by namedtuple. Signature() will also handle C-coded functions if and when Argument Clinic (new in 3.4) adds signature information.

from collections import namedtuple
from inspect import signature

Point = namedtuple('Point', 'x y')
print(signature(Point.__new__))
# '(_cls, x, y)'
print(signature(Point))
# '(x, y)'

Note that str (called by print) is needed to get the desired string form.

There are tests in CallTips.py, but they should be converted to unittest, moved to idle_test/test_calltips.py, changed to match new output detail, and expanded to include new examples.

[terryjreedy]

*17481 is about changing getargspec to use signature, at least as a backup. But that would not fix the example here. The calltips have been moved in 20122.

[terryjreedy]

I have not decided yet whether to apply to 3.4.

[terryjreedy]

My understanding is that when a builtin is converted, a) the docstring signature disappears, and b) getfullargspec calls str(signature). I need to check what now happens with .getfullargspec version .signature for uncoverted builtins and if any changes should be made now.

[terryjreedy]

.signature also handles wrapped functions better.

>>> def deco(f):
        import functools
        @functools.wraps(f)
        def ret(*args, **kw):
                return f(*args, **kw)
        return ret

>>> @deco
def f(n, k):
        return n + k*2

>>> import inspect as ip
>>> isf=ip.signature(f)
>>> print(isf)
(n, k)
>>> ip.formatargspec(*ip.getfullargspec(f))
'(*args, **kw)'

[terryjreedy]

The above suggests that the change might be applied to 3.4. The difference surprises me because bpo-17481 patched getfullargspec() to 'use' signature() 'behind the scenes', so I expected the result to be the same. It seems that 'use' does not mean what I thought. My hesitation is that only some builtins have been converted to use Argument Clinic, and Idle currently treats builtins special to get the signature from the docstring. Treating just some builting special will be slightly awkward.

[terryjreedy]

An example should make my concern clearer. str(signature(int)) is '()' because int (and all other builtins) has not been converted whereas the Idle calltip is
int(x=0) -> integer
int(x, base=10) -> integer
I do not want to simply add a line with '()' to the top of the calltip, so a little thought is needed to improve the calltip for the rare cases above without degrading them for existing, more common cases.

[terryjreedy]

Several builtin functions have recently gotten the Arg clinic treatment. So I think it is now time to switch. Before closing bpo-29653 as a duplicate, I discovered that inspect.signature also handles functools.partials correctly, while the other inspect functions do not.

[mlouielu]

I'm now testing to change getfullargspect to signature. It came out that signature can't accept bound method with only _VAR_KEYWORD. This test case will gave a ValueError:

    >>> class C:
        def m2(**kw):
            pass
    >>> c = C()
    >>> ip.signature(c.m2)
   Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/inspect.py", line 3002, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/usr/lib/python3.6/inspect.py", line 2752, in from_callable
    follow_wrapper_chains=follow_wrapped)
  File "/usr/lib/python3.6/inspect.py", line 2169, in _signature_from_callable
    return _signature_bound_method(sig)
  File "/usr/lib/python3.6/inspect.py", line 1759, in _signature_bound_method
    raise ValueError('invalid method signature')
ValueError: invalid method signature

[terryjreedy]

Thanks for the report. As Yuri pointed out in msg292701, and I verified, c.m2() raises "TypeError: meth() takes 0 positional arguments but 1 was given".

The purpose of get_argspec is to tell the user how to call the function without getting such a TypeError. But this is not possible, at least in this case. When signature raises "ValueError: invalid method signature", get_argspec should catch ValueError and put, for instance, "Function has an invalid method signature" in the box, instead of a signature that does not work, so users will know that the function cannot be called properly. (Should message be red?)

An initial patch could just comment out this test. I would like to see whether everything else passes. Or have you, Louie, already done that?
Signature can raise "TypeError if that type of object is not supported". Does .signature return for all the other test cases? I expect it should if only called after current guard.

try:
    ob_call = ob.__call__
except BaseException:
    return argspec

Before committing a patch using .signature, the known possible exception (ValueError) must be caught. This test should then be changed to verify the new behavior.

Possible future enhancement. Current nothing happens after code like "1(" or "''(". Should a box pop up saying (in red?) something like "Non-callable object before ("?

[mlouielu]

Terry, only this case was failed, others in unittest work very well.