functools.lru_cache omits __defaults__ attribute from wrapped function

[gpshead]

BPO	44003
Nosy	@rhettinger, @gpshead, @ncoghlan, @serhiy-storchaka
PRs	bpo-44003: expose default args on the wrapped functools.lru_cache function #25800

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-05-02.04:35:18.116>
created_at = <Date 2021-05-01.19:49:53.450>
labels = ['type-feature', 'library', '3.11']
title = 'functools.lru_cache omits __defaults__ attribute from wrapped function'
updated_at = <Date 2021-05-02.06:04:28.545>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2021-05-02.06:04:28.545>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-05-02.04:35:18.116>
closer = 'gregory.p.smith'
components = ['Library (Lib)']
creation = <Date 2021-05-01.19:49:53.450>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 44003
keywords = ['patch', '3.5regression']
message_count = 12.0
messages = ['392623', '392627', '392643', '392644', '392645', '392646', '392647', '392649', '392651', '392652', '392662', '392668']
nosy_count = 4.0
nosy_names = ['rhettinger', 'gregory.p.smith', 'ncoghlan', 'serhiy.storchaka']
pr_nums = ['25800']
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue44003'
versions = ['Python 3.11']

[gpshead]

When the C implementation of functools.lru_cache was added in bpo/issue14373, it appears to have omitted setting .__defaults__ on its wrapped function.

Python 3.10.0a7+ (heads/master-dirty:823fbf4e0e, May  1 2021, 11:10:30) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import functools
>>> def func(b=5): pass
... 
>>> @functools.lru_cache
... def cached_func(b=5): pass
... 
>>> func.__defaults__
(5,)
>>> cached_func.__defaults__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'functools._lru_cache_wrapper' object has no attribute '__defaults__'

functools.update_wrapper() does set __defaults__ so this appears to just be an oversight in Modules/_functoolsmodule.c.

[serhiy-storchaka]

Where does functools.update_wrapper() set __defaults__?

[gpshead]

That was anecdotal evidence:

Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:10:52) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def func(arg=1, *, kwarg=2): pass
... 
>>> import functools
>>> from functools import lru_cache, update_wrapper
>>> @lru_cache
... def cached_func(arg=1, *, kwargs=2): pass
... 
>>> def x(*args, **kwargs): func(*args, **kwargs)
... 
>>> updated_x = update_wrapper(func, x)
>>> x
<function x at 0x7feff5892b80>
>>> updated_x
<function x at 0x7feff5828a60>
>>> updated_x.__defaults__
(1,)
>>> updated_x.__kwdefaults__
{'kwarg': 2}
>>> func.__defaults__
(1,)
>>> func.__kwdefaults__
{'kwarg': 2}
>>> cached_func.__defaults__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'functools._lru_cache_wrapper' object has no attribute '__defaults__'
>>> cached_func.__kwdefaults__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'functools._lru_cache_wrapper' object has no attribute '__kwdefaults__'

[gpshead]

the pure python functools.lru_cache doesn't get this right either. here's a desirable testcase for this bug:

    def test_lru_defaults_bug44003(self):
        @self.module.lru_cache(maxsize=None)
        def func(arg='ARG', *, kw='KW'):
            return arg, kw

        self.assertEqual(func.__wrapped__.__defaults__, ('ARG',))
        self.assertEqual(func.__wrapped__.__kwdefaults__, {'kw': 'KW'})
        self.assertEqual(func.__defaults__, ('ARG',))
        self.assertEqual(func.__kwdefaults__, {'kw': 'KW'})

results in

ERROR: test_lru_defaults_bug44003 (test.test_functools.TestLRUC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/greg/oss/python/gpshead/Lib/test/test_functools.py", line 1738, in test_lru_defaults_bug44003
    self.assertEqual(func.__defaults__, ('ARG',))
AttributeError: 'functools._lru_cache_wrapper' object has no attribute '__defaults__'

======================================================================
FAIL: test_lru_defaults_bug44003 (test.test_functools.TestLRUPy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/greg/oss/python/gpshead/Lib/test/test_functools.py", line 1738, in test_lru_defaults_bug44003
    self.assertEqual(func.__defaults__, ('ARG',))
AssertionError: None != ('ARG',)

[rhettinger]

I don't think this should be done. We want the lru_cache to be a pass-through. Applying defaults or keyword-only/positional-only restrictions is the responsibility of the inner function.

FWIW, here are the fields that Nick selected to be included in update_wrapper(): ('__module__', '__name__', '__qualname__', '__doc__', '__annotations__').

Those are sufficient to get help() to work which is all we were aiming for:

>>> from functools import *
>>> @lru_cache
def cached_func(b=5):
    pass

>>> help(cached_func)
Help on _lru_cache_wrapper in module __main__:

cached_func(b=5)

[gpshead]

__defaults__ and __kwdefaults__ get used for code introspection. Just as __annotations__ does. __annotations__ is already available on the lru_cache wrapped function. All of those seem to go together from a runtime inspection point of view.

[gpshead]

An inner function can't know if somebody else might want to inspect it.

This is a decorator that does not change anything about the argument signature of the wrapped function, carrying over the reference to meta-information about that by default seems to make sense.