Inconsistent __args__ between typing.Callable and collections.abc.Callable

[Zac-HD]

BPO	42195
Nosy	@gvanrossum, @serhiy-storchaka, @ilevkivskyi, @corona10, @Zac-HD, @miss-islington, @isidentical, @hauntsaninja, @Fidget-Spinner
PRs	bpo-42195: Ensure consistency of Callable's __args__ in collections.abc and typing #23060 [3.9] bpo-42195: Ensure consistency of Callable's __args__ in collections.abc and typing (GH-23060) #23765 bpo-42675: Document collections.abc.Callable changes #23839 [3.9] bpo-42675: Document collections.abc.Callable changes (GH-23839) #23852 bpo-42195: Override _CallableGenericAlias's __getitem__ #23915 [3.9] bpo-42195: Override _CallableGenericAlias's __getitem__ (GH-23915) #23916 bpo-42195: Disallow isinstance/issubclass for subclasses of genericaliases in Union #24059

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-12-14.16:34:44.125>
created_at = <Date 2020-10-29.12:25:19.859>
labels = ['type-bug', 'library', '3.9', '3.10']
title = 'Inconsistent __args__ between typing.Callable and collections.abc.Callable'
updated_at = <Date 2021-01-02.16:19:18.916>
user = 'https://github.com/Zac-HD'

bugs.python.org fields:

activity = <Date 2021-01-02.16:19:18.916>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2020-12-14.16:34:44.125>
closer = 'gvanrossum'
components = ['Library (Lib)']
creation = <Date 2020-10-29.12:25:19.859>
creator = 'Zac Hatfield-Dodds'
dependencies = []
files = []
hgrepos = []
issue_num = 42195
keywords = ['patch']
message_count = 27.0
messages = ['379869', '379913', '379918', '380031', '380040', '380048', '380059', '380175', '380181', '381064', '381065', '381068', '381073', '381402', '381642', '381643', '381644', '381647', '382103', '382108', '382213', '382936', '382991', '382992', '383668', '383669', '384227']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'serhiy.storchaka', 'levkivskyi', 'corona10', 'Zac Hatfield-Dodds', 'miss-islington', 'BTaskaya', 'hauntsaninja', 'kj']
pr_nums = ['23060', '23765', '23839', '23852', '23915', '23916', '24059']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42195'
versions = ['Python 3.9', 'Python 3.10']

[Zac-HD]

The two ways of getting a parametrised Callable have inconsistent __args__:

    >>> import collections.abc, typing
    >>> typing.Callable[[int, int], int].__args__
    (int, int, int)
    >>> collections.abc.Callable[[int, int], int].__args__
    ([int, int], int)

I discovered this while working on PEP-585 support in Hypothesis [1], where it is easy enough to work around but carries a potentially serious performance cost - the list means we cannot use the type as a cache key for non-... argument types.

https://bugs.python.org/issue40494 and https://bugs.python.org/issue40398 may be related.

[1] HypothesisWorks/hypothesis#2653

[gvanrossum]

Good find! I see that typing.Callable has adopted this structure precisely to enable caching.

We should see if we can fix _collections_abc.Callable. It's still early in the life of 3.9 so I think this is reasonable.

We'll need a subclass of GenericAlias so that the repr() of Callable[[int, int], int] still comes out correctly. This is similar to how typing.Callable solves it.

Do you feel up to submitting a PR for this? Otherwise maybe Batuhan feels like contributing a fix for this?

[Zac-HD]

Unfortunately I'm overcommitted for the next few weeks-months :-/

[gvanrossum]

@corona10 Do I hear that you'd like to work on this?

[corona10]

@gvanrossum

Sorry, Not this time. I just add myself to observe how to solve this issue.

Maybe Batuhan is the proper member to handle this issue.

Just question to this issue.
Since GenericAlias does not have a Py_TPFLAGS_BASETYPE flag, IMHO we have to add Py_TPFLAGS_BASETYPE to type declaration and this behavior does not occur any regression?

[serhiy-storchaka]

Would not be better to change typing.Callable?

Attributes __origin__, __args__ and __parameters__ are not documented in the typing module. They are also not mentioned in any PEP except PEP-585. So I don't know what is intention and correct value of these attributes.

[Fidget-Spinner]

Hi, I submitted a PR for a proof-of-concept on how the code which subclasses GenericAlias might look like for everyone's consideration.

Personally, I'd lean more towards what Serhiy said and change typing.Callable rather than GenericAlias which affects many other classes' __class_getitem__. However, considering that typing.Callable has been around for years, I'm hesitant to change its __args__ which might break programs which rely on it.