TypedDict: no way to tell which (if any) keys are optional at runtime

[Zac-HD]

BPO	38834
Nosy	@ilevkivskyi, @Zac-HD
PRs	bpo-38834: enable runtime inspection of TypeDict classes #17214

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-11-24.13:06:26.688>
created_at = <Date 2019-11-18.00:53:54.937>
labels = ['3.8', 'library']
title = 'TypedDict: no way to tell which (if any) keys are optional at runtime'
updated_at = <Date 2019-11-24.13:06:26.687>
user = 'https://github.com/Zac-HD'

bugs.python.org fields:

activity = <Date 2019-11-24.13:06:26.687>
actor = 'levkivskyi'
assignee = 'none'
closed = True
closed_date = <Date 2019-11-24.13:06:26.688>
closer = 'levkivskyi'
components = ['Library (Lib)']
creation = <Date 2019-11-18.00:53:54.937>
creator = 'Zac Hatfield-Dodds'
dependencies = []
files = []
hgrepos = []
issue_num = 38834
keywords = ['patch']
message_count = 1.0
messages = ['356836']
nosy_count = 2.0
nosy_names = ['levkivskyi', 'Zac Hatfield-Dodds']
pr_nums = ['17214']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue38834'
versions = ['Python 3.8']

[Zac-HD]

Consider the following cases:

class A(typing.TypedDict):
    a: int  # a is required

class B(A, total=False):
    b: bool  # a is required, b is optional

class C(B):
    c: str  # a is required, b is optional, c is required again

PEP-589 is clear about the semantics, and this is obvious enough when reading the code. At runtime the __annotations__ attribute of each class gives us the set of allowed keys and the type of each corresponding value, but we have a problem:

• C has __total__==True, but b is not actually required.
• B has __total__==False, but a *is* required.
• I can't see any way to get the parent classes of a TypedDict class!

The _TypedDictMeta metaclass updates the attributes, but leaves no record of the parent type - at runtime A, B, and C all appear to inherit directly from dict.

After discussion on the typing-sig mailing list, I propose to add __required_keys__ and __optional_keys__ attributes to TypedDict subclasses, as frozensets of strings.

This will be very useful for Hypothesis' from_type() strategy, as well as for type-based validation frameworks like pydantic or typeguard.