1

I’d like us to think carefully about this.

I agree that this might be nice for “simple” type annotations, where an argument can e.g. only be a str or whatever. But I’d be wary of starting a large-scale project to add type hints in lots of places.

As a typeshed maintainer and a codeowner for typing.py, I’m obviously pro-typing. But I’ve also seen how “apparently simple” functions can get very complicated to add annotations for. There will be a lot of functions where we won’t be able to add type annotations without it becoming overwhelming, and it might be pretty confusing for some people to see that some functions in a certain file are fully annotated whereas others have no annotations at all.

Until the stdlib has “type annotation coverage” matching typeshed, type checkers will also have to continue using the typeshed stubs for type-checking user code; they’ll essentially ignore any type hints in the runtime stdlib. That might be highly confusing for end users, if the annotations in the stdlib and the annotations in typeshed diverge for some reason. We’d have to think carefully about how to keep the annotations in the two repos in sync (unless the long-term goal is to get rid of the stdlib stubs from typeshed altogether).

10 Likes
2

In general, having a third-party substitute (typeshed) for a first-party feature (typing) of first-party modules (the standard library) doesn’t feel cohesive and adds to a sense of jankiness.

Not confusing readers is certainly something to strive for, but considering that those readers are now likely to be more developers, I don’t think it takes high priority.

I think type-annotating the standard library is something to which will bring further benefits (such as better documentation for std-lib developers and readers), and is therefore worthwhile.

I think it makes sense to have that as a repeat blocker: if any module attributes are annotated, then all must be. I don’t think it should be a blocker on starting to annotate.

I don’t think it makes sense to keep the stubs in typeshed. We can just *-import after the Python release:

import sys as _sys
import typing as _typing

if _sys.version_info < (3, 12):
     ...  # original stubs
else:
    if _typing.TYPE_CHECKING:
        from mod import *
3 Likes
3

Type checkers would (all) need to be able to interpret that amount of runtime-dynamic code. That doesn’t seem impossible at first glance, but might be more difficult than it looks. It’s hard for me to tell, as someone who has no idea how type checkers handle this sort of thing.

1 Like
4

I’ve only contributed a few times to typeshed but one of the great advantages I noticed is that improvements aren’t bound to Python versions. Once a new typing PEP is approved (and implemented in the major type checkers), it’s quite common that the improvements are rolled out through the whole code base and shipped with the next release for all supported versions. That isn’t really possible with inline annotations as those will depend on the usual release cycle and, since new PEPs aren’t implemented in old versions, those can’t benefit from any improvements like it’s possible today.

So even if type hints are added to the stdlib, I believe there might always be a need for externally shipped stubs. That is somewhat different for simple applications or libraries as they usually only support one concurrent version.

5 Likes
5

Also factor in that the stdlib wasn’t implemented under typing’s constraints, and frequently does things that will fail even a basic type check. typeshed gets around this by lying, essentially, and matching the documentation rather than the implementation, but I’ll be surprised if we get to keep doing that (there are already many many people running fuzzers and filing bugs, and people will definitely start running type checkers and filing bugs for things that have been working fine for decades).

We’d also have to develop compatibility rules that factor in people relying on our annotation. For example, it’s not currently a compatibility break to expand the accepted range of values in most of our APIs (because nobody could’ve been using them today without getting an error). But if we’ve published “the full extent of accepted types” in the form of an annotation, then people may have coded to that in a way that no longer validates against the new implementation.[1]

I’d much rather keep the stubs separate and have them reflect a “strictly correct” usage of the stdlib for those who want to adhere to it, rather than the flexibly correct usage we allow.

  1. Yes, this seems backwards, and in most cases of working code it will be. I’m thinking more of people copying our annotations into their own mocks (e.g. “my function accepts everything that os.fsdecode accepts”), and the potential for causing breakage by then changing what they interpreted as a maximal guarantee, rather than a minimal one. ↩︎

2 Likes
6

Yes, and learn how to correctly interpret what looks like a recursive import :wink:

7

(I’m a maintainer of typeshed and mypy)

There are three use cases for type annotations in the standard library:

  • used by type checkers to type check downstream user code (aka make type checkers use stdlib instead of typeshed)
  • used for the purpose of alerting core devs to potential bugs (aka let core devs run mypy on stdlib)
  • used by end users as documentation

These three things actually have somewhat different requirements, so +1 to the suggestion to tread carefully.

The thing I feel strongest about is that it would be a mistake to have type checkers use annotations from the standard library instead of typeshed. Here are some reasons:

  • typeshed has a much faster pace of development and lower stability than the standard library. All typeshed changes are made available to users of all versions of Python.
  • Moreover, we’re able to use typing features added to typing.py in new versions of Python on all versions of Python
  • The development cycle of typeshed and type checkers is linked in several ways. It is common for typeshed to workaround specific issues in type checkers. It is common for type checkers to patch typeshed (mypy definitely does this, I think pyright does as well).
  • The tests and tooling we have for type annotations in typeshed would be hard to get merged into CPython (e.g. at a minimum you’d want to make sure that type annotations make sense to a type checker)
  • The expertise is split. Core devs typically know less about typing than typeshed maintainers and suddenly core devs would be responsible for typing of their modules (when many core devs aren’t sold on typing to begin with)
  • Accurate typing is hard! Adding typeshed quality annotations to the standard library would add e.g. a bunch of new protocols and aliases and annoying Literal types. typeshed is also forced to be opinionated in certain ways (think union returns, collection types, use of Any, deciphering intentions) and I think these decisions would be harder and harder to reverse if part of standard library
  • Accurate type annotations may not be zero runtime cost. At a minimum you’re going to have more import cycles. You could if TYPE_CHECKING some of those away, but then runtime users of annotations might be surprised that typing.get_type_hints or whatever raises on stdlib
  • No type checker authors are asking for this. This would necessitate changes in all type checkers
  • You’d still need to use stubs for all the modules written in C, so not sure how much “cohesiveness” this would actually get you
  • etc etc etc

Seems great if core devs find typing useful enough that they’d like to use it in their workflow. Some thoughts:

  • There are already a couple modules in the standard library that have type annotations for this purpose
  • This workflow seems to work best for modules that are developed somewhat separately from CPython. In particular, in repos where type checking is run as part of tests
  • Alex has a fair point about users mistaking such type hints for “type hints that type checkers use”, but for now this issue isn’t too bad, and at the current scale of use, other issues that crop up seem solvable with tooling

Thoughts about type hints as documentation:

15 Likes
8

A difficulty with this is that often the Python code in the stdlib wasn’t written with type consistency in mind, and often the annotations found in typeshed are only an approximation of the truth. If we have a stdlib function foo(arg) that was written with the intention to take strings, but which accidentally also supports certain other argument types, should we encode that in the annotation or not? Typeshed can take a strict stance, so that users who depend on non-string arguments are reminded that perhaps their usage is not supported. But if we wanted to officially change the function to only support strings, we might be required to introduce a two-release deprecation period.

In many cases researching such situations is time consuming and not a very good use of core developers’ time. Separating the decisions into typeshed empowers a separate group of people to make such judgment calls, adjusting them based on user feedback, and they can move faster than the stdlib.

I am not against having new stdlib modules contain annotations, if they are of sufficiently high quality to satisfy all users and type checkers. Even there, though, we’d probably still need a stub in typeshed, since type checkers are often reluctant to even look in the stdlib code – unannotated code there often would just confound type checkers because it was written with other goals in mind, e.g. performance.

7 Likes
9

Thanks for the discussion, everyone! Let me throw some questions and ideas around.

I’m mostly coming at this from a documentation angle. Type hints can be good in docs, at least in some cases, and it’s a shame to have them locked away in a separate repo.

The tomllib.loads docs start with:

tomllib.loads(s, /, *, parse_float=float)
Load TOML from a str object. Return a dict.

In this case, the information can be put into the signature, which would mean:

  • it shows up in help()
  • it can be checked against the code.

Of course, this is a best-case scenario – a place to start.

When – if ever – do you think we should start merging typeshed into stdlib?

There’s a lot of points in this topic that, to me, look like typeshed growing pains. But it also looks like a lot of the issues are known at this point. Is it too early to start thinking about the next steps?

+1, new modules are good place to start, slowly and carefully.

Should typeshed have a mechanism to say “this module is safe to look at“?
Or should stdlib have some module-level equivalent of py.typed?

It seems to me that Python can take a hard stance as well. If str works for both documentation and type checking, why not use that? Can we say the types only represent “best practice”, rather than what we support for backwards compatibility (or other reasons)? Would it be useful to have separate type info for the gnarly reality?
If the type info says str but we still want to support other types, we can always express that as a comment in the docs, or in the source. It wouldn’t be worse than what we have today.
If we don’t want to support other types (e.g. to rewrite the function in C for speed), then there needs to be a deprecation period. But IMO it’s better to try to collect the extra supported types and write it down, so it’s there before someone goes to touch the code. And so we can decide what’s just an implementation accident.

If it’s not a good use of core devs’ time, can we let more typeshed maintainers join the team? :‍)

What if the next iteration of Argument Clinic could either generate stubs, or even take them as input?

10

Well, help will show you the docstring which already describes the types in prose :wink:

1 Like
11

Doesn’t tomllib already have this? cpython/_parser.py at main · python/cpython · GitHub

Which is great because it also shows up e.g. in IDLE:

image
image1612×342 26.6 KB

For comparison:

image
image1408×420 31.3 KB

1 Like
12

In the code, yes. But not in the docs. And the info is duplicated in typeshed, so their test suite doesn’t help catch mistakes in stdlib.

1 Like
13

In my experience, fully type hinted libraries where the types are exposed in the documentation often end up with long and complex type hints that harm readability rather than helping. Unfortunately, I can’t find a good example to link to right now (most of the examples I tried don’t include type hints in the docs, which says something itself, IMO…)

Good (where I mean complete, flexible, and not unnecessarily restrictive) type hints are often complex, and use concepts that are not particularly obvious to people unfamiliar with type theory. Using such types in the Python documentation and help will make it much harder to claim that types are “optional”, as users will need to understand types to interpret the basic documentation.

For example, in typeshed, asyncio.as_completed is typed as

def as_completed(fs: Iterable[_FutureLike[_T]], *, timeout: float | None = ...) -> Iterator[Future[_T]]: ...

That’s not incredibly difficult if you know the concepts and a bit about generic types, but it would be pretty daunting for a newcomer wanting to learn about asyncio.

On the other hand, asyncio.gather is a huge list of overloads, with the comment:

# `gather()` actually returns a list with length equal to the number
# of tasks passed; however, Tuple is used similar to the annotation for
# zip() because typing does not support variadic type variables. See
# typing PR #1550 for discussion.

So my feeling is that while having type annotations for the stdlib is great, they are fine in typeshed, and they absolutely should not be exposed in the documentation. Narrative prose, while harder to write initially, is a far better way of explaining how to use all but the simplest functions.

Please, no. We already have enough problem with people arguing excessively (IMO) strict positions like “if it can’t be expressed in the type annotation, it should be illegal”. If we did this, IDEs would red-flag perfectly valid (but “not best practice”) use of stdlib functions, users would submit PRs with elaborate workarounds for valid code that doesn’t typecheck, CI that runs mypy as part of a lint check would either fail, or wouldn’t be using the documented stdlib annotations.

8 Likes
14

This reminds me a lot of the discussion on how to debundle stdlib from the core interpreter and potentially make them upgradable with e.g. pip. And the general discussion “should we add X to stdlib” and the usual “stdlib is where code comes to die” sentiment. I’d take these as hints to typing has still not been developed to the level where using it in stdlib makes sense, as stdlib is currently managed.

It seems to me that the most reasonable approach would be to keep type hints in typeshed for now, and work on either (both?) a. developing typing to reach a level where it can at least express most of the stdlib, and b. debundling stdlib so each module can decide separately when’s the best time to include type annotations, instead of being blocked by the most dynamic parts of the entire stdlib (and potentially forever).

4 Likes
15

Types in stdlib would slow compilation. That is enough reason to keep them in typeshed forever.

16

I think that depends on how you display them. For instance, sphinx.ext.autodoc has multiple options for how to display type information. It would be totally possible to list the type information optionally via a toggle, via a collapsed-by-default drop-down at the end of the callable, etc. That would let those who want the type info to have them easily available while making quick glances and general reading not have them get in the way visually.

“Compilation” of what specifically? If you’re talking about .pyc files then it’s negligible and cached. Plus with PEP 649 now tentatively accepted, that also helps deal with any overhead which typically shows up from importing.

4 Likes
17

If you’re thinking of starting a migration plan, yes, I think it’s too early. A very tiny fraction of stdlib code is currently annotated, and anything that would encourage contributors to start submitting PRs that add annotations will quickly overwhelm the core dev team.

Maybe we could start encouraging authors of new modules to add annotations (if they even need that encouragement), without requiring it yet. But even here I would advise caution, based on my experience over several years at Dropbox, where we introduced type annotations in a large legacy code base. It’s easy to add annotations. But it’s not easy to ensure that the annotations are correct and complete.

Think about this: how do you test annotations? Running the type checker over the unit tests is a huge pain, because you’d need to annotate the unit tests first (otherwise the type checker will see most variables as type Any and not provide useful feedback). Given the difficulty of testing annotations, the quality of annotations is often questionable, and incorrect annotations are a huge liability.

There’s also the issue of strictness. In tomllib (since we’re picking on it :slight_smile: I found several uses of dict, which doesn’t tell me what the key and value types are. Is that acceptable?

Yeah, those are good ideas.

I suppose at some point in the future we may end up distributing the stubs with the stdlib rather than as part of typeshed (although there are issues with this too, as some have already pointed out).

4 Likes
18

That pretty much sums up my caution. I have added some super trivial ones within internal stdlib code, but generally avoid it when not a dead simple obvious concrete builtin types like str or int without any |ing or []s or TypeVar-ing or abc protocols.

I currently recommend assuming we have a high bar for shipping annotations in stdlib code because I presume we implicitly need to satisfy all possible consumers at once: all static analysis type checkers (pytype, mypy, vscode’s pyright, jetbrain’s pycharm thing, etc, etc etc…) and all runtime type checkers (pydantic and similar). Granted we could adopt a policy of saying: type checkers should detect stdlib code and only make use of in-stdlib annotations as a last resort source of information if they don’t just ignore them entirely. Which is what I believe the static analysis checkers do today via typeshed.

Why such a high bar? because inaccuracies in those annotations could lead to existential bugs in peoples tooling objecting to code based on an errant stdlib annotation that cannot be fixed in a timely manner (because python releases and stdlib and non-trivial updates for many). This is the beauty of typeshed and the analyzers that use it being projects moving at their own pace outside of CPython.

Regardless of all of this, we should get to the point where we run multiple static analyzers on the stdlib as an automated CI/bot job so that we at least have some confidence in what we do author. But a wholesale merging of typeshed data in as annotations within the stdlib - as much as the pedantic part of me would love to see that happen - it feels dangerous right now.

(Or am I’m over-blowing things and type checkers already detect and ignore stdlib code annotations because they use typeshed? I don’t know how to answer that, there are too many possible annotation consumers)

19

Runtime checkers only see what’s in the stdlib, so they are the most sensitive here. Making assumptions on what they do would be dangerous, we should just ask them – does it matter if annotations start appearing in parts of the stdlib, does it matter if those annotations are not 100% correct, and what if stdlib annotations are updated? If it matters, what are the mechanisms by which runtime checkers consume these annotations? (Knowing how makes it easier to do the right thing.)

IIUC static type checkers currently all ignore the stdlib – PEP 561 makes them look in site-packages but other than that they get all their info from typeshed stubs (usually bundled with the checker, sometimes with checker-specific additions). We’d first need to standardize a mechanism for indicating which parts of the stdlib should be read by static checkers. I don’t even have an intuition about whether that data should be included with the stdlib or with typeshed, although the PEP 561 precedent makes me think probably the former. Standardization here is important to ensure buy-in of the various static checkers. Presumably a PEP, unless the discussion about PEP categories ends with a quick discussion.

1 Like
20

Hmm. Is this “multiple consumers” a problem of typing annotations in general? Do type hints need a way to say “show bytes in docs and IDE hints, but actually allow any iterable of int (and by the way: check that bytes are iterables of int)”?

And is that a problem with annotations in general? :‍)

When I asked if it’s too early to start thinking about the next steps, I didn’t mean a wholesale migration should be the next step. Rather, I’d like to know about the long-term direction and what’s blocking it.

Sounds like typeshed should override stdlib (and any other library)? So tools that need accurate typing should use typeshed, but e.g. IDLE should be fine with inspect.signature.

Oh, definitely!

Yes. Write a PEP if there’s no obviously better alternative. The PEP discussion is about a long-term direction.

1 Like