Generic type subscription is a huge toll on Python performance

[RuslanDautkhanov]

BPO	39168
Nosy	@gvanrossum, @ilevkivskyi, @davidism, @ZackerySpytz, @miss-islington, @calebj, @wouterdb
PRs	bpo-39168: Remove the __new__ method of typing.Generic #21327 [3.9] bpo-39168: Remove the __new__ method of typing.Generic (GH-21327) #21335

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-07-05.16:03:53.763>
created_at = <Date 2019-12-30.17:17:10.478>
labels = ['3.7', '3.8', 'performance']
title = 'Generic type subscription is a huge toll on Python performance'
updated_at = <Date 2020-11-16.00:12:17.193>
user = 'https://bugs.python.org/RuslanDautkhanov'

bugs.python.org fields:

activity = <Date 2020-11-16.00:12:17.193>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-05.16:03:53.763>
closer = 'gvanrossum'
components = []
creation = <Date 2019-12-30.17:17:10.478>
creator = 'Ruslan Dautkhanov'
dependencies = []
files = []
hgrepos = []
issue_num = 39168
keywords = ['patch']
message_count = 20.0
messages = ['359049', '359050', '359061', '359072', '359074', '359081', '359124', '359126', '359127', '359129', '359130', '359131', '359132', '373013', '373014', '373036', '381036', '381041', '381046', '381047']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'levkivskyi', 'davidism', 'ZackerySpytz', 'miss-islington', 'Ruslan Dautkhanov', 'navdevl', 'calebj', 'Wouter De Borger']
pr_nums = ['21327', '21335']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue39168'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[RuslanDautkhanov]

Reported originally here -
https://twitter.com/__zero323__/status/1210911632953692162

See details here
https://asciinema.org/a/290643

In [4]: class Foo: pass
In [5]: %timeit -n1_000_000 Foo()
88.5 ns ± 3.44 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [6]: T = TypeVar("T")
In [7]: class Bar(Generic[T]): pass
In [8]: %timeit -n1_000_000 Bar()
883 ns ± 3.46 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Same effect in Python 3.6 and 3.8

[RuslanDautkhanov]

Python typing gives an order of magnitude slow down in this case

[RuslanDautkhanov]

In [12]: cProfile.run("for _ in range(100_000): Bar()")
200003 function calls in 0.136 seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.047 0.047 0.136 0.136 <string>:1(<module>)
100000 0.079 0.000 0.089 0.000 typing.py:865(new)
100000 0.010 0.000 0.010 0.000 {built-in method __new__ of type object at 0x55ab65861ac0}
1 0.000 0.000 0.136 0.136 {built-in method builtins.exec}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}

In [13]: # And typing.py:865 points to

In [14]: inspect.getsourcelines(Generic.__new__)
Out[14]:
([' def __new__(cls, *args, **kwds):\n',
' if cls in (Generic, Protocol):\n',
' raise TypeError(f"Type {cls.__name__} cannot be instantiated; "\n',
' "it can be used only as a base class")\n',
' if super().__new__ is object.__new__ and cls.__init__ is not object.__init__:\n',
' obj = super().__new__(cls)\n',
' else:\n',
' obj = super().__new__(cls, *args, **kwds)\n',
' return obj\n'],
865)

[gvanrossum]

What Python version was used for the timings? If not 3.8, please do over in 3.8.

[gvanrossum]

Sorry, you already said 3.6 and 3.8 give the same effect. But what if you add a minimal __new__() to Foo?

[gvanrossum]

Hm, here's what I measure in Python 3.8.1. (I don't use IPython or notebooks so this looks a little different.)

>>> timeit.timeit('Foo()', 'class Foo: pass')
0.37630256199999934


>>> timeit.timeit('Foo()', 'class Foo:\n  def __new__(cls): return super().__new__(cls)')
1.5753196039999864


>>> timeit.timeit('Foo()', 'from typing import Generic, TypeVar\nT = TypeVar("T")\nclass Foo(Generic[T]): pass')
3.8748737150000068

From this I conclude that adding a minimal __new__() method is responsible for about 4x slowdown, and the functionality in typing.py for another factor 2.5.

While this isn't great I don't see an easy way to improve upon this without rewriting the entire typing module in C. (Some of this may or may not happen for PEP-604.)

PS. I just realized my Python binary was built with debug options, so absolute numbers will look different (better) for you -- but relative numbers will look the same, and I get essentially the same factors with 3.9.0a1+.

[ilevkivskyi]

This issue came up few times before (although I can't find an issue here on b.p.o., maybe it was on typing-sig list). Although in micro-benchmarks the impact may seem big, in vast majority of applications it is rarely more that a percent or so.

On the other hand, IIRC the only reason Generic.__new__() exists is so that one can't write Generic() (i.e. instantiate a plain Generic). I would be totally fine if we just remove it in 3.9. Hopefully, people already learned what typing is for and don't need so much "protection" against not very meaningful things. Also, the error can be given by static type checkers, there is probably no need for a runtime error.

[gvanrossum]

If that solves the perf issue I am fine with it.

[ilevkivskyi]

OK, here is the original issue python/typing#681. I asked the author to open an issue here instead, but likely they didn't open one.