Generic __new__ has serious performance overhead

[bartv]

In our code base (https://github.com/inmanta/inmanta) we are in progress of typing our code. We progressed pretty far and now noticed in our compiler that it has a serious overhead. In a certain performance benchmark Generic.__new__ amounts to 8.7% of the execution time. (It is called ~1M times)

The following piece of code demonstrates the large difference in performance:

import time
from typing import *

T = TypeVar("T")

class Test(Generic[T]):
    def __init__(self, x: T) -> None:
        pass

class Test2():
    def __init__(self, x) -> None:
        pass

a = time.time()
[Test("test") for i in range(1, 1000000)]
b = time.time()
[Test2("test") for i in range(1, 1000000)]
c = time.time()

print(f"Generic {b-a}")
print(f"Non-Generic {c-b}")

With python 3.7.4 the results are:

Generic 0.8733069896697998
Non-Generic 0.3357818126678467

With python 3.6 the results are slightly worse (but not so relevant due to the new implementation)

(#196 seems related but because the implementation is entirely different I did not post it there)

[gvanrossum]

Interesting. There's not that much code in Generic.__new__ (source), but there is definitely more than in the default object.__new__ (and it's written in Python). I gather you are creating objects whose __init__ does not do much?

It also looks like typing.py in CPython has more code in Generic.__new__ than the backport typing package (source).

I see some ways to micro-optimize the CPython version a bit, but I'm not sure that it will help much. Does it affect the performance of your real code? Usually micro-benchmarks don't really reflect reality in typical code bases (but perhaps yours isn't typical in this respect).

[ilevkivskyi]

There is one dirty workaround that may work on Python 3.7+. I think you can try simply del Generic.__new__ at startup. In Python 3.7+ __new__ exists only to prohibit instantiating Generic(), Generic[T](), Protocol() etc directly.

[gvanrossum]

Assuming type checkers will catch that, do we need those checks at runtime?

[ilevkivskyi]

Assuming type checkers will catch that, do we need those checks at runtime?

Actually mypy behaves a bit weird here:

Generic()  # Error: "_SpecialForm" not callable
Generic[int]()  # No error
Generic[T]()  # No error

Anyway, I don't think these checks are very important. I assume they exist mostly for historical reasons to emphasize Generic[T] is rather a special syntactic form, not a normal class.

[bartv]

This code is part of a model compiler. We have a benchmark where we scale the model. Once the model is large enough so make the startup negligible we see about 8% of the time spent on Generic.__new__. For example for a total runtime of 43.8s: 8.43s is spent on Generic.__new__ of which 7.76 is self time. So indeed most of the overhead is in Generic.__new__ and not in object.__new__

[gvanrossum]

Is the package open source? If so, could you link to the actual class definition? I'd like to understand what else is in your class -- apparently not much if you see such a large overhead. I'm not trying to minimize the problem -- I'm just trying to understand what code patterns make especially heavy use of Generic.__new__.

I think it's fine to just drop Generic.__new__ in CPython. Since it appears to be basically a perf enhancement we might be able to roll this out in Python 3.8.1.

[bartv]

Sure, it is ResultCollector (https://github.com/inmanta/inmanta/blob/c41b54a19e4ad8dc6b163d842b63a8af3b5dad5c/src/inmanta/execute/runtime.py#L55) and its subclasses and IPromise right below ResultCollector.

Attached is also a pdf version of the callgraph.

profile.pdf

[gvanrossum]

Cool. Would it be terribly inconvenient to re-run that exact same benchmark including Ivan's suggestion of del Generic.__new__ at startup? If that confirms that you are then running in ~36 seconds that would be a huge argument for just deleting this method from the source in 3.8.1.

I also note that this code uses __slots__, which makes the overhead of __new__ all the more pronounced. As typing reaches mainstream we cannot ignore performance in such cases.

[bartv]

How would you do the del in the example of the ticket? Because I get errors about wrong arguments to object.__new__

TypeError: object.__new__() takes exactly one argument (the type to instantiate)

We have indeed started introducing slots because once the models we compile become large enough construction of objects has some serious overhead.

[ilevkivskyi]

Yes, you are right, you need something more tricky like copying the __dict__:

ns = typing.Generic.__dict__.copy()
del ns['__new__']
typing.Generic = type('Generic', (), ns)

I however checked and this doesn't work anymore because of argumentless super() in Generic.__init_subclass__(). I will try to see if there is another simple way.

[bartv]

I tried this:

diff --git a/src/inmanta/execute/runtime.py b/src/inmanta/execute/runtime.py
index b414811b..96f74279 100644
--- a/src/inmanta/execute/runtime.py
+++ b/src/inmanta/execute/runtime.py
@@ -52,20 +52,23 @@ if TYPE_CHECKING:
 T = TypeVar("T")
 
 
 class ResultCollector(Generic[T]):
     """
         Helper interface for gradual execution
     """
 
     __slots__ = ()
 
+    def __new__(cls, *args, **kwargs):
+        return object.__new__(cls)
+
     def receive_result(self, value: T, location: Location) -> None:
         """
             receive a possibly partial result
         """
         raise NotImplementedError()
 
 
 class IPromise(Generic[T]):
     __slots__ = ()

This works and speeds up a little bit, however it only replaces the Generic.__new__ with an empty method instead of eliminating the method call entirely.

[ilevkivskyi]

I didn't have much time to think about a better way to experiment with this. In any case, it looks like there is no action items for the backport here, while there are potential action item for Python 3.8. @bartv Could you please instead open an issue on bugs.python.org?