Optimize function annotation

[methane]

BPO	42202
Nosy	@methane, @ambv, @markshannon, @serhiy-storchaka, @jstasiak, @corona10, @sweeneyde, @uriyyo
PRs	bpo-42202: Store func annotations as single tuple at bytecode level #23316 Improve the description of the improvements in bpo-42202 #24738

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-11-25.10:44:33.126>
created_at = <Date 2020-10-30.06:47:15.429>
labels = ['interpreter-core', '3.10', 'performance']
title = 'Optimize function annotation'
updated_at = <Date 2022-01-03.04:32:27.675>
user = 'https://github.com/methane'

bugs.python.org fields:

activity = <Date 2022-01-03.04:32:27.675>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2020-11-25.10:44:33.126>
closer = 'methane'
components = ['Interpreter Core']
creation = <Date 2020-10-30.06:47:15.429>
creator = 'methane'
dependencies = []
files = []
hgrepos = []
issue_num = 42202
keywords = ['patch']
message_count = 17.0
messages = ['379923', '379928', '379929', '379930', '379936', '379947', '381200', '381251', '381253', '381255', '381264', '381272', '381309', '381320', '381818', '388067', '409557']
nosy_count = 8.0
nosy_names = ['methane', 'lukasz.langa', 'Mark.Shannon', 'serhiy.storchaka', 'jstasiak', 'corona10', 'Dennis Sweeney', 'uriyyo']
pr_nums = ['23316', '24738']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue42202'
versions = ['Python 3.10']

[methane]

Look this example:

code:

# code
def foo(x: int, /, y, *, z: float) -> Hoge:
    pass

# dis
 2          12 LOAD_CONST               2 ('int')
            14 LOAD_CONST               3 ('float')
            16 LOAD_CONST               4 ('Hoge')
            18 LOAD_CONST               5 (('x', 'z', 'return'))
            20 BUILD_CONST_KEY_MAP      3
            22 LOAD_CONST               6 (<code object foo at ...>)
            24 LOAD_CONST               7 ('foo')
            26 MAKE_FUNCTION            4 (annotations)
            28 STORE_NAME               2 (foo) 

Four LOAD_CONST and BUILD_CONST_KEY_MAP are used to generate annotation dict. This makes program load slow and eat more memory.

Annotation information can be stored in some compact form. And creating annotation dict can be postponed to when func.__annotation__ is accessed.

Ideas for the compact form:

1. Tuple.
   In above example, ('int', None, 'float', 'Hoge') can be used. None means no annotation for the 'y' parameter.

2. Serialize into str or bytes.
   JSON like format can be used, like x:int,z:float;Hoge. Compact. But the string/bytes has lower chance to be shared with other constants in same module.

[serhiy-storchaka]

Are annotations now always known at compile time?

As for representation, it can also be a sequence of pairs (('x', 'int'), ('z', 'float'), ('return', 'Hoge')) or a pair of sequences (('x', 'z', 'return'), ('int', 'float', 'Hoge')). It would be better to save a dict directly in pyc files, but it needs changing the marshal protocol.

Also, it makes sense to make annotations attribute of the code object, so avoid the overhead at function creation time.

I have a dream to split the pyc file into several files or sections and save docstrings and annotations (and maybe line numbers) separately from the main code. They should be loaded by demand, when you read __doc__ or __annotation__. Most code does not use them at run time, so we can save memory and loading time. It can also help with internationalization.

[corona10]

I like the 1st option which uses a tuple

[corona10]

@serhiy race condition sorry ;)

[methane]

Are annotations now always known at compile time?

Yes, because from __future__ import annotations is enabled by default from Python 3.10.

As for representation, it can also be a sequence of pairs (('x', 'int'), ('z', 'float'), ('return', 'Hoge')) or a pair of sequences (('x', 'z', 'return'), ('int', 'float', 'Hoge')). It would be better to save a dict directly in pyc files, but it needs changing the marshal protocol.

Yes, but it is bit larger than my single tuple idea in most cases.
Since most annotations are not used by runtime, we don't need to create a dict until func.__annotation__ is read.

Also, it makes sense to make annotations attribute of the code object, so avoid the overhead at function creation time.

I am not sure this is the best option because there are many code object without annotation.

I have a dream to split the pyc file into several files or sections and save docstrings and annotations (and maybe line numbers) separately from the main code. They should be loaded by demand, when you read __doc__ or __annotation__. Most code does not use them at run time, so we can save memory and loading time. It can also help with internationalization.

I have same dream.

[serhiy-storchaka]

Yes, but it is bit larger than my single tuple idea in most cases.

Yes, but the code for creating a dict can be simpler. In any case we will better see what format is better when try to write a code.

I am not sure this is the best option because there are many code object without annotation.

In this case it can be None or NULL.

I like your idea. It is easy to implement it now. Later we can make annotations an attribute of the code object.

[methane]

Yes, but the code for creating a dict can be simpler. In any case we will better see what format is better when try to write a code.

Note that many annotations are not accessed. RAM usage of annotation information is important than how easy to create dict.

I don't like (('x', 'int'), ('z', 'float'), ('return', 'Hoge')) because it creates 4 tuples. It means use more memory, load pyc slower.

Please use ('x', 'int', 'z', 'float', 'return', 'Hoge') instead.

[markshannon]

For top level functions (functions created once) this isn't going to make any real difference. There might be a small speedup for function creation, but it isn't going to be measurable.

For nested functions with annotations, where many functions are created from a single code object, this could be worthwhile.

However, before we add yet another attribute to code objects, I'd like to see some evidence of a speedup.

[uriyyo]

I have just implemented co_annotations field for CodeObject.
I wrote a simple benchmark to measure the time required to import black module (I took black because it contains a log of annotations).

Benchmark simply run python -m timeit -n 5000000 "import black".

Results:

Python 3.6.8
5000000 loops, best of 3: 0.0983 usec per loop
Python 3.7.6
5000000 loops, best of 5: 102 nsec per loop
Python 3.8.3
5000000 loops, best of 5: 97.4 nsec per loop
Python 3.9.0
5000000 loops, best of 5: 99.5 nsec per loop
Python 3.10.0a2+ with co_annotations
5000000 loops, best of 5: 92.4 nsec per loop
Python 3.10.0a2+
5000000 loops, best of 5: 98.9 nsec per loop

[jstasiak]

Yurii, I don't believe that benchmark measures what you need to measure (once imported module is kept imported forever until unloaded, so successive imports are no-ops).

See how the side effects of importing bbb only happen once:

% cat bbb.py
import time
time.sleep(1)
with open('bbb.log', 'a') as f:
written = f.write('hello\n')
assert written == 6

% time python -m timeit "import bbb"
1 loop, best of 5: 515 nsec per loop
python -m timeit "import bbb" 0.03s user 0.01s system 4% cpu 1.050 total

% cat bbb.log
hello

[serhiy-storchaka]

If you want to measure import time, use

python -m timeit -s "from sys import modules; modules_copy = modules.copy()" "import black; modules.clear(); modules.update(modules_copy)"

But I would be surprised to see significant difference in this case.

What Mark means, measure the time of creation of nested function.

python -m timeit "def f(a: int, b: str) -> None: pass"

And maybe test with different number of arguments if there is a difference.