"Zero cost" exception handling

[markshannon]

BPO	40222
Nosy	@gvanrossum, @rhettinger, @scoder, @vstinner, @encukou, @cjerdonek, @markshannon, @serhiy-storchaka, @gvanrossum, @ammaraskar, @corona10, @pablogsal, @sweeneyde, @erlend-aasland, @Jongy, @hauntsaninja, @rkm, @iritkatriel, @iritkatriel
PRs	bpo-40222: "Zero cost" exception handling #25729 bpo-40222: Prevent access outside buffer #26012 bpo-40222: Mention zero-cost exceptions in whats-new for 3.11 #26021 bpo-40222: Remove PyTryblock struct #26059 bpo-40222: update doc entry with respect to the change in WITH_EXCEPT… #29975 bpo-40222: update doc entry with respect to the change in WITH_EXCEPT… #29975

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/markshannon'
closed_at = <Date 2022-01-06.12:51:43.175>
created_at = <Date 2020-04-08.09:41:03.282>
labels = ['3.11', 'performance']
title = '"Zero cost" exception handling'
updated_at = <Date 2022-04-05.23:22:48.550>
user = 'https://github.com/markshannon'

bugs.python.org fields:

activity = <Date 2022-04-05.23:22:48.550>
actor = 'vstinner'
assignee = 'Mark.Shannon'
closed = True
closed_date = <Date 2022-01-06.12:51:43.175>
closer = 'Mark.Shannon'
components = []
creation = <Date 2020-04-08.09:41:03.282>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 40222
keywords = ['patch']
message_count = 45.0
messages = ['365974', '365975', '366976', '370612', '370642', '370645', '391509', '391513', '391522', '392305', '392306', '392309', '393321', '393322', '393324', '393342', '393372', '393433', '393448', '393455', '393456', '393458', '393460', '393461', '393462', '393463', '393464', '393470', '394060', '394272', '394273', '394274', '394276', '399503', '399520', '399535', '399536', '399538', '399547', '408004', '409720', '409838', '416480', '416498', '416833']
nosy_count = 18.0
nosy_names = ['gvanrossum', 'rhettinger', 'scoder', 'vstinner', 'petr.viktorin', 'chris.jerdonek', 'Mark.Shannon', 'serhiy.storchaka', 'Guido.van.Rossum', 'ammar2', 'corona10', 'pablogsal', 'Dennis Sweeney', 'erlendaasland', 'Yonatan Goldschmidt', 'hauntsaninja', 'rkm', 'iritkatriel', 'iritkatriel']
pr_nums = ['25729', '26012', '26021', '26059', '29975', '29975']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue40222'
versions = ['Python 3.11']

[markshannon]

C++ and Java support what is known as "zero cost" exception handling.
The "zero cost" refers to the cost when no exception is raised. There is still a cost when exceptions are thrown.

The basic principle is that the compiler generates tables indicating where control should be transferred to when an exception is raised. When no exception is raised, there is no runtime overhead.

(C)Python should support "zero cost" exceptions.

Now that the bytecodes for exception handling are regular (meaning that their stack effect can be statically determined) it is possible for the bytecode compiler to emit exception handling tables.

Doing so would have two main benefits.

1. "try" and "with" statements would be faster (and "async for", but that is an implementation detail).
2. Calls to Python functions would be faster as frame objects would be considerably smaller. Currently each frame carries 240 bytes of overhead for exception handling.

[serhiy-storchaka]

+1! I was going to implement this, but first I wanted to implement support of line number ranges instead of just line numbers (co_lineno). We need to design some compact portable format for address to address mapping (or address range to address mapping if it is more efficient).

Are you already working on this Mark? I would be glad to make a review.

[rhettinger]

This is an exciting prospect. Am looking forward to it :-)

[corona10]

+1

[cjerdonek]

To clarify, would there be any observable difference in behavior aside from speed? And would there be any limitations in when the speedup can be applied?

[serhiy-storchaka]

The only observable changes will be changes in the code object: new attributes and constructor parameters, changed .pyc format, dis output, etc.

[markshannon]

The changes to pyc format aren't user visible so shouldn't matter,
but what about the dis output?

Consider this program:

def f():
    try:
        1/0
    except:
        return "fail"

Currently it compiles to:
2 0 SETUP_FINALLY 7 (to 16)

3 2 LOAD_CONST 1 (1)
4 LOAD_CONST 2 (0)
6 BINARY_TRUE_DIVIDE
8 POP_TOP
10 POP_BLOCK
12 LOAD_CONST 0 (None)
14 RETURN_VALUE

4 >> 16 POP_TOP
18 POP_TOP
20 POP_TOP

5 22 POP_EXCEPT
24 LOAD_CONST 3 ('fail')
26 RETURN_VALUE

With zero-cost exception handling, it will compile to something like:
2 0 NOP
3 2 LOAD_CONST 1 (1)
4 LOAD_CONST 2 (0)
6 BINARY_TRUE_DIVIDE
8 POP_TOP
10 LOAD_CONST 0 (None)
12 RETURN_VALUE

None 14 PUSH_EXCEPT

4 16 POP_TOP
18 POP_TOP
20 POP_TOP

5 22 POP_EXCEPT
24 LOAD_CONST 3 ('fail')
26 RETURN_VALUE

(There are additional optimizations that should be applied, but those are a separate issue)

The problem is that the exception handling flow is no longer visible.
Should we add it back in somehow, or just append the exception jump table?

[serhiy-storchaka]

We can add a new column for the offset or the index of the error handler. Or add pseudo-instructions (which do not correspond to any bytecode) at boundaries of the code with some error handler.

[gvanrossum]

I like Serhiy’s idea.

BTW, what are the three POP_TOP op codes in a row popping?

[markshannon]

BTW, what are the three POP_TOP op codes in a row popping?

When exceptions are pushed to the stack, they are pushed as a triple: (exc, type, traceback)
so when we pop them, we need three pops.

[markshannon]

Responding to Serhiy's suggestions:
1 Add another column:
Adding another column makes for lots of repetition in larger try blocks, and pushes useful information further to the right.
2 Add pseudo-instructions
I find those misleading, as they aren't really there, and probably won't even correspond to the original SETUP_XXX instructions.