Fatal Python error "XXX block stack overflow" when exception stacks >10

[myzhang1029]

BPO	39934
Nosy	@ambv, @markshannon, @serhiy-storchaka, @pablogsal, @myzhang1029, @iritkatriel
PRs	bpo-39934: Account for control blocks in 'except' in compiler. #22395 [3.9] bpo-39934: Account for control blocks in 'except' in compiler. (GH-22395) #23303
Files	exception_nest.py: Run exception_nest.py and run the generated python code

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-05-05.10:53:27.428>
created_at = <Date 2020-03-11.13:24:52.447>
labels = ['interpreter-core', '3.10', '3.8', '3.9', 'type-crash']
title = 'Fatal Python error "XXX block stack overflow" when exception stacks >10'
updated_at = <Date 2021-05-05.10:53:27.427>
user = 'https://github.com/myzhang1029'

bugs.python.org fields:

activity = <Date 2021-05-05.10:53:27.427>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-05-05.10:53:27.428>
closer = 'iritkatriel'
components = ['Interpreter Core']
creation = <Date 2020-03-11.13:24:52.447>
creator = 'myzhang1029'
dependencies = []
files = ['48969']
hgrepos = []
issue_num = 39934
keywords = ['patch']
message_count = 18.0
messages = ['363914', '377433', '377434', '377437', '377439', '377440', '377441', '377444', '377445', '377490', '378178', '378192', '381280', '385035', '385037', '385039', '385042', '393002']
nosy_count = 6.0
nosy_names = ['lukasz.langa', 'Mark.Shannon', 'serhiy.storchaka', 'pablogsal', 'myzhang1029', 'iritkatriel']
pr_nums = ['22395', '23303']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue39934'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[myzhang1029]

I apologize for describing this issue badly, but I'll try anyway.
The code to demonstrate the issue is attached, so it might be better to read that instead.
I noticed that when more than 10 exceptions are raised sequentially (i.e. one from another or one during the handling of another), the interpreter crashes saying "Fatal Python error: XXX block stack overflow".
This happens in python 3.7, 3.8 and development(git 39c3493) versions, but not in python2.7. Using ipython also fixes this issue.
I know this case is rare, but the maximum number of recursions is more than 2000, and the maximum number of statically nested blocks sepcified in frameobject.c is 20, so I'm pretty sure this isn't intended behavior.

[iritkatriel]

The error is coming from here:

cpython/Objects/frameobject.c

Line 958 in cb9879b

Py_FatalError("block stack overflow");

and CO_MAXBLOCKS is defined in Include/cpython/code.h
#define CO_MAXBLOCKS 20 /* Max static block nesting within a function */

This is not about recursion or about exception, it's about static nesting level.

There is an example here showing an input that gives the same error with >20 nested while blocks:

cpython/Lib/test/test_syntax.py

Line 545 in c8f29ad

[iritkatriel]

In summary, I think this is not-a-bug.

[serhiy-storchaka]

It is a bug. Compiler explicitly checks if the number of nested "try" blocks does not exceed the limit of CO_MAXBLOCKS, but it does not count implicit "try" blocks inserted when your assign an exception in the "except" clause.

try:
    ...
except Exception as e:
    ...

is actually translated to

try:
    ...
except Exception:
    try:
        e = ...
        ...
    finally:
        e = None
        del e

So we have double number of nested "try" blocks.