Skip to content

Python/compile.c:7482: int compute_code_flags(struct compiler *): Assertion `IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)' failed #121637

Closed
Closed
Python/compile.c:7482: int compute_code_flags(struct compiler *): Assertion `IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)' failed#121637
Assignees
JelleZijlstra
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dump

Description

@alex
alex
opened on Jul 12, 2024

Crash report

What happened?

~/p/cpython ❯❯❯ ./python.exe -c 'compile("assert await u", "", "exec", optimize=2)'
Assertion failed: (IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)), function compute_code_flags, file compile.c, line 7482.
fish: Job 1, './python.exe -c 'compile("asser…' terminated by signal SIGABRT (Abort)

Full stacktrace:


fuzz_pycompile: Python/compile.c:7482: int compute_code_flags(struct compiler *): Assertion `IS_TOP_LEVEL_AWAIT(c) \|\| _PyST_IsFunctionLike(ste)' failed.
--
  | ==24717== ERROR: libFuzzer: deadly signal
  | #0 0x59033612cf71 in __sanitizer_print_stack_trace /src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:87:3
  | #1 0x59033602fd98 in fuzzer::PrintStackTrace() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerUtil.cpp:210:5
  | #2 0x590336012ac3 in fuzzer::Fuzzer::CrashCallback() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:231:3
  | #3 0x79353882e41f in libpthread.so.0
  | #4 0x79353862800a in __libc_signal_restore_set /build/glibc-SzIz7B/glibc-2.31/sysdeps/unix/sysv/linux/internal-signals.h:86:3
  | #5 0x79353862800a in raise /build/glibc-SzIz7B/glibc-2.31/sysdeps/unix/sysv/linux/raise.c:48:3
  | #6 0x793538607858 in abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:79:7
  | #7 0x793538607728 in __assert_fail_base /build/glibc-SzIz7B/glibc-2.31/assert/assert.c:92:3
  | #8 0x793538618fd5 in __assert_fail /build/glibc-SzIz7B/glibc-2.31/assert/assert.c:101:3
  | #9 0x5903364b52da in compute_code_flags cpython3/Python/compile.c:7482:9
  | #10 0x5903364b52da in optimize_and_assemble cpython3/Python/compile.c:7578:22
  | #11 0x5903364af709 in compiler_mod cpython3/Python/compile.c:1597:10
  | #12 0x5903364af709 in _PyAST_Compile cpython3/Python/compile.c:422:24
  | #13 0x59033660c8a0 in Py_CompileStringObject cpython3/Python/pythonrun.c:1450:10
  | #14 0x59033660c994 in Py_CompileStringExFlags cpython3/Python/pythonrun.c:1463:10
  | #15 0x590336161d2e in fuzz_pycompile cpython3/Modules/_xxtestfuzz/fuzzer.c:551:24
  | #16 0x590336161d2e in _run_fuzz cpython3/Modules/_xxtestfuzz/fuzzer.c:570:14
  | #17 0x590336161d2e in LLVMFuzzerTestOneInput cpython3/Modules/_xxtestfuzz/fuzzer.c:711:11
  | #18 0x590336013fe0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:614:13
  | #19 0x590335ffe774 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:327:6
  | #20 0x59033600420a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:862:9
  | #21 0x590336030602 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
  | #22 0x793538609082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
  | #23 0x590335ff524d in _start
  |  

This regressed somewhere in f621618...690b935

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

No response

Linked PRs

Activity

  1. Imagealex
    added
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Jul 12, 2024

  2. alex commented on Jul 12, 2024

    @alex
    alex
    on Jul 12, 2024
    MemberAuthor

    cc: @iritkatriel -- it looks like you've got a few commits in this file in that range, though I don't see one that's obviously relevant.

  3. JelleZijlstra commented on Jul 12, 2024

    @JelleZijlstra
    JelleZijlstra
    on Jul 12, 2024
    Member

    Feels related to #121297.

  4. Imagesobolevn
    added
    topic-asyncio
    on Jul 12, 2024
  5. Imagegithub-project-automation
    added this to asyncioon Jul 12, 2024
  6. Imagegithub-project-automation
    moved this to Todo in asyncioon Jul 12, 2024
  7. Imagesobolevn
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    on Jul 12, 2024

  8. sobolevn commented on Jul 12, 2024

    @sobolevn
    sobolevn
    on Jul 12, 2024
    Member

    I will add both asyncio and interpreter-core, because both are related.

  9. ImageJelleZijlstra
    self-assigned this
    on Jul 12, 2024

  10. JelleZijlstra commented on Jul 12, 2024

    @JelleZijlstra
    JelleZijlstra
    on Jul 12, 2024
    Member

    Irit is on vacation, I'll look into this over the next few days.

  11. JelleZijlstra commented on Jul 13, 2024

    @JelleZijlstra
    JelleZijlstra
    on Jul 13, 2024
    Member

    The problem is arguably this:

    % python -c 'assert await 42'   
  File "<string>", line 1
SyntaxError: 'await' outside function
% python -O -c 'assert await 42'
%

    It used to be that we didn't raise a SyntaxError if there is an await but it is optimized away, and the code would work fine.

    Now, however, the symtable sets ste_coroutine when it sees the await, but it still doesn't raise an error about the await, leading to the crash reported here.

    I can see two solutions:

    The latter seems like the more intuitive behavior for users.

    It's also consistent with other cases where optimized-away code leads to a compile error:

    % python -O
Python 3.12.1 (main, Apr 22 2024, 00:22:15) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
...     assert (x := 3)
...     global x
... 
  File "<stdin>", line 3
SyntaxError: name 'x' is assigned to before global declaration

    So I'll send a PR to make python -O -c 'assert await 1' a SyntaxError.

  12. alex commented on Jul 13, 2024

    @alex
    alex
    on Jul 13, 2024
    MemberAuthor

    Always doing a syntax error, even if the assert is optimized out, feels like the right behavior to me.

  13. JelleZijlstra commented on Jul 13, 2024

    @JelleZijlstra
    JelleZijlstra
    on Jul 13, 2024
    Member

    I realized there's a few more similar cases (essentially, every place we call compiler_error in compile.c while inside an expression):

    % python -c 'assert (yield)'
  File "<string>", line 1
SyntaxError: 'yield' outside function
% python -O -c 'assert (yield)'
% python -O -c 'assert (__debug__ := True)'
% python -c 'assert (__debug__ := True)' 
  File "<string>", line 1
SyntaxError: cannot assign to __debug__
% python -O -c 'assert (__debug__ := True)'
% python -c 'assert [x async for x in ()]' 
  File "<string>", line 1
SyntaxError: asynchronous comprehension outside of an asynchronous function
% python -O -c 'assert [x async for x in ()]'

    The latter also crashes on current main:

    % ./python.exe -O -c 'assert [x async for x in ()]'
Assertion failed: (IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)), function compute_code_flags, file compile.c, line 7482.
zsh: abort      ./python.exe -O -c 'assert [x async for x in ()]'

  14. JelleZijlstra commented on Jul 13, 2024

    @JelleZijlstra
    JelleZijlstra
    on Jul 13, 2024
    Member

    The linked PR makes symtable.c detect incorrect await and async comprehensions, fixing this issue.

    I didn't bother detecting other compiler errors (e.g., invalid yields) because they don't cause similar crashes. I wouldn't be surprised though if there's a crash lurking somewhere with e.g. assert (yield) being optimized away, so maybe if you point your fuzzer in that direction it will find something :)

  15. gvanrossum commented on Jul 13, 2024

    @gvanrossum
    gvanrossum
    on Jul 13, 2024
    Member

    Always doing a syntax error, even if the assert is optimized out, feels like the right behavior to me.

    Of course! It must have been an oversight that we didn't do this from the start.

    I'm removing the topic-asyncio label -- just because it mentions await doesn't make it an asyncio issue.

  16. Imagegvanrossum
    removed
    topic-asyncio
    on Jul 13, 2024
  17. ImageJelleZijlstra
    added a commit that references this issue on Jul 22, 2024

    gh-121637: Syntax error for optimized-away incorrect await (#121656)

    2762c6c

  18. JelleZijlstra commented on Jul 22, 2024

    @JelleZijlstra
    JelleZijlstra
    on Jul 22, 2024
    Member

    Thanks for the report! This will now be a SyntaxError.

  19. ImageJelleZijlstra
    closed this as completedon Jul 22, 2024
  20. Imagegithub-project-automation
    moved this from Todo to Done in asyncioon Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dump

Projects

  • asyncio
    Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions