Improve Python 2 only syntax detection#2592
Merged
Conversation
First of all this fixes a mistake I made in Python 2 deprecation PR
using token.* to check for print/exec statements. Turns out that
for nodes with a type value higher than 256 its numeric type isn't
guaranteed to be constant. Using syms.* instead fixes this.
Also add support for the following cases:
print "hello, world!"
exec "print('hello, world!')"
def set_position((x, y), value):
pass
try:
pass
except Exception, err:
pass
raise RuntimeError, "I feel like crashing today :p"
`wow_these_really_did_exist`
10L
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
ichard26
force-pushed
the
improve-python-2-detection
branch
from
0b23189 to
78d32bf
Compare
JelleZijlstra
approved these changes
Nov 8, 2021
Collaborator
|
Thank you! |
Collaborator
Author
|
cc @Zac-HD Traceback (most recent call last):
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesmith/syntactic.py", line 110, in draw_symbol
compile(
SystemError: Negative size passed to PyUnicode_New
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/runner/work/black/black/fuzz.py", line 73, in <module>
test_idempotent_any_syntatically_valid_python()
File "/home/runner/work/black/black/fuzz.py", line 20, in test_idempotent_any_syntatically_valid_python
max_examples=1000, # roughly 1k tests/minute, or half that under coverage
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesis/core.py", line 1199, in wrapped_test
raise the_error_hypothesis_found
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesmith/syntactic.py", line 89, in do_draw
result = super().do_draw(data)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesis/extra/lark.py", line 153, in do_draw
self.draw_symbol(data, start, state)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesmith/syntactic.py", line 107, in draw_symbol
super().draw_symbol(data, symbol, draw_state)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesis/extra/lark.py", line 181, in draw_symbol
self.draw_symbol(data, e, draw_state)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesmith/syntactic.py", line 107, in draw_symbol
super().draw_symbol(data, symbol, draw_state)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesis/extra/lark.py", line 181, in draw_symbol
self.draw_symbol(data, e, draw_state)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesmith/syntactic.py", line 107, in draw_symbol
super().draw_symbol(data, symbol, draw_state)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesis/extra/lark.py", line 181, in draw_symbol
self.draw_symbol(data, e, draw_state)
File "/home/runner/work/black/black/.tox/fuzz/lib/python3.9/site-packages/hypothesmith/syntactic.py", line 129, in draw_symbol
raise type(err)(
SystemError: compile('delA\\ \n#\n', '<string>', 'single') raised SystemError: Negative size passed to PyUnicode_Newhttps://github.com/psf/black/runs/4184619816?check_suite_focus=true#step:5:85 Seems like the fuzz issue might be related to https://bugs.python.org/issue45738. Anyhow we can't do anything about it ourselves :) |
Contributor
|
Thanks, Zac-HD/hypothesmith#16 is open and I guess I'm adding another workaround for an upstream bug. |
JelleZijlstra
added a commit
that referenced
this pull request
Nov 16, 2021
* Improve Python 2 only syntax detection
First of all this fixes a mistake I made in Python 2 deprecation PR
using token.* to check for print/exec statements. Turns out that
for nodes with a type value higher than 256 its numeric type isn't
guaranteed to be constant. Using syms.* instead fixes this.
Also add support for the following cases:
print "hello, world!"
exec "print('hello, world!')"
def set_position((x, y), value):
pass
try:
pass
except Exception, err:
pass
raise RuntimeError, "I feel like crashing today :p"
`wow_these_really_did_exist`
10L
* Add octal support, more test cases, and fixup long ints
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
First of all this fixes a mistake I made in Python 2 deprecation PR
using token.* to check for print/exec statements. Turns out that
for nodes with a type value higher than 256 its numeric type isn't
guaranteed to be constant. Using syms.* instead fixes this.
Also add support for the following cases:
This should also fix the Python 2 related failures in #2586.
Checklist - did you ...