Asynchronous comprehensions don't work in asyncio REPL

[jack1142]

BPO	39562
Nosy	@asvetlov, @ambv, @serhiy-storchaka, @Carreau, @asottile, @hroncok, @pablogsal, @miss-islington, @isidentical, @jack1142
PRs	bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL #18968 bpo-39965: Correctly raise SyntaxError if await is used outside async functions when PyCF_ALLOW_TOP_LEVEL_AWAIT is set #19010 [3.8] bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL (GH-18968) #19070 bpo-39562: Prevent collision of future and compiler flags #19230 [3.8] bpo-39562: Prevent collision of future and compiler flags (GH-19230) #19835 [3.8] bpo-39562: Correctly updated the version section in the what's new document #19838

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-07-06.20:18:05.903>
created_at = <Date 2020-02-05.17:28:25.689>
labels = ['type-bug', '3.8', '3.9', 'release-blocker', 'expert-asyncio']
title = "Asynchronous comprehensions don't work in asyncio REPL"
updated_at = <Date 2021-09-07.20:14:54.080>
user = 'https://github.com/jack1142'

bugs.python.org fields:

activity = <Date 2021-09-07.20:14:54.080>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-06.20:18:05.903>
closer = 'pablogsal'
components = ['asyncio']
creation = <Date 2020-02-05.17:28:25.689>
creator = 'jack1142'
dependencies = []
files = []
hgrepos = []
issue_num = 39562
keywords = ['patch']
message_count = 28.0
messages = ['361443', '361445', '364600', '364601', '365311', '365312', '365313', '365347', '365881', '366838', '366841', '366843', '366849', '366851', '367015', '367018', '367734', '367847', '367855', '367890', '367892', '367897', '368334', '368835', '373131', '373159', '373168', '400234']
nosy_count = 10.0
nosy_names = ['asvetlov', 'lukasz.langa', 'serhiy.storchaka', 'mbussonn', 'Anthony Sottile', 'hroncok', 'pablogsal', 'miss-islington', 'BTaskaya', 'jack1142']
pr_nums = ['18968', '19010', '19070', '19230', '19835', '19838']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39562'
versions = ['Python 3.8', 'Python 3.9']

[jack1142]

asyncio REPL doesn't allow using asynchronous comprehensions outside of async func. Same behavior can also be observed when using ast.PyCF_ALLOW_TOP_LEVEL_AWAIT flag in compile()

Example with `async for`:
>>> async def async_gen():
...     for x in range(5):
...         yield await asyncio.sleep(1, x)
... 
>>> [x async for x in async_gen()]
  File "<console>", line 0
SyntaxError: asynchronous comprehension outside of an asynchronous function


Example with `await`:
>>> [await asyncio.sleep(1, x) for x in range(5)]
  File "<console>", line 0
SyntaxError: asynchronous comprehension outside of an asynchronous function

[jack1142]

I also noticed that putting await before the async comprehension will make the code inside the comprehension run (though after it runs, it will fail on awaiting list):

>>> await [await asyncio.sleep(1, print(x)) for x in range(5)] 
0
1
2
3
4
Traceback (most recent call last):
  File "C:\Python38\lib\concurrent\futures\_base.py", line 439, in result
    return self.__get_result()
  File "C:\Python38\lib\concurrent\futures\_base.py", line 388, in __get_result
    raise self._exception
  File "<console>", line 1, in <module>
TypeError: object list can't be used in 'await' expression

[pablogsal]

New changeset 9052f7a by Batuhan Taşkaya in branch 'master':
bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL (GH-18968)
9052f7a

[miss-islington]

New changeset ec8a973 by Miss Islington (bot) in branch '3.8':
bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL (GH-18968)
ec8a973

[vstinner]

This change caused python-gmpy2 tests to fail:
https://bugzilla.redhat.com/show_bug.cgi?id=1817710

Reproducer:
---

from __future__ import print_function, division
import doctest

filename = "doctest.txt"
with open(filename, "w") as fp:
    print("""
Test

====

    >>> all(x == 1 for x in [1, 1, 1])
    True
    """.strip(), file=fp)

result = doctest.testfile(filename, globs=globals())
print("result:", result)

---