bpo-43751: Fix anext() bug where it erroneously returned None

[sweeneyde]

https://bugs.python.org/issue43751

[pablogsal]

There are a bunch of problems in the CI. The main one is:

======================================================================
ERROR: test_async_gen_anext (test.test_asyncgen.AsyncGenAsyncioTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/cpython/cpython/Lib/test/test_asyncgen.py", line 401, in test_async_gen_anext
    res = self.loop.run_until_complete(consume())
  File "/Users/runner/work/cpython/cpython/Lib/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/Users/runner/work/cpython/cpython/Lib/test/test_asyncgen.py", line 399, in consume
    results.append(await anext(g, 'buckle my shoe'))
TypeError: anext() argument was not async iterable.

----------------------------------------------------------------------

and there is also some trailing whitespace. Could you check these?

[sweeneyde]

This should probably have been a draft PR. I wrote a test for the erroneous None returns, then fixed it. And then I applied the same test to async iterables that are not generators, and that is still failing.

[sweeneyde]

This should be ready for review now.

[jab]

Thanks for catching this!

[gvanrossum]

Side comment on the PR title: please don't use "should no longer". Say something like "Fix anext() bug where it erroneously returned None".

[gvanrossum]

I'm requesting that someone else reviews this instead of me. @pablogsal? @1st1?

[pablogsal]

This can raise, you need to check for errors and propagate accordingly

[pablogsal]

Also, can you add a test that checks this code path?

[sweeneyde]

I added the check, but I couldn't figure out how to make a test to check the code path, since awaitable is always a coroutine, so getter points to this in genobject.c:

static PyObject *
coro_await(PyCoroObject *coro)
{
    PyCoroWrapper *cw = PyObject_GC_New(PyCoroWrapper, &_PyCoroWrapper_Type);
    if (cw == NULL) {
        return NULL;
    }
    Py_INCREF(coro);
    cw->cw_coroutine = coro;
    _PyObject_GC_TRACK(cw);
    return (PyObject *)cw;
}

[pablogsal]

You need to add a test to the _testcapi module, but this is going to be a bit too much for just this, so is fine if we don't add such a test.

[pablogsal]

Wait, isn't this something you can get with a custom __await__ in a class?
 

[sweeneyde]

_PyCoro_GetAwaitableIter calls am_await and ensures it's iterable; this if block is only for coroutines.

[sweeneyde]

test_anext_bad_await covers __await__ methods that raise

[pablogsal]

👍

[pablogsal]

I think this is leaking awaitable.

[pablogsal]

You need to decrement awaitable once you are done with it

[sweeneyde]

./python -m test test_asyncgen -R3:3 now passes

[pablogsal]

Thanks for the PR @sweeneyde