asyncio.Task doesn't propagate CancelledError() exception correctly.

[pagliariccim]

BPO	45390
Nosy	@gvanrossum, @asvetlov, @cjerdonek, @serhiy-storchaka, @1st1, @graingert, @bensimner
PRs	bpo-45390: Propagate CancelledError's message from cancelled task to its awaiter #31383
Files	task_bug.py task_bug.py task_bug.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-02-22.09:46:40.916>
created_at = <Date 2021-10-06.11:30:42.256>
labels = ['3.11', 'type-bug', 'expert-asyncio']
title = "asyncio.Task doesn't propagate CancelledError() exception correctly."
updated_at = <Date 2022-02-23.18:04:42.655>
user = 'https://bugs.python.org/pagliariccim'

bugs.python.org fields:

activity = <Date 2022-02-23.18:04:42.655>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2022-02-22.09:46:40.916>
closer = 'asvetlov'
components = ['asyncio']
creation = <Date 2021-10-06.11:30:42.256>
creator = 'pagliaricci.m'
dependencies = []
files = ['50328', '50333', '50334']
hgrepos = []
issue_num = 45390
keywords = ['patch']
message_count = 20.0
messages = ['403294', '403297', '403350', '403360', '403521', '403530', '403531', '403532', '403533', '403570', '413392', '413395', '413397', '413679', '413779', '413794', '413808', '413830', '413836', '413838']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'asvetlov', 'chris.jerdonek', 'serhiy.storchaka', 'yselivanov', 'graingert', 'bjs', 'pagliaricci.m']
pr_nums = ['31383']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45390'
versions = ['Python 3.11']

[pagliariccim]

I've spotted a little bug in how asyncio.CancelledError() exception is propagated inside an asyncio.Task.

Since python 3.9 the asyncio.Task.cancel() method has a new 'msg' parameter, that will create an asyncio.CancelledError(msg) exception incorporating that message.

The exception is successfully propagated to the coroutine the asyncio.Task is running, so the coroutine successfully gets raised an asyncio.CancelledError(msg) with the specified message in asyncio.Task.cancel(msg) method.

But, once the asyncio.Task is cancelled, is impossible to retrieve that original asyncio.CancelledError(msg) exception with the message, because it seems that *a new* asyncio.CancelledError() [without the message] is raised when asyncio.Task.result() or asyncio.Task.exception() methods are called.

I have the feeling that this is just wrong, and that the original message specified in asyncio.Task.cancel(msg) should be propagated even also asyncio.Task.result() is called.

I'm including a little snippet of code that clearly shows this bug.

I'm using python 3.9.6, in particular:
Python 3.9.6 (default, Aug 21 2021, 09:02:49)
[GCC 10.2.1 20210110] on linux

[bensimner]

This seems to be present in both the Python implementation as well as the accelerated C _asyncio module.

It looks like that when a Task awaits a cancelled future,
the task itself is cancelled but the cancellation message is not propagated to the task.
https://github.com/python/cpython/blob/main/Lib/asyncio/tasks.py#L242

[cjerdonek]

But, once the asyncio.Task is cancelled, is impossible to retrieve that original asyncio.CancelledError(msg) exception with the message, because it seems that *a new* asyncio.CancelledError() [without the message] is raised when asyncio.Task.result() or asyncio.Task.exception() methods are called.

You say it's "impossible", but isn't the message accessible via the exception chain (and visible in the traceback)? One benefit of not duplicating the message on the internal call to cancel() is that it makes it easier to pinpoint which CancelledError object is associated with the user's call to cancel(), and which is associated with the call done by asyncio internals, which is a different cancellation. Another benefit is that it prevents info from being duplicated in the traceback.

[graingert]

afaik this is intentional https://bugs.python.org/issue31033

[pagliariccim]

OK, I see your point.
But I still can't understand one thing and I think it's very confusing:

1. if you see my example, inside the job() coroutine, I get correctly
   cancelled with an `asyncio.CancelledError` exception containing my message.
2. Now: if I re-raise the asyncio.CancelledError as-is, I lose the message,
   if I call the `asyncio.Task.exception()` function.
3. If I raise a *new* asyncio.CancelledError with a new message, inside the
   job() coroutine's `except asyncio.CancelledError:` block, I still lose the
   message if I call `asyncio.Task.exception()`.
4. But if I raise another exception, say `raise ValueError("TEST")`, always
   from the `except asyncio.CancelledError:` block of the job() coroutine, I
   *get* the message!
   I get `ValueError("TEST")` by calling `asyncio.Task.exception()`, while I
   don't with the `asyncio.CancelledError()` one.

Is this really wanted? Sorry, but I still find this a lot confusing.
Shouldn't it be better to return from the asyncio.Task.exception() the
old one (containing the message) ?
Or, otherwise, create a new instance of the exception for ALL the
exception classes?

Thank you for your time,
My Best Regards,

M.

On Thu, Oct 7, 2021 at 10:25 AM Thomas Grainger <report@bugs.python.org>
wrote:

Thomas Grainger <tagrain@gmail.com> added the comment:

afaik this is intentional https://bugs.python.org/issue31033

----------
nosy: +graingert

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45390\>

---

[cjerdonek]

2. Now: if I re-raise the asyncio.CancelledError as-is, I lose the message,
   if I call the asyncio.Task.exception() function.

Re-raise asyncio.CancelledError where? (And what do you mean by "re-raise"?) Call asyncio.Task.exception() where? This isn't part of your example, so it's not clear what you mean exactly.

[pagliariccim]

Chris,
I'm attaching to this e-mail the code I'm referring to.
As you can see, in line 10, I re-raise the asyncio.CancelledError exception
with a message "TEST".
That message is lost, due to the reasons we've talked about.

My point is that, if we substitute that line 10, with the commented line
11, and we comment the line 10, so we raise a ValueError("TEST") exception,
as you can see, the message "TEST" is NOT LOST.
I just find this counter-intuitive, and error-prone.

AT LEAST should be very well specified in the docs.

Regards,
M.

On Sat, Oct 9, 2021 at 2:51 PM Chris Jerdonek <report@bugs.python.org>
wrote:

Chris Jerdonek chris.jerdonek@gmail.com added the comment:

2. Now: if I re-raise the asyncio.CancelledError as-is, I lose the
   message,
   if I call the asyncio.Task.exception() function.

Re-raise asyncio.CancelledError where? (And what do you mean by
"re-raise"?) Call asyncio.Task.exception() where? This isn't part of your
example, so it's not clear what you mean exactly.

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45390\>

---

[cjerdonek]

I still don't see you calling asyncio.Task.exception() in your new attachment...

[pagliariccim]

Chris,
ok, I have modified the snippet of code to better show what I mean.
Still here, the message of the CancelledError exception is lost, but if I
comment line 10, and uncomment line 11, so I throw a ValueError("TEST"),
that "TEST" string will be printed, so the message is not lost.
Again, I just find this behavior very counter-intuitive, and should be VERY
WELL documented in the docs.

Thanks,
M.

On Sat, Oct 9, 2021 at 3:06 PM Chris Jerdonek <report@bugs.python.org>
wrote:

Chris Jerdonek <chris.jerdonek@gmail.com> added the comment:

I still don't see you calling asyncio.Task.exception() in your new
attachment...

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45390\>

---