Attempting to use class with both __enter__ & __exit__ undefined yields __exit__ attribute error

[ellingtonjp]

BPO	27100
Nosy	@rhettinger, @ncoghlan, @benjaminp, @ned-deily, @serhiy-storchaka, @1st1, @zhangyangyu, @TeamSpen210, @maggyero
PRs	bpo-39048: Look up __aenter__ before __aexit__ in the async with statement #17609
Files	with.patch fix_with_refleak.diff: Fix reference leak regexp-deprecated.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/1st1'
closed_at = <Date 2017-04-04.15:13:02.650>
created_at = <Date 2016-05-24.03:28:37.689>
labels = ['interpreter-core', 'type-bug', '3.7']
title = 'Attempting to use class with both __enter__ & __exit__ undefined yields __exit__ attribute error'
updated_at = <Date 2019-12-14.20:02:46.174>
user = 'https://bugs.python.org/ellingtonjp'

bugs.python.org fields:

activity = <Date 2019-12-14.20:02:46.174>
actor = 'maggyero'
assignee = 'yselivanov'
closed = True
closed_date = <Date 2017-04-04.15:13:02.650>
closer = 'ncoghlan'
components = ['Interpreter Core']
creation = <Date 2016-05-24.03:28:37.689>
creator = 'ellingtonjp'
dependencies = []
files = ['42962', '45604', '45618']
hgrepos = []
issue_num = 27100
keywords = ['patch']
message_count = 24.0
messages = ['266219', '266222', '266226', '266227', '266232', '277570', '277592', '277666', '277735', '281426', '281427', '281454', '281455', '281497', '281505', '281509', '281510', '281602', '281606', '281610', '281643', '291092', '291124', '358404']
nosy_count = 11.0
nosy_names = ['rhettinger', 'ncoghlan', 'benjamin.peterson', 'ned.deily', 'python-dev', 'serhiy.storchaka', 'yselivanov', 'xiang.zhang', 'Spencer Brown', 'ellingtonjp', 'maggyero']
pr_nums = ['17609']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27100'
versions = ['Python 3.6', 'Python 3.7']

[ellingtonjp]

Attempting to use class that has both __exit__() and __enter__() undefined as a context manager yields an AttributeError referring to __exit__:

>>> class A():
...     pass
...
>>> with A():
...     pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute '__exit__'

Knowing that the 'with' statement should first execute __enter__, this is unexpected.

The patch ensures the attribute error refers to __enter__() when both methods are undefined.

[zhangyangyu]

This seems to be designed. From PEP-343 it tells clearly:

If either of the relevant methods are not found
as expected, the interpreter will raise AttributeError, in the
order that they are tried (__exit__, __enter__).

But currently I don't find out why.

[rhettinger]

Nick, I've seen the come-up several times with people learning about context managers. The current error message seems cause confusion because it checks for __exit__ before __enter__, implying that the __enter__ is optional which it isn't. Do you have any objections testing for __enter__ first?

[benjaminp]

I don't completely understand. If you get an AttributeError for __enter__, will you think only an __enter__ is required?

[ncoghlan]

As Jonathan notes, the cause for confusion here is that we currently retrieve the attributes in the opposite order from the way we execute them, prompting the "Why is it looking for __exit__ first?", "Oh, __enter__ must be optional, so it didn't care that it was missing" train of thought.

If I recall correctly, rather than stemming from any particular design principle, the current behaviour is an artifact of the initial implementation that didn't use any dedicated opcodes: since CPython uses a stack-based VM, we needed __enter__ to be above __exit__ on the stack, and the most efficient way to achieve that was to do the attribute lookups in the reverse order of invocation.

When SETUP_WITH was introduced, that original method lookup ordering (and associated error message precedence) was preserved, even though the technical rationale for that ordering no longer applied.

Given that, I don't see any problem with switching the lookup order now, and I agree that should be less confusing, given that the most obvious explanation for getting an error message about __enter__ when both are missing is that the interpreter tries to call __enter__ before it tries to do anything with __exit__. (That intuitive explanation still isn't 100% accurate, but it's closer than thinking __enter__ is optional)

[rhettinger]

I don't see any problem with switching the lookup order now,
and I agree that should be less confusing,

Thanks Nick. I concur as well and will apply this.

Would it be reasonable to apply this to 3.6? IMO, it isn't a new feature, it is more of a usability bug due to an oversight.

[ncoghlan]

+1 for 3.6 from me - as you say, this is fixing a usability bug in the error handling rather than being a new feature.

[TeamSpen210]

Alternatively, SETUP_WITH could additionally check for the other method, and raise an AttributeError with a custom message mentioning both methods.

[ncoghlan]

Due to the expected semantics of AttributeError being relatively precise, changing to a protocol check rather than just changing the order we look up the protocol attributes would also involving changing from AttributeError to TypeError.

Since that's a potentially more disruptive change and just changing the order of the checks is likely sufficient to reduce confusion, the latter approach is preferable, at least for now (if folks are still getting confused by the time 3.7 rolls around, then that option would be worth reconsidering).

[python-dev]

New changeset 3aafb232f2db by Raymond Hettinger in branch '3.6':
Issue bpo-27100: With statement reports missing __enter__ before __exit__. (Contributed by Jonathan Ellington.)
https://hg.python.org/cpython/rev/3aafb232f2db

[rhettinger]

Thanks for the patch Jonathan. Nick, thanks for chiming in.