typing forward references and module attributes

[mjpieters]

BPO	26477
Nosy	@gvanrossum, @mjpieters, @agronholm, @ztane, @ilevkivskyi

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-09-27.22:26:39.856>
created_at = <Date 2016-03-03.18:07:47.335>
labels = ['library']
title = 'typing forward references and module attributes'
updated_at = <Date 2016-09-27.22:26:39.854>
user = 'https://github.com/mjpieters'

bugs.python.org fields:

activity = <Date 2016-09-27.22:26:39.854>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2016-09-27.22:26:39.856>
closer = 'gvanrossum'
components = ['Library (Lib)']
creation = <Date 2016-03-03.18:07:47.335>
creator = 'mjpieters'
dependencies = []
files = []
hgrepos = []
issue_num = 26477
keywords = []
message_count = 7.0
messages = ['261172', '261173', '261174', '261175', '261176', '261177', '277561']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'mjpieters', 'alex.gronholm', 'ztane', 'levkivskyi']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue26477'
versions = ['Python 3.5', 'Python 3.6']

[mjpieters]

Forward references to a module can fail, if the module doesn't yet have the required object. The "forward references" section names circular dependencies as one use for forward references, but the following example fails:

$ cat test/__init__.py
from .a import A
from .b import B
$ cat test/a.py
import typing
from . import b

class A:
    def foo(self: 'A', bar: typing.Union['b.B', None]):
        pass
$ cat test/b.py
import typing
from . import a

class B:
    def spam(self: 'B', eggs: typing.Union['a.A', None]):
        pass
$  bin/python -c 'import test'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/__init__.py", line 1, in <module>
    from .a import A
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/a.py", line 2, in <module>
    from . import b
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/b.py", line 4, in <module>
    class B:
  File "/Users/mjpieters/Development/venvs/stackoverflow-3.5/test/b.py", line 5, in B
    def spam(self: 'B', eggs: typing.Union['a.A', None]):
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 537, in __getitem__
    dict(self.__dict__), parameters, _root=True)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 494, in __new__
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 494, in <genexpr>
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 185, in __subclasscheck__
    self._eval_type(globalns, localns)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/typing.py", line 172, in _eval_type
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
AttributeError: module 'test.a' has no attribute 'A'

The forward reference test fails because only NameError exceptions are caught, not AttributeError exceptions.

[mjpieters]

Sorry, that should have read "the forward references section of PEP-484".

The section uses this example:

# File models/a.py
from models import b
class A(Model):
    def foo(self, b: 'b.B'): ...

# File models/b.py
from models import a
class B(Model):
    def bar(self, a: 'a.A'): ...

# File main.py
from models.a import A
from models.b import B

which doesn't fail because the forward references are not being tested until after all imports have completed; creating a Union however triggers a subclass test between the different types in the union.

[mjpieters]

A temporary work-around is to use a function to raise a NameError exception when the module attribute doesn't exist yet:

def _forward_A_reference():
    try:
        return a.A
    except AttributeError:
        # not yet..
        raise NameError('A')

class B:
    def spam(self: 'B', eggs: typing.Union['_forward_A_reference()', None]):
        pass

[agronholm]

I wonder why they forward references are evaluated *at all* at this point. Seems senseless to me. This should be the job of the static type checker or the get_type_hints() function.

[ztane]

Indeed, the assumption is be that if a string is used, it is used there because the actual thing cannot be referenced by name at that point. Then trying to evaluate it at all would be an optimization in only those cases where it is used incorrectly / needlessly.

[mjpieters]

I wonder why they forward references are evaluated *at all* at this point.

The Union type tries to reduce the set of allowed types by removing any subclasses (so Union[int, bool] becomes Union[int] only). That's all fine, but it should not at that point fail if a forward reference is not available yet.

Arguably, the except NameError there should be converted to a except Exception, since forward references are supposed to be *a valid Python expression [...] and it should evaluate without errors once the module has been fully loaded.* (from the PEP); anything goes, and thus any error goes until the module is loaded.

[gvanrossum]

Fixed by 09cc43df4509.