Type context not working for second argument of dict.get

[JukkaL]

The new signature for dict.get introduced in python/typeshed@6008b9d causes wrong types to be inferred for some arguments. Here is an example that demonstrates the issue and doesn't rely on the typeshed commit:

from typing import Any, Generic, overload, Optional, TypeVar, Union

_KT = TypeVar('_KT')
_VT_co = TypeVar('_VT_co', covariant=True)
_T = TypeVar('_T')

class D(Generic[_KT, _VT_co]):  # Mimics Dict
    @overload
    def get(self, k: _KT) -> Optional[_VT_co]: ...
    @overload
    def get(self, k: _KT, default: _T) -> Union[_VT_co, _T]: ...

def f(x: Any, y: D[int, D[int, Any]]) -> None:
    a = y.get(x, D())
    reveal_type(a)  # Union[t.D[builtins.int, Any], t.D*[builtins.None, builtins.None]]

The expected type is t.D[builtins.int, Any].

Similar example using typeshed (note that the commit that introduced the problem is not yet used in mypy master):

from typing import Any, Dict

def f(x: Any, y: Dict[int, Dict[int, Any]]) -> None:
    a = y.get(x, {})
    reveal_type(a)  # Union[builtins.dict[builtins.int, Any], builtins.dict*[builtins.None, builtins.None]]

The reason for the behavior is that mypy doesn't realize that type context should propagate from the return type to the argument somehow. Actually, it's unclear whether type context should generally work like this -- perhaps we should modify the signature of get instead.

[gvanrossum]

The first example doesn't even need the 1-arg overload. The covariance of the value type also seems unrelated.

But I'm not sure I understand what you think the behavior should be. The type of default is intentionally a different type variable than D's value type. So what is the type of D() passed as the default? Why should it conform to the value type?

I tried to rewrite the example using a separate assignment:

    d = D()
    a = y.get(x, d)

but this says Need type annotation for variable on the d = D() line. But somehow when I change to

    d = D()  # type: D
    a = y.get(x, d)

all is well. So it looks like there's something half-hearted being done to the D() call in your original version, where it is not quite given the type D[Any, Any]?

If you think there's simply a bug I'm happy to sit back and look at your fix to see if I can understand the issue that way. :-) (But your last sentence made it sound like you're not sure yourself?)

[JukkaL]

The explicit annotation works because D is a shorthand for D[Any, Any], so the None types get never inferred.

There are at least two ways in which we could fix this:

1. Somehow propagate information back from the union return type to the type context, with the goal of simplifying away the union in the return type. The rationale here is that it's preferable to have a non-union return than a union return type if we are free to choose either by substituting type variables suitably.
2. Update the second argument to get to have type Union[_VT_co, _T] and pick up the _VT_co context from there. I checked that this doesn't work yet, so this will also require a mypy change.

The first option sounds pretty complicated, so I'm planning to try the second option first.

[gvanrossum]

OK, now I see. It looks like the current state of Mapping.get() etc. causes lots of issues with real-world code, so I agree on the priority. Solution (2) sounds fine.