Unexpected "error: Slice index must be an integer or None"

[sametmax]

Despite having:

    class g():
    ...
        def __getitem__(self, index):
            # type: (Union[int, slice, Callable]) -> Union[IterableWrapper, Any]

Running mypy triggers "error: Slice index must be an integer or None" for g()[lambda x: x > 4:].

[elazarg]

You did not pass a callable to __getitem__. You pass a slice to __getitem__, but this slice is built with a lambda instead of an integer. You do not control the signature of slice.__init__.

[sametmax]

Oh, right, so we need to update the typeshed for slice ? Indeed, I can't imagine other people won't have this problem. Numpy comes to mind, since it uses slices containing tuples to mimic multi-dimensional slicing.

[elazarg]

I don't know. Passing a callable to slice instead of an integer is most likely an error:

elems[:x.count]  # oops, should be x.count()

[gvanrossum]

Maybe slice() needs to become generic?

…

--Guido (mobile)

[elazarg]

Sounds good to me. It might make existing annotation less safe in the short term, though (since slice will mean slice[Any]).

[JukkaL]

I think that somebody proposed making slice generic some time a ago, but it was rejected because we didn't have use case.

[elazarg]

Found it: python/typing#159

[elazarg]

Perhaps we should have Slice[T], and slice may be an alias for Slice[int].

[JukkaL]

That sounds reasonable to me, as potentially 90+% of slice objects have integer indices, and it would also be backward compatible.

[sametmax]

@elazarg: I'd go for your solution. Indeed, my need to pass a callable to the slice is a niche, and numpy slicing is not very comon, so this allow the best of both worlds. The reason I allow callables into slices is that I am creating an object you can slice to do the equivalent of itertools.dropwhile and takewhiles as a shorcut.

[elazarg]

(Just as a side note, there's a discussion in python-ideas of using slice[] as a slice constructor: https://mail.python.org/pipermail/python-ideas/2016-November/043630.html)

[shoyer]

I work on several projects in the scientific computing space that use slice objects with non-int arguments.

I don't like implicitly assuming that slice means a slice of integers, because this would be surprising to those of us who really want a generic slice object.

In NumPy, it's natural to pass NumPy integers into a slice, which do not subclass int (e.g., because they have fixed width), but otherwise are duck types that work almost exactly the same. The proper slice type for NumPy arrays would have elements of type SupportsIndex (i.e., values that implement __index__).

In pandas and xarray, slice objects can be used with non-integer values, e.g., for indexing along an axis with string labels. But there are no valid string strides, so instead we accept integer strides with the usual meaning. The fully specified valid slice type for a StringIndex would be something like Slice[str, str, int], where each field is presumed optional. At the very least, we need separate types for start/stop and step.

[gvanrossum]

So we should probably just bite the bullet and make Slice generic. It doesn't look like it'll be too hard. Elazar, do you want to be point on this?

[elazarg]

I am not entirely sure what is the decision.

Should we write the type of slice everywhere as Slice[int, int, int]?

[gvanrossum]

Hopefully it can just be Slice[int]? There shouldn't be too many places where this is needed. Or do you really want to support Slice[datetime, datetime, timedelta]? (The only example I can think of where the type of the difference isn't the same as the base value type.)