Would it make sense to add ShapeError and BroadcastingError

[NeilGirdhar]

In 2011, one of the NumPy authors proposed the exceptions ShapeError, BroadcastingError, and ConvergenceError here. Would it make sense to consider these for the array API? They could inherit from ValueError for ease of porting?

[rgommers]

Thanks for the suggestion @NeilGirdhar. I'd be +1 for the first two for NumPy. For the array API standard I'm less sure - we've shied away from specifying any specific exceptions or what should or shouldn't happen for incorrect/undefined usage of the API. The problems are that (a) it's impossible to get exhaustive or even uniform coverage or draw a line somewhere, and (b) it's pretty hard to align on this kind of thing.

E.g., for invalid shape= input, that can already raise ValueError, TypeError or NameError within numpy.ndarray.reshape alone. And what's an invalid type for one library may not be so for another (spec say tuple of ints, number accepts list of ints).

So I'd be inclined to not go into this.

[NeilGirdhar]

And what's an invalid type for one library may not be so for another (spec say tuple of ints, number accepts list of ints).

One benefit to pushing libraries to accept the same thing is that code that's written and tested with one library will probably work for another. If code is written and tested with library A that accepts a list of ints, and then someone tries to use that code with library B, which doesn't, then they have to go through the trouble of bug reports.

[rgommers]

All libraries are implementing a superset of the standard though - our answer to that is that there's one strictly conforming, minimal implementation (numpy.array_api) that can be used to test whether or not you're using library-specific constructs.

[NeilGirdhar]

Okay, thanks for explaining!

[asmeurer]

For the spec, I'd say we should specify a specific exception if it benefits downstream consumer library authors so that they can catch a uniform exception. If on the other hand, the point of a specific exception subclass is just to make things more legible for users, then I can see the benefit of an individual library like NumPy implementing it, but I don't see why that needs to be specified in the standard.

With that being said, thinking about this, there might be a benefit to specifying that the broadcasting functions should at least raise ValueError (or a subclass of it), if not adding BroadcastingError as you suggest. There's no clear way in the standard to check if two arrays are broadcast compatible. The best you can do is call broadcast_arrays, but it doesn't specify what exception is raised when they aren't https://data-apis.org/array-api/draft/API_specification/generated/array_api.broadcast_arrays.html. The spec is also missing broadcast_shapes (probably because it is still very new in NumPy), although that would also raise an exception on failure.

ShapeError (and AxisError) seem to be more about user error. Library code can easily check the actual condition rather than relying on an exception, and in most cases the most appropiate thing is to let the exception bubble up to the user. Also note that AxisError is a subclass of IndexError in NumPy, which is already required in a few places https://data-apis.org/array-api/draft/search.html?q=indexerror (perhaps all out of bounds axis arguments should require IndexError or a subclass for consistency?)

For ConvergenceError, what functions would this specifically apply to? This seems like something that is not appropriate to the array API, because it would tie a function to a given implementation. But we generally want libraries to be free to use different algorithms or implementations for any given function.

[rgommers]

With that being said, thinking about this, there might be a benefit to specifying that the broadcasting functions should at least raise ValueError (or a subclass of it), if not adding BroadcastingError as you suggest. There's no clear way in the standard to check if two arrays are broadcast compatible.

Right now it's except Exception: I think. It'd be nice of course if that could change to ValueError, however a quick check shows that NumPy and JAX indeed yield ValueError but PyTorch doesn't:

>>> import torch
>>> torch.broadcast_to(torch.ones((4, 3)), (2,5,2))
Traceback (most recent call last):
  Cell In[5], line 1
    torch.broadcast_to(torch.ones((4, 3)), (2,5,2))
RuntimeError: The expanded size of the tensor (2) must match the existing size (3) at non-singleton dimension 2.  Target sizes: [2, 5, 2].  Tensor sizes: [4, 3]

>>> torch.broadcast_shapes((4, 3), (2,5,2))
Traceback (most recent call last):
  Cell In[6], line 1
    torch.broadcast_shapes((4, 3), (2,5,2))
  File ~/mambaforge/envs/torch/lib/python3.11/site-packages/torch/functional.py:126 in broadcast_shapes
    raise RuntimeError("Shape mismatch: objects cannot be broadcast to a single shape")
RuntimeError: Shape mismatch: objects cannot be broadcast to a single shape

This would be quite a pain to get changed, I don't think it'll happen in practice - it's too low-prio to work on, and it'd be bc-breaking if it were to change. It's also not even completely clear-cut that RuntimeError is wrong.

So I think I'll stay with my opinion that we should avoid specifying any exception types. At most we can recommend a type for some common causes of exceptions.

[rgommers]

There is exactly one exception to that rule now: __dlpack__ has a Raises section that says _"should raise a BufferError". xref gh-498, where it discusses why SHOULD rather than MUST - most libraries don't behave like this, hence it's a recommendation only.

We should probably add a more explicit discussion of exceptions and how to deal with them somewhere.

[NeilGirdhar]

Right now it's except Exception:

That's fairly universally considered a code smell, so I agree with you that would be good to improve the situation.

It's also not even completely clear-cut that RuntimeError is wrong.

Why do you say that? Isn't RuntimeError for things that should never happen? How can library know that something should never happen?

This would be quite a pain to get changed, I don't think it'll happen in practice - it's too low-prio to work on,

Fair point. You could always pepper your standard with "recommended exceptions" so that at least implementers have some indication of what they should be doing?

[leofang]

Isn't RuntimeError for things that should never happen?

That would be AssertionError, not RuntimeError.

[rgommers]

RuntimeError is kinda like that, but it's also a catch-all for things that don't clearly fit elsewhere I'd say. For this one it's not obvious to me - it's not about the values in the array, but another property. There is also ongoing work to have static typing support for array dimensions and dtype, so also TypeError could be defensible here.

[NeilGirdhar]

That would be AssertionError, not RuntimeError.

I don't think so? AssertionError is for assertions, which can be disabled. Usually, it's raised in code by assert and never caught, or raised in tests and caught by the test library.

RuntimeError is kinda like that, but it's also a catch-all for things that don't clearly fit elsewhere I'd say.

Okay, that makes sense.

There is also ongoing work to have static typing support for array dimensions and dtype, so also TypeError could be defensible here.

Good point. But isn't that all the more reason to make a decision and provide some guidance? Otherwise, libraries are going to make guesses and code written with one won't work with another.

[asmeurer]

The docs say RuntimeError is just a catchall for exceptions that don't fit anywhere else https://docs.python.org/3/library/exceptions.html#RuntimeError:

Raised when an error is detected that doesn’t fall in any of the other categories. The associated value is a string indicating what precisely went wrong.

PyTorch uses RuntimeError for almost everything for technical reasons, but it seems like it's usually the wrong one based on the standard Python exception meanings. But that would be an upstream issue in pytorch to fix. I don't know if it necessarily means it's something that needs to be addressed in the standard.

[NeilGirdhar]

PyTorch uses RuntimeError for almost everything for technical reasons, but it seems like it's usually the wrong one based on the standard Python exception meanings. But that would be an upstream issue in pytorch to fix. I don't know if it necessarily means it's something that needs to be addressed in the standard.

I understand if this is low priority, but isn't the main point of the standard so that people can write code that works on all the libraries?

[rgommers]

@NeilGirdhar do you actually have real-world use cases for this? I think normally the broadcasting-related exception just bubbles up to the user, because the input shape is wrong and needs fixing. I think any try-except is a sign of a problem.

An analogy is dtype handling. There we may have special paths, so have a function can_cast. And then there's if can_cast(...); ...; else: ... usage. I can't think of cases though where you'd want can_broadcast(x, y).