NotImplemented needs to be documented

[pkch]

BPO	15997
Nosy	@loewis, @terryjreedy, @bitdancer, @pkch
Dependencies	bpo-28785: Clarify the behavior of eq() returning NotImplemented

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-09-21.05:50:31.754>
labels = ['type-feature', 'docs']
title = 'NotImplemented needs to be documented'
updated_at = <Date 2017-01-22.04:54:48.308>
user = 'https://github.com/pkch'

bugs.python.org fields:

activity = <Date 2017-01-22.04:54:48.308>
actor = 'martin.panter'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2012-09-21.05:50:31.754>
creator = 'max'
dependencies = ['28785']
files = []
hgrepos = []
issue_num = 15997
keywords = []
message_count = 8.0
messages = ['170860', '170861', '170862', '170880', '170883', '170937', '170946', '171015']
nosy_count = 6.0
nosy_names = ['loewis', 'terry.reedy', 'Arfrever', 'r.david.murray', 'docs@python', 'max']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue15997'
versions = ['Python 3.2']

[pkch]

Quoting from http://docs.python.org/reference/datamodel.html#the-standard-type-hierarchy:

NotImplemented
This type has a single value. There is a single object with this value. This object is accessed through the built-in name NotImplemented. Numeric methods and rich comparison methods may return this value if they do not implement the operation for the operands provided. (The interpreter will then try the reflected operation, or some other fallback, depending on the operator.) Its truth value is true.

This is not a sufficient description of NotImplemented behavior. What does it mean "reflected operation" (I assume it is other.__eq__(self), but it needs to be clarified), and what does it mean "or some other fallback" (wouldn't developers need to know?). It also doesn't state what happens if the reflected operation or the fallback again return NotImplemented.

The rest of the documentation doesn't seem to talk about this either, despite several mentions of NotImplemented, with references to other sections.

This is particularly serious problem because Python's behavior changed in this respect not that long ago.

[loewis]

This must not be documented for NotImplemented, but for the operations itself. On the same page, it says

"There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does); rather, __lt__() and __gt__() are each other’s reflection, __le__() and __ge__() are each other’s reflection, and __eq__() and __ne__() are their own reflection."

So I'd say it's there already.

[pkch]

I agree about reflected operation, although the wording could be clearer ("will try reflected operation" is better worded as "will return the result of the reflected operation called on the swapped arguments".)

But what does it mean "or some other fallback"? And what if the reflected operation or the fallback again return NotImplemented or is actually not implemented. Is it somewhere else in the docs?

[loewis]

The main point is: it depends on the operation. NotImplemented is a way to signal that an operation is not implemented. It can be used for whatever you want to use it for. You can design to call an operation "foo", and, if NotImplemented is returned, call "bar" instead.

If you want to know how a specific operation performs its fallback, you have to look in the documentation of the specific operation.

As an example for a method where some other fallback is used, see

http://docs.python.org/py3k/library/abc.html#abc.ABCMeta.\_\_subclasshook__

[bitdancer]

The mention of NotImplemented in library/stdtypes cross references to the 'comparisions' section of the reference guide chapter on expressions. That might be a useful cross link in the datamodel section as well.

[terryjreedy]

The first three sentences are fine. The problem I have is with the 4th: 'may return' is rather vague. Such methods may raise TypeError instead (the old way), seemingly to the same effect. (See msg170936 in issue bpo-12067, which is about clarifying the comparisons section of the expression chapter.). So the 5th sentence could be misinterpreted wrongly to mean that NotImplemented is needed to get the alternate method try. (Since it is not, I wonder why it was added, since it complicates the internal logic of arithmetic and comparison ops.)

Perhaps a better place for any clarification on this point would be in 3.3 in the entry for __le__, etc.

[loewis]

Terry: can you propose an alternative wording? I don't think that the discussion of TypeError belongs into the description of NotImplemented, though.

IIUC, NotImplemented was added for performance reasons; returning a value is much faster than raising an exception. The return only requires an INCREF/DECREF pair, and a pointer identity test. The exception requires to create a new object, possibly a traceback, and a subclass check on exception matching.

[terryjreedy]

I was wrong about TypeError; forget that. The seeming equivalent to 'return NotImplement' is not defining the method. My confusion stemed from the fact that NotImplemented returned from special methods gets intercepted and converted to TypeError exceptions by the operator functions that call them -- unless there is an alternate path *and* the alternate does not also return NotImplemented.

Changing "will try reflected operation" to "will return the result of the reflected operation called on the swapped arguments" is wrong as the latter is not completely correct. If the fallback also returns NotImplemented, it is ignored and TypeError is raised the same as if the fallback did not exist. But as Martin said, the details belong in operator documentation.

I would like to more closely parallel the None entry, which has

"It is used to signify the absence of a value in many situations, e.g. it is returned from functions that don’t explicitly return anything."

by replacing

"Numeric methods and rich comparison methods may return this value if they do not implement the operation for the operands provided. (The interpreter will then try the reflected operation, or some other fallback, depending on the operator.)"

with

"It is used to signify that a method does not implement the operation requested for the operands provided. For example, the special methods for arithmetic and and rich comparison may return NotImplemented, and when they do, the interpreter will try the reversed or reflected operation."