%c, %o, %x, %X accept non-integer values instead of raising an exception

[ethanfurman]

BPO	19995
Nosy	@gvanrossum, @warsaw, @rhettinger, @terryjreedy, @mdickinson, @pitrou, @vstinner, @larryhastings, @ericvsmith, @ezio-melotti, @bitdancer, @skrah, @ethanfurman, @serhiy-storchaka, @MojoVampire
Files	issue19995.stoneleaf.01.patch issue19995.stoneleaf.02.patch issue19995.stoneleaf.03.patch issue19995.stoneleaf.04.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ethanfurman'
closed_at = <Date 2014-03-21.20:48:10.612>
created_at = <Date 2013-12-16.10:29:04.037>
labels = ['type-bug']
title = '%c, %o, %x, %X accept non-integer values instead of raising an exception'
updated_at = <Date 2015-06-22.19:46:27.672>
user = 'https://github.com/ethanfurman'

bugs.python.org fields:

activity = <Date 2015-06-22.19:46:27.672>
actor = 'barry'
assignee = 'ethan.furman'
closed = True
closed_date = <Date 2014-03-21.20:48:10.612>
closer = 'ethan.furman'
components = []
creation = <Date 2013-12-16.10:29:04.037>
creator = 'ethan.furman'
dependencies = []
files = ['33284', '33286', '33289', '33414']
hgrepos = []
issue_num = 19995
keywords = ['patch']
message_count = 66.0
messages = ['206290', '206293', '206303', '206321', '206322', '206323', '206325', '206326', '206331', '206332', '206333', '206336', '206338', '206339', '206341', '206363', '206364', '206367', '206368', '206369', '206371', '206378', '206379', '206381', '206382', '206383', '206409', '206465', '206467', '206715', '206858', '207049', '207069', '207098', '207099', '207100', '207101', '207114', '207115', '207117', '207303', '207378', '207379', '207380', '207382', '207383', '207384', '207385', '207386', '207387', '207389', '207393', '207835', '207895', '207929', '207940', '207966', '208099', '210417', '214104', '214149', '214155', '214365', '214411', '245643', '245644']
nosy_count = 17.0
nosy_names = ['gvanrossum', 'barry', 'rhettinger', 'terry.reedy', 'mark.dickinson', 'pitrou', 'vstinner', 'larry', 'eric.smith', 'ezio.melotti', 'Arfrever', 'r.david.murray', 'skrah', 'ethan.furman', 'python-dev', 'serhiy.storchaka', 'josh.r']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19995'
versions = ['Python 3.4', 'Python 3.5']

[ethanfurman]

Using Enum to illustrate:

--> class Grade(enum.Enum):
...   A = 4
...   B = 3
...   C = 2
...   D = 1
...   F = 0
...   def __index__(self):
...     return self._value_

--> ['miserable'][Grade.F]
'miserable'

    --> '%x' % Grade.F
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: %x format: a number is required, not Grade

--> hex(Grade.F)
'0x0'

I suggest that hex() and oct() have the same check that %x and %o do so that non-numbers are not representable as hex and octal.

While we're at it, we should do the same for bin(). Are there any others?

I'll create a patch once we have a decision on which way to solve this issue.

[vstinner]

Calls:

• hex()/oct() => PyNumber_ToBase() => PyNumber_Index().
• PyUnicode_Format() => mainformatlong() => PyNumber_Long()

I never understood the difference between "long" (int method) and "index" (index method). Is the difference on the behaviour of floating point numbers?

[ericvsmith]

It seems to me that by giving it an __index__ method, you're saying it can be used as an integer. It's not surprising to me that hex(), oct(), and bin() would work with a Grade.F object. If anything, I'd say that more places should use __index__ than currently do.

[ethanfurman]

Victor Stinner commented:
-------------------------

I never understood the difference between "long" (int method)
and "index" (index method). Is the difference on the behaviour
of floating point numbers?

__index__ was originally added so that non-int integers, such as NumPy's int16, int32, etc, integer types could be used as indices and slices.

Now it means "if your type can produce a lossless integer, use __index__", which is why float and similar types don't define it.

The current meaning is unfortunate in that it is possible to want a type that can be used as an index or slice but that is still not a number, and in fact won't be used as a number in any scenario _except_ bin(), hex(), and oct().

It seems to me that by having those three functions check that the argument is a number, and bailing if it is not, is a decent way to ensure consistency.

One question I do have, since I don't have NumPy installed, is what happens with:

--> "NumPy's int's work here? %x" % uint16(7)

[vstinner]

$ python
Python 2.7.5 (default, Nov 12 2013, 16:18:42) 
>>> import numpy
>>> hex(numpy.uint16(257))
'0x101'
>>> "%x" % numpy.uint16(257)
'101'
>>> x=numpy.uint16(257)
>>> x.__int__()
257
>>> x.__index__()
257

[skrah]

Ethan Furman <report@bugs.python.org> wrote:

The current meaning is unfortunate in that it is possible to want a type that
can be used as an index or slice but that is still not a number, and in fact
won't be used as a number in any scenario _except_ bin(), hex(), and oct().

memoryview, struct and probably also array.array accept __index__.

[ethanfurman]

Did I mention __index__ is an unfortunate name for the current trend for this method?

Stefan Krah commented:
----------------------

memoryview, struct and probably also array.array accept __index__.

When you say "accept __index__" do you mean for use as indices, or for use as values in the data structure itself?

[skrah]

Did I mention __index__ is an unfortunate name for the current trend for this method?

Yes, but it's probably too late to change that now. Also, a fully precise
name would be something like:

__to_int_exact_iff_object_has_integer_nature__ :)

When you say "accept __index__" do you mean for use as indices, or for use as
values in the data structure itself?

The latter, see Lib/test/test_buffer.py:2489 .

[ethanfurman]

Hmmm...

Well, much as I hate to say it, it's sounding like the correct solution here is to have %o and %x work when __index__ is available, instead of the other way around. :(

.format is not an issue because one must specify one's own if inheriting from object.

So the complete list of spcecifiers then is d, i, o, u, U, and c [1], and they should work if __index__ works.

Are we in agreement?

[1] http://docs.python.org/dev/library/stdtypes.html#printf-style-string-formatting

[vstinner]

Are we in agreement?

Start maybe on writing unit tests :-)

IMO all int-like objects should behave the same. I don't see any good
reason why hex(value) would succeed whereas "%x" % value fails. Both
should succeed (or both should fail).

[serhiy-storchaka]

> Did I mention __index__ is an unfortunate name for the current trend for
> this method?
Yes, but it's probably too late to change that now. Also, a fully precise
name would be something like:

__to_int_exact_iff_object_has_integer_nature__ :)

Perhaps in future (may be in 4.0) __index__ should be renamed to __int__ and
__int__ to __trunc__.

[ericvsmith]

Yes, I think adding __index__ to d, i, o, u, U, and c is the correct thing to do here.

[gvanrossum]

Not so fast. Currently, even in Python 3, '%d' % 3.14 returns '3'. "Fixing" this will likely break a huge amount of code.

[gvanrossum]

Also (the tracker email interface swallowed this):

it is possible to want a type that can be used as an index or slice but that is still not a number

I'm sorry, but this requirement is absurd. An index *is* a number. You
have to make up your mind. (I know, in the context of the example that
started this, this is funny, but I still stand by it.)

---

Finally, the correct name should perhaps have been __integer__ but I don't see enough reason to change it now.