Add AutoNumberedEnum to stdlib

[johnthagen]

BPO	26988
Nosy	@warsaw, @rhettinger, @vstinner, @ethanfurman, @vedgar, @Vgr255, @johnthagen, @kennethreitz
Files	issue26988.stoneleaf.01.patch issue26988.stoneleaf.02.patch issue26988.stoneleaf.03.patch issue26988.stoneleaf.05.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ethanfurman'
closed_at = <Date 2016-08-20.07:02:02.617>
created_at = <Date 2016-05-10.01:34:31.690>
labels = ['type-feature', 'library']
title = 'Add AutoNumberedEnum to stdlib'
updated_at = <Date 2016-08-20.07:02:02.616>
user = 'https://github.com/johnthagen'

bugs.python.org fields:

activity = <Date 2016-08-20.07:02:02.616>
actor = 'ethan.furman'
assignee = 'ethan.furman'
closed = True
closed_date = <Date 2016-08-20.07:02:02.617>
closer = 'ethan.furman'
components = ['Library (Lib)']
creation = <Date 2016-05-10.01:34:31.690>
creator = 'John Hagen'
dependencies = []
files = ['43694', '43743', '43972', '43986']
hgrepos = []
issue_num = 26988
keywords = ['patch']
message_count = 62.0
messages = ['265214', '265215', '265216', '265218', '265237', '269476', '269521', '270039', '270048', '270049', '270051', '270054', '270056', '270071', '270088', '270107', '270117', '270137', '270142', '270148', '270151', '270199', '270201', '270212', '270219', '270238', '270523', '270524', '271811', '271855', '271868', '272071', '272874', '272876', '272885', '272901', '272932', '272941', '272951', '272952', '272971', '272982', '273006', '273039', '273040', '273064', '273065', '273076', '273084', '273092', '273103', '273105', '273109', '273118', '273125', '273126', '273127', '273131', '273134', '273145', '273154', '273188']
nosy_count = 11.0
nosy_names = ['barry', 'rhettinger', 'vstinner', 'eli.bendersky', 'ethan.furman', 'python-dev', 'veky', 'abarry', 'John Hagen', 'David Hagen', 'kennethreitz']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26988'
versions = ['Python 3.6']

[johnthagen]

I suggest that the AutoNumberedEnum be added to the standard library for the following reasons:

1. Provides a fundamental tool for defining a unique, abstract set of coupled members
2. Avoids boilerplate @enum.unique for a very common use case of enumerations
3. The code already exists in the Python documentation, so it has been vetted at some level

The AutoNumberedEnum also allows the developer to make a clearer distinction
between enumerations whose values have special meaning and those that do not.

Consider:

@enum.unique
class Color(enum.Enum):
    red = 1
    blue = 2
    green = 3

@enum.unique
class Shape(enum.Enum):
    """Member values denote number of sides."""
    circle = 1
    triangle = 3
    square = 4

With AutoNumberedEnum it's possible to better express the intent that
the value of Color members does not hold special meaning, while
Shape members do:

class Color(enum.AutoNumberedEnum):
    red = ()
    blue = ()
    green = ()

@enum.unique
class Shape(enum.Enum):
    """Member values denote number of sides."""
    circle = 1
    triangle = 3
    square = 4

For enumerations with many members (10s), there becomes a maintenance
issue when inserting new enumerations into the list:

@enum.unique
class Color(enum.Enum):
    aquamarine = 1
    blue = 2
    fushia = 3
    # inserting a member here (perhaps because it's clearest to keep these in alphabetic order)
    # results in having to increment all following members
    ...
    green = 40
    red = 41

Most other languages have support for naming enumerations
without explicitly declaring their values (albeit with
specialized syntax that makes it cleaner):

C++: http://en.cppreference.com/w/cpp/language/enum
C#: https://msdn.microsoft.com/en-us/library/sbbt4032.aspx
Java: https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html
Rust: https://doc.rust-lang.org/book/enums.html

Currently, a developer can copy the code from the Python docs into
his or her project, or add a dependency on aenum. I would argue
that it belongs in the standard library.

If there are objections to it being too magical, I would argue
it's already spelled out in the docs, so it's being prescribed
as a solution already. I think it should be very clear when you
derive from "AutoNumberedEnum" what is going on.

The code is very simple, so I would
hope maintenance would not be difficult.

[ethanfurman]

On the other hand, if you use aenum you can do:

class Color(AutoNumber):
    red
    green
    blue

And isn't that better*? ;)

• For those in danger of swallowing their tongue over the magic involved: The magic is turned off as soon as any descriptor is defined (property, function, etc.).

[warsaw]

Wow Ethan, that's quite interesting. I'm not sure if I like it or not. :)

It's better than having to assign a dummy value to the enum items, and if we did require that, I'd suggest just throwing away the values for autonumbering.

But not having to specify any values at all is probably better, although it does look weird!

/me goes to look at the magic you're using...

[ethanfurman]

It's in _EnumDict.__getitem__; there's some duplication in __setitem__ for supporting Python 2 (although with 2 you have to use either the empty tuple, or some other handy thing that may go in the __doc__ attribute, for example).

[johnthagen]

@Ethan, I hadn't tried to use the aenum AutoNumberEnum that way, but I agree with Barry that I like it. To me that is the ideal case we should shoot for as I think it's the best* long term and deviate only if practical concerns prevent it.

So I am +1 for empty member assignment and if that is rejected, +1 for = () assignment as at least it is a big step forward. I feel both solutions already have some "magic", so would lean toward the one that leads to the least amount of boilerplate.

As for the empty assignment, I have played around with something similar before and will throw out one con for it: static analyzers get really confused. PyCharm, for example, thinks this is has unresolved references in it:

class Color(AutoNumberEnum):
    red
    green
    blue

But the counter point is that if this is in the stdlib, static analyzer authors are much more likely to add a special case for it than if in a "non-official" third party package (PyCharm example: https://youtrack.jetbrains.com/issue/PY-19150).

Another piece of evidence to support inclusion is that Python already provides specialized Enum subclasses (like IntEnum). I've written a lot of Python code that uses Enums and haven't personally needed IntEnum yet, but would have used an AutoEnum many, many times.

• I am assuming here that a true "enum" keyword is out of the question at this point for Python, which would probably be even better.

[johnthagen]

@Ethan/Barry what needs to be done now to accept or reject this for Python 3.6? Should I propose it onto python-dev? If accepted, would be nice to get it in before alpha 3 (~2 weeks). What's nice about this proposal is Ethan has already written the code, so it's just a matter of getting consensus to add it.

[warsaw]

I don't really think much more needs to be done other than agree on this tracker issue that it's something we want, and that Ethan's implementation is good enough. It's been a while since I looked that the code but it seemed good to me before, so if Ethan is happy with it going into the stdlib, so am I. As for the feature, it *is* a little magical, but not so much that it would be confusing or ambiguous, so I'm +1 on it.

I'll defer to Ethan for the final decision, but if we have tests and documentation, I'd be very happy to see this in 3.6.

[johnthagen]

Is this something we want to get in before the next alpha in two days? Just wanted to bring up the deadline since this may be a feature people want to play around with during the alpha phase.

Ethan, I'm happy to help with documentation or anything else.

[ethanfurman]

There is one wrinkle with auto-numbering. Consider the following code:

class Color(AutoNumberEnum):
  red
  green
  blue
  @property
  def cap_name(self):
    return self.name.upper()

What happens with property?

• property is looked up in the class namespace
• it's not found, so is given the number 4
• it's then called to handle cap_name
• an exception is raised

As far as I can tell there is only one "good" way around this:

• store any names you don't want auto-numbered into an _ignore_
  attribute (this only needs to be global and built-in names that
  are used before the first method is defined)

another option is to build a proxy around any found global/built-in objects and decide what to do based on whether those objects are immediately called, but that fails when the object is simply assigned for later use.

So, barring any other ideas to handle this problem, the above example should look like this:

class Color(AutoNumberEnum):
  _ignore_ = 'property'
  red
  green
  blue
  @property
  def cap_name(self):
    return self.name.upper()

Another advantage of using ignore is the ability to have temporary variables automatically discarded:

  class Period(timedelta, Enum):
    '''
    different lengths of time
    '''
    _ignore_ = 'Period i'
    Period = vars()
    for i in range(31):
      Period['day_%d' % i] = i, 'day'
    for i in range(15):
      Period['week_%d' % i] = i*7, 'week'
    for i in range(12):
      Period['month_%d' % i] = i*30, 'month'
    OneDay = day_1
    OneWeek = week_1
    def __new__(self, value, period):
      ...

and the final enumeration does not have the temp variables Period nor i.

Thoughts?

[warsaw]

On Jul 09, 2016, at 03:20 PM, Ethan Furman wrote:

As far as I can tell there is only one "good" way around this:

• store any names you don't want auto-numbered into an _ignore_
  attribute (this only needs to be global and built-in names that
  are used before the first method is defined)

Seems reasonable, though why not a dunder, e.g. __ignore__?

_ignore_ = 'Period i'

Why space separated names inside a string instead of a sequence of strings?

[ethanfurman]

A standard feature of Enum is that either space separated strings or a list of strings is accepted any where either is.

_sunder_ names are the ones reserved for Enum use (such as _value_, _name_, and soon _order_ (for py2/py3 compatible code).

[johnthagen]

What happens with property?

• property is looked up in the class namespace

Perhaps you've already considered this, I'm not intimately familiar with how classes are parsed and constructed but is it possible to determine if the object is a decorator? It already determines to stop auto-numbering when it hits the first method, could it stop when it hits the first decorator or method?

Being able to use temporaries is an interesting side effect, but I feel like that would be used less often than a @staticmethod, @Property, or @classmethod over a method, in which case it becomes a little more complex.

That being said, I think either solution is valid.

[ethanfurman]

The problem with testing the type of object a name refers to outside the class is it then becomes more difficult to make that an Enum member:

class AddressType(Enum):
    pobox
    mailbox  # third-party po box
    property

Having to assign a value to property pretty much negates the value of the magic.

I'll go with _ignore_.