C++ enum to Python namedtuple (cont.)

Here is a way to convert a C++ enum with values to Python.
C++ code:

typedef enum
{
   LADYBUG_RAW_CAM0           = ( 0x1 << 0 ),
   LADYBUG_RAW_CAM1           = ( 0x1 << 1 ),
   LADYBUG_RAW_CAM2           = ( 0x1 << 2 ),
   LADYBUG_RAW_CAM3           = ( 0x1 << 3 ),
   LADYBUG_RAW_CAM4           = ( 0x1 << 4 ),
   LADYBUG_RAW_CAM5           = ( 0x1 << 5 ),
   LADYBUG_ALL_RAW_IMAGES     =  0x0000003F,

   ...

   LADYBUG_PANORAMIC          = ( 0x1 << 12 ),

   LADYBUG_DOME               = ( 0x1 << 13 ),

   LADYBUG_SPHERICAL          = ( 0x1 << 14 ),

   LADYBUG_ALL_CAMERAS_VIEW   = ( 0x1 << 15 ),

   LADYBUG_ALL_OUTPUT_IMAGE   = 0x7FFFFFFF,

} LadybugOutputImage

The Python version:

LadybugOutputImage = dict(
   LADYBUG_RAW_CAM0           = ( 0x1 << 0 ),
   LADYBUG_RAW_CAM1           = ( 0x1 << 1 ),
   LADYBUG_RAW_CAM2           = ( 0x1 << 2 ),
   LADYBUG_RAW_CAM3           = ( 0x1 << 3 ),
   LADYBUG_RAW_CAM4           = ( 0x1 << 4 ),
   LADYBUG_RAW_CAM5           = ( 0x1 << 5 ),
   LADYBUG_ALL_RAW_IMAGES     =  0x0000003F,

   ...

   LADYBUG_PANORAMIC          = ( 0x1 << 12 ),

   LADYBUG_DOME               = ( 0x1 << 13 ),

   LADYBUG_SPHERICAL          = ( 0x1 << 14 ),

   LADYBUG_ALL_CAMERAS_VIEW   = ( 0x1 << 15 ),

   LADYBUG_ALL_OUTPUT_IMAGE   = 0x7FFFFFFF  )
LadybugOutputImage = namedtuple('Enum',
                       LadybugOutputImage.keys() )(**LadybugOutputImage)

Or even better, just get rid of all the packaging and commas on this type if it’s not too long and you have the patience. But you will also have to correct the indenting. Maybe there is a program out there that can do this editing for you.

   LADYBUG_RAW_CAM0           = ( 0x1 << 0 )
   LADYBUG_RAW_CAM1           = ( 0x1 << 1 )
   LADYBUG_RAW_CAM2           = ( 0x1 << 2 )
   LADYBUG_RAW_CAM3           = ( 0x1 << 3 )
   ...
   LADYBUG_ALL_CAMERAS_VIEW   = ( 0x1 << 15 )
   LADYBUG_ALL_OUTPUT_IMAGE   = 0x7FFFFFFF

Python Enum continued

I needed to convert an enum to python so that I could decode some responses from C code. Here is an example of quickly converting a C enum to python.
The C code I am working with has several enum structures, one is a long list of error types:

enum LadybugError
{
   LADYBUG_OK,
   LADYBUG_FAILED,
   LADYBUG_INVALID_ARGUMENT,
   ...
   LADYBUG_INVALID_FRAMERATE,
   LADYBUG_INVALID_OFFSCREEN_BUFFER_SIZE,
   LADYBUG_INVALID_JPEG_IMAGE_STRUCTURE,

   LADYBUG_NUM_LADYBUG_ERRORS,

   LADYBUG_ERROR_FORCE_QUADLET = 0x7FFFFFFF,
}

There are about 70 items in this list.
Here is the conversion using namedtuple:

LadybugError = namedtuple('Enum', '''
   LADYBUG_OK,
   LADYBUG_FAILED,
   LADYBUG_INVALID_ARGUMENT,
   ...
   LADYBUG_INVALID_FRAMERATE,
   LADYBUG_INVALID_OFFSCREEN_BUFFER_SIZE,
   LADYBUG_INVALID_JPEG_IMAGE_STRUCTURE,

   LADYBUG_NUM_LADYBUG_ERRORS''')
LadybugError = LadybugError._make(range(len(LadybugError._fields)))

What is nice about namedtuple is that the keywords can be given as one long string. It parses the string for you by spaces and commas. This saves me the trouble adding quotes around each item, just add quotes around the whole list. I overwrite the original ‘LadybugError’ string with the enumeration because it is no longer needed.
When a program returns an integer representing an error, it can be passed to _fields[] to get the error string.

err = libc.ladybugGetStreamHeader( pContext, streamHeaderInfo, None )
print err, LadybugError._fields[err]

The output from this is:

20 LADYBUG_NOT_INITIALIZED

The value of err is 20 and ‘LADYBUG_NOT_INITIALIZED’ is the 21st item in the LadybugError enumeration.

Because I only need to interpret numerical feedback, I can drop the “_make(range” line and just take advantage of namedtuple’s builtin parsing:

LadybugError = namedtuple('Enum', '''
   LADYBUG_OK,
   LADYBUG_FAILED,
   LADYBUG_INVALID_ARGUMENT,
   ...
   LADYBUG_INVALID_FRAMERATE,
   LADYBUG_INVALID_OFFSCREEN_BUFFER_SIZE,
   LADYBUG_INVALID_JPEG_IMAGE_STRUCTURE,
   LADYBUG_NUM_LADYBUG_ERRORS''')

This helps me easily find out what an error of 12 means:

print error, LadybugError._fields[error]

prints

12 LADYBUG_COULD_NOT_OPEN_FILE

This works smoothly if the enumeration starts with zero. Otherwise, getting the name through _fields[i] will not be correct. Also, remember that exceptions can be added using ._replace(field=new_value). Here’s an example:

LadybugBusSpeed = LadybugBusSpeed._replace(LADYBUG_SPEED_UNKNOWN=-1)
err = -1
print LadybugBusSpeed._fields[LadybugBusSpeed.index(err)]

Python Enum with namedtuple

I saw a question asking how to do enum in Python and because I just learned about namedtuple I thought it should be an easy thing to do.

For an enum starting with 0, you can write it like this:

from collections import namedtuple as ntup

animalnames = 'Dog','Cat','Mouse','Horse','Bunny'

Animal = ntup('Enum', animalnames)._make(range(len(animalnames)))

Then you can

print Animal.Horse

and get 3 returned.
Also, because it is also a tuple, you can access it by the index:

Animal.Horse == Animal[3]

For an enum starting at 1 or higher there are a few ways.
You can use arange and add an offset:

from numpy import arange
animalnames = 'Dog','Cat','Mouse','Horse','Bunny'
Animal = ntup('Enum', animalnames)._make(3+arange(len(animalnames)))

Or stay with range:

animalnames = 'Dog','Cat','Mouse','Horse','Bunny'
Animal = ntup('Enum', animalnames)._make(range(3,3+len(animalnames)))

What is be nice about using this method is that it is still a tuple. Here are few lines and their output:

print Animal
print Animal._fields
print Animal[:]
print list(Animal)
print zip(Animal._fields, Animal)
Enum(Dog=3, Cat=4, Mouse=5, Horse=6, Bunny=7)
('Dog', 'Cat', 'Mouse', 'Horse', 'Bunny')
(3, 4, 5, 6, 7)
[3, 4, 5, 6, 7]
[('Dog', 3), ('Cat', 4), ('Mouse', 5), ('Horse', 6), ('Bunny', 7)]

Data Structures in brief

Making a list of data structures for reference. I probably won’t finish this. I mainly was trying to remember what ‘structured array’ was called and decided to make a list and maybe investigate a couple of them. In the process of searching for it I found ‘namedtuple’ which looks very useful.

List

Stack (list)

Queue (list)

Tuple

Namedtuple

# IMPORT NAMEDTUPLE
from collections import namedtuple
# CREATE A NAMEDTUPLE STRUCTURE
Rock = namedtuple('Rock', 'q w e r t')
# CREATE AN INSTANCE
n2 = Rock(5,4,3,2,1)
# SAME AS
n2 = Rock(t=1, e=3, q=5, r=2, w=4)
# CHANGE A VALUE BY COPYING TO NEW INSTANCE
n2 = n2._replace(w=234, r=[3,4,5])

Deque

Set

Frozenset

Dictionary

Counter (dict)

OrderedDict (dict)

Array

Structured Array

# IMPORT NUMPY ARRAY
from numpy import array
# CREATE A DATA LIST, TUPLE VALUES WILL RECEIVE NAMES
dl = [[(1,2,3,4),(3,2,5,6),(7,6,9,8)],[(1,2,3,4),(3,2,5,6),(7,6,9,8)]]
# CREATE THE DATA TYPE LIST
dt = [('1st','float'),('2nd','int'),('3rd','float'),('4th','int')]
# CREATE A STRUCTURED ARRAY, TUPLES BECOME NP.VOID OBJECTS
d = array(dl, dtype=dt)
# SET ALL VALUES OF A CATEGORY
d['1st'] = 6
# SET ONE VALUE
d['1st'][0,0] = 5
# same as
d[0,0]['1st'] = 5
# GET LISTING OF DTYPE, SIMILAR TO: print dt
print d.dtype
# JUST GET CATEGORY NAMES
print d.dtype.names
# GET A DICTIONARY OF CATEGORY(KEYS) WITH THE DATA TYPE AND BYTE POSITION (I BELIEVE)
print d.dtype.fields

Matrix

Vector

Create a class

Documentation

I decided to explore what is considered to be the proper or best documentation methods. Up til now I just adopted a few things I saw in other scripts or carried over ideas I learned from java programming.

Header

#!/usr/bin/env python
# -*- coding: utf-8 -*-

These should be the top two lines. The first line is necessary for programs to be cross-platform. The second line is optional, where the default encoding is ASCII. The ‘utf-8’ can be replaced with other encoding types.

Next should be a multi-line docstring. It must come next and before all code in order for python to recognize it as the main docstring and attach it to the ‘__doc__’ identifier. Thus,

print objname.__doc__

will print out the contents of this string. The first line should be a short description followed by a blank line. This brief description will show up after the filename (or ‘__name__’ if it is explicitly defined below the docstring but doing this could cause other problems).

In [4]: import SIFT_pack_v3 as SIFT

In [5]: help SIFT
------> help(SIFT)
Help on module SIFT_pack_v3:

NAME
    SIFT_pack_v3 - Created on Sat Dec 24 22:52:12 2011

FILE
    c:\dropbox\my programming\sift_pack_v3.py
...

This shows how the help output is formatted. The first line of my docstring is a creation date statement instead of a description and this is printed out after the file name and dash. The help format follows this up with the file location. After skipping a line from the short description or statement you can write a more verbose description and instructions.

Epydoc seems to be a prominent auto documenting program. If you write you docstrings conforming to their standards, then it can produce nice looking html documentation pages for the code. Details on how to do epydoc formatting are at epydoc.

After the main docstring, it is recommended that you have import statements, beginning with built-in modules and followed by any others.

I’m not completely clear about the difference between using @version in the docstring versus declaring the __version__ variable outside the docstring. They are two different conventions, where @version conforms to the epydoc style, but epydoc will still recognize __version__. You can make up as many of these as you like but only a few are recognized by epydoc. The recognized ones are:

__author__
__authors__
__contact__
__copyright__
__license__
__deprecated__
__date__
__version__

This is described as using module variables as module metadata. There is a way to get epydoc to recognize other metadata (variables) so you can have the best of both worlds. See adding new fields for epydoc.

The next section should have global variables, but it is recommended that globals be avoided. I don’t remember why and I still use them for setting the working directory or testing for the existence of necessary files.

All the functions follow. The first item within the class or function block should also be a docstring. Again, the first line should be a short summary. Then describe the inputs and outputs in the epydoc style,

def example():
    """
    @arg x: This is a description of
        the parameter x to a function.
        Note that the description is
        indented four spaces. Also
        can use @param instead.
    @type x: This is a description of
        x's type.
    @return: This is a description of
        the function's return value.
    @rtype: This is a description of
        the return value's type.
    """

Apart from epydoc formatting, a docstring can be arranged anyway you want. Here is an example found on PEP 257:

def complex(real=0.0, imag=0.0):
    """Form a complex number.

    Keyword arguments:
    real -- the real part (default 0.0)
    imag -- the imaginary part (default 0.0)

    """
    if imag == 0.0 and real == 0.0: return complex_zero
    ...

Finally, the very last item should be:

if __name__ == '__main__':
    main()

This will automatically run ‘main()’ when the program is ran and not when the module is simply imported. This needs to be the last item to ensure that all functions are loaded prior to running the main method regardless of order they are arranged.

On a side note, dir(object) in IPython console will print out a list of all namespaces, functions, and variables(outside of functions) within the object.

Branching river model

This model extends the semi-implicit scheme to include a branching river. It would be easy to extend it to multiple tributaries but I haven’t tried it yet.

In the animation above, the ocean tide enters at the left. The main river is blue and the branching river is cyan. The model is 50km long with a long wave tide (M2) and a smaller oscillation.