Arlie's Journal

In a fit of optimism, I decided it would be interesting to convert a global variable, defined as a TypedDict, into a Class.

This variable is a collection of bits of information used by a parser to keep track of the state of its input. If this were C, it would probably have started life as a collection of global variables at file scope. If the local coding style limited the use of module globals, it would have been a struct.

Python's handling of non-mutable (aka pass-by-value) variables above function or class scope is basically garbage. You can't reliably modify them, without various workarounds. If "foo" is global, and I assign a value to "foo" in a function, the value is assigned to a local variable of the same name, which goes out of scope at the end of the function. If you use the "global" keyword to fix this, the type checkers have hysterics, and the code still doesn't always work. etc. etc. ad nauseam.

So I originally implemented the parser state as a dictionary. Dictionaries are "mutable", which means pass-by-reference in python, which means they work sensibly at global scope. Of course they also have the delightful "benefit" that typos in assignments will generally silently do the wrong thing. bar[name] = x; bar[Name] = y leaves bar with two fields. (But that's true for individual variables too.) I created an ugly workaround for that issue, but then I discovered the TypedDict class - dictionary at run time, but with extra testing when running type checkers. (You *almost* get all the maintainability of a proper C struct, though not quite.)

Well, it turned out that I wanted to enhance the program to run the parser multiple times, each on its own input file, with its own parser state. I could reuse the same variable, re-initializing it after each input file.

But conceptually this looks a lot like a 'class' which had been singleton, and would now have one instance per input file. (I'm used to doing OO things in non-OO languages; this doesn't bend my brain at all.) However, Python is an OO language. Maybe I should make it into a (singleton) class, before converting the code to handle several of them.

I checked, and the type checkers handle classes about as well as they handle TypedDict - I lose type checking on the two fields previously labelled NotRequired, now described as "attributes not set by the class' __init__ method." I gain slightly in readability. There's also a negligible-in-this-case improvement in memory use and performance - not from using a class, but from using python's __slots__ function with my class, which I'd do anyway since it gets me type-checker warnings for typoed field names.

I re-implemented it, and ran the various static check modules. The first problem was the names of my attributes. Dictionary tags are supposed to be Camel Case. FooBarBaz. Class attribute names are supposed to be snake case. foo_bar_baz. The syntax for accessing either a class attribute or a dictionary member is the same, so I'd expected not to have to change the 95 places where I used one of these fields. No such luck. Every ParserState.FooBar has to be changed to ParserState.foo_bar.

Of course I could simply suppress the warning, but I'm trying to train myself to read and write "normal python".

Then I got the next pair of warnings.

parser_state.py:14:0: R0902: Too many instance attributes (9/7) (too-many-instance-attributes)
parser_state.py:14:0: R0903: Too few public methods (0/2) (too-few-public-methods)

I have 11 fields in this not-a-struct. The error says 9, but that's presumably because only 9 are set in __init__. Whoops! Can't have that.

And even though it's always legal to access class attributes from non-class code (ugh! ugh! ugh!) it's not OK to have nothing but an __init__ method.

How is this good language design? In what world should "objects" be totally unprotected from users messing with their guts - best you get is a convention that users should not mess with variables with names like __foo__ - but having less than 2 methods or more than 7 instance attributes should be a problem treated as harshly as modifying a conventionally private class/instance attribute?!