New pylibftdi release - 0.11

'pylibftdi' is a library for talking to FTDI devices via the libftdi library. FTDI make a wide range of chipsets and modules for interfacing to a number of protocols via USB, including 8-bit parallel and RS232 serial modes. They're a great way of interfacing to other electronics from your computer.

I've just released pylibftdi 0.11. I'm at the point where I'm looking at getting to RC and then stable status, which I'll release as 1.0 - at which point the API will be considered stable. While it isn't yet, I've taken the opportunity to tidy a couple of things, as well as add some improvements.

Raspberry Pi support; better documentation

Though it worked previously, I've taken the opportunity to test it a bit on Raspberry Pi, and I've updated the docs describing udev rules which allow access to the devices without needing sudo / root access. I think this is now a good option if you want a bidirectional 8 bit port for your Raspberry Pi, and it's certainly lower risk of damaging your Pi than using the GPIO pins directly.

BitBangDevice changes

The new latch property

BitBangDevices provide a simple abstraction of a parallel IO device; a 'direction' property which controls which line is an input or output, and a 'port' property for the actual reads and writes. This is based on systems going all the way back to the BBC micro user port and earlier. direction maps to the 'Data Direction Register' of the Beeb, the 'TRISx' register of the Microchip PIC, and so on. port maps to the 'data' register of the Beeb, or the PORTx register of the Microchip PIC. Just as the PIC18F series introduced the 'LATx' register, so too does pylibftdi 0.11 introduce the latch. Read the documentation for more information - in most cases you simply don't need to care about this.

Initialisation

If a physical FTDI device is not reset between program runs, then it retains its output register state; a pin set high in one run of the program would still be high when the device was opened in a subsequent program run. Prior to pylibftdi v0.11, this was not taken into account, and the assumed state of all output pins was that they were at the reset state, i.e. all low. This meant that operations such as read-modify-write on port bits would not reflect the current state, as they do not do a read based on the output state of the port, but based on the internal view of what output values are set to.

With the change, the following will work as expected:

$ python
>>> from pylibftdi import BitBangDevice
>>> bb = BitBangDevice()
>>> bb.port |= 1
>>> ^D
$ python
>>> from pylibftdi import BitBangDevice
>>> bb = BitBangDevice()
>>> bb.port |= 2
>>> ^D

Previously, the final state of the device pins would have been '2'; the read-modify-write implied by |= 2 would have used '0' is its 'read' source, and have output '2'. The new code initialises the internal latch state to the value read from the pins (it's possible to read the actual state of output pins as well as input pins). With the latest version, the final state of the pins after the above will be '3' - both D0 and D1 set high.

API changes

I've always said in the README for pylibftdi that the API won't be stable until version 1.0, and I've changed two parameters only introduced in 0.10.x to have clearer names.

The following two parameters to the Device constructor have changed name:

interface -> interface_select

I considered interface too generic and unintuitive here. The values and behaviour for this parameter (which selects which interface to use on a multi-interface device) haven't changed.

buffer_size -> chunk_size

This is the maximum number of bytes which will be written / read at a time in read/write calls to the libftdi library, designed to ensure we are regularly executing at least some Python byte code, which we can then interrupt (timely Ctrl-C interruption is the primary use-case for this parameter). It was never about buffering, so I've changed the name to reflect this.

Other changes

The bit_server example now works properly; this can be run as:

$ python -m pylibftdi.examples.bit_server

and will start a basic CGI-based web server, open a web-browser talking to it (on port 8008 by default), and allow you to control the state of each of the 8 output lines on the connected device (which it sets to async bit-bang mode).

This will be further developed in the future - it looks somewhat rough right now :)

The led_flash example has also gained a feature in taking a command line argument of the rate at which to flash - defaulting to 1 Hz. To cause an LED (or a piezo buzzer works just as well - and more annoyingly!) to flash at 10Hz, run:

$ python -m pylibftdi.examples.led_flash 10

Coming next

I'm still trying to improve test coverage. I spent some time trying to port the tests to the Mock library, though my efforts at effectively patching at the ctypes DLL level weren't very successful.

Documentation continues, and thanks to the wonderful readthedocs.org, the documentation isn't necessarily tied to the more sedate release cycle - it always shows the latest version from Bitbucket. If more API changes happen this could be counter-productive, but I'll try really hard to note if this is the case, and it makes things much nicer when updating things like installation instructions (which I have done, adding tested udev rules instructions etc).

libftdi 1.0 is just going through release candidate stage at the moment, so I'll test against that. I expect only the installation docs will need changes.

I've never tested pylibftdi on Windows, and I'm keen to do this in the near future, though I don't have regular access to a Windows machine, so no guarantees about this. I suspect it all 'just works'...

pylibftdi v0.10 released

I’ve recently released pylibftdi v0.10. pylibftdi is a ‘minimal Pythonic interface to libftdi’, and is intended to be (possibly?) the easiest way to get up and running for simple use cases of FTDI’s range of USB to serial and parallel interface chips and modules. v0.10 adds a couple of new features and bug fixes.

For a long time I suffered under the misapprehension that version numbers should follow the rules of decimal numbers, and that by all reasonable accounts v1.0 should have followed 0.9, and since I want(ed) 1.0 to be ‘stable’ (I currently classify it as ‘beta’), I’d reached an impasse. I can’t remember the exact moment, but I had a realisation that I didn’t have to approach 1.0 via smaller and smaller increments from 0.9 (as in Zeno’s race), but that I could go from 0.9 to 0.10. Anyway, I still want to do better documentation (and a few other things) before reaching 1.0.

Changes in v0.10:

• Running the unit tests is now easier due to some reorganisation - just run python -m unittest discover in the top level directory.

• Support for the FT232H device - this has a different USB product ID (PID) to the previous devices I’d been testing with and using - mainly FT245BM/RL, FT232R/RL. All those devices have a PID of 0x6001, while the newer FT232H has a PID of 0x6014. I experimented for a while with having (defaulted) extra parameters for specifying the VID and PID of the target device, but this pushed too much complexity up to the user - I really want pylibftdi to be something which can be used with default options and next-to-no set up code for most basic operations. The approach taken is to have two lists (USB_VID_LIST, USB_PID_LIST) and have the device finding code iterate over the cartesian product of these (i.e. a nested loop, but implemented through the wonderful itertools.product). So adding new PIDs in the future is as simple as appending to USB_PID_LIST, and a device can be opened with no parameters to the Device() constructor if it’s the only FTDI device on the USB bus.

• Resetting the device to serial mode on open. There’s been discussion about implementing this logic in the library on the libftdi mailing list, but doing it in pylibftdi as well doesn’t hurt. This fixes the unexpected condition that if a previous application had used a device in BitBang mode, reopening it just using Device() would leave it in BitBang mode, rather than the expected serial mode (for devices which have support both).

• Added a ‘buffer_size’ parameter to the Device() constructor (defaulted to zero, which retains previous behaviour) which chunks reads and writes into accesses of that length at most. This avoids the issue that a call of (for example) dev.write(‘hello’ * 100000) over a 9600 serial link would take an incredibly long time, and since it is all running in the library context (via a ctypes call), it wouldn’t be interruptible by Ctrl-C.

• Removed the deprecated use of Driver() to be a synonym for Device().

• Update: I've already done two maintenance releases in the hours since originally writing this - v0.10.2 is now current. One of the major changes is that the examples subpackage is now included in the sdist - so python -m pylibftdi.examples.led_flash should work if you have an LED attached to D0 on your device.

The plan for the next release is just more tidy-ups, examples and more documentation, but I might squeeze a few other things in there…

Bugs: they hide where you're not looking

I bought a new Pogoplug over the weekend (only £50 new at PC World), and after being genuinely impressed by the Pogoplug software, decided it was far too easy and put PlugApps Linux on it. These 'plug' type devices are fairly amazing - cheap, very low power (measured mine at just under 4 watts, with only a single USB flash stick), but with a decent 1.2GHz ARM processor. I'm already thinking my next computer might be another 'plug.

After hacking for I while (why won't my printer work?), I decided to check whether my pylibftdi package worked on it. To my shock, a quick 'pacman -S libftdi; pip install pylibftdi', installed fine, and I could open a device connection to a FTDI device! But then things got worse. Trying to run examples/lcd.py failed with an exception in BitBangDevice, and I quickly realised that the API changes I'd done in 0.8 to make device access more 'file-like' had broken things in BitBangDriver. I was slightly sad that I'd released something where the examples didn't even work, and part of the whole reason the package might be useful to people (the abstraction over bit-bang device operation) was broken.

pylibftdi is fairly simple, and basically consists of Driver, Device, and BitBangDevice classes. Most of the interesting is in the Device class - so this is where I started when I finally got round to adding some tests for the 0.8 release. Having achieved reasonable coverage (though shamefully less than the 100% Uncle Bob demands), I considered my initial testing 'done'. I knew there was more to add later, and had (and still have) full intentions to 'get around to it'.

What I failed to anticipate has the unintended side-effect of writing tests. In the same way teachers might teach how to pass an exam rather than the depth and breadth of a subject, once tests exist, the purpose can simply be to pass them. Old manual acceptance tests get ignored as the 'old way' of doing things. Ideally of course this isn't a problem, because full test cases exist for every feature and code-path in the system, but that was very far from the case here. So somehow, my standard acceptance test (do the example programs still work) got omitted, in preference for 'there are tests now, so it must be better! And the tests pass!'

So beware - a little testing can be a dangerous thing. The bugs hide where you're not looking for them. This is great motivation for achieving full test coverage, for automating acceptance testing (as well as unit / component level testing) so far as possible, and for being humble when it comes to writing tests. My motivations for writing them in the first place were two-fold: the feeling it was 'what I should do', and the idea that at some future point when I added or refactored things later I could be confident I hadn't broken things. I had no thought that the software was already broken; that I needed tests.

Anyway, pylibftdi 0.8.1 is now out, with trivial but important fixes and lots more tests.

pylibftdi 0.8 released; testing, coverage, and mocking

pylibftdi is a file-like wrapper to FTDI USB devices such as the UB232R (a USB<->serial converter) and the UM245R (8 bit parallel I/O).

No big changes for the 0.8 release, but a couple of new things:

• ability to specify the device identifier in the Device([device_id]) constructor as either a serial number (as previously), or a device description. So can now specify Device('UB232R'), and the first attached UB232R device will be opened. The code initially tries to open by serial number, and if that fails will try to open by description, which I'm fairly confident will be useful rather than annoying :-)
• more file-like API functions (flush, readline()/readlines()/writelines(), iterating over lines). These probably only make sense for text over serial lines, but that's a use-case worth supporting, considering pylibftdi already has unicode support.

As well as that, I finally got round to adding some tests, and discovered something wonderful: checking test coverage isn't just practical icing on the cake to make sure things are tested well, but is a powerful and effective motivation for writing tests. I'm using coverage, and have to say it's one of those things I wish I had got round to sooner.

Speaking of which, at some point I'll probably end up saying the same about Mock, which I've read around and know I should probably start using, but it's just so easy in Python to knock up something like this:

fn_log = []
class SimpleMock(object):
    """
    This is a simple mock plugin for fdll which logs any calls
    made through it to fn_log, which is currently rather ugly
    global state.
    """
    def __init__(self, name="<base>"):
        self.__name = name

    def __getattr__(self, key):
        # This makes me smile :)
        return self.__dict__.get(key, SimpleMock(key))

    def __call__(self, *o, **k):
        fn_log.append(self.__name)
        # most fdll calls return 0 for success
        return 0

def get_calls(fn):
    "return the called function names which the fdll mock object made"
    del fn_log[:]
    fn()
    return fn_log

Sometimes I think Python makes 'clever' things like that too easy, and is perhaps the reason that although in the Python language there is only-one-way-to-do-it, in the Python ecosystem there is perhaps a tendency to reinvent the wheel over and over again. Because it's easy - and it's fun.

As always code is at bitbucket. For the next release (0.9) I'm planning to add more tests and docs (which are rather scarce), as well as one or two of the other things I've got planned (possible D2XX support, or at least some notes on libftdi on Windows, more examples & protocol adapters, maybe even a web interface for 8 bit IO...)

pylibftdi 0.7 - multiple device support

pylibftdi has always been about minimalism, which means that if you wanted to do something it didn't support, things got tricky. One of it's glaring deficiencies until now was that it only supported a single FTDI device - if you had multiple devices plugged in, it would pick one - seemingly - at random.

With pylibftdi 0.7, that has finally changed, and devices can now be opened by name. Or at least by serial number, which is nearly as good. A new example script (which I've just remembered is hideously raw and lacks any tidying up at all) examples/list_devices.py in the source distribution will enumerate the attached devices, displaying the manufacturer (which should be FTDI in all cases), description, and serial number.

The API has changed slightly to cope with this; whereas previously there was just a single Driver class, now the primary interface is the Device class. Driver still exists, and holds the CDLL reference, as well as supporting device enumeration and providing backwards compatibility.

(As an aside, using ftdi_usb_find_all was (not) fun - it sets a pointer to pointer which is then used to traverse a linked list. Trivial in C, an hour of frustration in ctypes. Anyway, I got there in the end).

>>> from pylibftdi import Device
>>> import time
>>> 
>>> # make some noise
>>> with Device('FTE4FFVQ', mode='b') as midi_dev:
...     midi_dev.baudrate = 31250
...     for count in range(3):
...         midi_dev.write(b'\x90\x3f\x7f')
...         time.sleep(0.5)
...         midi_dev.write(b'\x90\x3f\x00')
...         time.sleep(0.25)
... 

Both Device() and BitBangDevice take device_id as the (optional) first parameter to select the target device. If porting from an earlier version, one of the first changes is probably to use named parameters for options when instantiating these classes. My intention is that device_id will always be the first parameter, but the order and number of subsequent parameters could change.

Another change is that Devices are now opened implicitly on instantiation unless told not to (see the docstrings). Previously the Driver class only opened automatically when used as a context manager. There is no harm in opening devices multiple times though - subsequent open()s have no effect.

I've also finally figured out that I need to set long_description in setup.py to get documentation to appear on the PyPI front page. After all, without docs, it doesn't exist.

It's only been a few days since 0.6, but I wanted to get this release out - I think it is a big improvement since 0.5, and It'll probably be a while till the next release. In the mean time, I'll try and get a vaguely useful example going - which will probably involve MIDI and an LCD...

pylibftdi 0.6 released: now with Python 3 goodness

pylibftdi 0.6 has been out the door and onto PyPI for the last few days, but I'm only just getting round to blogging about it. It's basically some minor work for Python 3 compatibility - the same code now works on both Python 2 (2.6/2.7) and Python 3. This means support for Python 2.5 has been dropped (due to use of bytearray/bytes types). I can always add it back in if people shout.

Other than trivially fixing a print statement to be a function call, the main change required was the expected bytes/string issue. The driver also gains a couple of parameters; mode = 't' ('t':text, 'b':binary) and encoding = 'latin1'.

In binary mode (the default - so no user code changes are required for this release), read() and write() take and return instances of type bytes. For text mode, write() will take either bytes/bytearray, or a string which it will encode with the given driver encoding, and read() will return a string. I've set the default to be latin1 rather than using utf-8 as it is an equivalence mapping over the first 256 code points.

Coming soon...

I've started work on 0.7 - the main feature of which is support for multiple devices. I had a few problems getting the right ctypes incantations to follow the linked-list which ftdi_usb_find_all sets, but that's sorted now. The bigger issue is that it really needs a split between driver and device, which could cause the API to change. I'm thinking of various ways to keep existing code working, and will probably go for something like:

• 0.7 - set pylibftdi.SUPPORT_MULTIPLE to True to use new API / support multiple devices
• 0.8 - set pylibftdi.SUPPORT_MULTIPLE to False to use old API / only support a single device / get a deprecation warning
• 0.9 - SUPPORT_MULTIPLE no longer used; old API disappears.

So 0.7 is all about multiple device support, 0.8 will probably be support for Windows (supporting D2XX, for example), and 0.9 (or maybe just 1.0) will be a tidy-up / bug-fix / improve docs release. In parallel with all of this I'm writing some test code which will gradually bring this side of things up-to-standard. I'm not allowing myself to do a 1.0 release without decent testing & docs. All that will probably take a two months; I only get a couple of hours a week looking at this. But it could be sooner - or later.

pylibftdi 0.7 should be out in within a week or so, and I'll elaborate more then, hence the lack of any examples here. I'm on the case!

pylibftdi updated to 0.5

I've done some tidying up of pylibftdi, fixing a few bugs, refactored the pylibftdi package to contain several modules instead of everything dumped in a __init__.py file, and generally made it a bit cleaner. It even has docstrings for most things now, and a test or two!

pylibftdi is a simple interface to libftdi, which in turn allows accessing FTDI's range of USB parallel and serial chips. See here for more details, but very briefly, serial access is provided by a file-like (read/write) interface with a baudrate property, and parallel access is provided by a pair of properties - direction (data direction register) and port (data IO register).

I haven't yet added any more examples, hopefully I'll get round to that in the next week or so. I have been using it as a MIDI interface though, which is fun - I'll get an appropriate example of that out in the next version, together with some diagrams / photos etc.

Announcing pylibftdi - a minimal Pythonic wrapper for libftdi

[edit 2013-11-25: note that recent versions of pylibftdi have deprecated and then removed the ability to use 'Driver' in this way; replace Driver and BitBangDriver with Device and BitBangDevice in the code below]

The playing-around I've done with FTDI devices seemed like a good opportunity to actually release something as open source, and so I present 'pylibftdi'. Undoubtedly not the greatest, but right now most likely the latest FTDI-Python bridge in the rather large open source field. There are a few features I know I want to add to it (primarily support for multiple devices), but for a flavour of things:

Toggling an LED or other device from pin D0

from pylibftdi import BitBangDriver
import time

with BitBangDriver() as bb:
    while True:
        bb.port ^= 1
        time.sleep(0.5)

Sending a string to a serial terminal

from pylibftdi import Driver

with Driver() as drv:
    drv.baudrate = 19200
    drv.write('Hello World!')

It's been tested on Linux (Ubuntu 10.10) and Mac OS X (10.6), with libftdi 0.17 and 0.18, but doesn't have any intended platform specific requirements other than having the libftdi library installed. The following goals for this project differentiate it from similar projects:

• Fully supported on Linux and Mac OS X, using Intra2net's open source libftdi driver.
• Simple things are very simple, as the example above shows. The basic functionality is made as simple as possible to use, with properties and context managers used where it makes sense. Most other FTDI Python wrappers are 'just' low level bindings of the underlying API.
• There will be an increasing library of examples showing interaction with various protocols and devices - note this is a goal, not yet an accomplishment, though there is an LCD example there.

pylibftdi is easy_installable (or equivalent) from PyPI, and the code is developed on bitbucket, where you can also raise tickets for problems and feature requests.

For reference, other similar projects include PyUSB, pyftdi, python-ftdi, and ftd2xx. There are probably others...