Driven by a desire for privacy, customization, and lower costs, there’s growing interest in AI models which can be run on local hardware. Few of them go as far as [Tim], though, who built an image generation diffusion model which can run on an RP2350 microcontroller.
As might be expected, its capabilities are limited. The resolution is 128×128, it only generates images of human faces, and it takes about twenty seconds per image – still impressive for such limited hardware. It runs on a Waveshare RP2350 development board, and it can output the generated image over USB or display it with the aid of a VGA adapter board.
The generative model doesn’t directly create an image. Rather, it generates a distribution in a latent space, which a variational auto-encoder’s decoder component translates into an image. The auto-encoder was trained in two parts: an encoder which transforms an image into a latent-space distribution, and a decoder to transform that distribution back to an image; once this was trained, only the decoder was used.
The generative portion of the model uses a latent flow diffusion transformer; this takes in noise to start with, then iteratively predicts changes which bring it toward the desired image. It can also take in a output class, which guides the generator’s direction (toward a smiling face, for example). [Tim] trained two models, one larger and one faster, and quantized the weights for both to 8-bit integers. Both models, along with the inference program, then fit into 4 MB of flash memory.
For such a small model, the results are remarkably good; they don’t look quite natural, but they’re quite recognizable. For more on how diffusion image generators work, check out our article on Stable Diffusion.
