Main UI (Light)
Main UI (Dark)
Fly Brain Neurons Firing
Training Results Over Time

Flyft

The complete FlyWire fruit fly connectome (139k neurons, frozen weights, real LIF dynamics) driving a car in a 2D simulator. A population of flies shares one brain; evolutionary strategies evolve only the sensory encoder (retina rays -> visual neuron currents) and motor decoder (descending neuron rates -> steering/throttle).

Inspiration

FlyWire gave the world something unprecedented: a complete synapse-level wiring diagram of an adult fruit fly brain. Most projects treat it as a map to study. We wanted to treat it as a machine to run. A fly's brain is a control system honed by hundreds of millions of years of evolution for fast visual navigation through clutter, and driving a car is, squinting a little, the same problem: look ahead, steer, manage speed. So the question became: if you keep the brain completely frozen, exactly as nature wired it, can you teach it to drive by evolving only the thin adapters that connect it to a new body?

What it does

Flyft loads the full FlyWire connectome and simulates it as a batched leaky integrate-and-fire (LIF) spiking network on the GPU, with every synaptic weight frozen. Around that fixed brain sit two small, evolvable pieces:

an encoder that turns the car's retina rays and speed into input currents on the brain's visual neurons, and
a decoder that reads spike counts from the descending neurons (the ones that would drive motor circuits in a real fly) into steering and throttle.

A whole population of cars shares the one brain and races around procedurally generated tracks in a vectorized 2D sim. Evolutionary strategies (OpenAI-ES, with an ARS elite variant and a (1+lambda) hillclimb mode) optimize only the encoder/decoder genome; the brain itself never learns a thing. The result: a real, unmodified fly brain completing laps.

Everything is watchable in the viz app: an HMI-style dashboard with a replay viewer, a 3D chase cam, per-generation ghost trails so you can see evolution happen, a live training dashboard, and a Race tab where you take the wheel and drive against the fly yourself.

How we built it

Contract-first, in parallel. We froze every module interface in docs/contracts.md up front, then built data, brain, environment, interface, and infra on separate branches against synthetic fixtures, merging through an integration branch.
Brain: fly/brain.py is a batched LIF engine in PyTorch, population dimension first, running the ~139k-neuron connectome for a whole population in one set of tensor ops. A calibration pass searches for a global weight scale that lands the network in a plausible firing-rate band, so the frozen brain is neither silent nor seizing.
Body: fly/env.py is a fully vectorized 2D car sim with lidar-style rays, centerline progress rewards, and procedural 16-waypoint tracks.
Evolution: fly/es.py implements OpenAI-ES with antithetic sampling, plus an ARS elite update, a hillclimb mode with 1/5-success sigma adaptation, and sigma pulses to hop out of basins on long stalls. Curriculum flags (--track-width, --max-speed, --episode-steps) let us start easy and anneal toward the real task.
Infra: island runs with different seeds and sigmas on an AWS GPU box (provisioning and teardown scripts in infra/), with champion replays and genomes banked in the repo.
Viz: a dependency-free JS app (viz/) that renders replays, ghost trails, a WebGL 3D chase cam, live training telemetry, and the human-vs-fly race mode.

Challenges we ran into

Waking the brain up. With naive input scaling the frozen connectome either stayed silent or blew up into runaway spiking. We had to calibrate a global synaptic scale against a target firing-rate range before any evolution could get traction.
The fitness plateau. Vanilla ES stalled hard. We fought through it with reward shaping (crash penalties that actually cost progress, wall-graze costs, speed-scaled crash cost, lap-finish bonuses), the ARS elite update, hillclimb mode, sigma pulses on stalls, and a survival-then-speed curriculum.
Making 139k neurons fast enough. Evolution needs thousands of rollouts, so the LIF step had to be ruthlessly batched; the whole population shares one brain simulation.
Reproducibility. Bit-exact replay of training rollouts meant carefully decoupling the brain init seed from the ES sampling seed and threading RNG state through the recorder.
The web being the web. Some hosts refused to serve raw binary replay files (we added a base64 fallback loader), and the 3D view had to degrade gracefully without WebGL float targets.

Accomplishments that we're proud of

A real, complete animal connectome, weights untouched, drives a car around a track.
The learnable surface is tiny: only the encoder and decoder evolve, and it still works.
A GPU LIF engine that simulates the full 139k-neuron brain for an entire ES population at training speed.
A viz suite we actually enjoy watching: ghost trails make the evolutionary progress visible generation by generation, and the Race tab makes the result tangible, because losing to a fruit fly is a memorable experience.

What we learned

Reward shaping and curriculum beat algorithm tweaks. Most of our progress came from making dying expensive and the early task survivable, not from fancier optimizers.
A frozen dynamical system is a usable computational reservoir: you can get meaningful control out of a brain you are not allowed to touch, if you learn the right way to talk to it.
Spiking networks are operating-point machines. Calibration (firing-rate targets, input current clamps) mattered as much as architecture.
Fixed interface contracts made parallel development actually work; integration was merges, not rewrites.

What's next for Flyft

Generalization: evolve on families of tracks and test on unseen ones, and push the curriculum until the fly handles narrow, fast circuits without hand-holding.
Better senses: richer retina models and letting evolution choose which visual neurons receive input, instead of a fixed grouping.
Baselines: race the frozen connectome against a same-size random reservoir and a tiny trained MLP to quantify how much the fly's wiring actually helps.
New tasks: same brain, different bodies. Parking, drone-style flight, and multi-car racing against other islands' champions.
Public demo: host the live dashboard and race mode so anyone can lose to the fly.