Matt Beton

@MattBeton

ml systems & algos | prev @exolabs, maths @Cambridge_Uni

london

mattbeton.com

Replies Reposts Media

Pinned
Matt Beton
@MattBeton
Feb 21, 2025
Training an LLM on 8 M4 Mac Minis Ethernet interconnect between Macs is 100x slower than NVLink so Macs can’t synchronise model gradients every training step. I got DiLoCo running so Macs synchronise once every 1000 training steps using 1000x less communication than DDP
166
Matt Beton
@MattBeton
Jul 21
I built a 50k parameter language model to run on an 8-bit 6502 processor from 1975. It learnt to spell words and construct basic grammar! The model is build on a Mamba backbone (attention’s KV cache not fit in 32KB of memory) with a character-level tokenizer. The MOS-6502
20
Matt Beton
@MattBeton
Jun 9
fable just smashed the prompt i've been using to test new models for the past year. nothing else has come close. i ask the model to generate a piece of artwork using javascript, inspired by artblocks.io. (prompt and source below)
4
Matt Beton
@MattBeton
Feb 28
It was a pleasure to be able to speak at @GPU_MODE today I spoke about some of my work at @exolabs, discussing ML on consumer devices, and how to do distributed inference and training on Apple Silicon with MLX youtube.com/watch?v=sV0PJC…
3
Matt Beton
@MattBeton
Jan 13
It’s possible to train models of any size using Macs. Pipeline parallelism enables this - putting part of the model on each device. Implementing this with MLX on Apple Silicon was quite a challenge; I’ve written a blog post detailing how it works. 🧵 mattbeton.com/blog/pipeline-…
Matt Beton
@MattBeton
Sep 18, 2025
Finetune DeepSeek 🐳 with two Mac Studios + MLX 🚀 We use pipeline parallelism to split the full 671GB model across two devices connected by a single TB5 cable. LoRA reduces the number of parameters to train from 671 billion down to 37 million, reducing the memory overhead from
3