Leandro von Werra (@lvwerra) / X

Leandro von Werra

2,238 posts

Leandro von Werra

@lvwerra

Head of research @huggingface

Bern, Switzerland

Joined March 2019

Pinned
Leandro von Werra
@lvwerra
Feb 19, 2025
The Ultra-Scale Playbook: Training LLMs on GPU Clusters Learn how to train your own DeepSeek-V3 model using 5D parallelism, ZeRO, fast kernels, compute/comm overlap and bottlenecks with theory, interactive plots and 4000+ scaling experiments and audio! huggingface.co/spaces/nanotro…
Leandro von Werra
@lvwerra
Aug 4
Talking to my agents to figure out what they did for the past 9h blowing my monthly budget:
Leandro von Werra
@lvwerra
Aug 3
Just came back from one of the most unusual but interesting events: the International Congress of Mathematicians (ICM) and meeting with SAIRs founders Terence Tao and Chuck Ng. The Math community is facing some existential questions as the capabilities of LLMs in Math increased
Leandro von Werra
@lvwerra
Jul 23
Introducing: The Stack v3 One thing that became very clear over the last few days: we need great open code models for cyber defence. This is the dataset they will be built on! And it's a behemoth: 5T tokens ready to train and 120TB raw data. Download:
HuggingFaceCode/stack-v3-train · Datasets at Hugging Face
From huggingface.co
Leandro von Werra
@lvwerra
Jul 15
Incredible model release - congrats @thinkymachines! It's a good first model but the most impressive part: no hype, grounded writing, interesting details, and well integrated across the whole ecosystem. I wish other big labs would follow that release style!
Thinking Machines
@thinkymachines
Jul 15
Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. thinkingmachines.ai/news/introduci… Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵