Arthur Douillard (@Ar_Douillard) / X

Arthur Douillard

5,238 posts

Arthur Douillard

@Ar_Douillard

Distributed Learning @ deepmind | DiLoCo, DiPaCo. Continual Learning PhD @ Sorbonne

London

Joined January 2016

Pinned
Arthur Douillard
@Ar_Douillard
Apr 23
The DiLoCo team at Google DeepMind and Google Research is proud to release Decoupled DiLoCo, the next frontier for resilient AI pre-training. Decoupled DiLoCo enables training with datacenters across the world, using heterogeneous hardware, and never halting the system despite
GIF
2.7M
Arthur Douillard
@Ar_Douillard
Oct 9, 2024
The Nobel prize website has a pretty well made 1-pager on how Alphafold works
475K
Arthur Douillard
@Ar_Douillard
Oct 24, 2025
Anthropic
@AnthropicAI
Oct 23, 2025
Today, we announced that we plan to expand our use of Google TPUs, securing approximately one million TPUs and more than a gigawatt of capacity in 2026.
182K
Arthur Douillard
@Ar_Douillard
Nov 6, 2025
I see a lot of bad takes on X about PhDs and frontier labs (not just this quoted tweet), so let me chime in. For context, I didn't do a prestigious undergrad, worked a bit in a startup as an applied ML engineer, then did a PhD, and now work in a frontier lab. A PhD isn't
Karan🧋
@kmeanskaran
Nov 6, 2025
HOT TAKE: Reality is, you can't actually work in top-quality ML research labs without a PhD. Top research labs still look for people with PhDs and excellence in maths, stats, PyTorch, neural networks, and CUDA kernels. In India, quality ML research labs are virtually
147K
Arthur Douillard
@Ar_Douillard
Feb 15, 2025
from @JeffDean at @dwarkesh_sp podcast: "asynchronous training where each copy of the model does local computation [...] it makes people uncomfortable [...] but it actually works" yep, i can confirm, it does work for real
00:00
101K
Arthur Douillard
@Ar_Douillard
Aug 26, 2021
I've released my course on deep learning for computer vision! It includes slides, google colab, and Anki decks for all 6 topics I'm covering. We code from the basics (backprop from scratch) to the SotA (transformers & MLP-mixer). arthurdouillard.com/deepcourse/ Feedback appreciated!
Arthur Douillard
@Ar_Douillard
May 18, 2021
Vision transformers are more biased towards shapes (as humans are) than Convolutional Networks: arxiv.org/abs/2105.07197
Arthur Douillard
@Ar_Douillard
Nov 29, 2024
Excellent explanation of RoPE embedding, from scratch with all the math needed: fleetwood.dev/posts/you-coul… And with beautiful 3blue1brown's style of animation: github.com/3b1b/manim. Original RoPE paper: arxiv.org/abs/2104.09864
57K
Arthur Douillard
@Ar_Douillard
Jan 31, 2025
We release today the next step for distributed training: --> Streaming DiLoCo with Overlapping Communication. TL;DR: train data-parallel across the world with low-bandwidth for the same performance: 400x less bits exchanged & huge latency tolerance
112K
Arthur Douillard
@Ar_Douillard
Oct 15, 2025
I'm AGI-pilled, but things like this make me think we may be in a bubble.
Karina
@karinanguyen
Oct 15, 2025
After a formative time at OpenAI, I’m launching Maison AGI (@maisonagi), a fashion house creating cultural artifacts for the AI era. Our first collection, Relic of Thought, is a collaboration with Ilya Sutskever (@ilyasut), featuring his original artworks alongside a signature
00:00
93K
Arthur Douillard
@Ar_Douillard
Jun 1, 2022
I am excited to share that, after my PhD 👨‍🎓, I will join @DeepMind this summer as a Research Scientist in the Continual Learning team led by Marc'Aurelio Ranzato! 🎉
Arthur Douillard
@Ar_Douillard
Oct 14, 2024
Distributed learning? Recently, @PrimeIntellect have announced their 10B distributed learning (x.com/PrimeIntellect…), what is it exactly? Going back to the origin, Federated Learning (FL, arxiv.org/abs/1602.05629) aims to train a model across a fleet of phones. To handle the
00:20
Prime Intellect
@PrimeIntellect
Oct 11, 2024
Announcing INTELLECT-1: the first-ever decentralized training of a 10B model Scaling decentralized training 10x beyond prior efforts. Anyone can join us to build open-source AGI 🦋
69K
Arthur Douillard
@Ar_Douillard
Oct 23, 2025
I’ve been promoted to Staff RS. Vain title etc. but feels good to see appreciation for distributed learning in DeepMind ☺️
40K
Arthur Douillard
@Ar_Douillard
Jul 30, 2024
"White House says no need to restrict ‘open-source’ artificial intelligence — at least for now"
White House says no need to restrict 'open-source' artificial intelligence — at least for now
From apnews.com
130K