really excited to head home for icml:) and attending the co-located @farairesearch's alignment workshop (for the first time)! would love to meet others interested in training & interpretability
check out @_emliu and our work on pretraining interp! we initially asked if we can predict from simple task learning, can we predict a mode complex learning behavior? super excited for follow-ups as well:)
Copying → morphology/translation → basic arithmetic → complex reasoning & math. Across every model family we tested, LLMs acquire skills in roughly the same order during pretraining.
Can we use this to predict what a model will learn next, just from its internals? 🧵
Our ICML 2025 workshop on Actionable Interpretability drew massive interest. But the same questions kept coming up: What does "actionable" mean? Is it achievable? How?
We're ready to answer.
🧵
@RISKshow best music everrr. whenever friends ask me how i found my favorite songs, i pretend like im one of those cool people who just know cool songs
