Millicent Li

@millicent_li

cs phd @ northeastern | ex-ugrad @uwcse and @uwnlp; ai resident @MetaAI (FAIR); @MSFTResearch x2

Cambridge, MA

millicentli.github.io

Joined November 2021

Pinned
Millicent Li
@millicent_li
Jul 3
NLAs look like they "work"---but compared to what? How can we know that they actually tell us anything about model internals? In my new blogpost (and paper, to be presented at ICML!), we investigate whether activation verbalizers (like NLAs) produce faithful explanations.
GIF
03:16
Anthropic
@AnthropicAI
May 7
New Anthropic research: Natural Language Autoencoders. Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read. Here, we train Claude to translate its activations into human-readable text.
Millicent Li
@millicent_li
May 20
How do we figure out whether a LM has learned the right skills, and in what order? We look at model internals :) Great work led by @_emliu!!
Emmy Liu
@_emliu
May 20
Copying → morphology/translation → basic arithmetic → complex reasoning & math. Across every model family we tested, LLMs acquire skills in roughly the same order during pretraining. Can we use this to predict what a model will learn next, just from its internals? 🧵
Millicent Li
@millicent_li
Sep 17, 2025
Wouldn’t it be great to have questions about LM internals answered in plain English? That’s the promise of verbalization interpretability. Unfortunately, our new paper shows that evaluating these methods is nuanced—and verbalizers might not tell us what we hope they do. 🧵👇1/9
Millicent Li
@millicent_li
Apr 23, 2025
I'll be at #ICLR25 to present our spotlight poster (w/ @nlpaxia @ctongfei @ben_vandurme) on Multi-Field Adaptive Retrieval on Friday at 3 PM (Hall 3 + Hall2B)! DM me to chat about this work, interpretability, science of AI, or just to generally chat and eat good food!
Millicent Li
@millicent_li
Oct 29, 2024
Documents 📃 on the internet 🌎 contain structure, such as 'author' and 'biography' fields from Wikipedia pages. Can we improve document ranking based on the field name and its associated text? Our answer: yes we can! 🧵 arxiv link:
arxiv.org
Multi-Field Adaptive Retrieval
Document retrieval for tasks such as search and retrieval-augmented generation typically involves datasets that are unstructured: free-form text without explicit internal structure in each...