Pinned
New paper w/ @benwu_ml and @NeelNanda5!
LLMs don’t just output the next token, they also output confidence. How is this computed?
We find two key neuron families: entropy neurons exploit final LN scale to change entropy, and token freq neurons boost logits proportional to freq 🧵
