LMs capture many factual associations, but how do they recall them internally during inference?
In a new preprint, we find that LMs build attribute-rich subject representations, from which attention heads extract the predicted attribute.
@jasmijnbastings @fajtak @amirgloberson 🧵
