MIT NLP

@nlp_mit

NLP Group at @MIT_CSAIL! PIs: @yoonrkim @jacobandreas @lateinteraction @pliang279 @david_sontag, Jim Glass, @roger_p_levy

Cambridge, MA

Joined March 2025

Pinned
MIT NLP
@nlp_mit
Mar 27, 2025
Hello everyone! We are quite a bit late to the twitter party, but welcome to the MIT NLP Group account! follow along for the latest research from our labs as we dive deep into language, learning, and logic 🤖📚🧠
MIT NLP
@nlp_mit
Aug 23
We do teacher forcing on input tokens, why not do it for RNN memory states as well?
Christopher Manning
@chrmanning
Aug 23
Pretraining Recurrent Networks without Recurrence by @akarshkumar0101 & @phillip_isola is a great paper! It makes an end-run around the problems of RNNs via a transformer teacher to learn good predictive state representations & supervised learning of a memory transition function
MIT NLP
@nlp_mit
Aug 23
new paper! A continual learner must build on an ever-growing history of experience, making adaptive compression of the past a fundamental part of intelligence
Jyo Pari
@jyo_pari
Aug 20
In-context continual learning requires models to accumulate experience and reuse it later in the same sequence. But an RNN compresses an ever-growing history into a fixed-size state, where each token gets a single write into memory. We study dynamic compression: letting the
MIT NLP
@nlp_mit
Jul 3
training LMs to be right in the right way: 👉
Mehul Damani
@MehulDamani2
Jul 3
Higher benchmark scores do not always mean better models for users. Why? We claim that RL teaches LMs to be correct but not how to be correct: code can pass tests but be unreadable; explanations can be right but unclear. How do we train LMs to be right in the right way? (1/n)
MIT NLP
@nlp_mit
Jun 20
variable length transformers!
Zhaofeng Wu
@zhaofeng_wu
Jun 18
Introducing ><former Most transformers are rectangles◻️: every layer has the same width But is that optimal?🤔 We propose variable-width transformers that have different widths across layers, improving loss while cutting compute & KV cache size 🧵