Yupei Du

@YupeiDu

Postdoc at @LstSaar @SIC_Saar. LLM reasoning. On industry job market

Saarbrücken, Germany

ypdu.github.io

Joined July 2017

813 Following

123 Followers

Replies Reposts Media

Pinned
Yupei Du
@YupeiDu
Nov 1, 2025
How do language models memorize noise while reason impressively well? Our #EMNLP2025 (poster, Nov 5, 11:00-12:30, Hall C) paper shows that memorization reuses internal mechanisms of generalization, even when they are not related to each other! arxiv.org/abs/2507.04782
1
Yupei Du
@YupeiDu
Nov 1
Great work! MTP predicts the data, but speculative decoding only needs them to agree with the model's own NTP. Still lots of room in checking whether we optimize what we actually want!
Jayden Teoh
@jayden_teoh_
Oct 30
Multi-Token Prediction (MTP) can be done much better. 🚀 Turn out,you can recover the gradient for KL self-distillation to next-token prediction (NTP) for free without ever computing the KL. You just need a one-line code change: h_ntp += h_mtp.detach() - h_mtp Soft Multi-Token
Yupei Du
@YupeiDu
Oct 2
Fascinating
Celeste 🇧🇪
@celestepoasts
Sep 24
Replying to @celestepoasts
results for all claudes
Yupei Du
@YupeiDu
Jul 5
At ICML in Seoul 🇰🇷 @icmlconf! Would love to chat about developing better supervision and training environments for long-horizon reasoning — especially how models can do more reasoning with fewer tokens and solve harder problems more reliably. Reply or DM if you’re interested!
2
Yupei Du
@YupeiDu
Jul 3
We characterize the types of planning tasks a transformer can and cannot verify ⬇️ looking forward to chatting in Seoul, about reasoning, planning, the capabilities of transformers, or just to have some fun!
Yash Sarrof
@yashYRS
Jul 3
Will be presenting our paper in Seoul, ICML along with the amazing @YupeiDu @Stein1Katharina Tuesday July 7th, 2:00 PM to 3:45 PM Poster Hall A: Number 4611 If you are interested in Transformer theory, generalization or planning , please drop by! Looking forward :)

Yupei Du

Yupei Du’s posts