Zaid Khan

@codezakh

NDSEG Fellow / PhD @uncnlp with @mohitban47 working on automating env/data generation + program synthesis formerly @allenai @neclabsamerica

Boston, USA

zaidkhan.me

Joined June 2023

1,692 Following

810 Followers

Replies Reposts Media

Pinned
Zaid Khan
@codezakh
Jun 2
Can an LLM act as a selective model of a GPU during evolutionary search, by reasoning + forecasting a kernel’s runtime but deferring to a GPU when unsure? We produced 12k kernels + runtimes from evolutionary search, costing 400M reasoning tokens + 600 GPU-hours to answer this.
5
Zaid Khan
@codezakh
Nov 2
Learned some interesting things about LLM CoTs while working with @XiaoYu_UNC on our recent paper: 1) LLM reasoning is often a post-hoc rationalization (the LLM likely didn't use the CoT to come up with a response)! 2) Standard RLVR often results in this sort of acausal
Justin Chih-Yao Chen
@cyjustinchen
Nov 2
🤔 Rewarding correct answers doesn’t guarantee the model relies on the CoT, and LLM reasoning can often be post-hoc rationalization. Can we improve both accuracy and CoT faithfulness by optimizing CoT causality? 💡 Our key idea: If reasoning causally contributes to the final
1
Zaid Khan
@codezakh
Oct 27
Error-targeted learning is a direction I'm excited about for agentic post-training. The idea is to find mistakes an agent is making, and focus compute on training away those mistakes. It's efficient + tells you how training is changing your agent. We show how to make it work
Runchu Tian
@Runchu_Tian
Oct 27
🤔 How can we efficiently train terminal agents to avoid misbehaviors? 😮 We try error-targeted learning on terminal agents and find that, with < 200 envs in TMax-15K & 35% of TMax's rollout, we can match TMax-4B full recipe. 🚨Introducing MiRL (Misbehavior-driven RL): MiRL
2
Zaid Khan
@codezakh
Oct 21
I thought ppl were exaggerating about ICLR slop deluge but I legitimately had a paper in my ICLR batch containing meta commentary from the authoring LLM saying that the results in the paper do not match the paper's claims and are not statistically significant LMAO
16
Zaid Khan
@codezakh
Oct 21
Creepypasta / SCP idea: When your coauthor accidentally generates a cursed video from the robotics world model containing an… entity
1

Zaid Khan

Zaid Khan’s posts