Yuandong Tian

@tydsh

Co-founder of @Recursive_SI. ex-Meta FAIR Director. ex-Google. Reasoning, Optimization and Understanding LLM. Novelist in spare time. PhD in @CMU_Robotics.

California, USA

yuandong-tian.com

Replies Reposts Media

Pinned
Yuandong Tian
@tydsh
Jun 11
Early results from Recursive 🚀🚀 SotA results from our open-ended knowledge discovery system: 1️⃣NanoChat 5min pre-training (0.9372 bpb -> 0.9109 bpb, 2.8% lower Bits-Per-Byte than long-standing community SoTA) 2️⃣NanoGPT SpeedRun (79.7s -> 77.5s, 2.8% faster than long-standing
Recursive
@Recursive_SI
Jun 11
Article
First Steps Toward Automated AI Research
Early results from Recursive’s automated AI research system on model training and GPU kernel benchmarks Today we are releasing early results from Recursive’s automated AI research system. Across three...
9
Yuandong Tian
@tydsh
Sep 2
We got exact the same comments when we published Coconut😀. However, even with explicit CoTs, the model can still learn to hide the true thinking process. The ultimate solution is always to understand how the model works.
Amir Efrati
@amir
Sep 2
new: OpenAI & others quietly using loop transformers that don't show their 'thinking' when scaled up a leap forward on performance, but sparking concerns inside & outside OpenAI re: security as this takes off
9
Yuandong Tian
@tydsh
Sep 1
😅 so our StreamingLLM (the original attention sink paper) strikes back again? 😅 -- Caveat: it is in post-training setting.
Alexia Jolicoeur-Martineau
@jm_alexia
Aug 31
Simple beats complicated: We show that switching to a sliding-window attention mask with attention sinks (at no cost) beats linear attention post-training. Huge thanks to my collaborators @RheaSukthanker, @CameronPashmina, and @Emy_Aze. Paper: arxiv.org/abs/2608.28444
4
Yuandong Tian
@tydsh
Aug 28
Interesting findings by our automatic system😆 It finds that some FlashInfer kernels have numerical issues, provides a PR and accepted by human experts🤓
Josh Tobin
@josh_tobin_
Aug 27
A fun small win for automated research: We found and helped fix some edge cases that could affect inference performance in vLLM and SGLang. In our last blog post, we described a reward hacking judge we developed for performance optimization tasks. While applying the judge to
2
Yuandong Tian
@tydsh
Aug 21
赠Zeyuan Allen-Zhu🫡 世言大模惟精调，君执初心问真章。动力学中观涨落，表达性里辨行藏。万里独征云和月，所慕岂在功与赏。他年若问登临处，格物致理照新航。突き破れ、扉の向こうへ！（source: 鋼の錬金術師 ED，《扉の向こうへ》）
Zeyuan Allen-Zhu, Sc.D.
@ZeyuanAllenZhu
Aug 21
I've decided it's time to resign from FAIR. 🫡 I'm especially grateful for the compute that made much of my research possible: 400 H100/200 GPUs allocated to me by FAIR; over a thousand H100s borrowed from FAIR Europe's CodeGen team led by Gabriel (@syhw ); thousands more
7