Weiyang Liu

@Besteuler

Directing SphereLab @CUHKofficial. Postdoc @MPI_IS. PhD @Cambridge_Uni & @GeorgiaTech. Previous Intern @Google & @nvidia. All opinions are my own.

wyliu.com

Joined May 2009

795 Following

2,583 Followers

Replies Reposts Media

Weiyang Liu
@Besteuler
23h
🤔 Do more loops actually make a language model better? If so, for what tasks? Can a looped language model keep benefiting from additional loops beyond the recurrence depth it was trained on? 🔥 In our new paper, we systematically study what makes recurrence effective in looped
1
Weiyang Liu
@Besteuler
Sep 30
We have evaluated all kinds of depth scaling methods, ranging from pre-LN scaling to Hyper-connections, mHC and attention residuals. Each shows quite different scaling behavior and computational depth. In our results, we found that Hyper-connections work surprisingly well. Check
Keyu Wang
@Keyuciallo
Sep 29
🔥 Frontier labs are redesigning the residual stream to beat the curse of depth: Kimi K3 uses AttnRes, DeepSeek V4 uses mHC, ByteDance proposed HC, and there are more: LNS, KEEL, MoDA… 🙋 But each method was tested with its own training budget, model design and codebase, so the
Weiyang Liu
@Besteuler
Sep 24
Excited that Pion is accepted to NeurIPS'26! See you in Sydney! 🎊
Weiyang Liu
@Besteuler
May 13
🚀 We are very excited to introduce Pion — a spectrum-preserving optimizer for LLM training. Pion shows strong training stability in practice. This is the project that we have been working on since POET. The central idea is to turn POET/POET-X (spherelab.ai/poet;
Weiyang Liu
@Besteuler
Sep 1
Great work led by Jiale! NoRA builds on LoRA’s parameterization, with the down-projection matrix normalized. Even more surprisingly, applying this normalization only at initialization is already sufficient to obtain considerable performance gains. More details in
J.L
@Jo1uck
Sep 1
1/ How can we make Low-Rank Matrices behave more like Full-Rank Matrices—and achieve higher rank efficiency? We release our new paper: NoRA: Normalized Low-Rank Adaptation. Paper: arxiv.org/abs/2608.31036
1
Weiyang Liu
@Besteuler
Aug 23
Our recent works, DiagDistill (ICLR’26) and XYZFlow (ICML’26), are built on a simple principle of progressively strengthening constraints. Less trajectory ambiguity → fewer-step generation. The key is to spend more compute early, when generation is uncertain. As context
2