Log inSign up
Image
user avatar
Weiyang Liu
@Besteuler
Assistant Professor of CSE @CUHKofficial. Postdoc @MPI_IS. PhD @Cambridge_Uni & @GeorgiaTech. Previous Intern @Google & @nvidia. All opinions are my own.
wyliu.com
Joined May 2009
791
Following
2,525
Followers
RepliesMedia
  • user avatar
    Weiyang Liu
    @Besteuler
    Jul 29
    Great work! Glad to see spectrum preservation serves as a central principle for improving RLVR. We also have proposed a new optimizer called Pion (spherelab.ai/pion/) that is designed with spectrum preservation as the first principle. Pion not only work well for RLVR, but
    user avatar
    Hanqing Zhu
    @zhu_hanqing666
    Jul 22
    People keep asking me: what's different about optimization in RL? Seemingly nothing — the pre-training stack just works (Adam, even SGD 👀 @saagnikkk). Bringing some answers from my last work (sorry for the delay — been cooking 🚀). We introduce ISO: Isospectral Optimization:
    Image
  • user avatar
    Weiyang Liu
    @Besteuler
    Jul 16
    Excited to share SymbOmni, our ECCV’26 work on a self-evolving agentic system for complex image and video generation/editing. SymbOmni learns new symbolic concepts by composing existing skills. Its self-evolution is driven by verbalized backpropagation, where successful past
    Image
  • user avatar
    Weiyang Liu
    @Besteuler
    Jun 8
    Great post. I think there is much more to unpack behind the spectrum-preserving update rule in Pion (spherelab.ai/pion). Jianlin derives a different spectrum-preserving update rule from a steepest-descent perspective, leading to an alterantive orthogonalization (msign).
    user avatar
    jianlin.su
    @Jianlin_S
    Jun 8
    Steepest Descent on Manifolds: 6. Muon + Double Rotation kexue.fm/archives/11777 Introduces MuonR — a Muon variant that constrains updates to left & right rotation matrices. This preserves the singular value distribution of weights, providing a clean, elegant way to maintain
  • user avatar
    Weiyang Liu
    @Besteuler
    Jun 7
    In PEFT-Arena (spherelab.ai/PEFT-Arena), we found a “free lunch” that improves both adaptation performance and preservation of general capabilities across PEFT methods, including full-parameter finetuning. The trick is simple: interpolate the weights and choose a midpoint between
    Image

  • See @Besteuler's full profile

    Sign up
    Log in