Weiyang Liu

I am an assistant professor of Computer Science and Engineering at The Chinese University of Hong Kong, heading the Scalable Principles for Learning and Reasoning Lab (SphereLab). Previously, I did my postdoc at Max Planck Institute for Intelligent Systems. I have received a Ph.D. in Machine Learning from University of Cambridge, and a Ph.D. in Computer Science from Georgia Tech. I have also spent wonderful time at Google and Nvidia.

I work primarily on principled modeling of inductive bias in learning algorithms. My research seeks to understand how inductive bias affects generalization, and to develop "light-yet-sweet" learning algorithms: (i) light: conceptually simple in methodology and easy to implement in practice, (ii) sweet: having clear intuitions and non-trivial theoretical guarantees.

Over the years, I always find myself fascinated by geometric invariance, symmetry, structures and how they can benefit generalization as guiding principles. Recently, I start rethinking inductive bias for foundation models, and develop a deep interest in large language models and generative modeling across different modalities. My current research focuses on

• developing principled, scalable algorithms and systems for stable and efficient training of foundation models:
  – OPT, OFT(v1,v2), BOFT, VML, POET, POET-X, Pion, Orbit;
• exploring scalable verification for large language models and how it can elicit generalizable reasoning:
  – MetaMath, SGP-Bench, SGP-Gen, BesiegeField, FormalMATH.

Throughout my research journey, I have long been drawn to

• Representation and weight-space geometry for principled deep learning:
  – L-Softmax, SphereFace(v1,v2,Rev), Hyperspherical uniformity, Generalized collapse, FDA, OrthoMerge;
• Data-centric modeling of inductive biases (curriculum learning, data augmentation / selection / distillation):
  – Iterative teaching, Label synthesis, MetaMath, Easy-to-hard.

I always believe in two principles in my research: (i) insight must precede application, and (ii) everything should be made as simple as possible, but not simpler. I try to follow certain research values.

    - Focus on creating novel ideas, not publishing papers
    - Follow curiosity and passion, not trends
    - Ideas are not owned, but come with debts to those who came before
    - Ideas become stronger when shared, discussed and criticized
    - Life is surprisingly short, so solve problems that interest and excite you most
    - It is good to be quick, but it is more important to be deep
    - Think like an amateur, do as an expert
    - This is not only about how to do research, but also how to live your life