Log inSign up
Machine Learning at Berkeley profile banner
user avatar

Machine Learning at Berkeley

@BerkeleyML
Students at UC Berkeley working on academic research, ML education, industry projects, and fostering a vibrant ML community 🧠💡
Berkeley, CA
ml.berkeley.edu
Joined December 2016
112
Following
3,722
Followers
RepliesMedia
  • user avatar
    Machine Learning at Berkeley
    @BerkeleyML
    Aug 14
    Check out this Mixture of Experts project by our member!!
    user avatar
    chloe chia
    @chloewchia
    Aug 13
    This week, I wanted to see if we could get the smallest possible Mixture of Experts going that takes a few hours to pretrain on a single GPU, is fast for experiments, but also somehow performs well compared to larger MoEs. For reference, we have Nanochat for transformer
    Image
  • user avatar
    Machine Learning at Berkeley
    @BerkeleyML
    Aug 13
    check out this Mixture of Experts project by one of our members!!
    user avatar
    chloe chia
    @chloewchia
    Aug 13
    This week, I wanted to see if we could get the smallest possible Mixture of Experts going that takes a few hours to pretrain on a single GPU, is fast for experiments, but also somehow performs well compared to larger MoEs. For reference, we have Nanochat for transformer
    Image
  • user avatar
    Machine Learning at Berkeley
    @BerkeleyML
    Aug 11
    Check out this awesome explainer of DiffusionGemma by one of our members!
    user avatar
    Timothy Gao
    @timg8710
    Aug 10
    This is Google’s new diffusion LLM, DiffusionGemma’s denoising canvas over time. Diffusion LLMs can generate tokens in flexible order. But in practice, do they just become autoregressive anyway? 1/ 🧵
    Image
    GIF
  • user avatar
    Machine Learning at Berkeley
    @BerkeleyML
    May 1
    our members ran a great reading group on major architecture changes in deepseek v4!! they covered: hyper connections + manifold hyper connections, KV cache + MQA/GQA intro, Deepseek Sparse attention (prerequisite to understanding the new CSA) & a walk through of CSA and HCA
    Image