our members ran a great reading group on major architecture changes in deepseek v4!!
they covered: hyper connections + manifold hyper connections, KV cache + MQA/GQA intro, Deepseek Sparse attention (prerequisite to understanding the new CSA) & a walk through of CSA and HCA
Students at UC Berkeley working on academic research, ML education, industry projects, and fostering a vibrant ML community 🧠💡
- join us for the last bioml seminar of this semester!!We'll be closing out this semester's Berkeley BioML Seminar on 4/28 with a talk from @antoinekoehl on PEINT, a powerful deep learning framework for both phylogenetic inference and protein engineering. Sign up below! luma.com/3fbbftwy
- super proud of members Avy Harish (@AvyukthH60737), Sahir Tandon and Chris John for hosting our first physics informed ml seminar! they covered: physics-informed nns, neural odes, sparse identification of nonlinear dynamics, fourier neural operators & other applications
- > intuitions of how to go from pytorch to jax > training language models from scratch > how to scale ur model pt 2 > lfgggWrote a deep dive on implementing a language model from scratch in JAX and scaling it with distributed training! If you’re coming from PyTorch and want to see how the same ideas look in JAX, or just want a hands-on intro to distributed training, check out this blog post:
- who are the best people to talk to about physics + ml?



