Dibya Ghosh

@its_dibya

@AnthropicAI | Previously @ UCBerkeley @ Google Brain Montreal, @physical_int

Joined July 2017

Pinned
Dibya Ghosh
@its_dibya
Jan 26, 2025
With R1, a lot of people have been asking “how come we didn't discover this 2 years ago?” Well... 2 years ago, I spent 6 months working exactly on this (PG / PPO for math+gsm8k), but my results were nowhere as good. Here’s my take on what blocked me and what’s changed: 🧵
Dibya Ghosh
@its_dibya
Apr 13, 2025
There are so many "guide to JAX for Pytorch users", but is there any good "guide to Pytorch for JAX users"? pytorch semantics / notation can be umm.... unintuitive
Dibya Ghosh
@its_dibya
Jan 20, 2025
Dumb hack I realized today: x = stopgrad(2 * x) - x lets you invert the gradient into part of the network. Cram min-max optimizations into otherwise standard loss functions!
Dibya Ghosh
@its_dibya
Oct 23, 2024
Ahhh my favorite number: -2.3819763e38 Anyone know who came up with this number? It's *not* bfloat16.min
Dibya Ghosh
@its_dibya
Sep 28, 2024
me @ my wandb runs before the iclr deadline