OP-Mix, neural neural scaling laws, and simulating human memory are all going to NeurIPS!
see you in sydney 🇦🇺 (? provided that i can successfully turn money into a conference registration)
What is the right data mix, and how do we find it as the data keeps changing?
This is a core, unsolved problem in continual learning. To tackle it, we built a data mixing algo that works everywhere — pretraining, midtraining, instruction tuning
Introducing: On-Policy Mix
🧵1/6
Heading to #ICML2026🇰🇷today! Top of mind for me:
- self-improvement & continual learning (what I'm doing this summer)
- user simulation
- data, as always (data mixing, synthetic pretraining, etc.)
Excited to meet new people! DM if you'd like to chat🥳
