The Top AI Papers of the Week (April 19 - 26)
- Skill-RAG
- DeepSeek V4
- Autogenesis
- Attention to Mamba
- Stateless Decision Memory
- Self-Evolving Logic Synthesis
- Self-Generated World Knowledge
Read on for more:
Heading to Rio 🇧🇷 to present our Celo line of work at #ICLR2026!
Get in touch if you are curious about new avenues in neural network training or how we scaled learned optimizers from CIFAR-10 to GPT-3 🚀
Details ⬇️
Attention → Mamba cross-architecture distillation is real
Transformer doesn't need to stay just a transformer – Apple showed how you can transfer it into a State Space Model (SSM)
▪️ It happens through a linearized-attention intermediate:
1. Distill the Transformer into a
So glad to see this release! Just look at that huge leap in CharXiv figure understanding 🤯
Our tiny team in London worked insanely hard to contribute in this area and even the initial results blew our minds completely. No surprise it's SOTA in this category :)
1/ today we're releasing muse spark, the first model from MSL. nine months ago we rebuilt our ai stack from scratch. new infrastructure, new architecture, new data pipelines. muse spark is the result of that work, and now it powers meta ai. 🧵
