We ran 1,373,020 benchmarks across 222 open ML kernel implementations. The result is Popcorn: the world’s largest collection of verified ML kernels.
Today, we’re releasing it to everyone.
Ben
@SolidlySheafy
Joined March 2024
- One Layer Deeper is officially live! The motivating idea is that some models just don’t want to learn. Most optimizer benchmarks ask how quickly you can train a given model, but baked into that question is the assumption that it can be trained (well) at all. We think thisThe beta period for OneLayerDeeper is over! We are very excited to collaborate on this competition with the great folks @CoreAutoAI Rules: onelayerdeeper.ai Blog post: blog.tilderesearch.com/blog/one-layer… Competition lasts until the end of August!
- This was incredibly fun! We shared a small preview of One Layer Deeper! Really excited for the competition to officially start and to see what people try.We had the @CoreAutoAI housewarming party a couple of weeks back. It was tons of fun and should hopefully give you a better idea of where we're spending some of our time improving model architectures and systems 00:00: Introduction 00:40: @MillionInt (@CoreAutoAI), Building the
- Really excited to put this out! Very cool to see elegant preconditioning and careful numerics lead to such strong empirical results.Introducing Online KL Shampoo (OKLS), an optimizer that brings a KL-optimal approximation of full-matrix AdaGrad to language-model training. Diagonal optimizers ignore correlations between gradient coordinates. Full-matrix AdaGrad captures this geometry but requires quadratic
- Very excited to see what people come up with! The task is fundamentally serial, so will hopefully lead to some quite wacky (and fun!) ideas!onelayerdeeper.ai There were (and still are) kernel competitions. Now we want to present something new: optimizer competition. Show your skills, and learn along the way optimizing through hard problems.
