Kevin Frans (@kvfrans) / X

Kevin Frans

526 posts

Kevin Frans

@kvfrans

phd @berkeley_ai prev mit, reflection, openai read my thoughts: kvfrans.com

Berkeley, CA

Joined August 2013

Pinned
Kevin Frans
@kvfrans
Nov 25, 2025
New notes: We've been building a research-friendly LLM-RL repo in JAX, and I recently took the time to optimize the sampling/training pipeline. We're able to match vLLM sampling and get decent training batchsizes now! notes.kvfrans.com/7-misc/rl-infr…
14K
Kevin Frans
@kvfrans
Aug 24, 2019
one step closer to AI animation!!!
00:00
Kevin Frans
@kvfrans
Oct 18, 2024
*Shortcut models* are a plug-and-play replacement for diffusion models that can generate in a single step (or more). This speeds up inference by up to 128x. Shortcut models are trained end-to-end, and do not require a separate distillation phase or learning schedules.
142K
Kevin Frans
@kvfrans
Nov 26, 2022
Powderworld is a lightweight simulation environment for understanding AI generalization, supporting: - modular & emergent ruleset - expressive RL task space - 10k timesteps/sec via GPU Paper: arxiv.org/abs/2211.13051 Blog + Online Demo: kvfrans.com/static/powder/
00:00
Kevin Frans
@kvfrans
Jun 29, 2021
CLIPDraw is a way to synthesize stroke-based drawings based on natural language input. New work w/ @crosslabstokyo @err_more @okw ! blog: kvfrans.com/clipdraw-explo… arxiv: arxiv.org/abs/2106.14843 Colab notebook: colab.research.google.com/github/kvfrans…
00:00
Kevin Frans
@kvfrans
Jun 10, 2025
Very excited for this one. We took a cautiously experimental view on NN optimizers, aiming to find something that just works. SPlus matches Adam within ~44% of steps on a range of objectives. Please try it out in your setting, or read below for how it works.
102K
Kevin Frans
@kvfrans
Jul 9, 2025
LLM RL code does not need to be complicated! Here is a minimal implementation of GRPO/PPO on Qwen3, from-scratch in JAX in around 400 core lines of code. The repo is designed to be hackable and prioritize ease-of-understanding for research: github.com/kvfrans/lmpo
32K
Kevin Frans
@kvfrans
Jun 17, 2024
Flow-matching implementation: github.com/kvfrans/jax-fl… Flow-matching is very similar to diffusion, but simplifies things. Noised images are linear interpolations between (data, noise) pairs, and the network predicts *velocity* of this trajectory.
00:00
39K
Kevin Frans
@kvfrans
Nov 3, 2025
By the way, one intuition I like about descent on the spectral norm is the update is *invariant to the input distribution* for certain loss landscapes (specifically, gradients that are orthogonal transformations of the input vector) From our recent blog: kvfrans.com/matrix-whiteni…
00:00
31K
Kevin Frans
@kvfrans
Nov 6, 2025
New notes on PSGD, focusing specifically on the mechanics of gradient descent on the Lie group (and leads into the idea of *relative* gradient descent): notes.kvfrans.com/7-misc/psgd.ht…
23K
Kevin Frans
@kvfrans
Oct 31, 2025
What really matters in matrix-whitening optimizers (Shampoo/SOAP/PSGD/Muon)? We ran a careful comparison, dissecting each algorithm. Interestingly, we find that proper matrix-whitening can be seen as *two* transformations, and not all optimizers implement both. Blog:
83K
Kevin Frans
@kvfrans
Jun 10, 2024
Implementation of Diffusion Transformers in JAX: Code: github.com/kvfrans/jax-di… DiT is pretty great! Things work without much hacking needed. For big images, it helps to do denoising in a latent space instead -- computationally easier, and the noise removes more signal.
00:00
28K
Kevin Frans
@kvfrans
May 30, 2025
Stare at policy improvement and diffusion guidance, and you may notice a suspicious similarity... We lay out an equivalence between the two, formalizing a simple technique (CFGRL) to improve performance across-the-board when training diffusion policies. arxiv.org/abs/2505.23458
53K
Kevin Frans
@kvfrans
Apr 25, 2023
excited to share I will be starting my PhD this fall at @berkeley_ai advised by @pabbeel ! looking forward to working with some amazing people. and, a giant thanks to all of you who have been supporting me so far :)
31K