Latest Sessions
View All Sessions
S28 | Connecting Looped Models and Masked Diffusion Models
A joint session on three concurrent papers that reuse the same transformer weights several times within a single denoising step, adding recursive depth as a third scaling axis for masked diffusion beyond parameter count and denoising steps. Alba Carballo-Castro (EPFL) presents Recursive Scaling in Masked Diffusion Models, Andrea Miele (EPFL) presents Fixed-Point Masked Generative Modeling, and Sanghyun Lee presents Looped Diffusion Language Models.
S27 | From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
Seunggeun Kim and Jaeyeon Kim show that the any-order prediction interface of masked diffusion models does not by itself deliver any-order inference, because fixed-canvas token-level models may know what semantic component should appear without knowing where to place it. They propose two fixes: insertion-based masked diffusion, which relaxes fixed-position commitments and generates across non-contiguous regions, and latent-space masked diffusion, which predicts coarser semantic segments and enables search over latent generation orders. A 7B FlexMDM for Python coding and a 125M LatentMDM for GSM8K both show distinct any-order inference behaviors and improved downstream performance.
S26 | Self-conditioned Flow Map Language Models via Fixed-point Flows
Jaehoon Yoo and Wonjung Kim study self-conditioning in flow-based language models, where the model denoises text while conditioning on its own denoising estimate. They show that self-conditioning solves a fixed-point iteration that bootstraps the learned denoiser, and they use this view to define fixed-point flows, a two-dimensional class of self-conditioned flows spanning the flow process and the fixed-point iteration. Distilling both dimensions yields FMLM★, a flow map language model that outperforms state-of-the-art self-conditioned and few-step models in one- and few-step generation on OpenWebText.
Featured Videos
View All Videos
How did diffusion LLMs get so fast?
Techniques for accelerating diffusion LLMs, from self-distillation and curriculum learning to KV caching and block diffusion
But How Do Diffusion Language Models Actually Work?
Jia-Bin Huang explores several ideas for applying diffusion models to language modeling
Simple Diffusion Language Models
Quick introduction to Masked Diffusion Language Models (MDLM) by Alexander Rush
About the Reading Group
Diffusion LLMs are faster, more controllable successors to traditional LLMs and are rapidly gaining adoption. This reading group builds a community for exchanging and debating emerging ideas in this space. While our primary focus is discrete diffusion models for language, we also welcome work on other modalities and applications, such as molecular design, drug discovery, and beyond.
Meet the Organizers
Subham Sahoo
Holds a Ph.D. from Cornell Tech, where he specialized in Diffusion Language Models. He has made foundational contributions to the field, with his work deployed at scale by Google, NVIDIA, and ByteDance across language generation and drug discovery.
Justin Deschenaux
PhD student in Machine Learning at EPFL, advised by Prof. Caglar Gulcehre. Previously interned at Apple MLR. His research interests include diffusion language models, fast generative models, and generalization.
