Latent Particle World Models

Self-supervised Object-centric Stochastic Dynamics Modeling

International Conference on Learning Representations (ICLR) 2026 Oral (Top 1.18%)

Tal Daniel¹ Carl Qi² Dan Haramati³ Amir Zadeh⁵ Chuan Li⁵ Aviv Tamar⁴ Deepak Pathak¹ David Held¹

¹Carnegie Mellon University

²UT Austin

³Brown University

⁴Technion

⁵Lambda

@inproceedings{
daniel2026latent,
title={Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling},
author={Tal Daniel and Carl Qi and Dan Haramati and Amir Zadeh and Chuan Li and Aviv Tamar and Deepak Pathak and David Held},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=lTaPtGiUUc}
}

OpenReview

arXiv

Code

Video

Abstract

We introduce Latent Particle World Model (LPWM), a self-supervised object-centric world model scaled to real-world multi-object datasets and applicable in decision-making. LPWM autonomously discovers keypoints, bounding boxes, and object masks directly from video data, enabling it to learn rich scene decompositions without supervision. Our architecture is trained end-to-end purely from videos and supports flexible conditioning on actions, language, and image goals. LPWM models stochastic particle dynamics via a novel latent action module and achieves state-of-the-art results on diverse real-world and synthetic datasets. Beyond stochastic video modeling, LPWM is readily applicable to decision-making, including goal-conditioned imitation learning, as we demonstrate in the paper.

Architecture

Latent Particle World Model (LPWM)

Left: Input frames are encoded into particle sets by the `Encoder` and decoded back to images by the `Decoder`. The `Context` module then processes the particles to sample latent actions, which are combined with the particles in the `Dynamics` module to predict next-step particle states. Right: The `Context` module models the per-particle latent action distribution. During training, we use the latent inverse dynamics head, while at inference, the latent policy is employed for sampling.