Excited to introduce PolaRiS, a real-to-sim recipe for turning short real-world videos into high fidelity simulation environments for scalable and reliable zeroshot generalist policy evaluation.
polaris-evals.github.io
(1/N 🧵)
(1/6) From @GoogleDeepMind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for
This robot has never seen a real-life gear, but assembles one with a 94% success rate! It can also thread an M16 nut and insert a 16mm peg. All trained with RL and transferred zero-shot sim2real from RGB. 🔈
Our method, Success-Guided Sampling (SGS), makes a small change to how
Today, we're introducing SimFoundry, our real2sim2real framework at NVIDIA GEAR that automatically turns real-world scenes into simulation-ready worlds from a single image or video.
Website: research.nvidia.com/labs/gear/simf…
Paper: arxiv.org/abs/2606.28276
This work marks a major step for
Policies trained on real robot data via imitation can be surprisingly capable. But for domains like dexterous manipulation, they are often not quite good enough: they move slowly, miss grasps, make unreliable contact, and fail under small perturbations.
Can we improve them