Sayan Deb Sarkar

I'm a 2nd-year PhD student at Stanford University in the Gradient Spaces Group, advised by Prof. Iro Armeni, part of the Stanford Vision Lab (SVL). In summer '25, I interned with the Microsoft Spatial AI Lab, working on efficient video understanding in spatial context.

Before starting PhD, I was a CS master student at ETH Zürich, supervised by Prof. Marc Pollefeys, working on aligning real-world 3D environments from multi-modal data. I graduated with a Bachelors in Information Technology from Manipal University, India, where I spent time working on face recognition and medical imaging problems.

In 2020-21, I spent a wonderful time working with Shreyas Hampali and Mahdi Rad at Prof. Vincent Lepetit's lab on hand-object pose estimation and monte carlo scene search for 3D scene understanding. I view them as mentors entering research, and strive to learn from them.

My research interests are on multimodal 3D scene understanding and interactive editing. I am always looking for research collaborations, get in touch if you have something relevant. If you're around the Bay Area, feel free to reach out for a cup of coffee!

Email  /  CV  /  Google Scholar  /  Github  /  Twitter  /  LinkedIn

• 2025-09 GuideFlow3D accepted to NeurIPS 2025. See you in San Diego!
• 2025-07 Invited talk(s) on "Scalable Cross-Modal 3D Scene Understanding" at Google XR Research Munich and Imagine Labs.
• 2025-06 This summer I will be a research intern at Microsoft Spatial AI Lab.
• 2025-03 CrossOver accepted to CVPR 2025 as ✨Highlight✨ (top 3%)!
• 2024-09 Career Update: I joined Stanford University for PhD in Computer Vision.
• 2023-08 I will be a research intern at Qualcomm XR Labs, Amsterdam for the Fall.
• 2023-07 SGAligner accepted to ICCV 2023. First first-author submission, first accept!
• 2022-09 I started Computer Science MSc at ETH Zurich.
---- show more ----
2022-07 Keypoint Transformer accepted to CVPR 2022 as Oral!
2021-05 I started at Mercedes-Benz R & D as a Computer Vision Research Engineer.
2021-03 MCSS accepted at CVPR 2021.
2020-06 General 3D Room Layout from a Single View by Render-and-Compare accepted at ECCV 2020.
2020-01 I started as a Research Assistant at ICG, TU Graz with Prof. Vincent Lepetit, supported by a Qualcomm fellowship.

My research interests lie at the intersection of Computer Vision and Machine Learning, specifically in the areas of multimodal data representations for spatial understanding.

A training-free method that steers pre-trained generative rectified flow with differentiable guidance for robust, geometry-aware 3D appearance transfer across shapes and modalities.

Cross-modal alignment method for 3D scenes that learns a unified, modality-agnostic embedding space, enabling scene-level alignment without semantic annotations.

3D Scene Graph Alignment robust to in-the-wild scenarios powering point cloud registration and map integration.

Efficient network for joint two-hand and object pose estimation in complex interactions, paired with the new H2O-3D dataset of two-hand interaction with YCB objects.

Monte-Carlo Tree Search (MCTS) based analysis-by-synthesis method to recover complete scene (3D layout+objects) from a noisy RGB-D scan.

3D layout estimation from a single perspective view, to recover complex non-cubiod layouts by solving a constrained discrete optimization problem.

Extension of SGAligner using open-vocabulary cues and learned joint embeddings, achieving robust performance under noise and low overlap.
Supervised master student project.

Ray Tracing
Computer Graphics Rendering Competition, Autumn Semester 2022

Implemented a ray tracer with functionalities such as advanced camera models, participating media, photon mapping, Disney BRDF, etc on the Nori framework.