Xibin Song

I'm a research scientist at Tencent XR vision Labs.

Before that, I work as senior researcher at Robotics and Autonomous Driving Laboratory (RAL) of Baidu Research in 2018-2022. I did my PhD and Bachelor at Shandong University, where I was advised by Prof. Xueying Qin. During my PhD, I worked as a joint training of Ph.D. student in the Research School of Engineering at the Australian National University in 2015-2016, where I was advised by Prof. Richard Hartley and Prof. Yuchao Dai.

Email  /  Scholar  / 

Research

My research interests lie in the intersection of 3D computer vision, artificial intelligence, particularly focusing on Perception Technology in Autonomous Driving, 3D AIGC and AR/VR, etc.

This report presents a comprehensive framework for generating high-quality 3D shapes and textures from diverse input prompts, including single images, multi-view images, and text descriptions. The framework consists of 3D shape generation and texture generation.

Large Image and Point Cloud Alignment Model (LAM3D), which utilizes 3D point cloud data to enhance the fidelity of generated 3D meshes.

3D shape generation aims to produce innovative 3D content adhering to specific conditions and constraints. Existing methods often decompose 3D shapes into a sequence of localized components, treating each element in isolation without considering spatial consistency. As a result, these approaches exhibit limited versatility in 3D data representation and shape generation, hindering their ability to generate highly diverse 3D shapes that comply with the specified constraints. In this paper, we introduce a novel spatial-aware 3D shape generation framework that leverages 2D plane representations for enhanced 3D shape modeling.

Night-time self-supervised Depth Estimation for Outdoor Scenes.

We propose a domain generalizable feature extraction network with adaptive guidance fusion (AGDF-Net) to fully acquire essential features for depth estimation at multi-scale feature levels.

We propose to dig into uncertainty estimation for robust stereo matching.

We propose an end-to-end self-regularized network (TUSR-Net) which exploits the contrastive peculiarity of different components of the hazy image, i.e, self-regularization (SR).

In this work, we propose an efficient multi-modal feature fusion based depth completion framework (MFF-Net), which can efficiently extract and fuse features with different modals in both encoding and decoding processes, thus more depth details with better performance can be obtained.

We propose PCW-Net, a Pyramid Combination and Warping cost volume-based network to achieve good performance on both crossdomain generalization and stereo matching accuracy on various benchmarks.

A wavelet spatial attention based multi-stream feedback network (WSAMF-Net) is proposed for effective single image dehazing.

We propose a novel framework, named MLDA-Net, to obtain per-pixel depth maps with shaper boundaries and richer depth details.

We propose a domain-separated network for self-supervised depth estimation of all-day images.

We propose a simple but practical detection framework to jointly predict the 3D BBox and instance segmentation.

We propose a new framework for real-world DSR, which consists of four modules : 1) An iterative residual learning module with deep supervision to learn effective high-frequency components of depth maps in a coarse-to-fine manner; 2) A channel attention strategy to enhance channels with abundant high-frequency components; 3) A multi-stage fusion module to effectively reexploit the results in the coarse-to-fine process; and 4) A depth refinement module to improve the depth map by TGV regularization and input loss.

In this paper, we contribute the first largescale database suitable for 3D car instance understanding – ApolloCar3D.

First, we propose to represent the task of depth map super-resolution as a series of novel view synthesis sub-tasks. The novel view synthesis sub-task aims at generating (synthesizing) a depth map from different camera pose, which could be learned in parallel. Second, to handle large up-sampling factors, we present a deeply supervised network structure to enforce strong supervision in each stage of the network. Third, a multi-scale fusion strategy is proposed to effectively exploit the feature maps at different scales and handle the blocking effect.