Xin Tao

Senior Researcher

Visual Generation Group
Large Model and Multimedia Technology
Kuaishou Technology
Shenzhen, Guangdong, China

Email: [at] gmail

Image Image

Image

Biography

I am currently a Senior Researcher and Tech Lead in Visual Generation Group (a.k.a. Kling Team) of Kuaishou, China. I joined Kuaishou in 2020, and previously held positions in both Video Technology Department and Y-Tech. I was responsible for spearheading the research and tech transfer of practical video editing algorithms, as well as a comprehensive system of 2D digital human for live streaming purpose.

Before that, I was a Senior Researcher in YouTu X-Lab of Tencent, since 2018. I obtained my Ph.D. degree in Computer Science and Engineering Department in the Chinese University of Hong Kong (CUHK) since 2013. My supervisor is Prof. Jiaya Jia. I received the B. Eng. degree in Information Engineering from Shanghai Jiao Tong University (SJTU) in 2013, supervised by Prof. Guangqiang He.

My research interest includes visual restoration and generation. Currently, I am currently focusing on the research and deployment of efficient systems for video generation and editing. We are actively seeking talented and self-motivated research interns or full-time researchers to join us. Please feel free to contact us via [email protected].

Work and Research Experience

Tech Lead
Visual Generation Group | Kuaishou
November 2024 - present | Shenzhen, China
Mixed Reality Group, Y-tech | Kuaishou
November 2023 - 2024 | Shenzhen, China
Video Processing and Analysis Group, Video Technology | Kuaishou
November 2020 - 2023 | Shenzhen, China
Image
Senior Researcher
Youtu Lab | Tencent
November 2018 - 2020 | Shenzhen, China
Image
PhD
Department of Computer Science and Engineering
August 2013 - September 2018 | The Chinese University of Hong Kong

    Advisor: Jiaya Jia

Image
Research Intern
Youtu Lab | Tencent
June 2017 - September 2017 | Shenzhen, China

    Advisor: Yu-Wing Tai

Image
Research Intern
Creative Technologies Lab | Adobe Research
June 2016 - August 2016 | Greater Seattle Area

    Advisor: Jue Wang

Image
Research Assistant
Laboratory of Quantum Nonlinear Photonics (QNP)
January 2011 - January 2013 | Shanghai Jiao Tong University

    Advisor: Guangqiang He

Image

Tech Transfers

Image [2024-06] Our Sora-like text-to-video tool, Kling, is available online for invitation-only users.
Image [2023-08] We have officially launched an out-of-the-box 2D digital human live streaming platform VirtualLive, which has now been used in e-commerce and advertising.
Image We have deployed dozens of cutting-edge video editing algorithms in Kuaiying App.
Image To be updated.

Publications [Google Scholar]

Image

Perception-Oriented Video Frame Interpolation via Asymmetric Blending

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.

Guangyang Wu, Xin Tao, Changlin Li, Wenyi Wang, Xiaohong Liu, Qingqing Zheng

Paper Arxiv

Image

Feature Decoupling-Recycling Network for Fast Interactive Segmentation

ACM International Conference on Multimedia (ACM MM), 2023.

Huimin Zeng, Weinong Wang, Xin Tao, Zhiwei Xiong, Yu-Wing Tai, Wenjie Pei

Paper Arxiv

Image

Scene-Generalizable Interactive Segmentation of Radiance Fields

ACM International Conference on Multimedia (ACM MM), 2023.

Songlin Tang, Wenjie Pei, Xin Tao, Tanghui Jia, Guangming Lu, Yu-Wing Tai

Paper Arxiv

Image

Compression-Aware Video Super-Resolution

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

Yingwei Wang, Takashi Isobe, Xu Jia, Xin Tao, Huchuan Lu, Yu-Wing Tai

Paper

Image

DeViT: Deformed Vision Transformers in Video Inpainting

ACM International Conference on Multimedia (ACM MM), 2022.

Jiayin Cai, Changlin Li, Xin Tao, Chun Yuan, Yu-Wing Tai

Paper Arxiv

Image

Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

Takashi Isobe, Xu Jia, Xin Tao, Changlin Li, Ruihuang Li, Yongjie Shi, Jing Mu, Huchuan Lu, Yu-Wing Tai

Project Paper Arxiv

Image

Text-guided Human Image Manipulation via Image-text Shared Space

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021.

Xiaogang Xu, Ying-Cong Chen, Xin Tao, Jiaya Jia

Project Paper

Image

MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

Liying Lu, Wenbo Li, Xin Tao, Jiangbo Lu, Jiaya Jia

Project Paper Arxiv

Image

VCNet: A Robust Approach to Blind Image Inpainting

European Conference on Computer Vision (ECCV), 2020.

Yi Wang, Ying-Cong Chen, Xin Tao, Jiaya Jia

Project Arxiv

Image

MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution

European Conference on Computer Vision (ECCV), 2020.

Wenbo Li, Xin Tao, Taian Guo, Lu Qi, Jiangbo Lu, Jiaya Jia

Project Paper Arxiv

Image

Particularity beyond Commonality: Unpaired Identity Transfer with Multiple References

European Conference on Computer Vision (ECCV), 2020.

Ruizheng Wu, Xin Tao, Yingcong Chen, Xiaoyong Shen, Jiaya Jia

Project Paper

Image

Wide-context semantic image extrapolation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Yi Wang, Xin Tao, Xiaoyong Shen, Jiaya Jia

Project Paper

Image

Attribute-Driven Spontaneous Motion in Unpaired Image Translation

IEEE International Conference on Computer Vision (ICCV), 2019.

Ruizheng Wu, Xin Tao, Xiaodong Gu, Xiaoyong Shen, Jiaya Jia

Project Paper Arxiv

Image

Dynamic Scene Deblurring with Parameter Selective Sharing and Nested Skip Connections

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Hongyun Gao, Xin Tao, Xiaoyong Shen, Jiaya Jia

Project Paper

Image

Semantic Regeneration Network

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Yi Wang, Xin Tao, Xiaoyong Shen, Jiaya Jia

Project Paper

Image

Image Inpainting via Generative Multi-column Convolutional Neural Networks

Conference on Neural Information Processing Systems (NeuIPS), 2018.

Yi Wang, Xin Tao, Xiaojuan Qi, Xiaoyong Shen, Jiaya Jia

Project Arxiv

Image

Scale-recurrent Network for Deep Image Deblurring

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, Jiaya Jia

Project Paper Arxiv

Image

Facelet-Bank for Fast Portrait Manipulation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

Ying-Cong Chen, Huaijia Lin, Michelle Shu, Ruiyu Li, Xin Tao, Xiaoyong Shen, Yangang Ye, Jiaya Jia

Project Arxiv

Image

Zero-order Reverse Filtering

IEEE International Conference on Computer Vision (ICCV), 2017.

Xin Tao, Chao Zhou, Xiaoyong Shen, Jue Wang, Jiaya Jia

Project Paper Arxiv

Image

Detail-revealing Deep Video Super-resolution

IEEE International Conference on Computer Vision (ICCV),Oral, 2017.

Xin Tao, Hongyun Gao, Renjie Liao, Jue Wang, Jiaya Jia

Project Paper Arxiv

Image

High-Quality Correspondence and Segmentation Estimation for Dual-Lens Smart-Phone Portraits

IEEE International Conference on Computer Vision (ICCV), 2017.

Xiaoyong Shen, Hongyun Gao, Xin Tao, Chao Zhou,Jiaya Jia

Arxiv

Image

Regional Foremost Matching for Internet Images

ACM Transactions on Graphics (Proc. SIGGRAPH ASIA), 2016.

Xiaoyong Shen, Xin Tao, Chao Zhou, Hongyun Gao, Jiaya Jia

Paper

Image

Deep Automatic Portrait Matting

European Conference on Computer Vision (ECCV),Spotlight, 2016.

Xiaoyong Shen, Xin Tao, Hongyun Gao, Chao Zhou, Jiaya Jia

Paper

Image

Break Ames Room Illusion: Depth from General Single Images

ACM Transactions on Graphics (Proc. SIGGRAPH ASIA), 2015.

Jianping Shi, Xin Tao, Li Xu, Jiaya Jia

Project Paper

Image

Video Super-Resolution via Deep Draft-Ensemble Learning

IEEE International Conference on Computer Vision (ICCV), 2015.

Renjie Liao, Xin Tao, Ruiyu Li, Ziyang Ma, Jiaya Jia

Project Paper

Image

Handling Motion Blur in Multi-Frame Super-Resolution

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.

Ziyang Ma, Renjie Liao, Xin Tao, Li Xu, Jiaya Jia, Enhua Wu

Project Paper

Image

Inverse Kernels for Fast Spatial Deconvolution

European Conference on Computer Vision (ECCV), 2014.

Li Xu, Xin Tao, Jiaya Jia

Project Paper

Education Background

Ph.D.
Computer Science and Engineering
August 2013 - September 2018 | The Chinese University of Hong Kong

    Advisor: Jiaya Jia

Image
B. Eng.
Information Engineering | Honored Class (TRC)
January 2011 - January 2013 | Shanghai Jiao Tong University
Image

Professional Activities

Journal Reviewer
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
Transaction on Graphics (TOG)
International Journal of Computer Vision (IJCV)
Transactions on Image Processing (TIP)
IEEE Transactions on Multimedia
Conference Reviewer
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018-2024
IEEE International Conference on Computer Vision (ICCV) 2017-2023
European Conference on Computer Vision (ECCV) 2018-2024
International Conference on Machine Learning (ICML) 2022-2023
International Conference on Learning Representations (ICLR) 2021-2023
Conference on Neural Information Processing Systems (NeurIPS) 2021-2023
Asian Conference on Computer Vision (ACCV) 2018

Teaching

CSCI3290 Computational Photography 2013-2014 / 2014-2015 / 2015-2016 / 2016-2017 Fall
CSCI1050 Hands-on Introduction to MATLAB 2013-2014 Spring
CSCI5280 Image Processing and Computer Vision 2014-2015 / 2015-2016 Spring