Xin Tao

Senior Researcher

Visual Generation Group
Large Model and Multimedia Technology
Kuaishou Technology
Shenzhen, Guangdong, China

Email: [at] gmail


Biography

I am currently a Senior Researcher and Tech Lead in Visual Generation Group (a.k.a. Kling Team) of Kuaishou, China. I joined Kuaishou in 2020, and previously held positions in both Video Technology Department and Y-Tech. I was responsible for spearheading the research and tech transfer of practical video editing algorithms, as well as a comprehensive system of 2D digital human for live streaming purpose.

Before that, I was a Senior Researcher in YouTu X-Lab of Tencent, since 2018. I obtained my Ph.D. degree in Computer Science and Engineering Department in the Chinese University of Hong Kong (CUHK) since 2013. My supervisor is Prof. Jiaya Jia. I received the B. Eng. degree in Information Engineering from Shanghai Jiao Tong University (SJTU) in 2013, supervised by Prof. Guangqiang He.

My research interest includes visual restoration and generation. Currently, I am currently focusing on the research and deployment of efficient systems for video generation and editing. We are actively seeking talented and self-motivated research interns or full-time researchers to join us. Please feel free to contact us via taoxin@kuaishou.com.

Work and Research Experience

Tech Lead
Visual Generation Group | Kuaishou
November 2024 - present | Shenzhen, China
Mixed Reality Group, Y-tech | Kuaishou
November 2023 - 2024 | Shenzhen, China
Video Processing and Analysis Group, Video Technology | Kuaishou
November 2020 - 2023 | Shenzhen, China
Senior Researcher
Youtu Lab | Tencent
November 2018 - 2020 | Shenzhen, China
PhD
Department of Computer Science and Engineering
August 2013 - September 2018 | The Chinese University of Hong Kong

    Advisor: Jiaya Jia

Research Intern
Youtu Lab | Tencent
June 2017 - September 2017 | Shenzhen, China

    Advisor: Yu-Wing Tai

Research Intern
Creative Technologies Lab | Adobe Research
June 2016 - August 2016 | Greater Seattle Area

    Advisor: Jue Wang

Research Assistant
Laboratory of Quantum Nonlinear Photonics (QNP)
January 2011 - January 2013 | Shanghai Jiao Tong University

    Advisor: Guangqiang He

Tech Transfers

[2024-06] Our Sora-like text-to-video tool, Kling, is available online for invitation-only users.
[2023-08] We have officially launched an out-of-the-box 2D digital human live streaming platform VirtualLive, which has now been used in e-commerce and advertising.
We have deployed dozens of cutting-edge video editing algorithms in Kuaiying App.
To be updated.

Publications [Google Scholar]

Perception-Oriented Video Frame Interpolation via Asymmetric Blending

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.

Guangyang Wu, Xin Tao, Changlin Li, Wenyi Wang, Xiaohong Liu, Qingqing Zheng

Paper Arxiv

Feature Decoupling-Recycling Network for Fast Interactive Segmentation

ACM International Conference on Multimedia (ACM MM), 2023.

Huimin Zeng, Weinong Wang, Xin Tao, Zhiwei Xiong, Yu-Wing Tai, Wenjie Pei

Paper Arxiv

Scene-Generalizable Interactive Segmentation of Radiance Fields

ACM International Conference on Multimedia (ACM MM), 2023.

Songlin Tang, Wenjie Pei, Xin Tao, Tanghui Jia, Guangming Lu, Yu-Wing Tai

Paper Arxiv

Compression-Aware Video Super-Resolution

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

Yingwei Wang, Takashi Isobe, Xu Jia, Xin Tao, Huchuan Lu, Yu-Wing Tai

Paper

DeViT: Deformed Vision Transformers in Video Inpainting

ACM International Conference on Multimedia (ACM MM), 2022.

Jiayin Cai, Changlin Li, Xin Tao, Chun Yuan, Yu-Wing Tai

Paper Arxiv

Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

Takashi Isobe, Xu Jia, Xin Tao, Changlin Li, Ruihuang Li, Yongjie Shi, Jing Mu, Huchuan Lu, Yu-Wing Tai

Project Paper Arxiv

Text-guided Human Image Manipulation via Image-text Shared Space

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021.

Xiaogang Xu, Ying-Cong Chen, Xin Tao, Jiaya Jia

Project Paper

MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

Liying Lu, Wenbo Li, Xin Tao, Jiangbo Lu, Jiaya Jia

Project Paper Arxiv

VCNet: A Robust Approach to Blind Image Inpainting

European Conference on Computer Vision (ECCV), 2020.

Yi Wang, Ying-Cong Chen, Xin Tao, Jiaya Jia

Project Arxiv

MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution

European Conference on Computer Vision (ECCV), 2020.

Wenbo Li, Xin Tao, Taian Guo, Lu Qi, Jiangbo Lu, Jiaya Jia

Project Paper Arxiv

Particularity beyond Commonality: Unpaired Identity Transfer with Multiple References

European Conference on Computer Vision (ECCV), 2020.

Ruizheng Wu, Xin Tao, Yingcong Chen, Xiaoyong Shen, Jiaya Jia

Project Paper

Wide-context semantic image extrapolation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Yi Wang, Xin Tao, Xiaoyong Shen, Jiaya Jia

Project Paper

Attribute-Driven Spontaneous Motion in Unpaired Image Translation

IEEE International Conference on Computer Vision (ICCV), 2019.

Ruizheng Wu, Xin Tao, Xiaodong Gu, Xiaoyong Shen, Jiaya Jia

Project Paper Arxiv

Dynamic Scene Deblurring with Parameter Selective Sharing and Nested Skip Connections

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Hongyun Gao, Xin Tao, Xiaoyong Shen, Jiaya Jia

Project Paper

Semantic Regeneration Network

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Yi Wang, Xin Tao, Xiaoyong Shen, Jiaya Jia

Project Paper

Image Inpainting via Generative Multi-column Convolutional Neural Networks

Conference on Neural Information Processing Systems (NeuIPS), 2018.

Yi Wang, Xin Tao, Xiaojuan Qi, Xiaoyong Shen, Jiaya Jia

Project Arxiv

Scale-recurrent Network for Deep Image Deblurring

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, Jiaya Jia

Project Paper Arxiv

Facelet-Bank for Fast Portrait Manipulation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

Ying-Cong Chen, Huaijia Lin, Michelle Shu, Ruiyu Li, Xin Tao, Xiaoyong Shen, Yangang Ye, Jiaya Jia

Project Arxiv

Zero-order Reverse Filtering

IEEE International Conference on Computer Vision (ICCV), 2017.

Xin Tao, Chao Zhou, Xiaoyong Shen, Jue Wang, Jiaya Jia

Project Paper Arxiv

Detail-revealing Deep Video Super-resolution

IEEE International Conference on Computer Vision (ICCV),Oral, 2017.

Xin Tao, Hongyun Gao, Renjie Liao, Jue Wang, Jiaya Jia

Project Paper Arxiv

High-Quality Correspondence and Segmentation Estimation for Dual-Lens Smart-Phone Portraits

IEEE International Conference on Computer Vision (ICCV), 2017.

Xiaoyong Shen, Hongyun Gao, Xin Tao, Chao Zhou,Jiaya Jia

Arxiv

Regional Foremost Matching for Internet Images

ACM Transactions on Graphics (Proc. SIGGRAPH ASIA), 2016.

Xiaoyong Shen, Xin Tao, Chao Zhou, Hongyun Gao, Jiaya Jia

Paper

Deep Automatic Portrait Matting

European Conference on Computer Vision (ECCV),Spotlight, 2016.

Xiaoyong Shen, Xin Tao, Hongyun Gao, Chao Zhou, Jiaya Jia

Paper

Break Ames Room Illusion: Depth from General Single Images

ACM Transactions on Graphics (Proc. SIGGRAPH ASIA), 2015.

Jianping Shi, Xin Tao, Li Xu, Jiaya Jia

Project Paper

Video Super-Resolution via Deep Draft-Ensemble Learning

IEEE International Conference on Computer Vision (ICCV), 2015.

Renjie Liao, Xin Tao, Ruiyu Li, Ziyang Ma, Jiaya Jia

Project Paper

Handling Motion Blur in Multi-Frame Super-Resolution

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.

Ziyang Ma, Renjie Liao, Xin Tao, Li Xu, Jiaya Jia, Enhua Wu

Project Paper

Inverse Kernels for Fast Spatial Deconvolution

European Conference on Computer Vision (ECCV), 2014.

Li Xu, Xin Tao, Jiaya Jia

Project Paper

Education Background

Ph.D.
Computer Science and Engineering
August 2013 - September 2018 | The Chinese University of Hong Kong

    Advisor: Jiaya Jia

B. Eng.
Information Engineering | Honored Class (TRC)
January 2011 - January 2013 | Shanghai Jiao Tong University

Professional Activities

Journal Reviewer
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
Transaction on Graphics (TOG)
International Journal of Computer Vision (IJCV)
Transactions on Image Processing (TIP)
IEEE Transactions on Multimedia
Conference Reviewer
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018-2024
IEEE International Conference on Computer Vision (ICCV) 2017-2023
European Conference on Computer Vision (ECCV) 2018-2024
International Conference on Machine Learning (ICML) 2022-2023
International Conference on Learning Representations (ICLR) 2021-2023
Conference on Neural Information Processing Systems (NeurIPS) 2021-2023
Asian Conference on Computer Vision (ACCV) 2018

Teaching

CSCI3290 Computational Photography 2013-2014 / 2014-2015 / 2015-2016 / 2016-2017 Fall
CSCI1050 Hands-on Introduction to MATLAB 2013-2014 Spring
CSCI5280 Image Processing and Computer Vision 2014-2015 / 2015-2016 Spring