Our Research



Members in MMLab@NTU conduct research primarily in low-level vision, image and video understanding, creative content creation, 3D scene understanding and reconstruction.


Our team is the first to introduce the use of deep neural networks to directly predict super-resolved images. Our journal paper on image super-resolution was selected as the `Most Popular Article' by IEEE Transactions on Pattern Analysis and Machine Intelligence in 2016. It remains as one of the top 10 articles to date. Popular image and video super-resolution methods developed by our team include SRCNN, ESRGAN, EDVR, GLEAN and BasicVSR.

Content Editing and Generation

We research new methods for generating high-resolution, realistic and novel contents in images and videos. We are also interested in investigating fundamental concepts in generative models. Some of our works include scene deocclusion, video inpainting, image generation, and image manipulation.

Image and Video Understanding

We explore effective and efficient methods to detect, segment and recognize objects in complex scenes. We were the champion in COCO 2019 Object Detection Challenge 2019, and Open Images Challenge 2019.

3D Scene Understanding

Our team has been working on various tasks related to 3D reconstruction and perception, e.g, 3D shape generation and 3D human recovery. Check out our recent work on Variational Relational Point Completion Network, Unsupervised 3D Shape Completion through GAN Inversion and LiDAR-based Panoptic Segmentation.

Deep Learning

We investigate new deep learning methods that are more efficient, robust, accurate, scalable, transferable, and explainable. We have been working on various problems like domain generalization, knowledge distillation, long-tailed recognition, and self-supervised learning. We participated in Facebook AI Self-Supervision Challenge 2019, and we won champions of all four tracks. Check out OpenSelfSup, our open-source project for self-supervised learning.

Media Forensics

The popularization of Deepfakes on the internet has set off alarm bells among the general public and authorities, in view of the conceivable perilous implications. We have proposed two large-scale datasets for face forgery detection. See DeeperForensics and ForgeryNet.



  • Outstanding Reviewers, ICCV 2021
    Chongyi Li, Jingkang Yang, Kelvin Cheuk Kit Chan, Liang Pan, Zhongang Cai, and Kaiyang Zhou
  • AISG PhD Fellowship, 2021
    Jiawei Ren, Zhaoxi Chen
  • Outstanding Reviewers, CVPR 2021
    Chongyi Li, Davide Moltisanti, Xiangyu Xu, Liang Pan, and Jiahao Xie
  • Three Champions in NTIRE 2021 Challenge on Video Restoration and Enhancement, 2021
    Kelvin Cheuk Kit Chan, Shangchen Zhou, Xiangyu Xu
    Team members: K. C. K. Chan, S. Zhou, X. Xu, C. C. Loy
  • Best neural planning metric (PKL), second runner-up, nuScenes Detection Challenge of 5th AI Driving Olympics, NeurIPS 2020
    Wenwei Zhang
    Team members: W. Zhang, T. Wang, K. Chen, D. Lin, C. C. Loy
  • Top 10% Outstanding Reviewer, NeurIPS 2020
    Shangchen Zhou
  • Champion, COCO 2019 Object Detection Challenge (Without External Data), 2019
    Jiaqi Wang, Wenwei Zhang, Kai Chen
    Team members: J. Wang, W. Zhang, Y. Cao, K. Chen, J. Pang, T. Gong, J. Shi, C. C. Loy, D. Lin
  • Champions of all four tracks, Facebook AI Self-Supervision Challenge, 2019
    Xiaohang Zhan, Jiahao Xie
    Team members: X. Zhan, J. Xie, Z. Liu, Y. S. Ong, C. C. Loy
  • Champion, Open Images Challenge (Object Detection), 2019
    Yuhang Zang
    Team members: Y. Liu, G. Song, Y. Zang, Y. Gao, J. Yan, C. C. Loy, X. Wang
  • Outstanding Reviewer, ICCV 2019
    Tak-Wai Hui
  • Outstanding Reviewer Honorable Mention, BMVC 2019
    Xintao Wang
  • Outstanding Reviewers, CVPR 2019
    Xintao Wang, Ke Yu
  • Champions of all four tracks, NTIRE 2019 Challenge on Video Restoration and Enhancement, 2019
    Xintao Wang, Kelvin Cheuk Kit Chan, Ke Yu
    Team members: X. Wang, K. C. K. Chan, K. Yu, C. Dong, C. C. Loy
  • Champion, PIRM Challenge on Perceptual Super-Resolution (Third Region), 2018
    Xintao Wang
    Team members: X. Wang, S. Wu, J. Gu, K. Yu, Y. Liu, C. Dong, Y. Qiao, C. C. Loy
  • First Runner-up, DAVIS Challenge on Video Object Segmentation, 2018
    Xiaoxiao Li
    Team members: X. Li and C. C. Loy


Media Coverage | Blog

Source: @Luke_Aaron


Video Demos

Scene De-occlusion
Face Swapping
Pareidolia Face Reenactment
Effect Interpolation
Video Inpainting
Image Generation and Extrapolation
Low-Light Image Enhancement
Point Cloud Completion
Image Restoration and Manipulation
Optical Flow Estimation
Audio-Driven Face Reenactment
Motion Retargeting
Anime Super-Resolution (Credit: Jan Bing)
Game Super-Resolution (Credit: Snouz)
Anime Video Interpolation