Chen Change Loy

President's Chair Professor, CCDS, NTU
Director, MMLab@NTU
Co-Associate Director, S-Lab

Blk N4-02b-45, Nanyang Avenue
School of Computer Science and Engineering
Nanyang Technological University
Singapore 639798

Chen Change Loy is a President's Chair Professor with the College of Computing and Data Science, Nanyang Technological University, Singapore. He is the Lab Director of MMLab@NTU and Co-associate Director of S-Lab. He received his Ph.D. (2010) in Computer Science from the Queen Mary University of London. Prior to joining NTU, he served as a Research Assistant Professor at the MMLab of The Chinese University of Hong Kong, from 2013 to 2018.

His research interests include computer vision and deep learning with a focus on image/video restoration and enhancement, generative tasks, and representation learning. He serves as an Associate Editor of the International Journal of Computer Vision (IJCV) and IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). He also serves/served as an Area Chair of top conferences such as ICCV, CVPR, ECCV, and NeurIPS. He is a senior member of IEEE.

Full Biography

MMLab@NTU

MMLab@NTU was formed on the 1 August 2018, with a research focus on computer vision and deep learning. Its sister lab is MMLab@CUHK. It is now a group with four faculty members and more than 40 members including research fellows, research assistants, and PhD students. Members in MMLab@NTU conduct research primarily in low-level vision, image and video understanding, creative content creation, 3D scene understanding and reconstruction.
Visit MMLab@NTU Homepage

Recent

Papers

Nighttime Smartphone Reflective Flare Removal using Optical Center Symmetry Prior
Y. Dai, Y. Luo, S. Zhou, C. Li, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR, Highlight)
[PDF] [arXiv] [Supplementary Material] [Project Page]

Reflective flare is a phenomenon that occurs when light reflects inside lenses, causing bright spots or a “ghosting effect” in photos. we propose an optical center symmetry prior, which suggests that the reflective flare and light source are always symmetrical around the lens’s optical center. This prior helps to locate the reflective flare’s proposal region more accurately and can be applied to most smartphone cameras.

Learning Generative Structure Prior for Blind Text Image Super-resolution
X. Li, W. Zuo, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]

Blind text image super-resolution is challenging as one needs to cope with diverse font styles and unknown degradation. The problem is further compounded given characters of complex structures, e.g., Chinese characters that combine multiple pictographic or ideographic symbols into a single character. In this work, we present a novel prior that focuses more on the character structure. In particular, we learn to encapsulate rich and diverse structures in a StyleGAN and exploit such generative structure priors for restoration.

Generating Aligned Pseudo-Supervision from Non-Aligned Data for Image Restoration in Under-Display Camera
R. Feng, C. Li, H. Chen, S. Li, J. Gu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]

We examine the intrinsic potential of CLIP for pixel-level dense prediction, specifically in semantic segmentation. With minimal modification, we show that MaskCLIP yields compelling segmentation results on open concepts across various datasets in the absence of annotations and fine-tuning. By adding pseudo labeling and self-training, MaskCLIP+ surpasses SOTA transductive zero-shot semantic segmentation methods by large margins.

Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion
Y. Lan, X. Meng, S. Yang, C. C. Loy, B. Dai
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]

StyleGAN has achieved great progress in 2D face reconstruction and semantic editing via image inversion and latent editing. While studies over extending 2D StyleGAN to 3D faces have emerged, a corresponding generic 3D GAN inversion framework is still missing, limiting the applications of 3D face reconstruction and semantic editing. In this paper, we study the challenging problem of 3D GAN inversion where a latent code is predicted given a single face image to faithfully recover its 3D shapes and detailed textures.

Correlational Image Modeling for Self-Supervised Visual Pre-Training
W. Li, J. Xie, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]

We introduce Correlational Image Modeling (CIM), a novel and surprisingly effective approach to self-supervised visual pre-training. Our CIM performs a simple pretext task: we randomly crop image regions (exemplars) from an input image (context) and predict correlation maps between the exemplars and the context.

Aligning Bag of Regions for Open-Vocabulary Object Detection
S. Wu, W. Zhang, S. Jin, W. Liu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]

Pre-trained vision-language models (VLMs) learn to align vision and language representations on large-scale datasets, where each image-text pair usually contains a bag of semantic concepts. However, existing open-vocabulary object detectors only align region embeddings individually with the corresponding features extracted from the VLMs. Such a design leaves the compositional structure of semantic concepts in a scene under-exploited, although the structure may be implicitly learned by the VLMs. In this work, we propose to align the embedding of bag of regions beyond individual regions.