[Paper Review] Face Recognition in Low Quality Images: A Survey

Pei Li, Loreto R. Prieto+2 more|arXiv (Cornell University)|May 29, 2018

Advanced Image Processing Techniques135 references47 citations

TL;DR

A comprehensive survey of low-quality face recognition (LQFR) approaches, datasets, and challenges, covering super-resolution, robust features, unified representations, and deblurring techniques, with discussion of human vs. machine performance and future directions.

ABSTRACT

Low-resolution face recognition (LRFR) has received increasing attention over the past few years. Its applications lie widely in the real-world environment when high-resolution or high-quality images are hard to capture. One of the biggest demands for LRFR technologies is video surveillance. As the the number of surveillance cameras in the city increases, the videos that captured will need to be processed automatically. However, those videos or images are usually captured with large standoffs, arbitrary illumination condition, and diverse angles of view. Faces in these images are generally small in size. Several studies addressed this problem employed techniques like super resolution, deblurring, or learning a relationship between different resolution domains. In this paper, we provide a comprehensive review of approaches to low-resolution face recognition in the past five years. First, a general problem definition is given. Later, systematically analysis of the works on this topic is presented by catogory. In addition to describing the methods, we also focus on datasets and experiment settings. We further address the related works on unconstrained low-resolution face recognition and compare them with the result that use synthetic low-resolution data. Finally, we summarized the general limitations and speculate a priorities for the future effort.

Motivation & Objective

• Define the LQFR problem and its degradation processes.
• Systematically categorize and analyze existing LQFR approaches.
• Summarize datasets, protocols, and experimental results in LQFR.
• Discuss limitations and propose priorities for future research.

Proposed method

• Review four main categories: super-resolution based methods, low-resolution robust features, unified representation learning, and remedies for blurriness.
• Discuss image processing steps prior to feature extraction and matching, including SR and deblurring.
• Differentiate real LR images from synthetically downsampled LR images and their impact on FR performance.
• Summarize both traditional (pre-deep learning) and deep learning based LQFR methods.
• Highlight human performance studies and compare them to machine performance on LQFR tasks.

Experimental results

Research questions

• RQ1 What degradation processes produce low-quality face images and how do they affect recognition?
• RQ2 How can recognition be improved when probe and gallery images have mismatched quality or resolution?
• RQ3 What are the relative merits of super-resolution, robust features, and unified representations for LQFR?
• RQ4 What datasets and evaluation protocols exist for assessing LQFR methods, and what are their limitations?

Key findings

• Super-resolution methods can improve recognition performance when input LR faces are above a certain size, but very small inputs may not benefit significantly.
• Learning-based SR approaches, including deep networks and GANs, can preserve or enhance facial priors to aid recognition.
• Methods that integrate recognition objectives into the SR process (simultaneous SR and recognition) show promise in learning inter-class discrimination.
• Real LR images pose additional challenges (noise, non-ideal degradation) beyond synthetic LR images, impacting transferability of SR-based gains.
• Human performance on LQFR degrades with lower resolution and unfamiliarity, highlighting a gap to advanced machine systems and a potential for human-computer collaboration.
• A wide range of datasets and evaluation protocols exist, but data scarcity and varying quality dimensions remain key limitations for benchmarking.