[Paper Review] NTIRE 2023 Challenge on Light Field Image Super-Resolution: Dataset, Methods and Results

Yingqian Wang, Longguang Wang+4 more|arXiv (Cornell University)|Apr 20, 2023

Advanced Vision and Imaging4 citations

TL;DR

This paper presents the NTIRE 2023 Challenge on Light Field Image Super-Resolution, introducing a new benchmark dataset (NTIRE-2023), the BasicLFSR toolbox for model development, and a comprehensive evaluation of 11 top-performing methods. The challenge achieved state-of-the-art results, with the winning method improving PSNR by ~1 dB over prior SOTA, leveraging advanced fusion of spatial and angular information in 4D light field data through deep learning.

ABSTRACT

In this report, we summarize the first NTIRE challenge on light field (LF) image super-resolution (SR), which aims at super-resolving LF images under the standard bicubic degradation with a magnification factor of 4. This challenge develops a new LF dataset called NTIRE-2023 for validation and test, and provides a toolbox called BasicLFSR to facilitate model development. Compared with single image SR, the major challenge of LF image SR lies in how to exploit complementary angular information from plenty of views with varying disparities. In total, 148 participants have registered the challenge, and 11 teams have successfully submitted results with PSNR scores higher than the baseline method LF-InterNet \cite{LF-InterNet}. These newly developed methods have set new state-of-the-art in LF image SR, e.g., the winning method achieves around 1 dB PSNR improvement over the existing state-of-the-art method DistgSSR \cite{DistgLF}. We report the solutions proposed by the participants, and summarize their common trends and useful tricks. We hope this challenge can stimulate future research and inspire new ideas in LF image SR.

Motivation & Objective

• To establish a standardized benchmark for light field (LF) image super-resolution (SR) under bicubic degradation with a ×4 magnification factor.
• To address the core challenge of effectively fusing spatial and angular information across 4D light field data, which is more complex than single-image or stereo SR.
• To stimulate research in cross-domain vision by bridging low-level vision and 3D scene reconstruction through a community-driven challenge.
• To provide a reproducible, open-source framework via the BasicLFSR toolbox to accelerate method development and comparison.
• To evaluate and compare diverse deep learning architectures in LF-SR, identifying effective design patterns and practical improvements.

Proposed method

• The challenge uses a new dataset, NTIRE-2023, for validation and test, derived from publicly available LF datasets with standardized bicubic downsampling at ×4.
• A custom toolbox, BasicLFSR, is provided to streamline data loading, preprocessing, and training for LF-SR models, supporting common deep learning frameworks.
• Participants employed a range of architectures including CNNs with channel attention, residual learning, and Transformer-based modules to exploit spatial and angular correlations.
• The winning method, and others, used hybrid loss functions combining L1 loss and EPI gradient loss to preserve fine details and edge structures.
• Networks were trained using a two-stage strategy: shallow feature extraction followed by deep feature learning via residual blocks or attention modules.
• The framework supports MacPI (Multi-Angle Patch-based Image) processing, enabling 2D convolutional operations on rearranged light field data for efficient feature learning.

Figure 1 : An illustration of the center-view images in the developed NTIRE-2023 LF dataset. Both validation and test sets contain 16 real-world and 16 synthetic LFs, respectively.

Experimental results

Research questions

• RQ1 What are the most effective architectures for fusing spatial and angular information in 4D light field data for super-resolution?
• RQ2 How do different loss functions—especially EPI gradient loss and L1—impact perceptual quality and structural fidelity in LF-SR?
• RQ3 To what extent can attention mechanisms and residual learning improve performance in LF-SR compared to standard CNNs?
• RQ4 What design patterns and training strategies lead to consistent improvements across diverse LF-SR models?
• RQ5 How does the proposed benchmark and toolbox facilitate reproducibility and fair comparison in LF-SR research?

Key findings

• The winning method achieved a PSNR improvement of approximately 1 dB over the prior state-of-the-art method, DistgSSR, demonstrating significant performance gains.
• Eleven teams submitted results with PSNR higher than the baseline LF-InterNet, indicating strong community engagement and methodological innovation.
• Common trends among top methods included the use of residual learning, channel attention, and hybrid loss functions combining L1 and EPI gradient losses.
• The BasicLFSR toolbox proved effective in accelerating model development, with most participants leveraging it for training and inference.
• The NTIRE-2023 dataset was found to be diverse and representative, supporting robust generalization across different light field scenes and complexities.
• The challenge successfully identified key architectural and training improvements that are now considered best practices in LF-SR research.

Figure 2 : The OpenMeow Team: The network architecture of the proposed DistgEPIT.