[Paper Review] Fast and Efficient Lenslet Image Compression

Hadi Amirpour, António Pinheiro+2 more|arXiv (Cornell University)|Jan 27, 2019

Advanced Vision and Imaging23 references4 citations

TL;DR

This paper proposes a fast and efficient lenslet image compression method using HEVC by organizing sub-aperture images into a serpentine-scan pseudo-sequence with a novel GOP structure and CTU depth prediction. By dividing images into four regions, using the central view as a reference, and enabling parallel encoding, the method achieves up to 80% reduction in encoding time while outperforming state-of-the-art methods in rate-distortion efficiency with minimal quality loss.

ABSTRACT

Light field imaging is characterized by capturing brightness, color, and directional information of light rays in a scene. This leads to image representations with huge amount of data that require efficient coding schemes. In this paper, lenslet images are rendered into sub-aperture images. These images are organized as a pseudo-sequence input for the HEVC video codec. To better exploit redundancy among the neighboring sub-aperture images and consequently decrease the distances between a sub-aperture image and its references used for prediction, sub-aperture images are divided into four smaller groups that are scanned in a serpentine order. The most central sub-aperture image, which has the highest similarity to all the other images, is used as the initial reference image for each of the four regions. Furthermore, a structure is defined that selects spatially adjacent sub-aperture images as prediction references with the highest similarity to the current image. In this way, encoding efficiency increases, and furthermore it leads to a higher similarity among the co-located Coding Three Units (CTUs). The similarities among the co-located CTUs are exploited to predict Coding Unit depths.Moreover, independent encoding of each group division enables parallel processing, that along with the proposed coding unit depth prediction decrease the encoding execution time by almost 80% on average. Simulation results show that Rate-Distortion performance of the proposed method has higher compression gain than the other state-of-the-art lenslet compression methods with lower computational complexity.

Motivation & Objective

• To address the high data volume of lenslet images from light field cameras, which hinders practical deployment.
• To improve compression efficiency of lenslet images by exploiting spatial and angular redundancy among sub-aperture images.
• To reduce encoding complexity without sacrificing rate-distortion performance.
• To enable parallel processing in lenslet compression by decoupling sub-aperture image groups.

Proposed method

• Sub-aperture images are rendered from raw lenslet data and organized into a pseudo-sequence for HEVC video coding.
• A serpentine scan order is introduced to reduce prediction distances by grouping sub-aperture images into four spatial regions.
• The most central sub-aperture image is used as the initial reference for each region to maximize similarity with neighboring views.
• A novel GOP structure selects spatially adjacent sub-aperture images as prediction references based on similarity, improving prediction accuracy.
• A CTU depth prediction method is proposed to exploit similarities among co-located coding units, reducing encoding complexity.
• Independent encoding of each region enables parallel processing, significantly reducing overall encoding time.

Experimental results

Research questions

• RQ1 Can a serpentine scan order among grouped sub-aperture images improve prediction efficiency in lenslet compression?
• RQ2 How does using the central sub-aperture image as a reference for each group affect rate-distortion performance?
• RQ3 To what extent can CTU depth prediction based on co-located unit similarity reduce encoding complexity?
• RQ4 Can independent region encoding enable effective parallel processing in lenslet compression?
• RQ5 How does the proposed method compare in rate-distortion and complexity to existing state-of-the-art lenslet compression techniques?

Key findings

• The proposed method achieves a 79.06% to 80.74% average reduction in encoding time (ΔTp) across test sequences, with an average of 80.13% reduction.
• The method reduces computational complexity by 37.07% on average when using the CTU depth prediction method (ΔTs), even without parallel processing.
• Rate-distortion performance exceeds that of LAP, MuLe, and WasP, with an average BDPSNR of -0.0132 dB and BDBR of 0.5495% under 4:2:2 chromatic subsampling.
• The method maintains high quality with negligible distortion loss, as shown by SSIM and PSNR-Y curves, outperforming baseline zigzag scan and other state-of-the-art methods.
• The use of central sub-aperture images as references and spatially adjacent references improves prediction accuracy, leading to better compression efficiency.
• Independent region encoding enables effective parallel processing, making the method highly suitable for real-time or high-throughput applications.