[Paper Review] Vector Nonlocal Euclidean Median: Principal Bundle Captures The Nature of Patch Space

Chen-Yun Lin, Arin Minasian+2 more|arXiv (Cornell University)|Nov 15, 2016

Image and Signal Denoising Methods38 references3 citations

TL;DR

This paper proposes Vector Nonlocal Euclidean Median (VNLEM), a novel image denoising algorithm that models patch space using a fiber bundle structure to capture rotational invariance via SO(2) group actions. By incorporating orientation features from SIFT and leveraging a Riemannian inner product distance (RID), VNLEM improves robustness and denoising performance over traditional NLEM, achieving superior results in both error-based and human visual system-based image quality assessments on a 1,361-image database.

ABSTRACT

We extensively study the rotational group structure inside the patch space by introducing the fiber bundle structure. The rotational group structure leads to a new image denoising algorithm called the extit{vector non-local Euclidean median} (VNLEM). The theoretical aspect of VNLEM is studied, which explains why the VNLEM and traditional non-local mean/non-local Euclidean median (NLEM) algorithm work. The numerical issue of the VNLEM is improved by taking the orientation feature in the commonly applied scale-invariant feature transform (SIFT), and a theoretical analysis of the robustness of the orientation feature in the SIFT is provided. The VNLEM is applied to an image database of 1,361 images and compared with the NLEM. Different image quality assessments based on the error-sensitivity or the human visual system are applied to evaluate the performance. The results confirmed the potential of the VNLEM algorithm.

Motivation & Objective

• To address the lack of theoretical explanation for why nonlocal filtering works in noisy patch spaces.
• To model rotational group structure in patch space using fiber bundle geometry, with SO(2) as the fiber and the base manifold representing rotationally invariant patches.
• To develop a robust, geometry-aware denoising algorithm that improves upon NLEM by incorporating rotational invariance and orientation features.
• To evaluate VNLEM using diverse image quality metrics, including both error-based (SNR) and human perception-based (FSIM) assessments.
• To demonstrate the algorithm’s superiority on a large-scale image database and discuss its generalization potential to color images, multi-image denoising, and texture analysis.

Proposed method

• Model the patch space as a fiber bundle where each fiber corresponds to the SO(2) rotation group, and the base manifold represents rotationally invariant patches.
• Define a Riemannian inner product distance (RID) on the fiber bundle to measure similarity between patches, accounting for rotational invariance.
• Integrate orientation features from SIFT into the VNLEM algorithm to enhance numerical stability and robustness in neighbor selection.
• Replace the mean in NLEM with a vector median operation over patches in the neighborhood defined by the RID, enabling outlier resilience.
• Use a weighted averaging scheme based on RID to compute the denoised pixel value, preserving edges and fine structures.
• Apply the algorithm to a database of 1,361 images and compare performance using SNR, PSNR, and FSIM metrics.

Experimental results

Research questions

• RQ1 Why do nonlocal filtering methods like NLM and NLEM work effectively in noisy patch spaces, and what underlying geometric structure enables this?
• RQ2 How can rotational invariance in image patches be mathematically modeled to improve denoising performance?
• RQ3 Can the incorporation of orientation features from SIFT enhance the robustness and numerical efficiency of nonlocal denoising algorithms?
• RQ4 How does VNLEM compare to NLEM in terms of denoising quality across different image quality assessment metrics?
• RQ5 What are the limitations of current manifold-based models in patch space, and how can fiber bundle structures generalize them?

Key findings

• VNLEM significantly outperforms NLEM in image denoising quality, particularly in preserving edges and fine textures, as confirmed by both error-based and perception-based metrics.
• The use of SIFT orientation features improves the numerical stability and robustness of neighbor selection in VNLEM, reducing sensitivity to noise in patch similarity computation.
• The fiber bundle model with SO(2) fibers provides a geometrically principled way to capture rotational invariance, explaining the success of nonlocal methods in patch space.
• FSIM evaluations show that VNLEM produces results closer to human visual perception than NLEM, despite not explicitly using FSIM features in the algorithm.
• Statistical analysis across 1,361 images confirms the performance gain of VNLEM over NLEM is both significant and consistent across diverse image types.
• The proposed method demonstrates strong potential for extension to color images and multi-image denoising, suggesting broader applicability beyond grayscale images.