[Paper Review] Manifold Gradient Descent Solves Multi-Channel Sparse Blind Deconvolution Provably and Efficiently

Laixi Shi, Yuejie Chi|arXiv (Cornell University)|Nov 25, 2019

Sparse and Compressive Sensing Techniques52 references4 citations

TL;DR

This paper proposes a manifold gradient descent (MGD) algorithm for provably and efficiently solving multi-channel sparse blind deconvolution (MSBD) by optimizing a smooth surrogate of the sparsity-promoting loss on the sphere manifold. It establishes that MGD with random initialization recovers the unknown filter up to scaling and shift ambiguity with high probability when the number of observations is sufficiently large under a random data model.

ABSTRACT

Multi-channel sparse blind deconvolution, or convolutional sparse coding, refers to the problem of learning an unknown filter by observing its circulant convolutions with multiple input signals that are sparse. This problem finds numerous applications in signal processing, computer vision, and inverse problems. However, it is challenging to learn the filter efficiently due to the bilinear structure of the observations with the respect to the unknown filter and inputs, as well as the sparsity constraint. In this paper, we propose a novel approach based on nonconvex optimization over the sphere manifold by minimizing a smooth surrogate of the sparsity-promoting loss function. It is demonstrated that manifold gradient descent with random initializations will provably recover the filter, up to scaling and shift ambiguity, as soon as the number of observations is sufficiently large under an appropriate random data model. Numerical experiments are provided to illustrate the performance of the proposed method with comparisons to existing ones.

Motivation & Objective

• To address the challenge of efficiently recovering an unknown filter and sparse inputs from multiple convolutions with bilinear structure and sparsity constraints.
• To develop a nonconvex optimization framework that provably recovers the filter up to scaling and shift ambiguity.
• To establish convergence guarantees for manifold gradient descent on the sphere manifold under a random data model.
• To provide a smooth, computationally efficient alternative to existing nonconvex or convex approaches for MSBD.

Proposed method

• Formulates the MSBD problem as minimizing a smooth surrogate of the sparsity-promoting loss function on the sphere manifold.
• Uses manifold gradient descent (MGD) to optimize over the sphere, avoiding the bilinear structure by leveraging the inverse filter property.
• Employs a smooth approximation of the $–ell_0$ norm via the hyperbolic tangent function to enable differentiability and efficient optimization.
• Imposes the invertibility condition on the filter to ensure identifiability and enables conversion of the bilinear problem into a linear one via inverse filtering.
• Analyzes the geometry of the empirical loss function and establishes local strong convexity around the true solution under appropriate conditions.
• Uses concentration inequalities and moment bounds to control the behavior of gradient and Hessian terms in high-dimensional settings.

Experimental results

Research questions

• RQ1 Can manifold gradient descent with random initialization provably recover the unknown filter in multi-channel sparse blind deconvolution?
• RQ2 What is the minimum number of observations required for successful recovery under a random data model?
• RQ3 How does the smooth surrogate loss function on the sphere manifold ensure convergence to the true solution despite nonconvexity?
• RQ4 What are the theoretical guarantees on the convergence rate and robustness of MGD for MSBD?

Key findings

• Manifold gradient descent with random initialization provably recovers the filter up to scaling and shift ambiguity with high probability when the number of observations $p$ is sufficiently large.
• The method achieves recovery with $p = O(n \log n)$ observations under the random data model, matching information-theoretic limits up to logarithmic factors.
• Theoretical analysis shows that the empirical loss function exhibits local strong convexity around the true solution, enabling fast convergence.
• Numerical experiments demonstrate that MGD outperforms existing methods in convergence speed and recovery accuracy on synthetic and image deblurring tasks.
• The smooth surrogate loss via the hyperbolic tangent function enables efficient and stable optimization without requiring exact sparsity enforcement.