[Paper Review] Atomic norm denoising with applications to line spectral estimation

Badri Narayan Bhaskar, Gongguo Tang+1 more|arXiv (Cornell University)|Apr 3, 2012

Sparse and Compressive Sensing Techniques50 references19 citations

TL;DR

This paper proposes a convex optimization framework based on atomic norm denoising for line spectral estimation, enabling robust frequency and phase recovery from noisy, undersampled data without prior knowledge of the model order. The method achieves superior mean-squared error (MSE) performance compared to classical approaches like MUSIC and Cadzow’s method, with a semidefinite programming (SDP) formulation and a fast L1-regularized least-squares approximation via FFT that scales efficiently to large problems.

ABSTRACT

Motivated by recent work on atomic norms in inverse problems, we propose a new approach to line spectral estimation that provides theoretical guarantees for the mean-squared-error (MSE) performance in the presence of noise and without knowledge of the model order. We propose an abstract theory of denoising with atomic norms and specialize this theory to provide a convex optimization problem for estimating the frequencies and phases of a mixture of complex exponentials. We show that the associated convex optimization problem can be solved in polynomial time via semidefinite programming (SDP). We also show that the SDP can be approximated by an l1-regularized least-squares problem that achieves nearly the same error rate as the SDP but can scale to much larger problems. We compare both SDP and l1-based approaches with classical line spectral analysis methods and demonstrate that the SDP outperforms the l1 optimization which outperforms MUSIC, Cadzow's, and Matrix Pencil approaches in terms of MSE over a wide range of signal-to-noise ratios.

Motivation & Objective

• To develop a theoretically grounded, convex optimization approach for line spectral estimation that is robust to noise and does not require prior knowledge of the number of sinusoids.
• To extend atomic norm denoising to the continuous frequency domain for line spectra, avoiding discretization errors common in grid-based methods.
• To provide finite-sample mean-squared error bounds for frequency and phase estimation under noise, generalizing Lasso-like recovery guarantees to infinite dictionaries.
• To design computationally efficient solvers—both an SDP-based method and an L1-regularized least-squares approximation—enabling scalability to large-scale problems.
• To empirically validate the proposed method against classical algorithms (e.g., MUSIC, Cadzow, Matrix Pencil) across a range of signal-to-noise ratios and model orders.

Proposed method

• Formulates line spectral estimation as a denoising problem using the atomic norm associated with the set of complex exponentials, which generalizes the L1 norm to continuous dictionaries.
• Derives a convex optimization problem—Atomic Norm Soft Thresholding (AST)—that minimizes the atomic norm of the signal subject to data fidelity, enabling exact recovery in the noiseless case.
• Shows that the AST problem can be solved via semidefinite programming (SDP) using duality and positive polynomial theory, with an ADMM-based implementation achieving sub-minute runtime for 1000-sample problems.
• Proposes a practical approximation by discretizing the frequency domain and solving a standard L1-regularized least-squares problem, which leverages the Fast Fourier Transform (FFT) for sub-second solution times on large instances.
• Demonstrates that the L1-approximation achieves nearly identical MSE to the SDP solution when the grid is sufficiently fine, even when true frequencies lie off-grid.
• Uses performance profiles to compare AST, L1-approximation, and classical methods across multiple experimental settings, quantifying relative MSE and localization accuracy.

Experimental results

Research questions

• RQ1 Can atomic norm denoising provide finite-sample mean-squared error guarantees for line spectral estimation in the presence of noise, without requiring knowledge of the model order?
• RQ2 How does the performance of the atomic norm-based approach compare to classical subspace methods (e.g., MUSIC, Cadzow, Matrix Pencil) in terms of MSE across varying signal-to-noise ratios?
• RQ3 To what extent does oversampling in the frequency grid improve the accuracy of the L1-regularized approximation to the atomic norm minimization?
• RQ4 What is the trade-off between computational complexity and estimation accuracy when using SDP versus L1-regularized least-squares for atomic norm denoising?
• RQ5 Can the atomic norm framework be extended to other signal models, such as those with atoms supported on the unit disk, for applications in system identification and control?

Key findings

• The proposed Atomic Norm Soft Thresholding (AST) algorithm outperforms classical methods such as MUSIC, Cadzow’s, and Matrix Pencil in terms of mean-squared error (MSE) across a wide range of signal-to-noise ratios (SNR), even when classical methods are provided with the true model order.
• The SDP-based AST method achieves the best MSE performance, with the L1-regularized least-squares approximation closely matching its error rate while being significantly faster and scalable to large problems.
• Performance of the L1-approximation improves with increasing oversampling of the frequency grid, achieving high localization accuracy even when true frequencies are not on the grid.
• The ADMM-based SDP solver can solve problems with 1000 observations in a few minutes, demonstrating practical feasibility of the convex optimization approach.
• Performance profiles show that AST is the top-performing algorithm, with the L1-approximation ranking second, while Cadzow’s method degrades sharply when the model order is misestimated.
• Theoretical analysis shows that the atomic norm denoising framework provides finite-sample MSE bounds that generalize Lasso-like recovery guarantees to infinite dictionaries, with rates dependent on geometric and Gaussian width properties of the atomic set.