[Paper Review] Denoising modulo samples: k-NN regression and tightness of SDP relaxation

Michaël Fanuel, Hemant Tyagi|arXiv (Cornell University)|Sep 10, 2020

Sparse and Compressive Sensing Techniques4 citations

TL;DR

This paper proposes a two-stage denoising method for modulo-1 samples of a Lipschitz function, using k-NN regression to denoise mod 1 estimates followed by sequential unwrapping, achieving a uniform error rate of $ O((\log n / n)^{1/(d+2)}) $ with high probability. It further establishes sufficient conditions for tightness of the SDP relaxation of a nonconvex QCQP formulation used in modulo sampling recovery.

ABSTRACT

Many modern applications involve the acquisition of noisy modulo samples of a function $f$, with the goal being to recover estimates of the original samples of $f$. For a Lipschitz function $f:[0,1]^d \ o \\mathbb{R}$, suppose we are given the samples $y_i = (f(x_i) + \\eta_i)\\bmod 1; \\quad i=1,\\dots,n$ where $\\eta_i$ denotes noise. Assuming $\\eta_i$ are zero-mean i.i.d Gaussian's, and $x_i$'s form a uniform grid, we derive a two-stage algorithm that recovers estimates of the samples $f(x_i)$ with a uniform error rate $O((\\frac{\\log n}{n})^{\\frac{1}{d+2}})$ holding with high probability. The first stage involves embedding the points on the unit complex circle, and obtaining denoised estimates of $f(x_i)\\bmod 1$ via a $k$NN (nearest neighbor) estimator. The second stage involves a sequential unwrapping procedure which unwraps the denoised mod $1$ estimates from the first stage. The estimates of the samples $f(x_i)$ can be subsequently utilized to construct an estimate of the function $f$, with the aforementioned uniform error rate. Recently, Cucuringu and Tyagi proposed an alternative way of denoising modulo $1$ data which works with their representation on the unit complex circle. They formulated a smoothness regularized least squares problem on the product manifold of unit circles, where the smoothness is measured with respect to the Laplacian of a proximity graph $G$ involving the $x_i$'s. This is a nonconvex quadratically constrained quadratic program (QCQP) hence they proposed solving its semidefinite program (SDP) based relaxation. We derive sufficient conditions under which the SDP is a tight relaxation of the QCQP. Hence under these conditions, the global solution of QCQP can be obtained in polynomial time.

Motivation & Objective

• Address the challenge of recovering original samples from noisy modulo-1 observations of a Lipschitz function.
• Develop a two-stage algorithm that first denoises mod 1 estimates and then unwraps them to recover original function values.
• Provide theoretical guarantees on the uniform error rate of the recovered samples under i.i.d. Gaussian noise.
• Establish sufficient conditions under which the semidefinite programming (SDP) relaxation of a nonconvex QCQP is tight, enabling polynomial-time global solution recovery.
• Extend the analysis to multivariate settings with uniform grids and smoothness constraints via graph Laplacian regularization.

Proposed method

• Use k-NN regression on embedded points on the unit complex circle to estimate $ f(x_i) \mod 1 $, reducing noise in modulo-1 observations.
• Apply a sequential unwrapping procedure to reconstruct original $ f(x_i) $ values from denoised mod 1 estimates, leveraging local smoothness.
• Formulate a smoothness-regularized least squares problem on the product manifold of unit circles using the Laplacian of a proximity graph.
• Relax the nonconvex QCQP into a semidefinite program (SDP) to enable polynomial-time optimization.
• Derive sufficient conditions under which the SDP relaxation is tight, ensuring the global solution of the original QCQP is recovered.
• Utilize the structure of the problem, including i.i.d. Gaussian noise and uniform grid sampling, to derive high-probability error bounds.

Experimental results

Research questions

• RQ1 Can a two-stage algorithm combining k-NN denoising and sequential unwrapping achieve a uniform error rate of $ O((\log n / n)^{1/(d+2)}) $ for recovering Lipschitz function samples from noisy modulo-1 observations?
• RQ2 Under what conditions is the SDP relaxation of the nonconvex QCQP formulation for modulo-1 denoising tight, ensuring global optimality?
• RQ3 How does the choice of regularization parameter $ \lambda $ affect the accuracy of the denoised estimates in the SDP-based approach?
• RQ4 Can the proposed method be generalized to multivariate settings with $ d \geq 1 $, and what role does the graph Laplacian play in this generalization?
• RQ5 What is the relationship between the $ \ell_2 $ error of the complex-valued estimator and the true noise level, and under what conditions is the estimator significantly better than noise-only baseline?

Key findings

• The proposed two-stage algorithm achieves a uniform error rate of $ O((\log n / n)^{1/(d+2)}) $ for estimating $ f(x_i) $, holding with high probability under i.i.d. Gaussian noise.
• The k-NN denoising step effectively reduces the error in $ f(x_i) \mod 1 $ estimates, which is critical for accurate sequential unwrapping.
• The SDP relaxation of the QCQP is tight under sufficient conditions, enabling polynomial-time computation of the global solution to the nonconvex problem.
• The analysis shows that the $ \ell_2 $ error of the denoised complex estimator $ \widehat{g}/|\widehat{g}| $ can be made significantly smaller than the noise baseline $ \|z - h\|_2^2 $, under appropriate $ \lambda $ choice.
• For the univariate case with i.i.d. Gaussian noise and uniform grid, the error bound $ \|\widehat{g}/|\widehat{g}| - h\|_2^2 \lesssim \sigma n + \lambda M^2 \Delta^3 / n $ holds with high probability.
• Future work suggests that tighter $ \ell_\infty $ bounds on $ \widehat{g} - h $ are needed, and an optimal $ \lambda $ choice minimizing $ \|\widehat{g} - h\|_\infty $ is expected to improve performance.