[论文解读] The LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements

Christos Thrampoulidis, Ehsan Abbasi+1 more|arXiv (Cornell University)|Jun 6, 2015

Fault Detection and Control Systems参考文献 25被引用 89

一句话总结

该论文证明了具有非线性测量的广义LASSO在渐近意义上等价于一个具有线性测量的LASSO，其中非线性性被吸收进有效线性系数和噪声中。关键结果表明，使用非线性连接函数时的估计性能，等价于一个信号和噪声参数经过缩放的线性模型，其缩放参数由连接函数的统计特性导出，且在1-bit压缩感知中，最优量化器为Lloyd-Max量化器。

ABSTRACT

Consider estimating an unknown, but structured, signal $x_0\in R^n$ from $m$ measurement $y_i=g_i(a_i^Tx_0)$, where the $a_i$'s are the rows of a known measurement matrix $A$, and, $g$ is a (potentially unknown) nonlinear and random link-function. Such measurement functions could arise in applications where the measurement device has nonlinearities and uncertainties. It could also arise by design, e.g., $g_i(x)= ext{sign}(x+z_i)$, corresponds to noisy 1-bit quantized measurements. Motivated by the classical work of Brillinger, and more recent work of Plan and Vershynin, we estimate $x_0$ via solving the Generalized-LASSO for some regularization parameter $λ>0$ and some (typically non-smooth) convex structure-inducing regularizer function. While this approach seems to naively ignore the nonlinear function $g$, both Brillinger (in the non-constrained case) and Plan and Vershynin have shown that, when the entries of $A$ are iid standard normal, this is a good estimator of $x_0$ up to a constant of proportionality $μ$, which only depends on $g$. In this work, we considerably strengthen these results by obtaining explicit expressions for the squared error, for the \emph{regularized} LASSO, that are asymptotically \emph{precise} when $m$ and $n$ grow large. A main result is that the estimation performance of the Generalized LASSO with non-linear measurements is \emph{asymptotically the same} as one whose measurements are linear $y_i=μa_i^Tx_0 + σz_i$, with $μ= Eγg(γ)$ and $σ^2 = E(g(γ)-μγ)^2$, and, $γ$ standard normal. To the best of our knowledge, the derived expressions on the estimation performance are the first-known precise results in this context. One interesting consequence of our result is that the optimal quantizer of the measurements that minimizes the estimation error of the LASSO is the celebrated Lloyd-Max quantizer.

研究动机与目标

• 理解当测量通过未知连接函数非线性变换时，广义LASSO的估计性能。
• 确定广义LASSO（假设测量为线性）在非线性测量模型下是否仍具有效性。
• 在具有结构化信号的高维设置下，推导估计误差的精确渐近表达式。
• 确定最小化LASSO重构误差的1-bit压缩感知最优量化器。

提出的方法

• 作者分析了形式为 $ y_i = g_i(\mathbf{a}_i^T \mathbf{x}_0) $ 的非线性测量下的广义LASSO估计器，其中 $ g_i $ 为独立同分布的非线性连接函数。
• 在随机高斯测量矩阵下，推导了当 $ m, n \to \infty $ 时，估计误差 $ \|\hat{\mathbf{x}} - \mu \mathbf{x}_0\|_2 $ 的渐近精确表达式。
• 关键洞见是，非线性测量模型在渐近意义上等价于一个信号缩放为 $ \mu = \mathbb{E}[\gamma g(\gamma)] $ 且有效噪声方差为 $ \sigma^2 = \mathbb{E}[(g(\gamma) - \mu \gamma)^2] $ 的线性模型，其中 $ \gamma \sim \mathcal{N}(0,1) $。
• 分析利用了高维渐近统计工具，包括近似消息传递（AMP）框架和凸高斯极小最大定理（CGMT）。
• 作者通过最小化比值 $ \sigma^2 / \mu^2 $ 推导出1-bit压缩感知的最优量化器，结果表明Lloyd-Max量化器为最优解。

实验结果

研究问题

• RQ1 尽管广义LASSO是为线性模型设计的，当测量被非线性变换时，它是否仍具有效性？
• RQ2 在高维渐近情形下，广义LASSO在非线性测量下的估计误差是否可以被精确表征？
• RQ3 是否存在一个具有有效信号和噪声参数的线性模型，其在渐近意义上等价于非线性测量模型？
• RQ4 最小化LASSO重构误差的1-bit压缩感知最优量化器是什么？

主要发现

• 具有非线性测量的广义LASSO的估计误差，在渐近意义上等价于一个信号缩放为 $ \mu = \mathbb{E}[\gamma g(\gamma)] $ 且噪声方差为 $ \sigma^2 = \mathbb{E}[(g(\gamma) - \mu \gamma)^2] $ 的线性模型。
• 所推导的估计误差表达式是目前在具有结构化信号的非线性测量背景下，首个已知的精确渐近表征。
• 对于1-bit量化测量，最小化LASSO重构误差的最优量化器为Lloyd-Max量化器，其通过最小化比值 $ \sigma^2 / \mu^2 $ 得到。
• 该等价性在i.i.d.标准高斯测量矩阵下成立，且在高维渐近情形下成立，即 $ m, n \to \infty $ 且 $ m/n $ 固定。