[论文解读] The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties

Martin Genzel, Gitta Kutyniok|arXiv (Cornell University)|Aug 20, 2018

Sparse and Compressive Sensing Techniques参考文献 23被引用 5

一句话总结

本文提出了不匹配原则（mismatch principle），这是一种在大规模模型不确定性下推导广义Lasso非渐近误差界的一般性框架。通过利用次高斯设计矩阵和凸约束，该框架在半参数模型（包括单指数模型和广义线性模型）中建立了鲁棒的估计保证，无需假设特定的观测模型，展示了Lasso对非线性失真和量化效应的鲁棒性。

ABSTRACT

We study the estimation capacity of the generalized Lasso, i.e., least squares minimization combined with a (convex) structural constraint. While Lasso-type estimators were originally designed for noisy linear regression problems, it has recently turned out that they are in fact robust against various types of model uncertainties and misspecifications, most notably, non-linearly distorted observation models. This work provides more theoretical evidence for this somewhat astonishing phenomenon. At the heart of our analysis stands the mismatch principle, which is a simple recipe to establish theoretical error bounds for the generalized Lasso. The associated estimation guarantees are of independent interest and are formulated in a fairly general setup, permitting arbitrary sub-Gaussian data, possibly with strongly correlated feature designs; in particular, we do not assume a specific observation model which connects the input and output variables. Although the mismatch principle is conceived based on ideas from statistical learning theory, its actual application area are (high-dimensional) estimation tasks for semi-parametric models. In this context, the benefits of the mismatch principle are demonstrated for a variety of popular problem classes, such as single-index models, generalized linear models, and variable selection. Apart from that, our findings are also relevant to recent advances in quantized and distributed compressed sensing.

研究动机与目标

• 在大规模模型不确定性下建立广义Lasso的理论误差界。
• 证明广义Lasso对非线性观测模型和模型误设的鲁棒性。
• 提供一个通用且可重用的框架，用于推导估计保证，而无需假设特定的连接函数或噪声模型。
• 将Lasso的适用性从线性模型扩展到半参数和非线性场景。

提出的方法

• 提出不匹配原则作为高维估计中推导误差界的一般性方法。
• 将该原则应用于具有凸约束的广义Lasso，使用次高斯设计矩阵。
• 通过高斯宽度和经验过程理论推导界，避免对给定x时y的条件分布作假设。
• 采用基于对偶的方法，将估计误差与约束集和设计矩阵的几何结构联系起来。
• 建立对输出变量y分布无关的误差界，仅依赖于设计矩阵的次高斯性。
• 在多种模型上验证该框架：单指数模型、广义线性模型以及量化压缩感知。

实验结果

研究问题

• RQ1 当真实观测模型未知或误设时，如何为广义Lasso推导非渐近误差界？
• RQ2 广义Lasso在观测过程存在非线性失真时，其鲁棒性在多大程度上成立？
• RQ3 不匹配原则是否能在不假设输入与输出之间参数化连接函数的前提下，提供显式的误差界？
• RQ4 在强特征相关性或高维设计下，该方法表现如何？
• RQ5 不匹配原则对量化和分布式压缩感知有何启示？

主要发现

• 不匹配原则在任意次高斯设计和凸约束下，为广义Lasso提供了非渐近误差界，即使在无已知观测模型的情况下也成立。
• 该方法实现的估计误差界与约束集的高斯宽度成比例，反映了其几何结构。
• 广义Lasso在非线性观测模型（包括量化输出和非线性失真输出）下具有可证明的鲁棒性。
• 该框架适用于单指数模型和广义线性模型，在较弱正则性条件下可获得显式的误差率。
• 结果可扩展至分布式和量化压缩感知，凸显凸松弛在高维估计中的强大能力。
• 该方法避免了对响应变量的强参数假设，使其适用于结构假设最少的半参数模型。