[论文解读] Model-Based Deep Learning

Nir Shlezinger, Jay Whang+2 more|arXiv (Cornell University)|Dec 15, 2020

Model Reduction and Neural Networks被引用 50

一句话总结

一篇教程式综述，统一基于模型的信号处理与深度学习，介绍两种混合策略—基于模型的网络和DNN辅助推断—and 以CS、通信和跟踪等领域的应用来说明它们。

ABSTRACT

Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Such model-based methods utilize mathematical formulations that represent the underlying physics, prior information and additional domain knowledge. Simple classical models are useful but sensitive to inaccuracies and may lead to poor performance when real systems display complex or dynamic behavior. On the other hand, purely data-driven approaches that are model-agnostic are becoming increasingly popular as datasets become abundant and the power of modern deep learning pipelines increases. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance, especially for supervised problems. However, DNNs typically require massive amounts of data and immense computational resources, limiting their applicability for some signal processing scenarios. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches. Such model-based deep learning methods exploit both partial domain knowledge, via mathematical structures designed for specific problems, as well as learning from limited data. In this article we survey the leading approaches for studying and designing model-based deep learning systems. We divide hybrid model-based/data-driven systems into categories based on their inference mechanism. We provide a comprehensive review of the leading approaches for combining model-based algorithms with deep learning in a systematic manner, along with concrete guidelines and detailed signal processing oriented examples from recent literature. Our aim is to facilitate the design and study of future systems on the intersection of signal processing and machine learning that incorporate the advantages of both domains.

研究动机与目标

• 提出一个用于混合基于模型/数据驱动系统的统一框架。
• 将基于模型的深度学习分为基于模型的网络和DNN辅助推断。
• 提供设计指南和具体实现方法。
• 用信号处理、通信和控制领域的应用来说明该框架。
• 讨论未来挑战与研究方向。

提出的方法

• 定义推断问题并对比基于模型、数据驱动和混合方法。
• 介绍两种主要策略：基于模型的网络和DNN辅助推断。"
• 讨论深度学习基础知识，包括网络、损失函数和优化器，作为前提。
• 回顾实现方法，如深度展开和神经增强。
• 提供设计基于模型的深度学习系统的指南。
• 提供跨应用的具体文献支撑示例。

实验结果

研究问题

• RQ1 如何将 principled model-based 方法与数据驱动的深度学习有效整合，以提升性能与鲁棒性？
• RQ2 对于基于模型的深度学习体系结构，系统性类别与设计原则是什么？
• RQ3 如何在训练数据有限和部分领域知识可用的情况下设计这类混合系统？
• RQ4 在不同任务中，基于模型的网络与DNN辅助推断之间有哪些权衡？

主要发现

• 该论文提出了一个两类化的基于模型的深度学习框架：基于模型的网络和DNN辅助推断。
• 它为研究、设计和比较混合系统提供了具体指南。
• 它对压缩感知、数字通信和跟踪等领域的文献进行了综述，以展示方法的广度。
• 它讨论了混合方法如何在部分领域知识下工作，且通常比纯数据驱动的方法需要更少的训练集。
• 它将基于模型的知识与深度学习结合起来，以提升相对于黑箱DNN的可解释性和可靠性。
• 它概述了将领域知识与学习结合的未来研究主题与挑战。