[Paper Review] Model-Based Deep Learning

Nir Shlezinger, Jay Whang+2 more|arXiv (Cornell University)|Dec 15, 2020

Model Reduction and Neural Networks50 citations

TL;DR

A tutorial-style survey that unifies model-based signal processing with deep learning, introducing two hybrid strategies—model-aided networks and DNN-aided inference—and illustrating them with applications in CS, communications, and tracking.

ABSTRACT

Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Such model-based methods utilize mathematical formulations that represent the underlying physics, prior information and additional domain knowledge. Simple classical models are useful but sensitive to inaccuracies and may lead to poor performance when real systems display complex or dynamic behavior. On the other hand, purely data-driven approaches that are model-agnostic are becoming increasingly popular as datasets become abundant and the power of modern deep learning pipelines increases. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance, especially for supervised problems. However, DNNs typically require massive amounts of data and immense computational resources, limiting their applicability for some signal processing scenarios. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches. Such model-based deep learning methods exploit both partial domain knowledge, via mathematical structures designed for specific problems, as well as learning from limited data. In this article we survey the leading approaches for studying and designing model-based deep learning systems. We divide hybrid model-based/data-driven systems into categories based on their inference mechanism. We provide a comprehensive review of the leading approaches for combining model-based algorithms with deep learning in a systematic manner, along with concrete guidelines and detailed signal processing oriented examples from recent literature. Our aim is to facilitate the design and study of future systems on the intersection of signal processing and machine learning that incorporate the advantages of both domains.

Motivation & Objective

• Present a unified framework for hybrid model-based/data-driven systems.
• Categorize model-based deep learning into model-aided networks and DNN-aided inference.
• Provide design guidelines and concrete implementation approaches.
• Illustrate the framework with applications in signal processing, communications, and control.
• Discuss future challenges and research directions.

Proposed method

• Define the inference problem and contrast model-based, data-driven, and hybrid approaches.
• Introduce two main strategies: model-aided networks and DNN-aided inference."
• Discuss deep learning basics, including networks, losses, and optimizers, as prerequisites.
• Review implementation approaches such as deep unfolding and neural augmentation.
• Present guidelines for designing model-based deep learning systems.
• Provide concrete literature-backed examples across applications.

Experimental results

Research questions

• RQ1 How can principled model-based methods be effectively integrated with data-driven deep learning to improve performance and robustness?
• RQ2 What are the systematic categories and design principles for model-based deep learning architectures?
• RQ3 How can such hybrid systems be designed to work with limited training data and partial domain knowledge?
• RQ4 What are the trade-offs between model-aided networks and DNN-aided inference across tasks?

Key findings

• The paper proposes a two-category framework for model-based deep learning: model-aided networks and DNN-aided inference.
• It provides concrete guidelines for studying, designing, and comparing hybrid systems.
• It surveys literature across compressed sensing, digital communications, and tracking to demonstrate breadth of the approach.
• It discusses how hybrid methods can operate with partial domain knowledge and often with smaller training sets than purely data-driven methods.
• It connects model-based knowledge with deep learning to improve interpretability and reliability relative to black-box DNNs.
• It outlines future research topics and challenges in integrating domain knowledge with learning.