[Paper Review] A Primer on PAC-Bayesian Learning

Benjamin Guedj|arXiv (Cornell University)|Jan 16, 2019

Machine Learning and Algorithms75 references53 citations

TL;DR

This paper provides a self-contained survey of the PAC-Bayes framework, outlining its generalisation properties and surveying key theoretical and algorithmic developments in PAC-Bayesian learning.

ABSTRACT

Generalised Bayesian learning algorithms are increasingly popular in machine learning, due to their PAC generalisation properties and flexibility. The present paper aims at providing a self-contained survey on the resulting PAC-Bayes framework and some of its main theoretical and algorithmic developments.

Motivation & Objective

• Motivate the use of generalized Bayesian learning due to PAC generalisation properties and flexibility.
• Summarize the PAC-Bayes framework and its core theoretical results.
• Review major algorithmic developments and practical implications of PAC-Bayesian methods.

Proposed method

• Explain the PAC-Bayesian framework and its relation to Bayesian learning and PAC guarantees.
• Survey main theoretical results and their implications for generalisation bounds.
• Discuss notable algorithmic approaches and practical considerations in PAC-Bayesian learning.

Experimental results

Research questions

• RQ1 What are the core PAC-Bayes principles that enable generalisation guarantees?
• RQ2 What are the key theoretical results and how do they translate into algorithms?
• RQ3 What are the prominent algorithmic developments and their practical implications in PAC-Bayesian learning.

Key findings

• Provides a self-contained survey of the PAC-Bayes framework.
• Reviews essential theoretical results and their implications for generalisation.
• Summarizes important algorithmic developments within PAC-Bayesian learning.