[Paper Review] On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes

Alexander Matthews, James Hensman+2 more|arXiv (Cornell University)|Apr 27, 2015

Gaussian Processes and Bayesian Inference13 references4 citations

TL;DR

This paper establishes a rigorous measure-theoretic foundation for sparse variational Gaussian processes by showing that the standard inducing point method minimizes a well-defined Kullback-Leibler divergence between stochastic processes. It generalizes previous work by allowing non-data inducing points, non-conjugate likelihoods, and infinite-dimensional models like Cox processes, while proving that marginal consistency alone is insufficient for valid variational inference unless additional conditions—such as deterministic conditioning on the full function—are met.

ABSTRACT

The variational framework for learning inducing variables (Titsias, 2009a) has had a large impact on the Gaussian process literature. The framework may be interpreted as minimizing a rigorously defined Kullback-Leibler divergence between the approximating and posterior processes. To our knowledge this connection has thus far gone unremarked in the literature. In this paper we give a substantial generalization of the literature on this topic. We give a new proof of the result for infinite index sets which allows inducing points that are not data points and likelihoods that depend on all function values. We then discuss augmented index sets and show that, contrary to previous works, marginal consistency of augmentation is not enough to guarantee consistency of variational inference with the original model. We then characterize an extra condition where such a guarantee is obtainable. Finally we show how our framework sheds light on interdomain sparse approximations and sparse approximations for Cox processes.

Motivation & Objective

• To clarify the theoretical justification behind the variational inducing point framework in Gaussian processes, which has been widely used but lacks a rigorous probabilistic foundation.
• To address the gap in the literature by formally defining the Kullback-Leibler divergence between stochastic processes in the context of sparse variational inference.
• To investigate whether marginal consistency in model augmentation is sufficient to ensure valid variational inference, particularly when introducing auxiliary variables for approximation.
• To extend the framework to non-conjugate likelihoods and infinite-dimensional models such as Cox processes, where likelihoods depend on all function values.
• To provide a principled derivation of the variational objective that justifies optimizing inducing point locations as variational parameters without overfitting.

Proposed method

• Reformulates the standard sparse variational inference objective as a proper Kullback-Leibler divergence between the approximate and true posterior processes over function values.
• Uses Radon-Nikodym derivatives to define the KL divergence in infinite-dimensional function spaces, avoiding reliance on Lebesgue measure.
• Generalizes the original Titsias (2009) framework to allow inducing points that are not necessarily data points, and to handle likelihoods depending on the entire function, not just observed points.
• Applies Fubini’s theorem to justify interchanging expectations and integrals in the KL divergence expression, particularly for Cox process likelihoods.
• Introduces a condition under which variational inference in an augmented model is equivalent to inference in the original model: when inducing points are deterministic functions of the full function.
• Demonstrates that interdomain sparse approximations and Cox process models can be consistently treated within this framework by ensuring proper measure-theoretic foundations.

Experimental results

Research questions

• RQ1 Is the standard sparse variational inference objective in Gaussian processes equivalent to minimizing a well-defined Kullback-Leibler divergence between stochastic processes?
• RQ2 Can the variational inducing point framework be rigorously justified when inducing points are not data points and likelihoods depend on unobserved function values?
• RQ3 Does marginal consistency of an augmented model guarantee that variational inference in the augmented model is equivalent to inference in the original model?
• RQ4 What additional conditions are required to ensure that variational inference in an augmented model is principled and consistent with the original model?
• RQ5 How can the framework be extended to handle non-conjugate likelihoods and point process models such as Cox processes?

Key findings

• The standard sparse variational inference objective is rigorously equivalent to minimizing a Kullback-Leibler divergence between the approximate and true posterior processes over function values.
• The framework generalizes to non-data inducing points and likelihoods that depend on all function values, such as in Cox processes, by using Radon-Nikodym derivatives and measure-theoretic foundations.
• Marginal consistency of model augmentation is not sufficient to guarantee valid variational inference; a stronger condition—deterministic conditioning of inducing points on the full function—is required.
• When inducing points are deterministic functions of the full function, the variational objective becomes principled and the inducing point parameters are protected from overfitting.
• The theory correctly handles interdomain sparse approximations and Cox processes by ensuring the KL divergence is well-defined and the evidence lower bound is valid.
• The results provide a solid theoretical basis for extending sparse variational methods to complex models, including those with infinite-dimensional likelihoods, and suggest new directions involving Hilbert space theory.