[Paper Review] Variational Inference for Gaussian Process Modulated Poisson Processes

Chris Lloyd, Tom Gunter+2 more|arXiv (Cornell University)|Nov 2, 2014

Gaussian Processes and Bayesian InferenceComputer Science21 references70 citations

TL;DR

This paper introduces a fully variational Bayesian inference method for Gaussian process-modulated Poisson processes that eliminates domain discretization, achieves O(N) computational scaling, and enables fast, accurate inference on continuous spatial-temporal point process data. The proposed Variational Bayes for Point Processes (VBPP) framework outperforms sampling-based and kernel smoothing methods in prediction accuracy and runtime across synthetic data, coal mining disasters, and malaria incidence in Kenya.

ABSTRACT

We present the first fully variational Bayesian inference scheme for continuous Gaussian-process-modulated Poisson processes. Such point processes are used in a variety of domains, including neuroscience, geo-statistics and astronomy, but their use is hindered by the computational cost of existing inference schemes. Our scheme: requires no discretisation of the domain; scales linearly in the number of observed events; and is many orders of magnitude faster than previous sampling based approaches. The resulting algorithm is shown to outperform standard methods on synthetic examples, coal mining disaster data and in the prediction of Malaria incidences in Kenya.

Motivation & Objective

• Address the computational intractability of Bayesian inference in continuous-domain Gaussian process-modulated Poisson processes due to doubly intractable posteriors.
• Overcome the O(N³) scaling of standard Gaussian process inference and the sensitivity to binning in discretized approaches.
• Develop a fully variational Bayesian framework that scales linearly with the number of observed events and avoids domain discretization.
• Enable joint modeling of point processes with continuous covariates (e.g., rainfall) in a fully generative, scalable Bayesian framework.
• Demonstrate superior predictive performance and efficiency on real-world datasets including disaster records and disease incidence data.

Proposed method

• Propose a variational inference scheme for inhomogeneous Poisson processes where the intensity is modeled as a Gaussian process transformed via a sigmoid or square link function.
• Introduce a non-Gaussian likelihood model using a square link function to avoid the need for latent thinning, unlike previous methods such as the Sigmoidal Gaussian Cox Process.
• Employ inducing points to approximate the full GP, enabling O(N) scaling by reducing the number of latent function evaluations.
• Derive a variational lower bound (ELBO) that approximates the intractable posterior using a Gaussian approximation to the GP latent function.
• Optimize the variational parameters, including inducing point locations, to improve approximation tightness and predictive performance.
• Use a structured variational approximation that maintains computational efficiency while capturing uncertainty in the intensity function.

Experimental results

Research questions

• RQ1 Can a fully variational Bayesian inference scheme be developed for continuous-domain Gaussian process-modulated Poisson processes without requiring domain discretization?
• RQ2 Does the proposed method achieve linear O(N) scaling with the number of observed events, significantly outperforming O(N³) sampling-based approaches?
• RQ3 How does the performance of the VBPP framework compare to kernel smoothing and SGCP in terms of predictive log-likelihood and runtime on real-world datasets?
• RQ4 Can the method effectively model complex, non-Poisson-like intensity functions in high-dimensional spatial-temporal domains without binning?
• RQ5 To what extent can the framework be extended to jointly model point processes with continuous covariates in a multi-output GP framework?

Key findings

• The VBPP method achieves O(N) computational scaling and runs in 0.7 seconds on the coal mining disaster dataset, compared to 417.6 seconds for the SGCP method.
• On the Twitter data, VBPP with optimized inducing points outperforms both kernel smoothing and SGCP in predictive log-likelihood, even with as few as 10 inducing points.
• For the 2D malaria incidence data in Kenya, VBPP achieved a test log-likelihood of 869.7, outperforming kernel smoothing (867.2) and SGCP.
• The variational lower bound (L₀) and Lp bounds become tighter with fewer inducing points when using optimized inducing points, indicating improved uncertainty quantification.
• VBPP provides more accurate predictions than standard methods on held-out data across all benchmark datasets, including synthetic, disaster, and disease incidence data.
• The method enables joint inference of real-valued covariates (e.g., rainfall) and point processes, opening new avenues for fully generative modeling in epidemiology and environmental science.