[Paper Review] Efficient Inference in Multi-task Cox Process Models

Virginia Aglietti, Theodoros Damoulas+1 more|arXiv (Cornell University)|May 24, 2018

Gaussian Processes and Bayesian Inference4 citations

TL;DR

This paper proposes a multi-task Cox process model (MCPM) that models correlated point processes by treating log intensities as linear combinations of shared Gaussian process (GP) latent functions with stochastic, GP-distributed mixing weights. By deriving closed-form moment expressions for the intensity, the method enables fast variational inference—15× faster than state-of-the-art methods—while improving uncertainty quantification and predictive performance on real and synthetic multi-task point process data.

ABSTRACT

We generalize the log Gaussian Cox process (LGCP) framework to model multiple correlated point data jointly. The observations are treated as realizations of multiple LGCPs, whose log intensities are given by linear combinations of latent functions drawn from Gaussian process priors. The combination coefficients are also drawn from Gaussian processes and can incorporate additional dependencies. We derive closed-form expressions for the moments of the intensity functions and develop an efficient variational inference algorithm that is orders of magnitude faster than competing deterministic and stochastic approximations of multivariate LGCP, coregionalization models, and multi-task permanental processes. Our approach outperforms these benchmarks in multiple problems, offering the current state of the art in modeling multivariate point processes.

Motivation & Objective

• Address the limitations of existing multivariate log-Gaussian Cox process (lgcp) models that fix the linear combination weights, leading to poor uncertainty quantification.
• Overcome the computational intractability of inference in multi-task lgcp models due to their doubly-stochastic nature and high-dimensional latent spaces.
• Develop a scalable inference framework that enables efficient posterior estimation over both latent functions and stochastic mixing weights.
• Provide the first comprehensive empirical comparison between lgcp-based and permanental process-based models for multi-task point processes.
• Achieve state-of-the-art performance in predictive accuracy and computational efficiency on large-scale, real-world multi-task point process datasets.

Proposed method

• Model multiple correlated point processes as realizations of separate lgcp processes, where the log intensity of each task is a linear combination of shared GP latent functions.
• Introduce stochastic mixing weights by modeling the combination coefficients as independent Gaussian processes, enabling uncertainty propagation in the linear combination.
• Derive analytical expressions for the moment generating functions (MGFs) of the log intensity, allowing closed-form computation of expectations in the variational objective.
• Develop a variational inference algorithm that avoids Monte Carlo sampling by leveraging these closed-form MGFs, significantly accelerating convergence.
• Use inducing points to approximate the full GP posterior, enabling scalable inference on large datasets.
• Optimize the variational lower bound (evidence lower bound) using analytical expectations, eliminating the need for stochastic gradient estimates.

Experimental results

Research questions

• RQ1 Can stochastic mixing weights in a multi-task lgcp model improve uncertainty quantification compared to fixed or deterministic weights?
• RQ2 Can closed-form moment expressions for the intensity distribution enable faster and more stable variational inference in multi-task lgcp models?
• RQ3 How does the proposed MCPM compare in predictive performance and computational efficiency to existing methods such as coregionalization models, multi-task permanental processes, and MCMC-based lgcp models?
• RQ4 Can the model effectively transfer information across spatially segregated tasks with complex cross-correlation structures, especially in missing data settings?
• RQ5 Does the model maintain robustness and avoid negative transfer when inter-task correlations are negative or weak?

Key findings

• The MCPM model achieves state-of-the-art predictive performance on both synthetic and real-world datasets, outperforming benchmark models including intrinsic coregionalization models (ICM) and multi-task lgcp (mlgcp) with MCMC.
• On the btb dataset, MCPM reduced inference time from ~30 hours (mlgcp with MCMC) to ~8 hours, demonstrating a 3.75× speedup over MCMC-based approaches.
• In missing data experiments, MCPM achieved the highest expected coverage (EC) both in-sample and out-of-sample, and outperformed ICM on 3 out of 4 tasks.
• The model converged in 3.18 hours (1500 epochs), compared to 18.63 hours for ICM (1000 epochs), showing a 6× speedup in training time.
• The predictive distributions of MCPM contained the true counts within their 90% credible intervals, while ICM failed to do so, confirming improved uncertainty calibration.
• The use of closed-form MGFs eliminated the need for Monte Carlo sampling in the variational objective, resulting in faster and more stable optimization compared to black-box variational inference methods.