[Paper Review] Efficient EM-Variational Inference for Hawkes Process

Feng Zhou, Li, Zhidong+4 more|arXiv (Cornell University)|May 29, 2019

Point processes and geometric inequalities23 references4 citations

TL;DR

This paper proposes a fully Bayesian nonparametric Hawkes process using Gaussian process transformations to model baseline intensity and triggering kernel as flexible, smooth functions. By embedding variational inference within an EM framework via a latent branching structure and introducing acceleration techniques, the method enables efficient, accurate inference without gridding, outperforming state-of-the-art methods on synthetic and real-world data.

ABSTRACT

In classical Hawkes process, the baseline intensity and triggering kernel are assumed to be a constant and parametric function respectively, which limits the model flexibility. To generalize it, we present a fully Bayesian nonparametric model, namely Gaussian process modulated Hawkes process and propose an EM-variational inference scheme. In this model, a transformation of Gaussian process is used as a prior on the baseline intensity and triggering kernel. By introducing a latent branching structure, the inference of baseline intensity and triggering kernel is decoupled and the variational inference scheme is embedded into an EM framework naturally. We also provide a series of schemes to accelerate the inference. Results of synthetic and real data experiments show that the underlying baseline intensity and triggering kernel can be recovered without parametric restriction and our Bayesian nonparametric estimation is superior to other state of the arts.

Motivation & Objective

• To overcome the limitations of classical Hawkes processes that assume constant baseline intensity and parametric triggering kernels.
• To develop a fully Bayesian nonparametric model that allows flexible, smooth estimation of baseline intensity and triggering kernel without domain gridding.
• To enable efficient inference for such a complex model by decoupling baseline and kernel estimation through a latent branching structure.
• To accelerate variational inference in the EM framework using dimensionality reduction and closed-form derivatives of the ELBO.

Proposed method

• Model baseline intensity and triggering kernel as transformations of Gaussian processes to ensure nonnegativity and allow nonparametric, smooth estimation.
• Introduce a latent branching structure to decouple the dependence between baseline intensity and triggering kernel in the likelihood, enabling EM-based inference.
• Embed variational inference within the EM framework to jointly infer posterior distributions over the intensity and kernel functions.
• Apply dimensionality reduction techniques to reduce computational complexity in high-dimensional optimization during variational inference.
• Derive a closed-form partial derivative of the evidence lower bound (ELBO) to accelerate optimization steps in the EM-Variational (EMV) algorithm.
• Use the EMV algorithm with O(RN²) complexity over R iterations to scale inference to large datasets.

Experimental results

Research questions

• RQ1 Can a fully Bayesian nonparametric model for Hawkes processes achieve better flexibility and accuracy than parametric or frequentist nonparametric alternatives?
• RQ2 How can variational inference be effectively embedded within an EM framework to handle the coupled likelihood of baseline intensity and triggering kernel?
• RQ3 What acceleration techniques can make Bayesian nonparametric inference for Hawkes processes computationally feasible and practical?
• RQ4 To what extent can the model recover complex, time-varying baseline intensity and non-exponential triggering kernels without parametric assumptions?

Key findings

• The proposed EM-variational inference (EMV) method successfully recovers underlying baseline intensity and triggering kernel without parametric restrictions, as demonstrated on synthetic data with varying true functions.
• On synthetic Case 1, EMV achieved an estimation error of 0.008 for baseline intensity and 0.015 for the triggering kernel, with a log-likelihood of -37.91, outperforming all baselines.
• In Case 2, EMV achieved the highest log-likelihood (96.85) and lowest estimation error for the triggering kernel (0.002), indicating superior goodness-of-fit.
• On real-world datasets, EMV achieved a prediction accuracy of 71.7% on vehicle collision data and 70.4% on taxi pickup data, significantly outperforming alternatives like PH (60.6%) and WH (67.3%).
• Q-Q plots showed that EMV's model fit was closest to the ideal diagonal line, confirming better calibration than inhomogeneous Poisson processes and models with fixed baseline intensity.
• The EMV algorithm achieved a 20-30% speedup over baseline variational inference via closed-form derivatives and dimensionality reduction, making it practical for large-scale applications.