[Paper Review] Efficient State-Space Inference of Periodic Latent Force Models

Steven Reece, Stephen Roberts+3 more|arXiv (Cornell University)|Oct 23, 2013

Gaussian Processes and Bayesian Inference40 references14 citations

TL;DR

This paper proposes a novel state-space inference method for periodic latent force models (LFMs) using kernel principal component analysis (KPCA)-derived eigenfunction basis models to efficiently represent periodic and quasi-periodic Gaussian process priors. The approach enables scalable, principled inference in LFMs—reducing RMSE to 17% of non-periodic models and 27% of the resonator model—while maintaining full Bayesian inference without manual hyperparameter tuning.

ABSTRACT

Latent force models (LFM) are principled approaches to incorporating solutions to differential equations within non-parametric inference methods. Unfortunately, the development and application of LFMs can be inhibited by their computational cost, especially when closed-form solutions for the LFM are unavailable, as is the case in many real world problems where these latent forces exhibit periodic behaviour. Given this, we develop a new sparse representation of LFMs which considerably improves their computational efficiency, as well as broadening their applicability, in a principled way, to domains with periodic or near periodic latent forces. Our approach uses a linear basis model to approximate one generative model for each periodic force. We assume that the latent forces are generated from Gaussian process priors and develop a linear basis model which fully expresses these priors. We apply our approach to model the thermal dynamics of domestic buildings and show that it is effective at predicting day-ahead temperatures within the homes. We also apply our approach within queueing theory in which quasi-periodic arrival rates are modelled as latent forces. In both cases, we demonstrate that our approach can be implemented efficiently using state-space methods which encode the linear dynamic systems via LFMs. Further, we show that state estimates obtained using periodic latent force models can reduce the root mean squared error to 17% of that from non-periodic models and 27% of the nearest rival approach which is the resonator model.

Motivation & Objective

• To address the computational infeasibility of existing latent force models (LFMs) when modeling periodic or quasi-periodic latent forces.
• To extend state-space inference in LFMs to accommodate stationary periodic, non-stationary periodic, and quasi-periodic Gaussian process priors.
• To develop a sparse, efficient representation of periodic latent forces using eigenfunctions derived from kernel principal component analysis (KPCA).
• To enable fully Bayesian inference without manual tuning of hyperparameters by embedding the GP prior structure directly into the state-space model.
• To demonstrate the method’s effectiveness on real-world problems involving periodic dynamics, such as home heating and call centre queueing.

Proposed method

• The method uses a linear basis model (LBM) with eigenfunctions derived from kernel principal component analysis (KPCA) in the temporal domain to represent periodic latent forces.
• The eigenfunctions are computed from the covariance function of the periodic GP prior, ensuring the basis captures the full prior structure.
• The state vector is augmented with coefficients of the LBM to represent the latent force, enabling efficient state-space inference via Kalman filtering.
• The approach models both stationary and non-stationary periodic processes by adapting the KPCA basis to the time-varying characteristics of the latent force.
• Quasi-periodic forces are modeled using a weighted sum of eigenfunctions with time-varying amplitudes, allowing for day-to-day variation in periodic patterns.
• All model parameters, including hyperparameters, are inferred via maximum likelihood or Bayesian methods, avoiding manual tuning.

Experimental results

Research questions

• RQ1 Can periodic and quasi-periodic latent forces be efficiently represented within a state-space framework for latent force models?
• RQ2 Does the proposed eigenfunction-based LBM representation outperform existing approaches like the resonator model in terms of predictive accuracy?
• RQ3 Can the method scale to real-world applications with complex, non-stationary periodic dynamics such as home heating and customer arrivals in call centres?
• RQ4 To what extent does incorporating periodic priors reduce prediction error compared to non-periodic models in both linear and non-linear systems?
• RQ5 Can the method maintain computational efficiency while fully encoding the information from the GP prior in the state-space representation?

Key findings

• The proposed method reduced root mean squared error (RMSE) to 17% of that from non-periodic models in thermal dynamics prediction, significantly outperforming baseline approaches.
• In the call centre application, periodic models reduced RMSE by up to 83% compared to non-periodic models, demonstrating strong performance on daily arrival rate patterns.
• The eigenfunction-based model reduced RMSE by up to 74% compared to the resonator model in the thermal dynamics application, indicating superior modeling of residual heat profiles.
• Quasi-periodic models (WQM and SQM) achieved the lowest mean-squared error and highest expected log-likelihood across both applications, especially in scenarios with variable amplitude periodicity.
• The method successfully enabled efficient inference in both linear (home heating) and non-linear (call centre) systems, demonstrating broad applicability.
• The approach achieved state-of-the-art performance, outperforming the resonator model (Särkkä et al., 2012; Hartikainen et al., 2012) by reducing RMSE to 27% of its value.