[Paper Review] Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data

Maximilian Karl, Maximilian Soelch+2 more|arXiv (Cornell University)|May 20, 2016

Gaussian Processes and Bayesian Inference182 citations

TL;DR

DVBF trains a probabilistic latent state-space model from raw sequential data by making transitions differentiable and constraining the latent space to reflect true dynamics, enabling accurate long-term prediction and controllable inference.

ABSTRACT

We introduce Deep Variational Bayes Filters (DVBF), a new method for unsupervised learning and identification of latent Markovian state space models. Leveraging recent advances in Stochastic Gradient Variational Bayes, DVBF can overcome intractable inference distributions via variational inference. Thus, it can handle highly nonlinear input data with temporal and spatial dependencies such as image sequences without domain knowledge. Our experiments show that enabling backpropagation through transitions enforces state space assumptions and significantly improves information content of the latent embedding. This also enables realistic long-term prediction.

Motivation & Objective

• Motivate learning of latent state-space models from raw, high-dimensional sequential data without domain knowledge.
• Develop DVBF to enforce state-space assumptions and enable reliable system identification.
• Bridge variational inference with differentiable latent transitions to backpropagate through time.
• Show that DVBF yields full-information latent representations suitable for control and prediction.
• Demonstrate scalability to large datasets via stochastic gradient optimization.

Proposed method

• Formulate a probabilistic generative model with latent states z1:T, emissions x1:T, controls u1:T, and transition parameters β1:T.
• Introduce a reparameterized transition z_t+1 = f(z_t, u_t, β_t) so the transition is differentiable and backpropagates through time.
• Separate transition parameters into w_t (sample-specific noise) and v_t (global, prior-regularized parameters) with a factorized recognition model qφ(β1:T|x1:T, u1:T) = qφ(w1:T|x1:T, u1:T) qφ(v1:T).
• Derive a DVBF lower bound to the marginal likelihood pθ(x1:T|u1:T) using an evidence lower bound that includes a KL term for β and an annealing schedule to stabilize training.
• Provide a locally linear transition example where z_t+1 = A_t z_t + B_t u_t + C_t w_t and A_t, B_t, C_t are mixtures of M candidate matrices controlled by α_t = fψ(z_t, u_t).
• Demonstrate how the approach enforces latent dynamics suitable for long-term generation and prediction.

Experimental results

Research questions

• RQ1 Can DVBF reliably identify latent state-space dynamics from raw, potentially non-Markovian observations?
• RQ2 Does enforcing the transition dynamics in the latent space improve information content and long-term predictive capabilities compared to prior models?
• RQ3 How does DVBF perform on high-dimensional sensory data (e.g., images) for dynamic systems like pendulums and bouncing balls?
• RQ4 What is the impact of the proposed transition reparameterization and priors on learning stability and model quality?

Key findings

• DVBF with locally linear transitions learns latent spaces that encode full state information (e.g., angle and angular velocity in pendulum) more completely than prior methods such as DKF.
• On pendulum data, DVBF-LL achieves high correlations between latent states and ground-truth variables (e.g., sin(phi), cos(phi), and dot(phi)); DKF struggles to recover angular velocity.
• DVBF provides stable long-term generative samples that remain accurate beyond training sequence lengths, demonstrating improved long-horizon prediction.
• In the bouncing-ball experiments, the latent space cleanly separates position and velocity components, forming a meaningful 2D manifold for position and a coherent encoding for velocity.
• DVBF achieves better compression (lower reconstruction error with DVBF-LL) than DKF in the pendulum task, as reflected in the reported lower bound metrics.
• Across experiments, the DVBF framework demonstrates scalability to larger datasets via stochastic gradient optimization and a trainable neural-network-based recognition model.