[Paper Review] Learning Stochastic Recurrent Networks

Justin Bayer, Christian Osendorfer|arXiv (Cornell University)|Nov 27, 2014

Machine Learning and Algorithms24 references195 citations

TL;DR

This paper introduces Stochastic Recurrent Networks (STORNs), a novel class of recurrent neural networks enhanced with latent variables to model complex, multi-modal sequence distributions. By leveraging stochastic gradient variational Bayes (SGVB), STORNs enable end-to-end training with reliable likelihood estimation, outperforming deterministic RNNs and prior models on polyphonic music and motion capture data, achieving state-of-the-art results in both reconstruction and generation tasks.

ABSTRACT

Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a reliable estimator of the marginal likelihood and iv) is a generalisation of deterministic recurrent neural networks. We evaluate the method on four polyphonic musical data sets and motion capture data.

Motivation & Objective

• To address the limitation of deterministic RNNs in modeling complex, multi-modal conditional distributions over sequence components.
• To enable reliable estimation of marginal likelihood in recurrent models with latent variables, overcoming intractable inference in previous approaches.
• To extend the representational capacity of RNNs by incorporating structured, stochastic latent variables while maintaining training efficiency.
• To provide a generalization of deterministic RNNs that supports both structured and multi-modal output distributions at each time step.
• To evaluate the model on high-dimensional, tightly coupled sequence data such as polyphonic music and 3D motion capture sequences.

Proposed method

• The model extends standard RNNs by introducing a latent variable z_t at each time step, enabling stochastic, multi-modal modeling of x_t given the history.
• The joint likelihood p(x_{1:T}, z_{1:T}) is modeled as a product of conditional distributions p(x_t | z_t, h_t) and p(z_t | h_t), with h_t being the hidden state.
• The recognition model q(z_{1:T} | x_{1:T}) is implemented as a bidirectional RNN to infer posterior distributions over latent variables.
• Stochastic Gradient Variational Bayes (SGVB) is used to train the model via a differentiable lower bound on the log-likelihood, enabling backpropagation through stochastic nodes.
• The reparameterization trick is applied to the latent variables, allowing gradient estimation via Monte Carlo sampling with low variance.
• The model is trained using the evidence lower bound (ELBO), which serves as the objective function for optimizing both the generative and recognition models.

Experimental results

Research questions

• RQ1 Can the integration of latent variables into RNNs significantly improve modeling of complex, multi-modal sequence distributions?
• RQ2 Does the use of SGVB enable effective and efficient training of stochastic RNNs with reliable likelihood estimation?
• RQ3 How does the proposed STORN model compare to deterministic RNNs and prior models like RNN-RBM or RTRBM in terms of reconstruction and generation performance?
• RQ4 Can STORNs effectively handle high-dimensional, tightly coupled sequence data such as motion capture and polyphonic music?
• RQ5 To what extent does the model’s stochasticity enhance its ability to capture uncertainty and diversity in sequence generation?

Key findings

• The STORN model achieved a mean squared error (MSE) of 4.94 on motion capture data, significantly outperforming previous models such as RNN-RBM (MSE 20.1) and RTRBM (MSE 16.2).
• The estimated negative log-likelihood on the validation set was 15.99, providing a tractable metric for comparison, unlike models such as RNN-RBM and RTRBM that lack tractable likelihood estimation.
• The model demonstrated strong generative capabilities, producing diverse and plausible samples after a 20-step stimulus prefix, with visible uncertainty in the output distribution.
• Missing value imputation using maximum a posteriori estimation of latent variables successfully reconstructed corrupted motion sequences, confirming the model’s robustness and inference capability.
• The STORN model effectively captured structured dependencies among high-dimensional sequence components, outperforming models with naive Bayes assumptions or fixed mixture components.
• The use of a bidirectional recognition model improved posterior inference, contributing to better reconstruction and generation performance.