[Paper Review] Targeted Neural Dynamical Modeling

Cole Hurwitz, Akash Srivastava+5 more|arXiv (Cornell University)|Oct 28, 2021

Neural dynamics and brain function25 references22 citations

TL;DR

This paper introduces Targeted Neural Dynamical Modeling (TNDM), a nonlinear state-space model that jointly models neural activity and behavior by decomposing latent dynamics into behaviorally relevant and irrelevant components. TNDM uses a sequential variational autoencoder to learn low-dimensional, predictive dynamics that reconstruct both neural data and behavior with flexible, causal temporal relationships, outperforming PSID and LFADS in behavior prediction while maintaining strong neural data fit.

ABSTRACT

Latent dynamics models have emerged as powerful tools for modeling and interpreting neural population activity. Recently, there has been a focus on incorporating simultaneously measured behaviour into these models to further disentangle sources of neural variability in their latent space. These approaches, however, are limited in their ability to capture the underlying neural dynamics (e.g. linear) and in their ability to relate the learned dynamics back to the observed behaviour (e.g. no time lag). To this end, we introduce Targeted Neural Dynamical Modeling (TNDM), a nonlinear state-space model that jointly models the neural activity and external behavioural variables. TNDM decomposes neural dynamics into behaviourally relevant and behaviourally irrelevant dynamics; the relevant dynamics are used to reconstruct the behaviour through a flexible linear decoder and both sets of dynamics are used to reconstruct the neural activity through a linear decoder with no time lag. We implement TNDM as a sequential variational autoencoder and validate it on simulated recordings and recordings taken from the premotor and motor cortex of a monkey performing a center-out reaching task. We show that TNDM is able to learn low-dimensional latent dynamics that are highly predictive of behaviour without sacrificing its fit to the neural data.

Motivation & Objective

• To address the limitation of existing latent dynamics models in disentangling behaviorally relevant neural dynamics from irrelevant variability.
• To overcome the constraints of linear models like PSID, which assume no time lag between dynamics and behavior and cannot capture nonlinear dynamics.
• To develop a method that jointly models neural activity and behavior with flexible, causal temporal dependencies between latent dynamics and behavior.
• To improve interpretability and predictive power of neural population dynamics by enforcing disentanglement between behaviorally relevant and irrelevant factors.
• To validate the model on both synthetic data and real neural recordings from monkey premotor and motor cortex during a center-out reaching task.

Proposed method

• TNDM employs a sequential variational autoencoder (sVAE) to perform amortized inference in a nonlinear state-space model, enabling efficient learning of complex dynamics.
• The model decomposes latent dynamics into two components: behaviorally relevant dynamics (driven by a causal linear decoder) and behaviorally irrelevant dynamics (unrelated to behavior).
• A disentanglement penalty is applied to the initial condition distributions of the relevant and irrelevant dynamics to encourage separation of sources of variability.
• The behaviorally relevant dynamics are used to reconstruct behavior via a linear, time-lagged causal decoder, allowing for flexible temporal relationships beyond instantaneous correspondence.
• Both relevant and irrelevant dynamics are combined through a linear decoder to reconstruct the observed neural activity with no time lag, preserving data fidelity.
• The model is trained end-to-end using a variational lower bound that balances reconstruction of neural data and behavior, with hyperparameters tuned to avoid biologically implausible oscillations.

Experimental results

Research questions

• RQ1 Can a nonlinear state-space model effectively disentangle behaviorally relevant from behaviorally irrelevant neural dynamics in population activity?
• RQ2 Does allowing for time-lagged, causal relationships between latent dynamics and behavior improve behavior prediction compared to models assuming instantaneous correspondence?
• RQ3 Can a disentanglement penalty on initial condition distributions enhance the interpretability and predictive power of learned dynamics?
• RQ4 How does TNDM compare to state-of-the-art models like PSID and LFADS in terms of behavior prediction and neural data reconstruction accuracy?
• RQ5 What is the optimal dimensionality of the behaviorally relevant latent space for capturing movement dynamics in a center-out reaching task?

Key findings

• TNDM achieved higher behavior prediction accuracy than PSID and LFADS on both synthetic and real neural data, with the highest predictive power observed at two relevant latent factors.
• The behaviorally relevant dynamics in TNDM were lower-dimensional and more predictive than those from PSID and LFADS, suggesting a potentially lower intrinsic dimensionality for 2D reaching dynamics.
• The model's behavior prediction saturation point occurred at two relevant factors, indicating that this dimensionality captures the core dynamics related to hand velocity.
• Irrelevant dynamics in TNDM showed consistent, task-agnostic patterns—such as an initial fluctuation followed by a steady rise—suggesting they may reflect general motor execution processes.
• The disentanglement penalty successfully separated relevant and irrelevant dynamics, with the relevant factors showing strong correlation to movement velocity and the irrelevant factors showing minimal task dependence.
• TNDM maintained strong neural data reconstruction performance comparable to LFADS, demonstrating that improved behavior prediction does not come at the cost of data fit.