[Paper Review] A Block Coordinate Descent Proximal Method for Simultaneous Filtering and Parameter Estimation

Ramin Raziperchikolaei, Harish S. Bhat|arXiv (Cornell University)|Oct 16, 2018

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes a block coordinate descent proximal (BCD-prox) algorithm for simultaneous filtering and parameter estimation in nonlinear ODE models, directly minimizing a unified objective that enforces ODE consistency. It achieves superior robustness to noise, initialization, and hyperparameters, faster convergence, and higher accuracy than state-of-the-art methods on ODEs up to 40 dimensions.

ABSTRACT

We propose and analyze a block coordinate descent proximal algorithm (BCD-prox) for simultaneous filtering and parameter estimation of ODE models. As we show on ODE systems with up to d=40 dimensions, as compared to state-of-the-art methods, BCD-prox exhibits increased robustness (to noise, parameter initialization, and hyperparameters), decreased training times, and improved accuracy of both filtered states and estimated parameters. We show how BCD-prox can be used with multistep numerical discretizations, and we establish convergence of BCD-prox under hypotheses that include real systems of interest.

Motivation & Objective

• Address the challenge of estimating states and parameters in nonlinear ODEs from noisy, high-dimensional data.
• Overcome limitations of existing methods—such as spline-based, Bayesian, and Kalman filter approaches—that degrade under high noise or poor initialization.
• Develop a joint estimation framework that avoids indirect state reconstruction via smoothing functions, enabling direct learning of clean states in the original space.
• Ensure convergence under realistic system hypotheses while minimizing hyperparameter sensitivity.
• Provide a scalable, efficient, and robust method for real-world ODE systems in biology and engineering with noisy observations.

Proposed method

• Formulates a unified objective function that measures how well the estimated states and parameters satisfy the ODE dynamics, directly linking state and parameter estimation.
• Employs block coordinate descent (BCD) to alternately optimize over the state vector X and parameter vector θ, improving convergence and stability.
• Integrates proximal methods to regularize updates and ensure slow, stable progress away from poor initial parameter values.
• Uses explicit Euler discretization as a base, with extension to higher-order multistep numerical schemes for improved accuracy.
• Solves the optimization problem in a batch, distribution-free manner, avoiding Gaussian assumptions common in Kalman filters.
• Reduces hyperparameter count to one (the step size), enhancing robustness and ease of use.

Experimental results

Research questions

• RQ1 Can a joint optimization framework for ODE states and parameters achieve better robustness and accuracy than decoupled or indirect methods?
• RQ2 How does BCD-prox perform under high noise levels, poor initialization, and varying hyperparameters compared to state-of-the-art methods?
• RQ3 To what extent can BCD-prox maintain convergence and accuracy when applied to high-dimensional ODE systems (up to d=40)?
• RQ4 Does the direct optimization of states in the original space, without smoothing, lead to more accurate filtering and parameter estimation?
• RQ5 How does the integration of proximal updates improve convergence behavior compared to standard BCD or gradient-based methods?

Key findings

• BCD-prox demonstrates increased robustness to noise, with prediction errors decreasing significantly across models like FitzHugh-Nagumo, Rössler, and Lorenz-96 even at high noise variances (σ² = 1.5).
• The method achieves faster training times and improved convergence speed, with parameter estimation errors dropping steadily over iterations, as shown in the animation and error plots.
• On the FitzHugh-Nagumo model, BCD-prox reduced prediction error by over 50% compared to mean-field and EKF methods under varying noise levels.
• For the Rössler attractor, parameter estimation errors converged to near-zero within 20 iterations, even with σ² = 1.5, outperforming EKF and mean-field baselines.
• BCD-prox maintained stable performance across different initializations and hyperparameters, unlike spline-based or Bayesian methods that required careful tuning.
• The method achieved high accuracy in estimating true parameters (θ₀=0.5, θ₁=0.2, θ₂=3) in the FitzHugh-Nagumo model, with parameter errors decreasing monotonically over iterations.