[Paper Review] Identification of Stochastic Wiener Systems using Indirect Inference

Bo Wahlberg, James S. Welsh+1 more|arXiv (Cornell University)|Jul 20, 2015

Control Systems and Identification10 references4 citations

TL;DR

This paper proposes an indirect inference method for identifying stochastic Wiener systems—linear dynamic systems with process noise and nonlinear output sensors—using the best linear approximation (BLA) as an auxiliary model. By applying optimal uncertainty weighting during the second-stage estimation, the method achieves statistical performance comparable to maximum-likelihood and prediction error minimization, with significantly reduced computational cost.

ABSTRACT

We study identification of stochastic Wiener dynamic systems using so-called indirect inference. The main idea is to first fit an auxiliary model to the observed data and then in a second step, often by simulation, fit a more structured model to the estimated auxiliary model. This two-step procedure can be used when the direct maximum-likelihood estimate is difficult or intractable to compute. One such example is the identification of stochastic Wiener systems, i.e.,~linear dynamic systems with process noise where the output is measured using a non-linear sensor with additive measurement noise. It is in principle possible to evaluate the log-likelihood cost function using numerical integration, but the corresponding optimization problem can be quite intricate. This motivates studying consistent, but sub-optimal, identification methods for stochastic Wiener systems. We will consider indirect inference using the best linear approximation as an auxiliary model. We show that the key to obtain a reliable estimate is to use uncertainty weighting when fitting the stochastic Wiener model to the auxiliary model estimate. The main technical contribution of this paper is the corresponding asymptotic variance analysis. A numerical evaluation is presented based on a first-order finite impulse response system with a cubic non-linearity, for which certain illustrative analytic properties are derived.

Motivation & Objective

• To address the challenge of identifying stochastic Wiener systems where direct maximum-likelihood estimation is computationally intractable due to nonlinearities.
• To investigate the use of indirect inference with the best linear approximation as an auxiliary model for consistent, sub-optimal parameter estimation.
• To derive and validate the asymptotic variance of the indirect inference estimator for Wiener systems.
• To demonstrate that optimal weighting in the second-stage estimation is essential for reliable performance, especially when using higher-order BLA models.
• To compare the proposed method’s statistical efficiency and computational efficiency against maximum-likelihood and weighted prediction error minimization methods.

Proposed method

• Use the best linear approximation (BLA) of the Wiener system as an auxiliary model, estimated from observed input-output data.
• Apply indirect inference: in a second step, fit the structured stochastic Wiener model to the estimated BLA parameters using simulation-based optimization.
• Incorporate uncertainty weighting in the second-stage cost function, using the inverse of the asymptotic covariance matrix of the BLA estimate.
• Derive the asymptotic variance of the indirect inference estimator using system identification theory, with explicit expressions for the weighting matrix.
• Use analytic solutions for the second-stage optimization in a first-order FIR system with cubic nonlinearity to enable exact performance evaluation.
• Validate the method via Monte Carlo simulations with both Gaussian and uniform input signals, comparing to ML and PEM benchmarks.

Experimental results

Research questions

• RQ1 Can indirect inference using the best linear approximation provide a consistent and computationally efficient alternative to maximum-likelihood estimation for stochastic Wiener systems?
• RQ2 What is the impact of uncertainty weighting in the second-stage estimation on the statistical performance of the indirect inference method?
• RQ3 How does the performance of indirect inference compare to maximum-likelihood and weighted prediction error minimization in terms of bias, variance, and computational cost?
• RQ4 Does the use of a first-order BLA model with optimal weighting outperform a zero-order BLA model, despite containing more information?
• RQ5 How does the presence of a cubic nonlinearity affect the estimation uncertainty compared to the linear case?

Key findings

• The indirect inference method with optimal uncertainty weighting achieves statistical performance comparable to maximum-likelihood and weighted prediction error minimization, with significantly reduced computational cost.
• For a first-order FIR system with cubic nonlinearity, the mean estimate of the true parameter θ₀ = 0.5 was 0.4982 (std: 0.0418) under the first-order indirect inference with weighting, close to the ML result (mean: 0.5025, std: 0.0303).
• The zero-order indirect inference method (without weighting) yielded a higher standard deviation (0.0446 for Gaussian input), indicating that unweighted estimation leads to less precise estimates.
• The first-order indirect inference without weighting produced a higher standard deviation (0.0554) than the weighted version, confirming that optimal weighting is essential for reliable performance.
• The simulation results show that the nonlinearity (cubic) increases estimation uncertainty compared to the linear case, where the theoretical standard deviation is 0.0173, while nonlinear cases have higher empirical standard deviations.
• The computational time for indirect inference is a small fraction of that required for maximum-likelihood estimation, making it highly efficient for practical implementation.