[Paper Review] Error analysis of a class of derivative estimators for noisy signals

Da‐Yan Liu, Olivier Gibaru+1 more|arXiv (Cornell University)|Mar 3, 2011

Target Tracking and Data Fusion in Sensor Networks35 references3 citations

TL;DR

This paper extends algebraic derivative estimators for noisy signals by generalizing their parameters to real numbers via truncated Jacobi orthogonal series, significantly reducing noise error variance. It proves that negative real parameters yield lower noise error variance than integer parameters, enabling optimized trade-offs between bias, noise, and time delay in real-time applications.

ABSTRACT

Recent algebraic parametric estimation techniques led to point-wise derivative estimates by using only the iterated integral of a noisy observation signal. In this paper, we extend such differentiation methods by providing a larger choice of parameters in these integrals: they can be reals. For this the extension is done via a truncated Jacobi orthogonal series expansion. Then, the noise error contribution of these derivative estimations is investigated: after proving the existence of such integral with a stochastic process noise, their statistical properties (mean value, variance and covariance) are analyzed. In particular, the following important results are obtained: a) the bias error term, due to the truncation, can be reduced by tuning the parameters, b) such estimators can cope with a large class of noises for which the mean and covariance are polynomials in time (with degree smaller than the order of derivative to be estimated), c) the variance of the noise error is shown to be smaller in the case of negative real parameters than it was for integer values. Consequently, these derivative estimations can be improved by tuning the parameters according to the here obtained knowledge of the parameters' influence on the error bounds.

Motivation & Objective

• Address the ill-posed nature of numerical differentiation under noisy conditions by improving derivative estimation stability.
• Extend existing algebraic derivative estimators to allow real-valued parameters instead of restricting to integers.
• Analyze the statistical properties of noise error contributions in derivative estimators using stochastic integrals.
• Optimize the trade-off between bias error, noise error variance, and time delay through parameter tuning.
• Demonstrate that negative real parameters reduce noise error variance compared to integer parameters.

Proposed method

• Use truncated Jacobi orthogonal series expansion to generalize the parameter domain of derivative estimators from integers to real numbers.
• Model the noisy signal as a stochastic process and define the derivative estimator as a stochastic integral with respect to the observation signal.
• Derive analytical expressions for the mean, variance, and covariance of the noise error in the derivative estimation.
• Apply the algebraic parametric estimation framework to express the derivative estimator as a weighted integral of the noisy signal over a time window.
• Use Legendre and Jacobi polynomial bases to construct causal and anti-causal estimators with tunable parameters for bias and noise control.
• Validate the theoretical error bounds through numerical simulations with synthetic signals and white Gaussian noise.

Experimental results

Research questions

• RQ1 How does extending the parameter domain of Jacobi-based derivative estimators from integers to real numbers affect the noise error variance?
• RQ2 Can the bias error due to truncation in the Jacobi series expansion be reduced by tuning the real-valued parameters?
• RQ3 What is the statistical behavior of the noise error (mean, variance, covariance) in the derivative estimator under general noise models?
• RQ4 How do negative real parameters compare to integer parameters in terms of noise error variance and time delay?
• RQ5 What is the optimal compromise between bias, noise error, and time delay achievable through parameter tuning?

Key findings

• The noise error variance is significantly reduced when using negative real parameters (e.g., μ = -0.75, κ = -0.7) compared to integer parameters (μ = κ = 0), with variance dropping from 2.2351 to 0.1855 in simulations.
• Estimators with parameters in ]-1, 0] produce lower total error and better SNR than those with non-negative integer parameters under identical SNR conditions.
• The theoretical time delay of the minimal causal estimator with μ, κ ∈ ]-1, 0] is reduced to 0.3021T compared to 0.3927T for μ = κ = 0, indicating improved real-time performance.
• The affine causal estimator with optimized parameters (e.g., μ = -0.66, κ = -0.7, ξ = 0.234) achieves a noise error variance of 0.0085, significantly lower than the 1.7919 for integer parameters.
• The bias error term due to truncation can be reduced by tuning the real-valued parameters μ and κ, enabling better approximation of the true derivative.
• The proposed method allows for a systematic optimization of the trade-off between bias, noise error, and time delay through parameter selection.