[Paper Review] A new approach to optimal designs for correlated observations

Holger Dette, Maria Konstantinou+1 more|arXiv (Cornell University)|Jan 1, 2015

Optimal Experimental Design Methods13 references2 citations

TL;DR

This paper proposes a novel continuous-time approach to constructing optimal designs and efficient estimators for linear regression models with correlated errors. By leveraging stochastic calculus and the Doob-Meyer decomposition, it derives the best linear unbiased estimator (BLUE) as a stochastic integral, enabling discrete approximations that are practically indistinguishable from weighted least squares estimators and their optimal designs—offering high efficiency and ease of implementation without non-convex optimization.

ABSTRACT

This paper presents a new and effcient method for the construction of optimal designs for regression models with dependent error processes. In contrast to most of the work in this field, which starts with a model for a finite number of observations and considers the asymptotic properties of estimators and designs as the sample size converges to infinity, our approach is based on a continuous time model. We use results from stochastic anal- ysis to identify the best linear unbiased estimator (BLUE) in this model. Based on the BLUE, we construct an efficient linear estimator and corresponding optimal designs in the model for finite sample size by minimizing the mean squared error between the opti- mal solution in the continuous time model and its discrete approximation with respect to the weights (of the linear estimator) and the optimal design points, in particular in the multi-parameter case. In contrast to previous work on the subject the resulting estimators and corresponding optimal designs are very efficient and easy to implement. This means that they are practi- cally not distinguishable from the weighted least squares estimator and the corresponding optimal designs, which have to be found numerically by non-convex discrete optimization. The advantages of the new approach are illustrated in several numerical examples.

Motivation & Objective

• To address the challenge of constructing optimal designs for regression models with dependent errors, which typically lead to non-convex optimization problems.
• To overcome the limitations of asymptotic and discrete-time approaches that require complex, non-convex numerical optimization for weighted least squares estimators.
• To develop a method that yields estimators and designs that are practically indistinguishable from the optimal weighted least squares estimator and its corresponding design.
• To provide a computationally efficient and implementable alternative to existing methods, especially in multi-parameter models.
• To establish a rigorous continuous-time framework based on stochastic integration and absolute continuity of measures to derive the BLUE.

Proposed method

• The method is based on a continuous-time linear regression model with correlated errors, using the Doob-Meyer decomposition and absolute continuity of measures on C([a,b]) to derive the best linear unbiased estimator (BLUE).
• The BLUE is represented as a stochastic integral ∫₀ᵇ ˙f(t) dYt, which serves as the continuous-time optimal solution.
• For finite samples, the method minimizes the mean squared error between the continuous-time BLUE and its discrete approximation, optimizing both weights μi and design points ti.
• The approach uses Ito’s formula and measure-theoretic tools to derive the covariance structure and prove optimality of the resulting estimator.
• In models with an intercept, the method separates the constant term and applies the BLUE derivation to the remaining components via a transformed model.
• The resulting estimators are matrix-weighted linear combinations of observations, with design points and weights chosen to minimize approximation error to the continuous-time solution.

Experimental results

Research questions

• RQ1 Can a continuous-time stochastic model yield optimal estimators and designs that are practically equivalent to the weighted least squares estimator and its optimal design in finite samples?
• RQ2 How can the best linear unbiased estimator (BLUE) be represented and computed in a continuous-time model with correlated errors?
• RQ3 What is the relationship between the continuous-time BLUE and its discrete approximation in terms of mean squared error and design efficiency?
• RQ4 Can the proposed method achieve high efficiency in multi-parameter models without requiring non-convex discrete optimization?
• RQ5 How does the inclusion of an intercept term affect the derivation and implementation of the optimal estimator and design?

Key findings

• The proposed estimator is asymptotically equivalent to the weighted least squares estimator and achieves the same precision as the best linear unbiased estimator (BLUE) derived from the full trajectory in continuous time.
• The method produces estimators and designs that are practically indistinguishable from the optimal weighted least squares estimator and its corresponding optimal design, even in multi-parameter models.
• The resulting estimators are easy to implement and do not require solving complex non-convex discrete optimization problems, unlike traditional approaches.
• For models with f(0) = 0, the BLUE simplifies to ˆθBLUE = M⁻¹₀ ∫₀ᵇ ˙f(t) dYt, with Var(ˆθBLUE) = M⁻¹₀, which is analytically tractable.
• In models with an intercept (1 ∈ span{f₁,…,fₘ}), the method separates the constant term and derives the BLUE for the remaining parameters via ∫₀ᵇ ˙˜f(t) dYt, with covariance matrix ˜M⁻¹₀.
• The method achieves high efficiency even with a uniform design, particularly when the error process is integrated or has smooth sample paths.