[Paper Review] Sampling and Remote Estimation for the Ornstein-Uhlenbeck Process through Queues: Age of Information and Beyond

Tasmeen Zaman Ornee, Yin Sun|arXiv (Cornell University)|Feb 10, 2019

Age of Information Optimization56 references4 citations

TL;DR

This paper studies optimal sampling policies for estimating an Ornstein-Uhlenbeck (OU) process via a queueing system, showing that the optimal policy is a threshold policy on instantaneous estimation error when causal signal knowledge is available. When no such knowledge exists, the optimal policy minimizes a nonlinear age of information metric and is a threshold policy on expected estimation error, both computable via low-complexity algorithms like bisection and Newton’s method.

ABSTRACT

The age of information, as a metric for evaluating information freshness, has received a lot of attention. Recently, an interesting connection between the age of information and remote estimation error was found in a sampling problem of Wiener processes: If the sampler has no knowledge of the signal being sampled, the optimal sampling strategy is to minimize the age of information; however, by exploiting causal knowledge of the signal values, it is possible to achieve a smaller estimation error. In this paper, we extend a previous study by investigating a problem of sampling a stationary Gauss-Markov process, namely the Ornstein-Uhlenbeck (OU) process. The optimal sampling problem is formulated as a constrained continuous-time Markov decision process (MDP) with an uncountable state space. We provide an exact solution to this MDP: The optimal sampling policy is a threshold policy on instantaneous estimation error and the threshold is found. Further, if the sampler has no knowledge of the OU process, the optimal sampling problem reduces to an MDP for minimizing a nonlinear age of information metric. The age-optimal sampling policy is a threshold policy on expected estimation error and the threshold is found. These results hold for (i) general service time distributions of the queueing server and (ii) sampling problems both with and without a sampling rate constraint. Numerical results are provided to compare different sampling policies.

Motivation & Objective

• To address the gap in optimal sampling strategies for general Gauss-Markov processes beyond Wiener processes.
• To analyze remote estimation of an OU process under queueing constraints with i.i.d. service times and finite mean.
• To derive optimal sampling policies that minimize time-average mean-squared estimation error (MSE) under sampling rate constraints.
• To establish structural insights into optimal sampling policies for more general signal models by analyzing the OU process as a canonical case.
• To demonstrate that the optimal policy is a threshold policy on estimation error, with computable thresholds via low-complexity algorithms.

Proposed method

• Formulates the sampling problem as a continuous-time Markov decision process (MDP) with an uncountable state space.
• Uses Lagrangian duality to transform the constrained MDP into an unconstrained one, introducing a geometric multiplier for the sampling rate constraint.
• Derives the optimal policy as a threshold policy on instantaneous estimation error when causal knowledge of the signal is available.
• For the case with no signal knowledge, reformulates the problem as minimizing a nonlinear age of information metric, leading to a threshold policy on expected estimation error.
• Employs bisection search and Newton’s method to compute the optimal threshold, circumventing the curse of dimensionality.
• Considers general service time distributions (finite mean only), making the results robust to long transmission delays.

Experimental results

Research questions

• RQ1 What is the optimal sampling policy for minimizing estimation error in a remote estimation system with an OU process and queueing delays?
• RQ2 How does causal knowledge of the signal affect the structure and performance of the optimal sampling policy?
• RQ3 Can the optimal sampling policy be computed efficiently despite the uncountable state space and continuous-time dynamics?
• RQ4 What is the relationship between the age of information metric and estimation error in the absence of signal knowledge?
• RQ5 How do general service time distributions impact the optimality and structure of the sampling policy?

Key findings

• The optimal sampling policy is a threshold policy on the instantaneous estimation error when causal knowledge of the OU process is available.
• When no signal knowledge is available, the optimal policy minimizes a nonlinear age of information metric and is a threshold policy on expected estimation error.
• The optimal threshold can be computed via bisection search or Newton’s method, ensuring low computational complexity.
• The duality gap between the constrained and dual problems is zero, confirming the optimality of the derived policies.
• The results hold for general service time distributions with finite mean, enhancing robustness to variable transmission delays.
• Numerical results show that exploiting causal signal knowledge leads to significantly lower estimation error than age-optimal policies.