[Paper Review] On recovering missing values in a pathwise setting

Nikolai Dokuchaev|arXiv (Cornell University)|Apr 18, 2016

Sparse and Compressive Sensing Techniques12 references3 citations

TL;DR

This paper proposes a frequency-domain criterion for error-free recovery of missing values in a single discrete-time sequence using only its intrinsic pathwise properties, without probabilistic assumptions. It establishes recoverability for sequences whose Z-transform vanishes at a point with arbitrary decay rate, derives explicit recovering kernels, and demonstrates robustness to noise contamination.

ABSTRACT

The paper suggests a frequency criterion of error-free recoverability of a missing value for sequences, i.e., discrete time processes, in a pathwise setting, without using probabilistic assumptions on the ensemble. This setting targets situations where we deal with a sole sequence that is deemed to be unique and such that we cannot rely on statistics collected from other similar samples. A missing value has to be recovered using the intrinsic properties of this sole sequence. In the paper, error-free recoverability is established for classes of sequences with Z-transform vanishing at a point with a rate that can be chosen arbitrarily. The corresponding recovering kernels are obtained explicitly. Some robustness with respect to noise contamination is established for the suggested recovering algorithm.

Motivation & Objective

• Address the challenge of recovering missing values in a single, unique time series where ensemble statistics are unavailable.
• Develop a deterministic framework for recoverability without relying on probabilistic models or multiple samples.
• Establish conditions under which a missing value can be recovered exactly using only the sequence's intrinsic structure.
• Derive explicit recovering kernels applicable to sequences with specific Z-transform decay properties.
• Investigate the robustness of the recovery algorithm under noise contamination in the observed data.

Proposed method

• Formulate a frequency-domain criterion based on the Z-transform of the sequence, requiring it to vanish at a specific point with a controllable decay rate.
• Construct explicit recovering kernels derived from the Z-transform properties, enabling direct reconstruction of missing values.
• Operate in a pathwise setting, avoiding ensemble averaging or probabilistic assumptions about the data source.
• Use the structure of the Z-transform to ensure that the recovery process is consistent and stable under small perturbations.
• Analyze the sensitivity of the recovery to noise by establishing bounds on error propagation under contamination.
• Apply the method to classes of sequences where the Z-transform vanishes at a point with arbitrary polynomial or exponential decay.

Experimental results

Research questions

• RQ1 Under what conditions can a missing value in a single time series be recovered exactly without probabilistic assumptions?
• RQ2 How can the recovery be achieved using only the intrinsic properties of the observed sequence?
• RQ3 What role does the Z-transform's behavior at a specific point play in ensuring recoverability?
• RQ4 How does the proposed method perform when the observed data is contaminated by noise?
• RQ5 Can the recovery kernel be explicitly constructed for sequences with controlled Z-transform decay?

Key findings

• A missing value can be recovered without error if the Z-transform of the sequence vanishes at a specific point with a decay rate that can be arbitrarily chosen.
• Explicit recovering kernels are derived based on the Z-transform structure, enabling direct computation of missing values.
• The method is robust to noise, with bounded error propagation under small contamination of the observed data.
• The framework operates in a pathwise setting, making it suitable for unique sequences where ensemble statistics are inapplicable.
• The recoverability criterion is formulated purely in terms of spectral properties, avoiding reliance on probabilistic models.
• The approach applies to a broad class of sequences characterized by controlled Z-transform decay at a point.