[Paper Review] A brief note on the Karhunen-Loève expansion

Alen Alexanderian|arXiv (Cornell University)|Sep 24, 2015

Mathematical Dynamics and FractalsMathematics8 references21 citations

TL;DR

This paper provides a rigorous, self-contained derivation of the Karhunen-Lofleve (KL) expansion for centered, mean-square continuous stochastic processes using spectral theory of compact operators and Mercer's theorem. It establishes the convergence of the KL expansion and illustrates its application through a numerical example of a Gaussian random field with exponential correlation, showing that higher-order modes significantly influence realizations and that low-order truncations may yield artificially smooth samples.

ABSTRACT

We provide a detailed derivation of the Karhunen-Loève expansion of a stochastic process. We also discuss briefly Gaussian processes, and provide a simple numerical study for the purpose of illustration.

Motivation & Objective

• To provide a clear, self-contained theoretical foundation for the Karhunen-Lofleve (KL) expansion of stochastic processes.
• To bridge functional analysis—specifically spectral theory of compact and Hilbert-Schmidt operators—with stochastic process theory.
• To demonstrate the convergence of the KL expansion using the spectral theorem and Mercer’s theorem.
• To illustrate the practical implications of truncating the KL expansion through a numerical study of a Gaussian random field with exponential correlation.
• To highlight the risk of premature truncation in uncertainty quantification, where higher-order modes can significantly affect system outputs.

Proposed method

• Derive the KL expansion via the spectral theorem for compact self-adjoint operators acting on $ L^2(D) $.
• Apply Mercer’s theorem to represent the autocorrelation kernel as a series of eigenfunctions and eigenvalues.
• Define the KL expansion as $ X(t,\omega) = \sum_{i=1}^{\infty} \sqrt{\lambda_i} \xi_i(\omega) e_i(t) $, where $ \xi_i $ are uncorrelated standard normal variables for Gaussian processes.
• Use the Hilbert-Schmidt integral operator $ Ku(x) = \int_D k(x,y)u(y)dy $ with kernel $ k(s,t) = R_X(s,t) $ to compute eigenfunctions and eigenvalues.
• Numerically compute eigenfunctions and eigenvalues for the exponential correlation kernel $ R_X(s,t) = \sigma^2 \exp(-|s-t|/L_c) $ on $[0,1] \times [0,1]$.
• Simulate realizations of the random field using truncated KL expansions with increasing $ N $, and compare with the full series and standard normal distribution.

Experimental results

Research questions

• RQ1 How can the Karhunen-Lofleve expansion be rigorously derived using functional analysis and spectral theory?
• RQ2 What conditions ensure the convergence of the KL expansion in $ L^2(\Omega) $ for mean-square continuous stochastic processes?
• RQ3 How do the eigenfunctions and eigenvalues of the autocorrelation kernel relate to the structure of the stochastic process?
• RQ4 What are the practical consequences of truncating the KL expansion too early in uncertainty quantification applications?
• RQ5 How do higher-order KL modes affect the sample path smoothness and statistical properties of simulated random fields?

Key findings

• The KL expansion converges in $ L^2(\Omega) $ for centered, mean-square continuous stochastic processes, with convergence proven via spectral theory of compact operators.
• For Gaussian processes, the coefficients $ \xi_i $ in the KL expansion are independent standard normal random variables, enabling efficient sampling.
• The eigenfunctions and eigenvalues of the exponential correlation kernel $ R_X(s,t) = \exp(-|s-t|) $ on $[0,1]^2$ can be computed analytically, with the first few modes capturing most of the correlation structure.
• With $ N=6 $, the truncated KL approximation of the autocorrelation function achieves an absolute error bounded by $ 8 \times 10^{-2} $.
• Low-order KL truncations ($ N=6 $) produce artificially smooth realizations, and increasing $ N $ reveals higher-order oscillatory modes that significantly alter sample paths.
• Higher-order modes have a non-negligible impact on system outputs, indicating that premature truncation in uncertainty propagation can lead to misleading results.