[Paper Review] A retrieval strategy for interactive ensemble data assimilation

Ross N. Hoffman|arXiv (Cornell University)|Sep 8, 2010

Meteorological Phenomena and Simulations13 references5 citations

TL;DR

This paper proposes a retrieval-based data assimilation strategy that uses retrieved atmospheric profiles, their averaging kernels (AK), retrieval error covariance, and prior information to transform observations into unbiased, uncorrelated, and prior-free data for ensemble assimilation. By leveraging the AK and EOFs, the method reduces data volume, eliminates smoothing errors, and enables interactive, modular assimilation with improved vertical interpolation and error characterization.

ABSTRACT

As an alternative to either directly assimilating radiances or the naive use of retrieved profiles (of temperature, humidity, aerosols, and chemical species), a strategy is described that makes use of the so-called averaging kernel (AK) and other information from the retrieval process. This AK approach has the potential to improve the use of remotely sensed observations of the atmosphere. First, we show how to use the AK and the retrieval noise covariance to transform the retrieved quantities into observations that are unbiased and have uncorrelated errors, and to eliminate both the smoothing inherent in the retrieval process and the effect of the prior. Since the effect of the prior is removed, any prior, including the forecast from the data assimilation cycle can be used. Then we show how to transform this result into EOF space, when a truncated EOF series has been used in the retrieval process. This provides a degree of data compression and eliminates those transformed variables that have very small information content. In both approaches a vertical interpolation from the dynamical model coordinate to the radiative transfer coordinate is required. We define an algorithm using the EOF representation to optimize this vertical interpolation

Motivation & Objective

• To address the limitations of direct radiance assimilation, such as high channel count, error correlations, and sensitivity to spectroscopic and surface emissivity biases.
• To develop a modular data assimilation framework that uses retrievals instead of raw radiances, reducing complexity and improving computational efficiency.
• To eliminate the influence of the prior and smoothing errors in retrievals through the use of averaging kernels and proper error covariance transformation.
• To enable interactive assimilation by allowing the prior to be updated dynamically using ensemble forecasts as the background.
• To reduce vertical interpolation errors and data volume by transforming observations into EOF space using truncated representations.

Proposed method

• Transform retrieved profiles into unbiased observations using the averaging kernel (A) and retrieval noise covariance (S_m), ensuring uncorrelated errors and removal of prior influence.
• Apply the Desroziers et al. (2005) bias correction method by computing background radiance (y_b) and simulated observation (y_A) for observation quality control.
• Use EOFs to compress data and reduce dimensionality, transforming the retrieval into a reduced space (α) via the EOF basis (E) and climate mean (x̄).
• Construct a vertical interpolation operator (α) from the EOF representation to minimize interpolation errors between model and retrieval coordinates.
• Implement three pathways to compute A and S_m: from K, S_a, and S_ε; from S_a and Ŝ; or from Ŝ and A, with the latter two reducing data transfer needs.
• Integrate the transformed observations (ŷ_A) and their sensitivity matrix (A_R) into the ensemble Kalman filter (LETKF) framework, using A_R for localization and vertical weighting.

Experimental results

Research questions

• RQ1 Can retrievals with proper error characterization and averaging kernels be used to replace raw radiances in ensemble data assimilation while maintaining accuracy and reducing computational load?
• RQ2 How can the influence of the prior and smoothing errors in retrievals be systematically removed in a data assimilation system?
• RQ3 What is the optimal way to compress retrieval data using EOFs without losing information content or introducing bias?
• RQ4 Can EOF-based vertical interpolation reduce errors in vertical coordinate mapping between models and retrieval grids?
• RQ5 How can the retrieval process be made modular and interactive, allowing the prior to be updated from ensemble forecasts in real time?

Key findings

• The method successfully transforms retrieved profiles into unbiased, uncorrelated observations by using the averaging kernel and retrieval error covariance, eliminating both smoothing and prior biases.
• Using the second and third pathways for computing A and S_m reduces data transfer requirements, as only the retrieved state and its error covariance need to be passed from the retrieval to the assimilation system.
• EOF-based compression reduces data volume and allows for the elimination of low-information-content variables, improving computational efficiency and focusing on physically relevant components.
• The proposed vertical interpolation using EOFs (via α) significantly reduces interpolation errors by aligning the model and retrieval coordinate systems through a physically consistent transformation.
• The implementation enables interactive assimilation where the prior is dynamically updated from the ensemble mean forecast, enhancing adaptability and accuracy.
• The method supports quality control via the Desroziers bias correction technique by enabling computation of background radiance (y_b) and simulated observation (y_A), improving observation error estimation.