[Paper Review] An Expectation-Maximization Algorithm for the Matrix Normal Distribution

Hunter Glanz, Luís Carvalho|arXiv (Cornell University)|Sep 25, 2013

Soil Geostatistics and Mapping4 references10 citations

TL;DR

This paper proposes an Expectation-Maximization (EM) algorithm for parameter estimation in the matrix normal distribution under missing data, leveraging a Kronecker product structure for efficient covariance modeling. The method enables accurate imputation and estimation in high-dimensional spatio-temporal data, validated through simulations and applied to satellite imagery for land-cover classification.

ABSTRACT

Dramatic increases in the size and dimensionality of many recent data sets make crucial the need for sophisticated methods that can exploit inherent structure and handle missing values. In this article we derive an expectation-maximization (EM) algorithm for the matrix normal distribution, a distribution well-suited for naturally structured data such as spatio-temporal data. We review previously established maximum likelihood matrix normal estimates, and then consider the situation involving missing data. We apply our EM method in a simulation study exploring errors across different dimensions and proportions of missing data. We compare these errors and computational running times to those from two alternative methods. Finally, we implement the proposed EM method on a satellite image dataset to investigate land-cover classification separability.

Motivation & Objective

• To address the lack of methods for maximum likelihood estimation of the matrix normal distribution when data are incomplete.
• To develop an EM algorithm that efficiently handles missing data in matrix-variate normal models with Kronecker-structured covariance.
• To evaluate the performance of the proposed EM method in terms of estimation accuracy and computational efficiency across varying missing data proportions and dimensions.
• To demonstrate practical utility through application to real-world satellite image data for land-cover classification.
• To provide a scalable and interpretable framework for modeling structured multivariate data with inherent separability in dimensions (e.g., space and time).

Proposed method

• The EM algorithm iteratively estimates the mean matrix μ, and the row and column covariance matrices Σs and Σc under the matrix normal model.
• In the E-step, the algorithm computes the conditional expectation of the complete-data log-likelihood using the current parameter estimates.
• The algorithm uses matrix identities involving the vec operator and Kronecker product to simplify the Mahalanobis distance expression in the likelihood function.
• Missing values are imputed conditionally using the inverse of the precision matrix (Σc⁻¹ ⊗ Σs⁻¹), with row and column indices of missing entries tracked via indicator matrices.
• The M-step updates the parameters by aggregating contributions across all observations, using masked submatrices to compute sufficient statistics.
• Normalization steps scale the estimated covariance matrices to ensure identifiability and numerical stability.

Experimental results

Research questions

• RQ1 How well does the proposed EM algorithm estimate parameters of the matrix normal distribution under varying proportions of missing data?
• RQ2 How does the estimation accuracy of the EM method compare to unconstrained multivariate normal models in high-dimensional settings?
• RQ3 What is the computational efficiency of the EM algorithm relative to alternative methods when handling missing data in structured matrices?
• RQ4 Can the matrix normal model with EM-based estimation effectively handle real-world spatio-temporal data, such as satellite imagery?
• RQ5 To what extent does the Kronecker covariance structure improve interpretability and estimation performance in structured data compared to full covariance models?

Key findings

• The EM algorithm provides accurate parameter estimates for the matrix normal distribution even with high proportions of missing data, particularly when the Kronecker structure holds.
• Simulation results show that the EM method outperforms a full multivariate normal model in terms of estimation error and computational efficiency under missing data.
• The method maintains stable convergence and produces interpretable covariance estimates that reflect the underlying physical separability of dimensions (e.g., space and time).
• In the satellite image application, the method successfully identified separability between land-cover classes, with results consistent with known spectral characteristics.
• The algorithm demonstrated robustness across different data dimensions and missingness patterns, with computational times scaling reasonably with sample size and missingness.
• The use of masked matrix operations and efficient vectorization techniques enabled scalable implementation on large-scale remote sensing data.