[Paper Review] Graphical Elastic Net and Target Matrices: Fast Algorithms and Software for Sparse Precision Matrix Estimation

Solt Kovács, Tobias Ruckstuhl+2 more|arXiv (Cornell University)|Jan 6, 2021

Statistical Methods and Inference44 references4 citations

TL;DR

This paper proposes a novel Graphical Elastic Net approach for sparse precision matrix estimation in high-dimensional Gaussian graphical models, combining L1 and L2 penalties with flexible diagonal target matrices. It introduces fast, efficient algorithms (gelnet and dpgelnet) implemented in the GLassoElnetFast R package, significantly improving computational speed and methodological flexibility over existing methods, especially with target matrices and varying regularization weights.

ABSTRACT

We consider estimation of undirected Gaussian graphical models and inverse covariances in high-dimensional scenarios by penalizing the corresponding precision matrix. While single $L_1$ (Graphical Lasso) and $L_2$ (Graphical Ridge) penalties for the precision matrix have already been studied, we propose the combination of both, yielding an Elastic Net type penalty. We enable additional flexibility by allowing to include diagonal target matrices for the precision matrix. We generalize existing algorithms for the Graphical Lasso and provide corresponding software with an efficient implementation to facilitate usage for practitioners. Our software borrows computationally favorable parts from a number of existing packages for the Graphical Lasso, leading to an overall fast(er) implementation and at the same time yielding also much more methodological flexibility.

Motivation & Objective

• To improve precision matrix estimation in high-dimensional settings where traditional methods fail due to singularity or poor performance.
• To extend the Graphical Lasso by incorporating an Elastic Net penalty (L1 + L2) for enhanced stability, particularly under high correlation among variables.
• To enable inclusion of diagonal target matrices to encode prior knowledge or structural constraints in the estimation process.
• To develop computationally efficient algorithms that outperform existing implementations in speed and scalability.
• To provide a unified, user-friendly R software package (GLassoElnetFast) integrating multiple regularization and target-based approaches.

Proposed method

• Proposes a penalized Gaussian log-likelihood optimization with a combined L1 and L2 penalty on the precision matrix, parameterized by a mixing parameter α ∈ [0,1].
• Introduces two algorithms: gelnet (working with the inverse precision matrix) and dpgelnet (working with the covariance matrix), both using block coordinate descent.
• Incorporates diagonal target matrices T to shrink the estimated precision matrix toward a known structure, improving estimation under high correlation or low sample size.
• Employs a warm-start strategy and efficient updates using Cholesky decomposition and thresholding to accelerate convergence.
• Borrows computational optimizations from existing glasso and glassoFast packages to enhance speed while maintaining numerical stability.
• Implements the full pipeline in the GLassoElnetFast R package using R and Fortran for high-performance computation.

Experimental results

Research questions

• RQ1 Can combining L1 and L2 penalties in precision matrix estimation improve stability and accuracy, especially under high correlation among variables?
• RQ2 How does incorporating a diagonal target matrix affect the performance and interpretability of sparse precision matrix estimation?
• RQ3 Can the proposed algorithms achieve faster computation times than existing Graphical Lasso and Iterative Ridge implementations?
• RQ4 Does the Graphical Elastic Net with target matrices outperform standard Graphical Lasso in terms of estimation accuracy and sparsity recovery?
• RQ5 To what extent does the choice of α (mixing parameter) influence the trade-off between sparsity and stability in high-dimensional settings?

Key findings

• The GLassoElnetFast package significantly improves computational speed, outperforming both standard glasso and the Iterative Ridge method by orders of magnitude when using target matrices.
• The inclusion of an L2 penalty (via Elastic Net) enhances stability, particularly in high-correlation scenarios, reducing estimation variance.
• Target matrices, especially when combined with Elastic Net, lead to more accurate and interpretable precision matrix estimates, especially when prior structural knowledge is available.
• The gelnet and dpgelnet algorithms are computationally competitive with glassoFast, with only a slight increase in time for Elastic Net compared to pure L1-penalized estimation.
• The software implementation supports a full range of α ∈ [0,1], enabling flexible exploration of the regularization path and improved model selection.
• Empirical results show that not penalizing the diagonal elements of the precision matrix leads to better estimation performance, consistent with theoretical findings.