[Paper Review] Group Lasso with Overlaps: the Latent Group Lasso approach

Guillaume Obozinski, Laurent Jacob+1 more|arXiv (Cornell University)|Oct 3, 2011

Statistical Methods and Inference47 references129 citations

TL;DR

This paper introduces the latent group Lasso, a novel group Lasso approach that enables structured sparsity by modeling the parameter vector as a linear combination of latent variables, each supported on predefined overlapping groups. The method ensures that the estimated model's support is a union of these groups, with theoretical guarantees for group-support recovery and improved interpretability in high-dimensional data, as demonstrated on gene expression data with network-structured groups.

ABSTRACT

We study a norm for structured sparsity which leads to sparse linear predictors whose supports are unions of prede ned overlapping groups of variables. We call the obtained formulation latent group Lasso, since it is based on applying the usual group Lasso penalty on a set of latent variables. A detailed analysis of the norm and its properties is presented and we characterize conditions under which the set of groups associated with latent variables are correctly identi ed. We motivate and discuss the delicate choice of weights associated to each group, and illustrate this approach on simulated data and on the problem of breast cancer prognosis from gene expression data.

Motivation & Objective

• To address the limitation of standard group Lasso in handling overlapping groups by introducing a new regularization framework.
• To enable sparse linear models whose supports are unions of predefined overlapping groups, enhancing interpretability in structured data.
• To provide theoretical conditions for consistent group-support recovery under the latent group Lasso penalty.
• To investigate the critical role of group weights in determining recoverable supports and model complexity.
• To empirically validate the method on simulated data and real-world gene expression data for cancer prognosis.

Proposed method

• The latent group Lasso applies the standard group Lasso penalty to a set of latent variables, each associated with a predefined group of covariates.
• The final parameter vector is reconstructed as a linear combination of these latent variables, enforcing sparsity patterns that are unions of the groups.
• The method introduces the concept of 'group-support' to describe the set of non-zero latent variables, which corresponds to the union of groups in the final model.
• A penalty norm is defined as the sum of ℓ₂ norms of the latent variables, with group-specific weights influencing the selection of union supports.
• Theoretical analysis derives sufficient and necessary conditions for consistent group-support recovery, depending on the design matrix and group weights.
• The approach is applied to regression problems, with empirical evaluation on simulated data and a breast cancer gene expression dataset using biological interaction networks as groups.

Experimental results

Research questions

• RQ1 Can a group Lasso formulation be extended to handle overlapping groups such that the resulting sparsity pattern is a union of groups rather than an intersection?
• RQ2 What conditions ensure consistent recovery of the true group-support (i.e., the union of active groups) in the latent group Lasso framework?
• RQ3 How do group weights influence the set of recoverable supports and the complexity of the model class?
• RQ4 Does the latent group Lasso improve predictive performance and interpretability compared to standard ℓ₁ and group Lasso in high-dimensional, structured data?
• RQ5 Can the method reliably identify biologically coherent gene sets in gene expression data when prior knowledge is encoded via overlapping groups?

Key findings

• The latent group Lasso achieves nearly identical prediction accuracy to standard ℓ₁ regularization on a breast cancer prognosis dataset, with a balanced classification error of approximately 0.36 across folds.
• Despite similar predictive performance, the latent group Lasso selects genes in larger, more connected components—averaging 8.6 to 10.2 genes in the largest connected component—compared to only 1.8–2.2 for ℓ₁, indicating enhanced biological coherence.
• The method successfully recovers the union of groups in simulated data, with theoretical conditions for group-support recovery derived and validated.
• The choice of group weights is critical: incorrect weights can prevent recovery of the true underlying group structure, even when the group structure is known.
• In real data, the latent group Lasso produces more interpretable models by favoring clusters of genes in functional networks, without sacrificing prediction accuracy.
• The method outperforms ℓ₁ in connectivity of selected features, suggesting improved potential for identifying biologically meaningful signatures in systems biology applications.