[Paper Review] A lasso for hierarchical testing of interactions

Jacob Bien, Noah Simon+1 more|arXiv (Cornell University)|Nov 6, 2012

Animal Disease Management and EpidemiologyAgricultural and Biological Sciences9 references4 citations

TL;DR

This paper proposes a hierarchical lasso-based framework for testing pairwise interactions in high-dimensional two-class problems, ensuring interactions are only considered when their main effects are nonzero. By integrating main effects and interactions through convex optimization, the method improves statistical power and interpretability compared to standard non-hierarchical interaction tests.

ABSTRACT

Weconsiderthetestingofallpairwiseinteractions inatwo-class problemwithmany features. We devise a hierarchical testing framework that only considers an interaction when one or more of its constituent features has a nonzero main effect. It is based on a convex optimization framework that seamlessly considers main effects and interactions together. We provide examples — both real and simulated – that show a potential gain in power and interpretability over a standard (non-hierarchical) interaction test.

Motivation & Objective

• To address the challenge of testing numerous pairwise interactions in high-dimensional two-class classification with many features.
• To improve statistical power and model interpretability by enforcing a hierarchy where interactions are only tested if their main effects are nonzero.
• To develop a unified convex optimization framework that simultaneously models main effects and interactions.
• To demonstrate the superiority of the hierarchical approach over standard non-hierarchical interaction testing in real and simulated data.

Proposed method

• Formulates a convex optimization problem that jointly estimates main effects and interaction terms using a hierarchical lasso penalty.
• Imposes a structural constraint such that an interaction term is only included if at least one of its corresponding main effects is nonzero.
• Uses a regularization path to systematically test interactions only in the context of active main effects.
• Employs a penalized likelihood framework with hierarchical sparsity to ensure interpretability and avoid overfitting.
• Applies the method to both simulated and real datasets to evaluate performance in high-dimensional settings.
• Leverages the convexity of the optimization problem to ensure computational efficiency and global convergence.

Experimental results

Research questions

• RQ1 Can a hierarchical testing framework for interactions improve statistical power compared to standard non-hierarchical interaction tests in high-dimensional two-class problems?
• RQ2 How does enforcing hierarchy between main effects and interactions affect model interpretability and false discovery rates?
• RQ3 To what extent does the proposed convex optimization framework maintain or improve prediction accuracy while ensuring hierarchical structure?
• RQ4 How does the method perform on real-world datasets with complex interaction patterns?

Key findings

• The hierarchical lasso framework achieves higher statistical power in detecting true interactions compared to non-hierarchical methods.
• The method improves interpretability by ensuring that interactions are only considered when their main effects are active.
• In simulated data, the approach reduces false positive interaction detections by enforcing structural hierarchy.
• Real data examples demonstrate that the method identifies biologically or statistically meaningful interactions more consistently than standard approaches.
• The convex optimization framework enables efficient computation and stable estimation even in high-dimensional settings.
• The hierarchical constraint leads to more parsimonious models without sacrificing detection power.