[Paper Review] Integrative Multi-View Reduced-Rank Regression: Bridging Group-Sparse and Low-Rank Models

Gen Li, Xiaokang Liu+1 more|arXiv (Cornell University)|Jul 26, 2018

Sparse and Compressive Sensing Techniques39 references3 citations

TL;DR

This paper proposes integrative reduced-rank regression (iRRR), a convex method that jointly models multiple high-dimensional predictor views with view-specific low-rank coefficient matrices to enable supervised latent feature extraction. It achieves faster convergence than group-sparse or low-rank methods by bridging group sparsity and low-rank structures via a composite nuclear norm penalty, with theoretical oracle bounds and empirical validation on simulated and aging study data.

ABSTRACT

Multi-view data have been routinely collected in various fields of science and engineering. A general problem is to study the predictive association between multivariate responses and multi-view predictor sets, all of which can be of high dimensionality. It is likely that only a few views are relevant to prediction, and the predictors within each relevant view contribute to the prediction collectively rather than sparsely. We cast this new problem under the familiar multivariate regression framework and propose an integrative reduced-rank regression (iRRR), where each view has its own low-rank coefficient matrix. As such, latent features are extracted from each view in a supervised fashion. For model estimation, we develop a convex composite nuclear norm penalization approach, which admits an efficient algorithm via alternating direction method of multipliers. Extensions to non-Gaussian and incomplete data are discussed. Theoretically, we derive non-asymptotic oracle bounds of iRRR under a restricted eigenvalue condition. Our results recover oracle bounds of several special cases of iRRR including Lasso, group Lasso and nuclear norm penalized regression. Therefore, iRRR seamlessly bridges group-sparse and low-rank methods and can achieve substantially faster convergence rate under realistic settings of multi-view learning. Simulation studies and an application in the Longitudinal Studies of Aging further showcase the efficacy of the proposed methods.

Motivation & Objective

• To address predictive modeling in multi-view data where only a subset of views are relevant and predictors within each view act collectively.
• To develop a unified framework that integrates group-sparse and low-rank modeling for multivariate regression with high-dimensional predictors.
• To enable supervised extraction of latent features from each view through view-specific low-rank coefficient matrices.
• To establish non-asymptotic oracle bounds under a restricted eigenvalue condition for theoretical performance guarantees.
• To extend the method to non-Gaussian and incomplete data settings for broader applicability.

Proposed method

• Proposes an integrative reduced-rank regression (iRRR) model with a distinct low-rank coefficient matrix for each predictor view.
• Uses a convex composite nuclear norm penalty to encourage low-rank structure within each view and group-wise sparsity across views.
• Employs the alternating direction method of multipliers (ADMM) for efficient optimization of the penalized likelihood problem.
• Derives theoretical oracle bounds under a restricted eigenvalue condition, generalizing bounds for Lasso, group Lasso, and nuclear norm regression.
• Extends the framework to non-Gaussian responses and incomplete data using appropriate likelihood and imputation strategies.
• Treats each view's coefficient matrix as a low-rank component, enabling collective contribution of predictors within each view.

Experimental results

Research questions

• RQ1 Can a unified regression framework effectively model multiple high-dimensional predictor views with collective, non-sparse contributions?
• RQ2 How can low-rank and group-sparse structures be simultaneously embedded in a convex optimization framework for multi-view learning?
• RQ3 What theoretical performance guarantees can be established for such a model under realistic high-dimensional conditions?
• RQ4 Does the proposed method achieve faster convergence rates compared to existing group-sparse or low-rank methods?
• RQ5 How well does the method perform on real-world data with non-Gaussian and incomplete observations?

Key findings

• The iRRR method achieves faster convergence rates than Lasso, group Lasso, and nuclear norm penalized regression under realistic multi-view learning settings.
• Theoretical analysis establishes non-asymptotic oracle bounds under a restricted eigenvalue condition, generalizing known bounds for special cases.
• The method successfully recovers oracle bounds of Lasso, group Lasso, and low-rank regression as special cases, confirming its theoretical robustness.
• Simulation studies demonstrate improved prediction accuracy and variable selection performance across diverse multi-view configurations.
• An application to the Longitudinal Study of Aging confirms the method's practical utility in identifying relevant multi-view predictors with collective effects.
• Extensions to non-Gaussian and incomplete data show stable performance, enhancing real-world applicability.