[Paper Review] Supervised Learning for Multi-Block Incomplete Data

Hadrien Lorenzo, Jérôme Saracco+1 more|arXiv (Cornell University)|Jan 14, 2019

Statistical Methods and Inference37 references4 citations

TL;DR

This paper proposes mdd-sPLS, a novel supervised multi-block PLS method that jointly performs variable selection, subspace estimation, and missing data imputation using a new algorithm called Koh-Lanta. It outperforms existing two-step methods like mean imputation, nipals, softImpute, and imputeMFA by achieving the lowest prediction error in high-dimensional, incomplete data settings, especially under high intra- and inter-block correlations.

ABSTRACT

In the supervised high dimensional settings with a large number of variables and a low number of individuals, one objective is to select the relevant variables and thus to reduce the dimension. That subspace selection is often managed with supervised tools. However, some data can be missing, compromising the validity of the sub-space selection. We propose a Partial Least Square (PLS) based method, called Multi-block Data-Driven sparse PLS mdd-sPLS, allowing jointly variable selection and subspace estimation while training and testing missing data imputation through a new algorithm called Koh-Lanta. This method was challenged through simulations against existing methods such as mean imputation, nipals, softImpute and imputeMFA. In the context of supervised analysis of high dimensional data, the proposed method shows the lowest prediction error of the response variables. So far this is the only method combining data imputation and response variable prediction. The superiority of the supervised multi-block mdd-sPLS method increases with the intra-block and inter-block correlations. The application to a real data-set from a rVSV-ZEBOV Ebola vaccine trial revealed interesting and biologically relevant results. The method is implemented in a R-package available on the CRAN and a Python-package available on pypi.

Motivation & Objective

• Address the challenge of high-dimensional data with missing covariates in supervised learning settings.
• Overcome the limitations of two-step approaches that separately impute data and then analyze it.
• Develop a unified method that simultaneously performs variable selection, subspace estimation, and missing data imputation.
• Improve prediction accuracy of response variables in incomplete, multi-block data by leveraging block-wise correlations.
• Provide a computationally efficient and statistically robust solution for real-world biological and biomedical datasets with missing values.

Proposed method

• Proposes mdd-sPLS, a supervised multi-block PLS method that integrates variable selection via L1-penalized optimization.
• Introduces the Koh-Lanta algorithm for joint imputation and prediction, alternating between SVD-based imputation and PLS-based estimation.
• Uses a data-driven approach to iteratively update missing values using low-rank approximations derived from current PLS components.
• Applies sparse PLS with L1-norm regularization to select relevant variables in both predictor and response blocks.
• Employs alternating optimization to solve the non-convex problem by reducing the concave component through a tuning parameter κ.
• Standardizes all data matrices to zero mean and unit variance before analysis to ensure comparability across blocks.

Experimental results

Research questions

• RQ1 Can a unified method jointly perform missing data imputation and supervised prediction in multi-block, high-dimensional data?
• RQ2 How does the performance of mdd-sPLS compare to two-step methods (e.g., impute-then-analyze) in terms of prediction error?
• RQ3 Does the method maintain superior performance under varying levels of intra-block and inter-block correlation?
• RQ4 How does the number of individuals affect the prediction accuracy of mdd-sPLS compared to existing methods?
• RQ5 Is mdd-sPLS robust in low-correlation settings where block-wise structure is weak?

Key findings

• mdd-sPLS achieves the lowest prediction error (RMSEP) across all simulation scenarios, especially under high intra- and inter-block correlations.
• The method significantly outperforms mean imputation, nipals, softImpute, and imputeMFA, with prediction errors consistently lower than competing methods.
• In low-correlation settings (ρd = 0.3, ρt = 0.5), all methods perform similarly with RMSEP ≈ 0.73, except Lasso-based methods which show higher error (≈ 0.83).
• As the number of individuals increases, mdd-sPLS (with Koh-Lanta) shows a marked improvement in prediction accuracy, outperforming even nipals + classic-sPLS.
• mdd-sPLS with Koh-Lanta maintains the lowest error across all conditions, particularly excelling when correlations are strong.
• The method successfully identified 4 relevant covariates (2 per component) in a real rVSV-ZEBOV Ebola vaccine dataset, achieving perfect group assignment in leave-one-out cross-validation.