[Paper Review] Longitudinal LASSO: Jointly Learning Features and Temporal Contingency for Outcome Prediction

Tingyang Xu, Jiangwen Sun+1 more|arXiv (Cornell University)|Oct 23, 2016

Tensor decomposition and applications17 references6 citations

TL;DR

This paper proposes Longitudinal LASSO, a novel method that jointly learns relevant features and temporal lags influencing an outcome in longitudinal data by decomposing model parameters into feature-selection and temporal-contingency components, both regularized via block-wise LASSO. The approach models within-individual correlations and uses accelerated gradient descent to achieve global convergence, outperforming GEE, RE-EM trees, and Granger modeling on synthetic and real-world datasets including NLSY.

ABSTRACT

Longitudinal analysis is important in many disciplines, such as the study of behavioral transitions in social science. Only very recently, feature selection has drawn adequate attention in the context of longitudinal modeling. Standard techniques, such as generalized estimating equations, have been modified to select features by imposing sparsity-inducing regularizers. However, they do not explicitly model how a dependent variable relies on features measured at proximal time points. Recent graphical Granger modeling can select features in lagged time points but ignores the temporal correlations within an individual's repeated measurements. We propose an approach to automatically and simultaneously determine both the relevant features and the relevant temporal points that impact the current outcome of the dependent variable. Meanwhile, the proposed model takes into account the non-{\em i.i.d} nature of the data by estimating the within-individual correlations. This approach decomposes model parameters into a summation of two components and imposes separate block-wise LASSO penalties to each component when building a linear model in terms of the past $τ$ measurements of features. One component is used to select features whereas the other is used to select temporal contingent points. An accelerated gradient descent algorithm is developed to efficiently solve the related optimization problem with detailed convergence analysis and asymptotic analysis. Computational results on both synthetic and real world problems demonstrate the superior performance of the proposed approach over existing techniques.

Motivation & Objective

• To address the gap in existing longitudinal models that fail to jointly identify relevant features and their temporal dependencies in outcome prediction.
• To model within-individual correlations in repeated longitudinal measurements, which are often ignored or oversimplified in standard methods.
• To develop a unified framework that simultaneously selects influential covariates and their relevant lagged time points for predicting current outcomes.
• To improve predictive accuracy over standard penalized GEE and graphical Granger methods by explicitly modeling temporal contingency and correlation structure.
• To provide a consistent, convergent optimization algorithm for solving the joint feature and temporal selection problem.

Proposed method

• The model decomposes the coefficient matrix into two components: one for feature selection and one for temporal contingency, each subject to separate block-wise LASSO penalties.
• The method uses a linear model based on the past τ measurements of features, with parameters estimated via an accelerated gradient descent algorithm.
• It incorporates a working correlation structure (e.g., AR(1), exchangeable, tri-diagonal) to account for within-subject correlation, improving model robustness.
• The optimization problem is solved using an accelerated gradient method with proven global convergence and quadratic rate.
• The approach estimates both the feature selection matrix (U) and temporal contingency matrix (V), with a third matrix (W) modeling residual correlations.
• The formulation allows for consistent estimation of model parameters under asymptotic analysis, ensuring statistical reliability.

Experimental results

Research questions

• RQ1 Which features and which past time points are most predictive of the current outcome in longitudinal data?
• RQ2 How can we jointly identify relevant features and their temporal dependencies while accounting for within-individual correlation?
• RQ3 Can a unified regularization framework outperform existing methods like GEE and graphical Granger modeling in longitudinal outcome prediction?
• RQ4 How does modeling within-subject correlation improve prediction accuracy compared to assuming independence or simple correlation structures?
• RQ5 What is the convergence behavior and statistical consistency of the proposed optimization algorithm?

Key findings

• On the NLSY dataset, Longitudinal LASSO (LGL) with a tri-diagonal working correlation structure achieved the best performance, outperforming GEE, RE-EM trees, and Granger modeling in all test scenarios.
• LGL achieved a test nMSE of 0.883617 with 3-year lagged data and tri-diagonal correlation, significantly lower than GEE’s 0.892633 under the same conditions.
• The method selected 12 key features from 26 in the NLSY dataset, including smoking, drug use, education, and family background, which were found to influence binge drinking behavior.
• The gray map of the V matrix revealed that only the past two years’ data were selected for prediction, indicating that older lags were less relevant, validating temporal contingency learning.
• LGL with independent correlation assumption performed worst, demonstrating that modeling within-subject correlation significantly improves predictive performance.
• The accelerated gradient descent algorithm achieved global convergence with a quadratic rate, and asymptotic analysis confirmed the consistency of the estimated parameters.