[Paper Review] Predictive support recovery with TV-Elastic Net penalty and logistic regression: an application to structural MRI

Mathieu Dubois, Fouad Hadj‐Selem+5 more|arXiv (Cornell University)|Jul 21, 2014

Medical Image Segmentation Techniques10 references4 citations

TL;DR

This paper proposes a novel optimization framework for logistic regression with combined $ε_{1}$, $ε_{2}$, and Total Variation (TV) penalties to improve spatial coherence and support recovery in neuroimaging. By using Nesterov's smoothing technique to handle the non-smooth TV penalty while preserving exact $ε_{1}$ sparsity, the method enables accelerated proximal gradient optimization, demonstrating superior support recovery of predictive brain regions in structural MRI despite no significant gain in prediction accuracy.

ABSTRACT

The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited relevance. A penalty like Total Variation (TV) that exploits the natural 3D structure of the images can increase the spatial coherence of the weight map. However, TV penalization leads to non-smooth optimization problems that are hard to minimize. We propose an optimization framework that minimizes any combination of l1, l2, and TV penalties while preserving the exact l1 penalty. This algorithm uses Nesterov's smoothing technique to approximate the TV penalty with a smooth function such that the loss and the penalties are minimized with an exact accelerated proximal gradient algorithm. We propose an original continuation algorithm that uses successively smaller values of the smoothing parameter to reach a prescribed precision while achieving the best possible convergence rate. This algorithm can be used with other losses or penalties. The algorithm is applied on a classification problem on the ADNI dataset. We observe that the TV penalty does not necessarily improve the prediction but provides a major breakthrough in terms of support recovery of the predictive brain regions.

Motivation & Objective

• To address the poor interpretability of predictive weight maps in neuroimaging due to irregular ($\ell_{2}$) or scattered ($\ell_{1}$) patterns.
• To incorporate spatial structure via Total Variation (TV) penalty to enhance spatial coherence of predictive regions in brain images.
• To develop an efficient optimization algorithm that handles non-smooth TV penalty while preserving exact $\ell_{1}$ sparsity for improved biomarker detection.
• To evaluate the method on the ADNI dataset for classifying Alzheimer’s disease status from structural MRI.
• To assess whether TV regularization improves support recovery of biologically relevant brain regions despite potential trade-offs in prediction accuracy.

Proposed method

• The method formulates a logistic regression problem with a combined $\ell_{1}$, $\ell_{2}$, and TV penalty to balance sparsity, regularization, and spatial smoothness.
• It applies Nesterov’s smoothing technique to approximate the non-smooth TV penalty with a smooth function, enabling gradient-based optimization.
• The smoothed problem is minimized using an exact accelerated proximal gradient algorithm (FISTA-style), ensuring fast convergence.
• A continuation algorithm is introduced, successively decreasing the smoothing parameter $\mu$ to achieve high precision while maintaining optimal convergence rates.
• The algorithm preserves the exact $\ell_{1}$ proximal operator, ensuring true sparsity, while numerically approximating the $\mathrm{TV}$ proximal operator via smoothing.
• The framework is general and can be extended to other differentiable losses and non-smooth penalties expressible as $p$-norms of linear transformations.

Experimental results

Research questions

• RQ1 Does incorporating Total Variation (TV) regularization improve the spatial coherence and interpretability of predictive weight maps in structural MRI?
• RQ2 Can a smooth approximation of the TV penalty be effectively combined with exact $\ell_{1}$ and $\ell_{2}$ penalties in logistic regression for neuroimaging?
• RQ3 Does the proposed optimization framework achieve high-precision support recovery of known neuroanatomical regions associated with Alzheimer’s disease?
• RQ4 Is there a trade-off between prediction accuracy and support recovery when using TV regularization?
• RQ5 Can the continuation algorithm with adaptive smoothing parameters achieve both fast convergence and high precision in non-smooth optimization?

Key findings

• The $\ell_{2}+\mathrm{TV}$ penalty produces a smooth, spatially coherent weight map that aligns with known Alzheimer’s-related brain regions, such as the hippocampus and temporal lobe atrophy.
• The $\ell_{1}+\ell_{2}+\mathrm{TV}$ combination achieves better support recovery than $\ell_{1}+\ell_{2}$, identifying key regions like the hippocampus while excluding less relevant areas.
• The $\ell_{1}+\mathrm{TV}$ and $\ell_{1}+\ell_{2}+\mathrm{TV}$ methods produced nearly identical weight maps, suggesting that $\ell_{1}$ acts as a selector on the $\ell_{2}+\mathrm{TV}$ pattern.
• Prediction accuracy did not significantly improve with $\mathrm{TV}$; the $\ell_{2}+\mathrm{TV}$ method had a balanced classification rate (BCR) of 0.784, comparable to $\ell_{2}$ (0.855) and $\ell_{1}+\ell_{2}$ (0.772).
• McNemar’s test showed no significant difference in prediction performance between $\ell_{2}+\mathrm{TV}$ and $\ell_{2}$ (p = 0.16), but $\ell_{1}+\mathrm{TV}$ significantly outperformed $\ell_{1}$ (p = 2e-4).
• The algorithm achieved convergence in approximately one hour on a standard workstation, performing around 13,000 FISTA iterations.