[Paper Review] Picasso: A Sparse Learning Library for High Dimensional Data Analysis in R and Python

Jason Ge, Xingguo Li+5 more|arXiv (Cornell University)|Jun 27, 2020

Sparse and Compressive Sensing TechniquesEngineering22 citations

TL;DR

Picasso is a high-performance C++-based sparse learning library implemented in R and Python, offering a unified framework for pathwise coordinate optimization with strong theoretical guarantees. It supports multiple regularizers (ℓ₁, MCP, SCAD) and efficiently solves high-dimensional sparse regression problems, outperforming existing libraries like glmnet and ncvreg in speed and convergence, especially for nonconvex regularized models.

ABSTRACT

We describe a new library named picasso, which implements a unified framework of pathwise coordinate optimization for a variety of sparse learning problems (e.g., sparse linear regression, sparse logistic regression, sparse Poisson regression and scaled sparse linear regression) combined with efficient active set selection strategies. Besides, the library allows users to choose different sparsity-inducing regularizers, including the convex $\ell_1$, nonconvex MCP and SCAD regularizers. The library is coded in C++ and has user-friendly R and Python wrappers. Numerical experiments demonstrate that picasso can scale up to large problems efficiently.

Motivation & Objective

• To develop a unified, efficient, and scalable sparse learning library for high-dimensional data analysis in R and Python.
• To implement pathwise coordinate optimization with theoretical guarantees for both convex and nonconvex regularized problems.
• To support multiple sparsity-inducing regularizers, including ℓ₁, MCP, and SCAD, for diverse sparse learning tasks.
• To improve computational efficiency and convergence stability over existing libraries like glmnet and ncvreg, especially for nonconvex regularizers.
• To provide modular, extensible C++ code with user-friendly R and Python interfaces for broad adoption.

Proposed method

• Employs a three-level nested loop structure: warm start initialization, active set selection with strong rule preselection, and active coordinate minimization.
• Uses pathwise optimization with decreasing regularization parameters, initializing each stage from the previous solution.
• Applies active set selection based on coordinate gradient thresholding (strong rule) to dynamically update the set of non-zero coefficients.
• Implements efficient 'naive update' and 'covariance update' rules to accelerate inner-loop optimization over active coordinates.
• Supports restricted strong convexity and integrates with Eigen3 for high-performance linear algebra operations.
• Provides extensible C++ architecture with virtual function interfaces for custom objective and regularizer functions.

Experimental results

Research questions

• RQ1 Can a unified pathwise coordinate optimization framework achieve superior computational efficiency and convergence for high-dimensional sparse learning problems?
• RQ2 How does the integration of active set selection and strong rules improve performance in nonconvex sparse learning compared to heuristic methods?
• RQ3 To what extent does Picasso outperform established libraries like glmnet and ncvreg in timing and objective value accuracy across various regularizers and data conditions?
• RQ4 Can the library maintain stable convergence and high performance for ill-conditioned and dense estimator scenarios?
• RQ5 How effective is the modular design in enabling extensibility for new objective functions and regularizers?

Key findings

• Picasso achieves significantly faster convergence than ncvreg for nonconvex regularized sparse logistic regression, especially with small tuning parameters, where ncvreg often fails to converge within reasonable time.
• For sparse linear regression with ℓ₁, MCP, and SCAD regularizers, Picasso outperforms glmnet and ncvreg in execution time, with speedups of up to 3× for ℓ₁ and 4× for MCP under well-conditioned settings.
• In ill-conditioned scenarios, Picasso maintains stable performance with minimal degradation, while ncvreg shows substantial slowdowns and convergence failures.
• For scaled sparse linear regression, Picasso reduces computation time by over 90% compared to flare and scalreg on a smaller problem size (n=1000, d=10000).
• The library achieves nearly identical objective values to competitors across all benchmarks, confirming its optimization accuracy despite faster runtime.
• Numerical experiments confirm Picasso’s scalability and robustness across diverse high-dimensional settings, including well- and ill-conditioned data.