[Paper Review] Convexification of Permutation-Invariant Sets and Applications

Jinhak Kim, Mohit Tawarmalani+1 more|arXiv (Cornell University)|Oct 7, 2019

Sparse and Compressive Sensing Techniques12 references4 citations

TL;DR

This paper introduces convexification techniques for permutation- and sign-invariant sets, enabling tighter relaxations in sparse optimization. It derives new convex envelopes, improves sparse PCA formulations, and closes 95% of the duality gap in exterior product relaxations, generalizing classical results for 0-1 logical constraints.

ABSTRACT

We develop techniques to convexify a set that is invariant under permutation and/or change of sign of variables and discuss applications of these results. First, we convexify the intersection of the unit ball of a permutation and sign-invariant norm with a cardinality constraint. This gives a nonlinear formulation for the feasible set of sparse principal component analysis (Sparse PCA), and an alternative proof of the $K$-support norm. Second, we develop new relaxations for the exterior product of sparse vectors and show numerically that, on several problem instances, our relaxation closes 95% of the gap left by the relaxation in the literature. Third, we derive convex and concave envelopes of various permutation-invariant nonlinear functions and their level-sets over hypercubes, with congruent bounds on all variables. Finally, we study permutation-invariant sets in $0-1$ variables, including those that arise in the formulation of various logical requirements. For these models, we project the convex hull descriptions in the space of original variables, generalizing several classical results.

Motivation & Objective

• To develop convexification techniques for sets invariant under permutation and sign changes of variables.
• To provide tighter convex relaxations for sparse optimization problems, particularly in sparse principal component analysis (Sparse PCA).
• To generalize classical convex hull descriptions for 0-1 logical constraints by projecting in the original variable space.
• To derive convex and concave envelopes of permutation-invariant nonlinear functions over hypercubes with symmetric bounds.

Proposed method

• Use of symmetry properties to convexify the intersection of the unit ball of a permutation- and sign-invariant norm with a cardinality constraint.
• Development of nonlinear convex relaxations for the exterior product of sparse vectors via symmetric structure exploitation.
• Derivation of convex and concave envelopes for permutation-invariant functions over hypercubes using symmetric bounds.
• Projection of convex hull descriptions from lifted spaces back to the original variable space for 0-1 permutation-invariant sets.
• Application of these techniques to re-derive the $K$-support norm via an alternative proof based on convexification.
• Numerical validation of relaxation quality by comparing duality gaps against existing literature.

Experimental results

Research questions

• RQ1 How can permutation- and sign-invariant sets be convexified to improve relaxations in sparse optimization?
• RQ2 To what extent can convex relaxations of the exterior product of sparse vectors close the duality gap compared to prior methods?
• RQ3 What are the tightest convex and concave envelopes for permutation-invariant nonlinear functions over symmetric hypercubes?
• RQ4 How can convex hulls of 0-1 permutation-invariant sets be projected back to the original variable space to generalize classical results?

Key findings

• The proposed relaxation for the exterior product of sparse vectors closes 95% of the duality gap left by existing literature on tested problem instances.
• The intersection of the unit ball of a permutation- and sign-invariant norm with a cardinality constraint yields a nonlinear convex formulation for Sparse PCA.
• An alternative proof of the $K$-support norm is derived through the convexification of symmetric sets.
• Convex and concave envelopes are derived for various permutation-invariant functions over hypercubes with congruent variable bounds.
• The method generalizes classical results by projecting convex hull descriptions into the original variable space for 0-1 logical constraints.
• The framework enables tighter relaxations for a range of symmetric optimization problems, enhancing solution quality in sparse and combinatorial settings.