[Paper Review] Self-scaled bounds for atomic cone ranks: applications to nonnegative rank and cp-rank

Hamza Fawzi, Pablo A. Parrilo|DSpace@MIT (Massachusetts Institute of Technology)|Apr 11, 2014

Mathematical Inequalities and Applications3 citations

TL;DR

This paper introduces a novel self-scaled atomic norm framework to derive tighter lower bounds for nonnegative rank and cp-rank using semidefinite programming with sum-of-squares relaxations. The method yields bounds that are invariant under scaling, improve upon existing norm-based bounds, and inherit structural properties like subadditivity and diagonal scaling invariance.

ABSTRACT

The nonnegative rank of a matrix A is the smallest integer r such that A can be written as the sum of r rank-one nonnegative matrices. The nonnegative rank has received a lot of attention recently due to its application in optimization, probability and communication complexity. In this paper we study a class of atomic rank functions defined on a convex cone which generalize several notions of "positive" ranks such as nonnegative rank or cp-rank (for completely positive matrices). The main contribution of the paper is a new method to obtain lower bounds for such ranks which improve on previously known bounds. Additionally the bounds we propose can be computed by semidefinite programming. The idea of the lower bound relies on an atomic norm approach where the atoms are self-scaled according to the vector (or matrix, in the case of nonnegative rank) of interest. This results in a lower bound that is invariant under scaling and that is at least as good as other existing norm-based bounds. We mainly focus our attention on the two important cases of nonnegative rank and cp-rank where our bounds satisfying interesting properties: For the nonnegative rank we show that our lower bound can be interpreted as a non-combinatorial version of the fractional rectangle cover number, while the sum-of-squares relaxation is closely related to the Lovász theta number of the rectangle graph of the matrix. We also prove that the lower bound inherits many of the structural properties satisfied by the nonnegative rank such as invariance under diagonal scaling, subadditivity, etc. We also apply our method to obtain lower bounds on the cp-rank for completely positive matrices. In this case we prove that our lower bound is always greater than or equal the plain rank lower bound, and we show that it has interesting connections with combinatorial lower bounds based on edge-clique cover number.

Motivation & Objective

• To develop tighter, scalable lower bounds for atomic rank functions such as nonnegative rank and cp-rank.
• To address the challenge of computing effective lower bounds for nonnegative rank, a key quantity in optimization, communication complexity, and statistics.
• To unify and generalize existing norm-based and combinatorial bounds through a self-scaled atomic norm framework.
• To ensure the bounds are computable via semidefinite programming using sum-of-squares relaxations.
• To establish theoretical properties such as invariance under diagonal scaling and subadditivity for the new lower bounds.

Proposed method

• Propose a self-scaled atomic norm approach where atoms are scaled according to the input matrix or vector, ensuring invariance under scaling transformations.
• Define a lower bound function τ₊(A) based on the atomic norm, which is minimized over convex combinations of rank-one nonnegative matrices.
• Use sum-of-squares (SOS) relaxations to transform the non-convex atomic rank minimization into a tractable semidefinite program.
• Derive an SDP relaxation τ₊^sos(A) that captures the self-scaled bound via a block-structured positive semidefinite matrix constraint.
• Leverage the structure of block-diagonal matrices to prove subadditivity and additivity of the bound under direct sum operations.
• Connect the SOS relaxation to known combinatorial bounds such as the Lovász ϑ-number and fractional rectangle cover number via graph-theoretic interpretations.

Experimental results

Research questions

• RQ1 Can a self-scaled atomic norm framework yield tighter and more invariant lower bounds for nonnegative rank than existing norm-based methods?
• RQ2 How does the proposed bound relate to combinatorial lower bounds such as the fractional rectangle cover number and edge-clique cover number?
• RQ3 To what extent do the new bounds inherit structural properties like subadditivity and invariance under diagonal scaling?
• RQ4 Can the sum-of-squares relaxation of the atomic norm be efficiently computed and shown to be equivalent to known SDP relaxations?
• RQ5 What is the relationship between the new bound and the Lovász ϑ-number in the context of the rectangle graph of a matrix?

Key findings

• The proposed self-scaled bound τ₊(A) is always at least as large as other norm-based lower bounds for nonnegative rank.
• The bound τ₊(A) is invariant under diagonal scaling of the matrix A, preserving its value when rows and columns are scaled by positive weights.
• The sum-of-squares relaxation τ₊^sos(A) is shown to be equivalent to the Lovász ϑ-number of the rectangle graph of A, linking it to a well-known combinatorial bound.
• For the nonnegative rank, the bound τ₊(A) can be interpreted as a non-combinatorial variant of the fractional rectangle cover number.
• The bound τ₊(A) is subadditive: τ₊(A ⊕ B) = τ₊(A) + τ₊(B), and the same holds for the SOS relaxation τ₊^sos.
• For cp-rank, the bound τ₊(A) is always greater than or equal to the plain rank and has connections to the edge-clique cover number of the matrix’s support graph.