[Paper Review] Optimization over Nonnegative and Convex Polynomials With and Without Semidefinite Programming

Georgina Hall|arXiv (Cornell University)|Jun 19, 2018

Advanced Optimization Algorithms Research136 references22 citations

TL;DR

This paper presents novel, SDP-free methods for optimizing over nonnegative and convex polynomials by replacing semidefinite programming with sequences of linear or second-order cone programs. It introduces the first theoretical framework for converging lower bounds on polynomial optimization problems using only polynomial multiplication and coefficient nonnegativity checks, achieving significant computational speedups while maintaining theoretical convergence.

ABSTRACT

The problem of optimizing over the cone of nonnegative polynomials is a fundamental problem in computational mathematics, with applications to polynomial optimization, control, machine learning, game theory, and combinatorics, among others. A number of breakthrough papers in the early 2000s showed that this problem, long thought to be out of reach, could be tackled by using sum of squares programming. This technique however has proved to be expensive for large-scale problems, as it involves solving large semidefinite programs (SDPs). In the first part of this thesis, we present two methods for approximately solving large-scale sum of squares programs that dispense altogether with semidefinite programming and only involve solving a sequence of linear or second order cone programs generated in an adaptive fashion. We then focus on the problem of finding tight lower bounds on polynomial optimization problems (POPs), a fundamental task in this area that is most commonly handled through the use of SDP-based sum of squares hierarchies (e.g., due to Lasserre and Parrilo). In contrast to previous approaches, we provide the first theoretical framework for constructing converging hierarchies of lower bounds on POPs whose computation simply requires the ability to multiply certain fixed polynomials together and to check nonnegativity of the coefficients of their product. In the second part of this thesis, we focus on the theory and applications of the problem of optimizing over convex polynomials, a subcase of the problem of optimizing over nonnegative polynomials. (See manuscript for the rest of the abstract.)

Motivation & Objective

• To address the computational bottleneck of semidefinite programming (SDP) in sum-of-squares optimization for large-scale polynomial problems.
• To develop a framework for constructing converging hierarchies of lower bounds on polynomial optimization problems without relying on SDP.
• To study the complexity of convexity testing for cubic polynomials and its implications for optimization.
• To apply the proposed methods to real-world problems in robotics, machine learning, and econometrics with monotonicity and convexity constraints.

Proposed method

• Proposes two adaptive, SDP-free algorithms that replace large-scale SDPs with sequences of linear or second-order cone programs.
• Develops a novel hierarchy of lower bounds for polynomial optimization using only polynomial multiplication and coefficient nonnegativity checks.
• Introduces a theoretical framework for constructing converging lower bounds based on fixed polynomial products and coefficient sign analysis.
• Proves that testing convexity of a cubic polynomial over a box is NP-hard, establishing a complexity boundary for convexity detection.
• Extends Reznick's sum-of-squares representation to positive definite biforms for constructing SDP hierarchies over norms generated by convex forms.
• Applies the framework to robotics (minimum-volume polynomial sublevel sets for 3D point clouds) and machine learning (monotone and DC regression).

Experimental results

Research questions

• RQ1 Can we construct converging lower bounds for polynomial optimization problems without solving semidefinite programs?
• RQ2 What is the computational complexity of testing convexity of a cubic polynomial over a box?
• RQ3 Can we replace sum-of-squares programming with simpler, non-SDP solvers while preserving convergence and accuracy?
• RQ4 How can we efficiently optimize over convex polynomials and norms generated by convex forms?
• RQ5 Can the proposed methods be practically applied to real-world problems such as monotone regression and motion planning?

Key findings

• The proposed SDP-free algorithms achieve better runtime performance than CAP and Fast CAP, with a degree-2 hybrid regression taking only 0.24 seconds compared to 12.8 seconds for CAP.
• The best-performing monotonically constrained polynomial regression achieved an average test RMSE of 4.09 with standard error 0.20 on a personal loan dataset.
• For weekly wage prediction, the hybrid constrained degree-2 polynomial achieved a test RMSE of 250.0 ± 39.2, significantly outperforming CAP (385.7 ± 20.8).
• Testing convexity of a cubic polynomial over a box is proven to be NP-hard, establishing a minimal degree boundary for this complexity.
• The framework enables constructing converging lower bounds using only polynomial multiplication and coefficient nonnegativity, bypassing SDP entirely.
• The method successfully reformulates difference of convex (DC) programs, enabling faster solution of some of the best-known algorithms in machine learning.