[Paper Review] Complexity aspects of local minima and related notions

Amir Ali Ahmadi, Jeffrey Zhang|arXiv (Cornell University)|Aug 14, 2020

Polynomial and algebraic computation26 references4 citations

TL;DR

This paper investigates the computational complexity of identifying critical points, second-order points, local minima, and strict local minima in multivariate polynomials, with a focus on cubic polynomials. It presents an efficiently checkable condition for local minimality and shows that local minima of cubic polynomials can be found via semidefinite programs of linear size in the number of variables, while deciding the existence of critical points in cubics is strongly NP-hard.

ABSTRACT

We consider the notions of (i) critical points, (ii) second-order points, (iii) local minima, and (iv) strict local minima for multivariate polynomials. For each type of point, and as a function of the degree of the polynomial, we study the complexity of deciding (1) if a given point is of that type, and (2) if a polynomial has a point of that type. Our results characterize the complexity of these two questions for all degrees left open by prior literature. Our main contributions reveal that many of these questions turn out to be tractable for cubic polynomials. In particular, we present an efficiently-checkable necessary and sufficient condition for local minimality of a point for a cubic polynomial. We also show that a local minimum of a cubic polynomial can be efficiently found by solving semidefinite programs of size linear in the number of variables. By contrast, we show that it is strongly NP-hard to decide if a cubic polynomial has a critical point. We also prove that the set of second-order points of any cubic polynomial is a spectrahedron, and conversely that any spectrahedron is the projection of the set of second-order points of a cubic polynomial. In our final section, we briefly present a potential application of finding local minima of cubic polynomials to the design of a third-order Newton method.

Motivation & Objective

• To characterize the computational complexity of determining whether a given point is a critical point, second-order point, local minimum, or strict local minimum in multivariate polynomials.
• To study the complexity of deciding whether a polynomial has any point of a given type (e.g., existence of a local minimum).
• To identify tractable cases, particularly for cubic polynomials, where efficient algorithms are possible despite general intractability.
• To explore algorithmic applications, such as designing a third-order Newton method using local minima of cubic approximations.
• To establish connections between polynomial optimization and spectrahedra, showing that second-order points of cubic polynomials form spectrahedra and vice versa.

Proposed method

• Uses the Turing model of computation, with input size measured by bit-length of rational coefficients and point coordinates.
• Applies semidefinite programming (SDP) to efficiently find local minima of cubic polynomials, with SDP size linear in the number of variables.
• Develops an efficiently checkable necessary and sufficient condition for a point to be a local minimum of a cubic polynomial.
• Proves that the set of second-order points of any cubic polynomial is a spectrahedron, and conversely, that any spectrahedron arises as the projection of such a set.
• Analyzes the third-order Taylor approximation of functions to design a potential third-order Newton method that moves to local minima of the cubic approximation.
• Employs numerical experiments and comparisons (e.g., with classical Newton’s method) to demonstrate convergence advantages of the proposed third-order method.

Experimental results

Research questions

• RQ1 What is the computational complexity of verifying whether a given point is a local minimum of a multivariate polynomial?
• RQ2 Can the existence of a local minimum in a polynomial be decided efficiently, and if so, for which degrees?
• RQ3 Is there a tractable characterization of local minimality for cubic polynomials?
• RQ4 What is the complexity of deciding whether a cubic polynomial has a critical point?
• RQ5 Can local minima of cubic polynomials be efficiently computed using convex optimization techniques like semidefinite programming?

Key findings

• For cubic polynomials, there exists an efficiently checkable necessary and sufficient condition for a point to be a local minimum.
• Local minima of cubic polynomials can be found by solving semidefinite programs of size linear in the number of variables.
• Deciding whether a cubic polynomial has a critical point is strongly NP-hard.
• The set of second-order points of any cubic polynomial is a spectrahedron, and every spectrahedron arises as the projection of such a set.
• Numerical experiments show that a third-order Newton method, which moves to local minima of the third-order Taylor approximation, exhibits global convergence and reduced sensitivity to initial conditions compared to classical Newton’s method.
• In a univariate test case, the third-order method converges globally to the minimum, while classical Newton’s method oscillates for initial points outside a bounded interval.