[Paper Review] Linear regression over the max-plus semiring: algorithms and applications

James Hook|arXiv (Cornell University)|Dec 10, 2017

Control Systems and Identification14 references3 citations

TL;DR

This paper introduces a novel framework for max-plus $2$-norm regression using classical optimization techniques, enabling maximum likelihood estimation for three inverse problems: identifying max-plus linear dynamical systems from noisy time series, inferring network edge lengths from shortest path data, and fitting max-plus polynomial functions to data. Despite the non-convex and non-smooth nature of the $2$-norm residual, a modified Newton’s method with undershooting achieves practical convergence to high-quality solutions.

ABSTRACT

In this paper we present theory, algorithms and applications for regression over the max- plus semiring. We show how max-plus 2-norm regression can be used to obtain maximum likelihood estimates for three different inverse problems. Namely inferring a max-plus linear dynamical systems model from a noisy time series recording, inferring the edge lengths of a network from shortest path information and fitting a max-plus polynomial function to data.

Motivation & Objective

• To develop a theoretically grounded and computationally feasible approach to max-plus $2$-norm regression, which is non-convex and non-smooth, unlike the well-understood $∞$-norm case.
• To apply max-plus $2$-norm regression to three inverse problems: system identification in max-plus linear dynamical systems, shortest path-based network edge length inference, and max-plus polynomial fitting.
• To bridge max-plus algebra with classical optimization by reformulating max-plus matrix-vector multiplication in terms of classical linear algebra, enabling use of gradient-based methods.
• To demonstrate that heuristic optimization methods, despite lacking global convergence guarantees, yield high-quality solutions in practical inverse problem settings.
• To explore the theoretical and algorithmic challenges of $2$-norm regression in the max-plus setting, particularly the NP-hardness of local optimality verification, and to suggest directions for future work.

Proposed method

• Reformulate the max-plus matrix-vector multiplication $A \otimes \bm{x}$ as a piecewise-linear, non-smooth map in classical linear algebra, enabling application of standard optimization techniques.
• Apply steepest descent and Newton’s method with undershooting to minimize the non-convex, non-smooth $2$-norm residual $\|A \otimes \bm{x} - \bm{y}\|_2$.
• Transform max-plus polynomial fitting into a linear regression problem via a change of variables, enabling maximum likelihood estimation using $2$-norm regression.
• Use min-plus algebra to model shortest path networks and infer edge lengths from observed shortest path times, framing the problem as a min-plus linear system.
• Utilize low-rank approximations and model order reduction in min-plus networks to extract structural insights, such as neighborhood patterns in social networks.
• Leverage the duality between max-plus and min-plus algebra to generalize the framework to different inverse problems, including network inference and system identification.

Experimental results

Research questions

• RQ1 Can $2$-norm regression over the max-plus semiring be effectively solved despite its non-convex and non-smooth nature?
• RQ2 How can max-plus $2$-norm regression be applied to estimate parameters of a max-plus linear dynamical system from noisy time series data?
• RQ3 Can the problem of inferring edge lengths in a network from shortest path measurements be formulated as a max-plus or min-plus regression problem?
• RQ4 To what extent can max-plus polynomial regression be used to fit data, and how does it compare to classical polynomial fitting in terms of accuracy and interpretability?
• RQ5 What are the algorithmic and theoretical challenges in ensuring convergence or optimality in max-plus $2$-norm regression, and how can they be mitigated in practice?

Key findings

• The max-plus $2$-norm regression problem is non-convex and non-smooth, and determining whether a point is a local minimum is an NP-hard problem.
• A modified Newton’s method with undershooting successfully converges to high-quality approximate solutions in practice, even without global convergence guarantees.
• For max-plus linear dynamical system identification, the method yields maximum likelihood estimates from noisy time series, with a residual of $1.8247$ in Example 3.2 and $1.7041$ in Example 5.3.
• In polynomial fitting, the method achieves a residual of $1.7041$ for a degree-4 max-plus polynomial, demonstrating effective data approximation.
• The min-plus network inference framework successfully recovers edge lengths from shortest path data, with model order reduction techniques revealing structural patterns in real-world social networks.
• Theoretical results, such as Theorem 6.2, highlight the inherent difficulty of local optimality verification, but suggest that structured problems may still allow for efficient optimization.