[Paper Review] Inexact Newton Method for M-Tensor Equations

Donghui Li, Hong-Bo Guan+1 more|arXiv (Cornell University)|Jul 27, 2020

Tensor decomposition and applications21 references4 citations

TL;DR

This paper proposes an inexact Newton method for solving M-tensor equations, leveraging a dimension-reduction technique to transform the original problem into a lower-dimensional one when the constant term is nonnegative. The method achieves global convergence with quadratic rate and demonstrates superior numerical performance compared to existing methods like QCA, as validated through extensive numerical experiments on various tensor sizes and orders.

ABSTRACT

We first investigate properties of M-tensor equations. In particular, we show that if the constant term of the equation is nonnegative, then finding a nonnegative solution of the equation can be done by finding a positive solution of a lower dimensional M-tensor equation. We then propose an inexact Newton method to find a positive solution to the lower dimensional equation and establish its global convergence. We also show that the convergence rate of the method is quadratic. At last, we do numerical experiments to test the proposed Newton method. The results show that the proposed Newton method has a very good numerical performance.

Motivation & Objective

• To investigate structural properties of M-tensor equations, particularly the existence and computation of nonnegative solutions.
• To develop an inexact Newton method for solving lower-dimensional M-tensor equations arising from dimension reduction.
• To establish global convergence and quadratic convergence rate for the proposed inexact Newton method.
• To numerically validate the efficiency and robustness of the method through comprehensive experiments on tensors of varying order and size.
• To compare the proposed method with the QCA method from [13] in terms of iteration count and CPU time.

Proposed method

• The method reduces the original M-tensor equation to a lower-dimensional problem when the constant term b is nonnegative, enabling efficient solution of the positive solution.
• An inexact Newton iteration is applied to solve the reduced equation, where the Newton step is computed approximately with a relative error tolerance.
• The algorithm ensures global convergence by enforcing a safeguarded Armijo line search with parameters σ = 0.1 and ρ = 0.5.
• A regularized Newton variant is introduced to improve stability, especially when b has zero components, using a regularization parameter and modified line search.
• The method exploits the M-tensor structure to ensure that the solution remains positive throughout iterations.
• Numerical comparisons are conducted using 100 random tensor and vector pairs per (m,n) size, with b drawn from uniform distribution in (0,1) or modified to include zero entries.

Experimental results

Research questions

• RQ1 Can the solution of a nonnegative M-tensor equation be reduced to solving a lower-dimensional M-tensor equation when b ≥ 0?
• RQ2 Does the inexact Newton method for the reduced problem achieve global convergence and quadratic convergence rate?
• RQ3 How does the performance of the inexact Newton method compare to the QCA method in terms of iteration count and CPU time?
• RQ4 Can the regularized Newton method improve numerical stability when b has zero components?
• RQ5 What is the scalability of the proposed method across different tensor orders (m) and dimensions (n)?

Key findings

• The inexact Newton method achieves global convergence and quadratic convergence rate for solving the reduced M-tensor equation.
• For all tested problems, the inexact Newton method required on average 90% or fewer iterations than the QCA method, with TR values below 90% across all sizes.
• In the case of b ∈ ℝ₊ⁿ, the inexact Newton method reduced CPU time by up to 88.2% (for (3,500)) and iteration count by up to 96.3% (for (5,30)) compared to QCA.
• The regularized Newton method outperformed QCA in both iteration count and CPU time, with TR values ranging from 60% to 99.6% and IR from 49.4% to 95.7%.
• For problems with zero components in b, the regularized Newton method showed significant improvement, with TR values as low as 14.2% and IR as low as 8.3% in some cases.
• The QCA method, though not theoretically guaranteed for b ∈ ℝ₊ⁿ, still successfully found solutions in all tested cases, but was consistently outperformed by the proposed methods.