[Paper Review] The W4 method: a new multi-dimensional root-finding scheme for nonlinear systems of equations

H. Okawa, Kotaro Fujisawa+5 more|arXiv (Cornell University)|Sep 12, 2018

Iterative Methods for Nonlinear Equations9 references3 citations

TL;DR

The W4 method is a novel iterative root-finding algorithm for nonlinear systems of equations that extends the Newton-Raphson (NR) method by incorporating a second-order time derivative term inspired by damped oscillator dynamics. It achieves the same local quadratic convergence as NR but exhibits significantly improved global convergence, successfully finding solutions even when NR fails due to poor initial guesses, as demonstrated in multi-dimensional test cases with divergent or oscillatory NR behavior.

ABSTRACT

We propose a new class of method for solving nonlinear systems of equations, which, among other things,has four nice features: (i) it is inspired by the mathematical property of damped oscillators, (ii) it can be regarded as a simple extention to the Newton-Raphson(NR) method, (iii) it has the same local convergence as the NR method does, (iv) it has a significantly wider convergence region or the global convergence than that of the NR method. In this article, we present the evidence of these properties, applying our new method to some examples and comparing it with the NR method.

Motivation & Objective

• To address the critical limitation of the Newton-Raphson (NR) method—sensitivity to initial guess selection—especially in multi-dimensional nonlinear systems.
• To develop a method that maintains the local quadratic convergence of NR while significantly expanding the convergence region.
• To propose a numerically stable, computationally efficient alternative to NR that avoids divergence or oscillation when initial guesses are far from the solution.
• To demonstrate that the new method can solve systems where NR fails, even when the Jacobian is well-conditioned.
• To establish theoretical and numerical evidence that the W4 method achieves global convergence in practical nonlinear systems.

Proposed method

• The W4 method introduces a second-order time derivative term into the iterative update rule, modeling the system as a damped oscillator with inertia.
• It reformulates the Newton-Raphson iteration by incorporating a momentum-like term, derived from the time evolution of a damped harmonic oscillator with damping coefficient λ.
• The method uses a UDL decomposition of the Jacobian matrix to compute updates efficiently, maintaining computational cost comparable to NR.
• The update rule is expressed as $ \bm{x}_{n+1} = \bm{x}_n - \frac{1}{4} P_n \Lambda_n^{-1} P_n^{-1} \bm{F}(\bm{x}_{n-1}) $, where $ P_n $ and $ \Lambda_n $ are derived from the Jacobian's spectral decomposition.
• The method ensures alignment of the update direction with the negative eigenvector of the Jacobian when degeneracy occurs, enabling escape from local traps.
• The time step $ \Delta\tau $ is fixed at 0.5 in the presented results, and the method is shown to be robust under varying initial conditions.

Experimental results

Research questions

• RQ1 Can a second-order extension of the Newton-Raphson method improve global convergence in nonlinear systems without sacrificing local convergence rate?
• RQ2 How does the inclusion of inertial and damping terms in the iteration process affect convergence behavior when the initial guess is far from the solution?
• RQ3 Does the W4 method successfully escape from divergence or oscillation that plagues the Newton-Raphson and DN methods in multi-dimensional problems?
• RQ4 What is the computational cost of the W4 method relative to Newton-Raphson, and can it be kept within the same order of magnitude?
• RQ5 To what extent does the W4 method eliminate Newton basins that lead to non-convergence in the NR method?

Key findings

• The W4 method achieves the same local quadratic convergence rate as the Newton-Raphson method, as confirmed by theoretical analysis and numerical experiments.
• In problems where the Newton-Raphson method diverges or oscillates due to poor initial guesses, the W4 method consistently converges to a solution, even when the Jacobian is well-conditioned.
• The method successfully escapes from the origin in multi-dimensional test cases where the DN (damped Newton) method becomes trapped, demonstrating enhanced global convergence.
• The W4 method eliminates problematic Newton basins that lead to non-convergence in the NR method, as visualized in Figs. 2, 3, and 4, indicating a broader convergence region.
• The computational cost of the W4 method, using UDL decomposition of the Jacobian, remains comparable to NR, with no significant increase in complexity.
• The method's convergence is linear in the long run, similar to the DN method, but it avoids stagnation by leveraging momentum from previous steps.