[Paper Review] Kantorovich's Theorem on Newton's Method

O. P. Ferreira, B. F. Svaiter|arXiv (Cornell University)|Sep 25, 2012

Iterative Methods for Nonlinear Equations10 references5 citations

TL;DR

This paper presents a simplified proof of Kantorovich's Theorem on Newton's method using a novel technique based on scalar majorant functions. By comparing the nonlinear operator $ F $ to a quadratic scalar function $ f(t) = \frac{L}{2}t^2 - t + b $, the authors define invariant regions where Newton iterations remain well-defined and converge Q-linearly to a unique solution in a Banach space setting, with quadratic convergence under stricter conditions.

ABSTRACT

In this work we present a simplifyed proof of Kantorovich's Theorem on Newton's Method. This analysis uses a technique which has already been used for obtaining new extensions of this theorem.

Motivation & Objective

• To provide a simplified proof of Kantorovich's Theorem on Newton's method using a new analytical technique.
• To establish the existence and uniqueness of a solution to $ F(x) = 0 $ under semi-local conditions in Banach spaces.
• To demonstrate convergence of Newton's iterates to a unique zero within a well-defined ball around the initial point.
• To extend the classical result by showing Q-linear and, under stricter conditions, Q-quadratic convergence rates.

Proposed method

• Define a scalar majorant function $ f(t) = \frac{L}{2}t^2 - t + b $ that bounds the nonlinear operator $ F $ in norm.
• Construct a sequence $ \{t_k\} $ via Newton iteration on $ f $, which serves as a majorant for the Newton iterates $ \{x_k\} $ of $ F $.
• Prove that if $ x_k \in B[x_0, t_k] $, then $ x_{k+1} = x_k - F'(x_k)^{-1}F(x_k) \in B[x_0, t_{k+1}] $, ensuring invariance of the region.
• Use the inequality $ \|x_* - x_{k+1}\| \leq \frac{t_* - t_{k+1}}{(t_* - t_k)^2} \|x_* - x_k\|^2 $ to derive convergence rates.
• Establish that $ t_k \to t_* $ as $ k \to \infty $, with $ t_* = \frac{1 - \sqrt{1 - 2bL}}{L} $, and $ t_{**} = \frac{1 + \sqrt{1 - 2bL}}{L} $, defining the convergence radius.
• Use the ratio $ \theta = t_*/t_{**} < 1 $ to derive the Q-quadratic convergence bound when $ 2bL < 1 $.

Experimental results

Research questions

• RQ1 Can a simplified proof of Kantorovich’s Theorem on Newton’s method be constructed using a scalar majorant function approach?
• RQ2 What conditions ensure that Newton iterates remain within a well-defined invariant region around the initial point?
• RQ3 How does the convergence rate of Newton’s method depend on the parameters $ b $ and $ L $ in the semi-local setting?
• RQ4 Under what conditions is the solution to $ F(x) = 0 $ unique in the ball $ B[x_0, t_*] $?
• RQ5 What is the precise quantitative bound on the convergence rate when $ 2bL < 1 $?

Key findings

• Newton’s iterates $ \{x_k\} $ are well-defined and remain within $ B[x_0, t_*] $ under the assumptions of Theorem 1.
• The solution $ x_* $ is the unique zero of $ F $ in $ B[x_0, t_*] $, and $ \|x_* - x_{k+1}\| \leq \frac{1}{2}\|x_* - x_k\| $, indicating Q-linear convergence.
• When $ 2bL < 1 $, the convergence becomes Q-quadratic: $ \|x_* - x_{k+1}\| \leq \frac{1 - \theta^{2^k}}{1 + \theta^{2^k}} \cdot \frac{L}{2\sqrt{1 - 2bL}} \|x_* - x_k\|^2 $, with $ \theta = t_*/t_{**} < 1 $.
• The solution $ x_* $ is also the unique zero of $ F $ in $ B[x_0, \rho] $ for any $ \rho \in [t_*, t_{**}) $, provided $ B[x_0, \rho] \subset C $.
• A closed-form expression for $ t_k $ is derived: $ t_k = t_* - \frac{\theta^{2^k}}{1 - \theta^{2^k}} \cdot \frac{2\sqrt{1 - 2bL}}{L} $, with $ \theta = t_*/t_{**} $.
• The proof technique using scalar majorants provides a unified framework that can be extended to generalize Kantorovich’s Theorem beyond the classical setting.