Newton’s Method for Non-Linear System of Equations

Presented by: Rahul Choudhary

Roll Number: 2303141013

Course Instructor: Dr. Vijay Kumar Sohani
Department of Mathematics
Indian Institute of Technology, Indore
Simrol, Indore 453 552

November 12,2024

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 1 / 25

Outline

1 Introduction

2 Newton’s Method for Nonlinear System of Equations

3 Example

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 2 / 25

Outline

1 Introduction

2 Newton’s Method for Nonlinear System of Equations

3 Example

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 3 / 25

Introduction

A system of n non-linear equations having n unknowns can be given by



f1 (x1 , x2 , . . . , xn ) = 0,







f (x , x , . . . , x ) = 0,

2 1 2 n
(1)
..
.








fn (x1 , x2 , . . . , xn ) = 0,


where each function fi can be thought of as mapping a vector

x = (x1 , x2 , . . . , xn ) of the n−dimensional space Rn into the real line R.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 4 / 25
This system of n nonlinear equations in n unknown can also be represented

by defining a function F mapping from Rn into Rn as

F = (f1 (x1 , . . . , xn ), . . . , fn (x1 , . . . , xn ))

If vector notation is used to represent the variable x1 , x2 , . . . , xn , then

system (1) assumes the form

F(x) = 0. (2)

The functions f1 , f2 , . . . , fn are called the coordinate functions of F.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 5 / 25

Fixed Points in Rn

Definition
A function G from D ⊂ Rn into Rn has a fixed point at p ∈ D if

G(p) = p.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 6 / 25

Outline

1 Introduction

2 Newton’s Method for Nonlinear System of Equations

3 Example

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 7 / 25

Newton’s Method for Nonlinear System of Equations

To construct the algorithm that led to an appropriate fixed point method in

one dimension case, we found a function ϕ with the property that

g(x) = x − ϕ(x)f (x)

gives a quadratic convergence to the fixed point p of the function g. From

1
this condition Newton’s method evolved by choosing ϕ(x) = f ′ (x) ,

assuming that f ′ (x) ̸= 0.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 8 / 25

Newton’s Method for Nonlinear System of Equations

A similar approach in the n−dimensional case involves a matrix

 
 a11 (x) a12 (x) · · · a1n (x) 
 
 a21 (x) a22 (x) · · · a2n (x) 
 
A((x)) =  .

.. .. 
 (3)
 .. ..
 . . .  
 
an1 (x) an2 (x) · · · ann (x)

where each of the entries aij (x) is a function from Rn into R.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 9 / 25

This requires that A(x) be found so that

G(x) = x − A(x)−1 F(x)

gives quadratic convergence to the solution of F(x) = 0, assuming that

A(x) is nonsingular at fixed point p of G.

Now, we will give a theorem that will help us to express G in terms of its

Taylor series in n variables about p.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 10 / 25

Newton’s Method for Nonlinear System of Equations

Theorem
Let p be a solution of G(x) = x. Suppose a number δ > 0 exists with
∂gi
1 ∂xj is continuous on Nδ = {x | ∥x − p∥ < δ}, for each i = 1, 2, . . . , n

and j = 1, 2, . . . , n;

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 11 / 25

Newton’s Method for Nonlinear System of Equations

Theorem
Let p be a solution of G(x) = x. Suppose a number δ > 0 exists with
∂gi
1 ∂xj is continuous on Nδ = {x | ∥x − p∥ < δ}, for each i = 1, 2, . . . , n

and j = 1, 2, . . . , n;
∂ 2 gi (x) ∂ 2 gi (x)
2 (∂xj ∂xk ) is continuous, and | (∂xj ∂xk ) |≤ M for some constant M,

whenever x ∈ Nδ , for each i = 1, 2, . . . , n, j = 1, 2, . . . , n, and

k = 1, 2, . . . , n.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 11 / 25

Newton’s Method for Nonlinear System of Equations

Theorem
Let p be a solution of G(x) = x. Suppose a number δ > 0 exists with
∂gi
1 ∂xj is continuous on Nδ = {x | ∥x − p∥ < δ}, for each i = 1, 2, . . . , n

and j = 1, 2, . . . , n;
∂ 2 gi (x) ∂ 2 gi (x)
2 (∂xj ∂xk ) is continuous, and | (∂xj ∂xk ) |≤ M for some constant M,

whenever x ∈ Nδ , for each i = 1, 2, . . . , n, j = 1, 2, . . . , n, and

k = 1, 2, . . . , n.
∂gi (p)
3 ∂xk = 0, for each i = 1, 2, . . . , n and k = 1, 2, . . . , n.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 11 / 25

Newton’s Method for Nonlinear System of Equations

Theorem
Let p be a solution of G(x) = x. Suppose a number δ > 0 exists with
∂gi
1 ∂xj is continuous on Nδ = {x | ∥x − p∥ < δ}, for each i = 1, 2, . . . , n

and j = 1, 2, . . . , n;
∂ 2 gi (x) ∂ 2 gi (x)
2 (∂xj ∂xk ) is continuous, and | (∂xj ∂xk ) |≤ M for some constant M,

whenever x ∈ Nδ , for each i = 1, 2, . . . , n, j = 1, 2, . . . , n, and

k = 1, 2, . . . , n.
∂gi (p)
3 ∂xk = 0, for each i = 1, 2, . . . , n and k = 1, 2, . . . , n.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 11 / 25

Theorem cntd.
Then a number δ̂ ≤ δ exists such that the sequence generated by

xk = G(xk−1 ) converges quadratically to p for any choice to p for any

choice of x(0) , provided that ∥x(0) − p∥ ≤ δ̂. Moreover,

n2 M
∥x(k) − p∥∞ ≤ 2 ∥x
(k−1) − p∥2∞ , for each k ≥ 1.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 12 / 25

Newton’s Method for Nonlinear System of Equations

To apply the above theorem, suppose that A(x) is an n × n matrix of

functions from Rn into Rn in the form of Eq. (3), where the specific

entries will be chosen later. Moreover, that A(x) is non singular near a

solution p of F(x) = 0, and let bij x denote the entry of A(x)−1 in the ith

row and jth column.

For G(x) = x − A(x)−1 F(x), we have

n
X
gi (x) = xi − bij (x)fj (x).
j=1

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 13 / 25

Newton’s Method for Nonlinear System of Equations


 
∂bij
1 − nj=1 bij (x) ∂x
∂fi
 P
∂gi

k
(x) + ∂xk (x)fj (x) if i = k,
(x) =
∂xk 
∂bij

− n ∂fi
 P
j=1 bij (x) ∂xk (x) + ∂xk (x)fj (x) if i ̸= k.


∂gi
Above theorem implies that we need ∂xk = 0, for each i = 1, 2, . . . , n and

k = 1, 2, . . . , n. This means for i = k,

n
X ∂fj
0=1− bij (p) (p),
xi
j=1

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 14 / 25

Newton’s Method for Nonlinear System of Equations

that is,
n
X ∂fj
bij (p) (p) = 1. (4)
xi
j=1

When k ̸= i,
n
X ∂fj
− bij (p) (p) = 0,
xi
j=1

n
X ∂fj
bij (p) (p) = 0. (5)
xi
j=1

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 15 / 25

Jacobian Matrix

Define the matrix J(x) by

 
∂f1 ∂f1 ∂f1
 ∂x1 (x) ∂x2 (x) ··· ∂xn (x)
 
 ∂f2 ∂f2 ∂f2
 ∂x1 (x) ∂x2 (x) ··· ∂xn (x)

J(x) = 
 .. .. ..

. (6)
..
 .
 . . . 

 
∂fn ∂fn ∂fn
∂x1 (x) ∂x2 (x) ··· ∂xn (x)

Then conditions (4) and (5) require that A(p)−1 J(p) = I, the identity

matrix, so A(p) = J(p).

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 16 / 25

Jacobian Matrix

An appropriate choice for A(x) is, consequently, A(x) = J(x) since this
satisfies condition (3) in the above theorem. The function G is defined by

G(x) = x − J(x)−1 F(x),

and the functional iteration procedure evolves from selecting x(0) and
generating, for k ≥ 1.

x(k) = G(x(k−1) ) = x(k−1) − J(x(k−1) )−1 F(x(k−1) ).

This is called Newton’s Method for nonlinear systems, and it is

generally expected to give quadratic convergence, provided that a
sufficiently accurate starting value known is known and that J(p)−1 exists.
This matrix J(x) is called the Jacobian matrix.

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 17 / 25

Outline

1 Introduction

2 Newton’s Method for Nonlinear System of Equations

3 Example

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 18 / 25

Example

Consider the nonlinear system

1
3x1 − cos(x2 x3 ) − = 0,
2

x21 − 81(x2 + 0.1)2 + sinx3 + 1.06 = 0,

10π − 3
e−x1 x2 + 20x3 + = 0.
3

Apply Newton’s method to this problem with x(0) = (0.1, 0.1, −0.1)t .

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 19 / 25

Solution

Define

F(x1 , x2 , x3 ) = (f1 (x1 , x2 , x3 ), f2 (x1 , x2 , x3 ), f3 (x1 , x2 , x3 ))t ,

where
1
f1 (x1 , x2 , x3 ) = 3x1 − cos(x2 x3 ) − ,
2

f2 (x1 , x2 , x3 ) = x21 − 81(x2 + 0.1)2 + sinx3 + 1.06,

10π − 3
f3 (x1 , x2 , x3 ) = e−x1 x2 + 20x3 + .
3

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 20 / 25

Solution

The Jacobian matrix J(x) for this system is

 
 3 x3 sinx2 x3x2 sinx2 x3 
 
J(x1 , x2 , x3 ) = 
 2x1 −162(x2 + 0.1) cosx3  
 
−x2 e−x1 x2 −x1 e−x1 x2 20

Let x(0) = (0.1, 0.1, −0.1)t . Then

F(x(0) ) = (−0.199995, −2.2698334147, 8.462025346)t and

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 21 / 25

Solution

 
 3 9.999833334 × 10−4 9.999833334 × 10−4 
(0)
 
J(x =
 0.2 −32.4 0.9950041653 

 
−0.09900498337 −0.09900498337 20

(0) (0)
Solving 
the linear system, J(x
 )y = −F(x(0) ) gives 
 0.3998996728   0.499869782 
(0)
  (1) (0) (0)
 
y = −0.08053315147 and x = x + y =  0.01946684853 
  
   
−04215204718 −0.5215204718
Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 22 / 25
Solution

Continuing for k = 2, 3, . . . , we have

 (k)   (k−1)   (k−1) 
x1 x1 y1
 (k)   (k−1)   (k−1) 
x2  = x2  + y2 ,
(k) (k−1) (k−1)
x3 x3 y3

where
 (k−1) 
y1
 (k−1)  (k−1) (k−1) (k−1) −1 (k−1) (k−1) (k−1)
y2  = −(J(x1 , x2 , x3 )) F(x1 , x2 , x3 ).
(k−1)
y3

Thus at the kth step, the linear system J(x(k−1) )y(k−1) = −F(x(k−1) )
must be solved, where
Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 23 / 25
Solution

J(x (k−1) ) =
 
(k−1) (k−1) (k−1) (k−1) (k−1) (k−1)
 3 x3 sinx2 x3 x2 sinx2 x3 
 (k−1) (k−1) (k−1)

 2x1 −162(x2 + 0.1) cosx3 ,
 
 
(k−1) −x(k−1) x(k−1) (k−1) −x(k−1) x(k−1)
−x2 e 1 2 −x1 e 1 2 20
 
(k−1)
y1 
(k−1)
 (k−1) 
y = y2
  and

 
(k−1)
y3

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 24 / 25

Solution
 
(k−1) (k−1) (k−1)
 3x1 − cos(x2 x3 ) − 21 
(k−1)
 (k−1) 2 
(k−1)
2
(k−1)

F(x )=  x
 1 − 81 x2 + 0.1 + sinx3 + 1.06

 
(k−1) (k−1) (k−1)
e−x1 x2 + 20x3 + 10π−3
3
The results using this iterative procedure is shown in the following table

Rahul Choudhary Numerical Analysis(MA 653) November 12,2024 25 / 25