7H Definite Matrix

Definition
A quadratic form is a polynomial with terms all of degree two.
Form is another name for a homogeneous polynomial.

Proposition
Every quadratic form of x1 , x2 , . . . , xn can be uniquely written in a
Xn
concise form aij xi xj = xT Ax with a real symmetric matrix
i,j=1
A = [aij ]n×n and x = [x1 , . . . , xn ]T . Actually,

half of the coefficient of xi xj if i ̸= j,
aij =
the coefficient of xi2 if i = j.

A is called the matrix associated with the quadratic form

Xn n
X
aij xi xj . Conversely, aij xi xj is called the quadratic form
i,j=1 i,j=1
associated with the matrix A. 1 / 19
Example 1
Write the quadratic form into matrix form:

x12 − 2x1 x2 + 2x22 + 4x2 x3 + 3x32 .

Solution:   
1 −1 0 x1
[x1 x2 x3 ]  −1 2 2   x2 
0 2 3 x3

2 / 19
Definite Matrix

Definition
Let A be a real symmetric matrix and its associated quadratic form
be f (x) = xT Ax, where x is a real column vector.
1 A is positive definite if f (x) > 0 for every nonzero x.
2 A is positive semi-definite if f (x) ⩾ 0 for every nonzero x.
3 A is negative definite if f (x) < 0 for every nonzero x.
4 A is negative semi-definite if f (x) ⩽ 0 for every nonzero x.
5 A is indefinite if A is neither positive semi-definite, nor
negative semi-definite.

Pay attention that f (x) = 0 when x=0.

3 / 19
Proposition
1 A is positive definite if and only if all of its eigenvalues are
positive.
2 A is positive semi-definite if and only if all of its eigenvalues
are non-negative.
3 A is negative definite if and only if all of its eigenvalues are
negaitive.
4 A is negative semi-definite if and only if all of its eigenvalues
are non-positive.
5 A is indefinite if and only if it has both positive and negative
eigenvalues.

Proof: Let y = Q T x.
f = xT Ax = xT QDQ T x = yT Dy = λ1 y12 + λ2 y22 + · · · + λn yn2

4 / 19
Example 2
Are the following matrices positive definite?
   
3 −2 0 3 −2 4
A =  −2 3 0  , B =  −2 6 2  .
0 0 5 4 2 3

5 / 19
Proposition
1 A real symmetric matrix is positive definite if and only if all
the following minors are positive.

a11 a12 a13

a11 a12
a11 > 0, > 0, a21 a22 a23 > 0, . . .
a21 a22
a31 a32 a33

2 A real symmetric matrix A is negative definite if and only if

−A is positive definite. Hence

a11 a12 a13

a11 a12
a11 < 0, > 0, a21 a22 a23 < 0, . . .
a21 a22
a31 a32 a33

6 / 19
Example 3
Find whether the following matrices are positive or negative
definite.
   
3 1 1 3 1 1
A= 0 4 2 ,B =  1 4 2 ,
1 1 5 1 2 5
   
1 0 3 −2 1 0
C = 0 1 2 ,D =  1 −2 1  .
3 2 1 0 1 −2

7 / 19
Example 4
Find whether the following matrices are positive or negative
definite.
   
3 1 1 3 1 1
A= 0  4 2 ,B =  1 4 2 ,
1 1 5 1 2 5
   
1 0 3 −2 1 0
C = 0 1 2 ,D =  1 −2 1  .
3 2 1 0 1 −2

Solution: A is not symmetric, thus it is neither positive nor

negative definite.
B is positive definite, as it is symmetric and satisfies

3 1 1
3 1
3 > 0, = 11 > 0, 1 4 2 = 43 > 0.
1 4
1 2 5

8 / 19
C is indefinite, as

1 0 3
1 0
1 > 0, = 1 > 0, 0 1 2 = −12 < 0.
0 1
3 2 1

D is negative definite, as it is symmetric and satisfies

−2 1 0
−2 1
−2 < 0, = 3 > 0, 1 −2 1 = −4 < 0.
1 −2
0 1 −2

9 / 19
§4 Relative Extremum and Optimization

Definition
Let f (x, y ) be a function of two independent variables.
A point (x0 , y0 ) is said to be a relative minimum point of f (x, y )
if f (x, y ) ⩾ f (x0 , y0 ) for all (x, y ) in some neighbourhood of
(x0 , y0 ).
A point (x0 , y0 ) is said to be a relative maximum point of f (x, y )
if f (x, y ) ⩽ f (x0 , y0 ) for all (x, y ) in some neighbourhood of
(x0 , y0 ).
A relative maximum or a relative minimum of a function is known
as a relative extremum of the function.

10 / 19
2nd Order Derivative Test

Definition
A point (x0 , y0 ) satisfying fx (x0 , y0 ) = 0 and fy (x0 , y0 ) = 0 is called
a stationary point of f (x, y ).

Theorem
Let (x0 , y0 ) be
 a stationary point of f (x,  y ), Hessian matrix
fxx (x0 , y0 ) fxy (x0 , y0 )
H(x0 , y0 ) be .
fxy (x0 , y0 ) fyy (x0 , y0 )
1 If H(x , y ) is positive semi-definite, then (x , y ) is a relative
0 0 0 0
minimum point.
2 If H(x0 , y0 ) is negative semi-definite, then (x0 , y0 ) is a relative
maximum point.
3 If H(x0 , y0 ) is indefinite, then (x0 , y0 ) is a saddle point.

11 / 19
Proof: Let ∆x = x − x0 ̸= 0 and ∆y = y − y0 ̸= 0.
According to the 2nd order Taylor series,

∆w =f (x, y ) − f (x0 , y0 )
≈ [fx (x0 , y0 ) ∆x + fy (x0 , y0 ) ∆y ]
1
fxx (x0 , y0 ) ∆x 2 + 2fxy (x0 , y0 ) ∆x∆y + fyy (x0 , y0 ) ∆y 2

+
2
1
= fxx (x0 , y0 ) ∆x 2 + 2fxy (x0 , y0 ) ∆x∆y + fyy (x0 , y0 ) ∆y 2

2   
1 fxx (x0 , y0 ) fxy (x0 , y0 ) ∆x
= [∆x, ∆y ]
2 fxy (x0 , y0 ) fyy (x0 , y0 ) ∆y
 
1 ∆x
= [∆x, ∆y ] H(x0 , y0 ) .
2 ∆y

12 / 19
According to the feature of quadratic form, we have
1 If H(x0 , y0 ) is positive semi-definite, ∆w ⩾ 0 for any (x, y ).
Therefore f (x0 , y0 ) is a relative minimum.
2 If H(x0 , y0 ) is negative semi-definite, ∆w ⩽ 0 for any (x, y ).
Therefore f (x0 , y0 ) is a relative maximum.
3 If H(x0 , y0 ) is indefinite, ∆w > 0 for some (x, y ) and ∆w < 0
for some (x, y ). Therefore (x0 , y0 ) is a saddle point.

13 / 19
Example 5
Find the stationary points of the function w = x 3 + y 3 − 3xy and
determine their nature.
Solution:

wx = 3x 2 − 3y = 0, wy = 3y 2 − 3x = 0

we obtain (0,0) and (1,1) as stationary points of the function.

wxx = 6x, wyy = 6y , wxy = −3

Stationary Point (0,0) (1,1)

|H| = (6x)(6y ) − 9 −9 < 0 36 − 9 > 0
wxx = 6x 6>0
H Indefinite Positive definite
Nature Saddle Minimum
Thus, (0,0) is a saddle point, while (1,1) is a relative minimum
point.
14 / 19
3D case
Definition
A point (x0 , y0 , z0 ) satisfying fx (x0 , y0 , z0 ) = 0, fy (x0 , y0 , z0 ) = 0
and fz (x0 , y0 , z0 ) = 0 is called a stationary point of f (x, y , z).

Theorem
Let (x0 , y0 , z0 ) be a stationary point of f (x, y , z) and Hessian
H(x0 , y0 , z0 ) be
 
fxx (x0 , y0 , z0 ) fxy (x0 , y0 , z0 ) fxz (x0 , y0 , z0 )
 fxy (x0 , y0 , z0 ) fyy (x0 , y0 , z0 ) fyz (x0 , y0 , z0 ) 
fxz (x0 , y0 , z0 ) fyz (x0 , y0 , z0 ) fzz (x0 , y0 , z0 )
1 If Hessian is positive semi-definite, (x0 , y0 , z0 ) is a relative
minimum point.
2 If Hessian is negative semi-definite, (x0 , y0 , z0 ) is a relative
maximum point.
3 If Hessian is indefinite, f (x0 , y0 , z0 ) is a saddle point.
15 / 19
Example 6
Find all the relative extremum of the function
2 −y 2 −z 2
f (x, y , z) = xe −x .

Solution:
∂f −x 2 −y 2 −z 2 − 2x 2 e −x 2 −y 2 −z 2 = 0
∂x = e
∂f −x 2 −y 2 −z 2 = 0
∂y = −2xye
2 −y 2 −z 2
∂f
∂z = −2xze −x =0
 
Hence, the stationary points are ± √12 , 0, 0 .
 
At the point √12 , 0, 0 ,

∂2f 2 2 2 2 2 2
= −6xe −x −y −z + 4x 3 e −x −y −z = −2 2/e,
p
∂x 2

∂2f 2 2 2 2 2 2
= −2xe −x −y −z + 4xy 2 e −x −y −z = − 2/e,
p
∂y 2

∂2f −x 2 −y 2 −z 2 2 −x 2 −y 2 −z 2
p
= −2xe + 4xz e = − 2/e,
∂z 2
16 / 19
 ∂2f ∂2f ∂2f
= = =0
∂x∂y ∂x∂z ∂y ∂z
Hessian is  p 
−2 2/e p0 0
 0 − 2/e p0

0 0 − 2/e
which isnegativedefinite.
Hence, √12 , 0, 0 is a relative maximum point.
 
Analogously, − √12 , 0, 0 is a relative minimum point.

17 / 19
Optimization
Example 7
A company manufactures a product at two locations. The cost of
producing x units at location A is CA = 0.02x 2 + 4x + 500 and the
cost of producing y units at location B is CB = 0.05y 2 + 4y + 275.
If the product sells at $15 per unit, find the quantity that should
be produced at each location so as to maximize the profit.

Solution:
P = 15(x + y ) − 0.02x 2 + 4x + 500 − 0.05y 2 + 4y + 275

= −0.02x 2 + 11x − 0.05y 2 + 11y − 775

Px = −0.04x +11 = 0, Py = −0.1y +11 = 0 ⇒ (x, y ) = (275, 110)
So (275,110) is the unique stationary point.
Pxx = −0.04, Pyy = −0.1, Pxy = 0 ⇒ |H| = −0.04(−0.1) > 0
and Pxx < 0, so (275,110) is the relative maximum point.
Therefore, producing 275 at A and 110 at B can maximize the
profit.
18 / 19
Example 8
Find the global minima and maxima of the function
f (x, y ) = x 2 + 2y 2 − 1 in the disk region D = {x 2 + y 2 ≤ 1}.

Solution:
The stationary points satisfy fx = 2x = 0, fy = 4y = 0. We have
the unique stationary point (x, y ) = (0, 0) ∈ D and f (0, 0) = −1.

On the boundary x 2 + y 2 = 1 we have f (x, y ) = y 2 and

−1 ⩽ y ⩽ 1. Clearly, 0 ⩽ f (x, y ) ⩽ 1, and (±1, 0) are the
minimum points and (0, ±1) are the maximum points on the
boundary.

By comparing the values of the stationary point and the extrema

on the boundary, f (0, 0) = −1 is the global minimum and
f (0, ±1) = 1 is the global maximum.

19 / 19