24MA201 - Unit III InnerProduct Spaces Digital Material
24MA201 - Unit III InnerProduct Spaces Digital Material
3
24MA201 - LINEAR ALGEBRA AND APPLICATIONS
DATE 17.02.2025
4
Table of Contents
S. NO. TOPICS Page No.
1 Course Objectives 6
2 Pre Requisites 7
3 Syllabus 8
4 Course Outcomes 9
5 CO – PO/PSO Mapping 10
6
Lecture Notes: UNIT III INNER PRODUCT SPACES
Lecture Plan 11
Activity Based Learning 12
3.1 Inner Product 13
5
COURSE OBJECTIVES
6
PREREQUISITES
PREREQUISITES
7
SYLLABUS
3024
UNIT I Matrices and System of Linear Equations 15
Matrices – Row echelon form – Rank of a matrix – System of linear equations –
Consistency – Gauss elimination method – Gauss Jordan method.
Experiments using C language:
1. Solve the system of equations using Gauss Elimination method.
2. Solve the system of equations using Gauss Jordan method.
TOTAL: 75 PERIODS
8
Course Outcomes
Highest
Course Outcomes Cognitive
CO’s
Level
After the successful completion of the course, the student will be able to:
9
CO-PO/PSO Mapping
CO’s PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12
CO1 3 2 - - - - - - - - - 1
CO2 3 2 - - - - - - - - - 1
CO3 3 2 - - 1 - - - - - - -
CO4 3 2 - 1 - - - - - - - -
CO5 3 2 - - 1 - - - - - - -
CO6 3 - 2 - - - - - - - - 1
CO1 - - -
CO2 - - -
CO3 - - -
CO4 - - -
CO5 - - -
CO6 - - -
10
LECTURE PLAN- UNIT III
CO3
5.3.2025 PPT,Board
9 Least square 1 K2 & Marker
6.3.2025
approximation
11
ACTIVITY BASED LEARNING
12
UNIT III – INNER PRODUCT SPACES
3.1Introduction
An inner product space or a Hausdorff pre-Hilbert space is a vector
space with an additional structure called an inner product. This additional structure
associates each pair of vectors in the space with a scalar quantity known as the
inner product of the vectors, often denoted using angle brackets. Inner products
allow the rigorous introduction of intuitive geometrical notions, such as the length
of a vector or the angle between two vectors. They also provide the means of
defining orthogonality between vectors (zero inner product). Inner product
spaces generalize Euclidean spaces (in which the inner product is the dot
product, also known as the scalar product) to vector spaces of any (possibly
infinite) dimension, and are studied in functional analysis. Inner product spaces
over the field of complex numbers are sometimes referred to as unitary spaces.
The first usage of the concept of a vector space with an inner product is due
to Giuseppe Peano, in 1898.
1. Inner Product
An inner product on a vector space V ( F ) is a function that assigns, to every
(ii) u v, w u , w v, w
(iii) u , v u , v
(iv) u , u 0 and u , u 0 if and only if u 0 .
13
Orthogonal Vectors
Let V be an inner product space. The vectors u and v in V are orthogonal (or
perpendicular) if u, v 0 .
A subset S of V is orthogonal if any two distinct vectors in S are orthogonal.
Orthonormal Set
A subset S of V is orthonormal if S is orthogonal and consists entirely of unit vectors.
Note:
The process of multiplying a non-zero vector by the reciprocal of its length is
Problem: 1
Let V (F) R3 (R) be a vector space. u (a 1, a 2, a 3), v (b1,b 2,b 3) defined by
Solution:
Let u (a1, a2 , a3 ), v (b1,b2 ,b3 ) and w (c1, c2 , c3 )
14
Problem:2
Let u (a , a , a ,..., a ), v (b , b , b ,..., b 1) 2 F n3(C) . n 1 2 3 n
Solution:
Problem:3
Let V be the set of all continuous real functions defined on the closed interval [0,1].
1
0 0
15
1
(iii) f g, h ( f (t ) g (t ))h(t) dt
0
1 1
f (t )h(t ) dt g (t )h(t ) dt
0 0
f g, h f , h g, h
f , g is an inner product space over R
Problem: 4
Prove that R 2 (R) is an inner product space with the inner product defined by
Solution:
Theorem: 1
Let V be an inner product space. Then for u,v, wV and F, the following
16
(iii)u, 0 0, u 0
(iv) If u, v u, w, u V then v w
Proof:
(i) u, v w v w, u
v, u w, u
v, u w, u
u, v w u, v u, w
(ii) u, v v, u
v, u
u, v u, v
(iii)u, 0 u, 0v 0 u, v 0
0, u 0v, u 0 v, u 0
u, 0 0, u 0
(iv) Consider, u, v w u, v u, w using(i)
=0 (given)
v w 0 using(iii)
vw
Theorem: 2
Let V be an inner product space over F, then u, v V and , F we have
(i) u u
(ii) u, v u v (Cauchy Schwartz inequality)
(iii) u v u v (Triangular inequality)
Proof:
(i) u 2
u,u
u, u
u, u
u, u
u, u
u 2 2 u 2
u u
17
(ii) Cauchy Schwartz Inequality
Case(i)
If u 0 (or) v 0 then u, v 0
Also u 0, v 0
Hence u, v u v
Case(ii)
Let u 0 and v 0.
Let w v v,u2 u
u
u 2 u 2
2
u, vu, v u, vu, v v, u
2 u 2
v
u 2 u 2 u 4
2 2 2
u, v
2 u, v u, v
v
u 2
u 2
u 2
2
u, v2
w, w v
u 2
We know that, w, w 0
2
u, v
2
v 0
u 2
2
2 u, v
v
u 2
2
u v 2 u, v 2
u, v u v
18
(iii)Triangular Inequality
2
uv u v, u v
u, u u, v v, u v, v
u 2 u, v u, v v 2
u 2 2 Reu, v v 2
u 2 2 u, v v 2
since, 2 Re(z) 2 z
u 2 2 u v v 2
since, u, v u v
u v 2 (u v )2
uv u v
Theorem: 3
In an inner product space V(R), x, y V ,
(i) x y x y
(ii) x y 2 x y 2
4x, y
Proof:
(i) x (x y) y
x x y y
x y x y (1)
y ( y x) x
y yx x
y x yx
( x y ) x y (2)
from(1)&(2),
x y x y
(ii) x y 2 x y 2
x y, x y x y, x y
x, x x, y y, x y, y x, x x, y y, x y, y
x, y y, x x, y y, x
x y 2 x y 2 4x, y sinceV is real,x, y y, x
Theorem: 4
In an inner product space V, any subset of non-zero orthogonal vectors are linearly
independent.
Proof:
19
Let S v1 ,v2 ,v3 ,...,vn be a set of orthogonal vectors, which are non-zero.
vi 0, vi , v j 0 if i j
Let 1v1 2 v2 3v3 ... n vn 0.
Consider 1v1 2 v 2 3v3 ... n v n , vi 0 0, vi 0 by theorem1
1 v1, vi 2 v2 , vi 3 v3 , vi ... n vn , vi 0
1 .0 2 .0 3 .0 ... n .0 0
i vi , vi 0
i vi 2
0
i 0 Hence S is linearly independent.
Orthonormal basis:
Let V be an inner product space. A subset S of V is orthonormal basis if it is an
1 2 2 1
Example: The set , , , 2
is an orthonormal basis for R .
5 5 5 5
Proof:
Let V(F) be a finite dimensional inner product space and dim(V ) n .
We prove by induction on n.
Take u1 v1 0
20
v , u
For, if u2 0 then v2 2 21 u 1 0
u1
v , u
v2 2 21 u 1
u1
Claim: u2 is orthogonal to u1
v , u
u2 , u1 v2 2 21 u 1,u 1
u1
u1
v2 , u1 v2 , u1
u2 , u1 0
u2 is orthogonal to u1
u1 ,u2 is an orthogonal set
v , u v , u
If uk 1 0, then vk 1 k 1 2 1 u 1 k 1 22 u 2 ... vk 1 ,u2k u k
u1 u2 uk
v , u v , u
uk 1, ui vk 1 k 1 21 u 1 k 1 22 u 2 ... vk 1 ,u2k u k, u i
u1 u2 uk
21
vk 1 , ui vk 1,u21 u 1, u i .... vk 1,uk2 u ,ku i
u1 uk
uk 1 ,ui 0
Let V be an inner product space and S v1 ,v2 ,v3,...,vn be an orthogonal subset of V
n
v, vi
consisting of non-zero vectors . If v L(S ) , then V
i1 vi 2 i
v.
Proof:
Given: V is an inner product space over F.
v, vi
Where, i , i 1, 2, 3,...
vi 2
n n
v, vi
V i vi 2 vi
i1 i1 vi
Theorem: 7
Let V be an inner product space over F. Let v1 ,v2 ,v3,...,vn be an orthonormal set of
Proof:
22
To Prove: v1 ,v2 ,v3,...,vn are linearly independent.
Where,
u1 v1
v2 , u1 u
u2 v 2 2 1
u1
v3 , u1 u v3 , u2 u .
u3 v 3 2 1 2 2
u1 u2
Where, w1 u1 , w 2 u2 , w 3 u3 .
u1 u2 u3
Note: The Fourier coefficients of v relative to the orthonormal basis w1 , w2 , w3are
given as v, w1 ,v, w2 ,v, w3 .
23
Problem 1:
In R2 (R), test whether S (v ,v )where v (1,1) and v (1,1) is a basis. Find
1 2 1 2
the orthogonal basis with the standard inner product. Also find the Fourier
Solution:
Let S (v1 , v2 ) (1,1), (1, 1)
1 1
Let A
1 1
1 1
A (1 1) 2 0 and dim R 2 2.
1 1
Where
u1 v1 (1,1)
v2 , u1 u
u2 v 2
u 2 1
1
0
u 2 (1, 1) (1,1) (1, 1) (0,0 ) (1, 1)
2
The orthogonal basis u1 , u2 (1,1), (1, 1)
To find orthonormal basis:
w2 u2
u2
24
Here u2 u2 ,u 2 (1,1),(1,1) 2
w 2 (1, 1) 1 , 1 .
2 2 2
Fourier coefficients:
The Fourier coefficients of the vector v relative to the orthonormal basis are
1 1 3 4 7
v, w1 (3, 4 ), ,
2 2 2 2 2
1 1 3 4 1
v, w 2 (3, 4 ), ,
2 2 2 2 2
Problem 2:
In R 4 , let w1 (1, 0,1, 0 ), w2 (1,1,1,1 ) and w3 (0,1, 2,1). Then w1 , w2 , w3 is linearly
independent. Use Gram-Schmidt process to compute orthogonal vectors v1, v2 , v3 and then
Solution:
Where
v1 w1 (1,0,1,0)
w , v
v2 w 2 2 21 v 1
v1
2
v 2 (1,1,1,1) (1,0,1,0) (1,11,1) (1,0,1,0) (0,1,0,1)
2
v2 (0,1,0,1).
25
w3 , v1 v w3 , v2 v
v3 w3 2 1 2 2
v1 v2
2
v 3 (0,1, 2,1) (1,0,1, 0) 2 (0,1, 0,1) (0,1, 2,1) (1,0,1, 0) (0,1,0,1) (1, 0,1, 0)
2 2
v3 (1,0,1,0 ).
The orthogonal basis v1 , v2 , v3 (1, 0,1, 0), (0,1, 0,1), (1, 0,1, 0).
v
u2 2
(0,1, 0,1)
v2 2
u 3 v3
(1, 0,1, 0 ) .
v3 2
The orthonormal basis u ,1u ,2u 3 1 (1,0,1,0), 1
(0,1,0,1), 1
( 1, 0,1, 0 ) .
2 2 2
Problem 3:
1
Let V P (R) with the inner product f (x ), g (x) f (t) g (t)dt and consider the
1
subspace P2(R) with the standard basis . Use Gram-Schmidt process to replace
by an orthogonal basis (v1 ,v2 ,v3 ) for P2 (R ) and the use the orthogonal basis to obtain
Solution:
The basis of P 2 (R ),
1, x, x 2
Let w1 1, w2 x and w3 x 2 .
26
To find orthogonal basis:
Where v1 w1 1
w2 , v1 v
v2 w 2 2 1
v1
1 1
1 1
0
v2 x (1) x 0 x
2
v2 x.
1 1 1
t3 1 1 2
w3 , v1 x ,1 t .1dt t dt
2 2 2
1 1 3 1 3 3 3
1 1
w3 , v2 x , x
2
1
t .t dt t
2
1
3
dt 0 t 3 is an odd function.
1
2 1 t3 1 1 1 2 2
v2 x, x t.t dt t dt
1 1 3 1 3 3 3
2
0 1
v 3 x 3 (1)
2
( x) x
2
2 2 3
3
v3 x 2 1 .
3
1
The orthogonal basis v1 , v2 , v3 1, x, x .
2
3
v1 1
Where u1
v1 2
27
v x 3x
u2 2
v2 2 2
3
u3 v3
v3
1 1
1 1 1 1 2 1 8
x 2 , x 2 t 2 . t 2 dt t 4 t 2 dt
2
v3
3 3 1 3 3 1
3 9 45
x2 1
3 45 x 2 1 .
u3
8 8 3
45
3 x, 45 x 2 1 .
The orthonormal basis for P2 (R) is u1,u 2,u 3 1,
8 3
2 2
Problem 4:
Apply Gram-Schmidt process to find the orthogonal basis and orthonormal basis for
the subset S (1,i, 0),(1 i, 2, 4i). Also find the Fourier coefficients of the vector
Solution:
Given S (1,i, 0 ), (1 i, 2, 4i )
Where
u1 v1 (1, i,0)
v2 , u1 u
u2 v 2 2 1
u1
28
1 3i
u 2 (1 i, 2,4i) (1,i,0) (1i, 2,4i) 1 3i , i 3 ,0 1 i ,1
i
, 4i
2 2 2 2 2
1 i , 1 i , 4i
u1 (1,i,0 ) and u2
2 2
1 i 1 i
The orthogonal basis u 1, u 2 (1, i, 0 ), , , 4i .
2 2
To find orthonormal basis:
w2 u2
u2
1 i ,1 i , 4i , 1 i ,1 i , 4i
u2 u2 , u2 2
2 2 2
Here
(1 i )(1 i) (1 i)(1 i) 4i (4i ) 17
4 4
1 i ,1 i , 4i
2 2 1 i 1 i 4i
w2 , ,
17 2 17 2 17 2 17 .
1 i , 0 , 1 i , 1 i , 4i .
The orthonormal basis w1 , w2 ,
2 2 2 17 2 17 2 17
Fourier coefficients:
Let v (3i,4i,4)
1 i 1 i 4i
v, w 2 (3 i, 4i, 4 ), , ,
2 17 2 17 2 17
17i
29
Problem 5:
Apply Gram-Schmidt process to find the orthogonal basis and orthonormal basis for
3 5 , 1 9 , 7 17
the subset S . Also find the Fourier coefficients of
1 1 5 1 2 6
1 27
the vector A relative to orthonormal basis.
4 8
Solution:
3 5 , w 1 9 , w 7
Let w1
17
2 5 2
1 1 1 6
3
To find Orthogonal basis:
Where
3 5
v1 w1
1 1
w2 , v1 v
v2 w 2 2 1
v1
3 1 . 1 9 8 28
trace trace
5 1 5 1 0 44
sumof diagonal elements 8 44 36
3 1. 3 5 10 14
trace trace
5 1 1 1 14 26
sum of diagonal elements 10 26 36
1 9 36 3 5 4 4
v2 1 36 1 1 6
5 2
4 4 .
v2
6 2
30
3 1. 7 17 trace 19 45
trace 6 37 91
5 1 2
sum of diagonal elements 72
4 6 . 4 4 52 28
trace trace
4 2 6 2 28 20
sum of diagonal elements 72
5 , 4 4 , 9 3 .
The orthogonal basis v 1,v 2,v 3 3
1 1 6 2 6 6
Where v1 1 3 5
u1
v1 6 1 1
v 1 4 4 1 4 4
u2 2
v2 72 6 2 6 2 6 2
u3 v3
v3
31
5 , 1 4 4 , 1 9 3 .
The orthonormal basis u ,1u ,2u 3 1 3
6 1 1 6 2 6 2 9 2 6 6
Fourier coefficients:
1 27
Let A
4 8
1 73
trace 1
6 9 143
1
sum of diagonal elements (144 ) 24
6
A,u1 24
1 4 6 1 27
A,u 2 trace (u 2 A ) trace .
6 2 4 2 4 8
1 20 60
trace
6 2 4 92
1
sum of diagonal elements (72 ) 6 2
62
A,u2 6 2
1 9 6 1 27
A,u3 trace (u 3 A ) trace
.
9 2 3 6 4 8
1 9 6 . 1 27
trace
9 2 3 6 4 8
1 33 291
trace 2 27 129
9
1
sum of diagonal elements (162 ) 9 2
62
A,u3 9 2 .
32
3.4 Adjoint Of A Linear Operator
()t
is defined by A* . That is A* A . A* is an n m matrix.
1 i i 1i i
For example, if A . Then A
1 2 3i 1 2 3i
() 1i
A* A
' 1
i 2 3i
Definition:
Let A Mnn (F ) be the vector space where F is R or C. The inner product of A
and B in V defined as A,B tr (B* A)is called the Frobeniusinner product on
V.
Problem
1 2 i , B 1 i 0
If A belong to M 22 (C ). Using Frobenius inner
3 i
0 i
product compute A,B , A , B .
Solution:
1 2 i , B 1 i 0
Given A
3 i
0 i
By the definition of Frobenius inner product : A,B tr (B* A)
1 i 0
Now B *
0 i
B * A 1 i 0 1 2 i
0 i 3 i
33
1 i 3 i
3i 1
A,B tr (B* A ) 1 i 1 i
A A,A
A,A tr (A* A )
1 3 1 2 i
A* A
2 i i 3 i
10 2 4i
2 4i
6
tr (A* A ) 10 6 16
A 4
And B B,B
B,B tr (B*B )
B*B 1 i 0 1 i 0
0 i 0 i
2 0
0 1
tr (B*B ) 2 1 3
B 3
34
Note: Let V (F ) be a finite dimensional inner product space and g:V F be a linear
functional. Then there exists a unique vector � ∈ � such that g(x)=x,y,x V
Proof:
Given T is a linear operator on V and T* is defined as T(u),v=u,T* (v) u,vV
First we prove T* is linear.
If u,v V , then
u,T* (v w ) T (u ), (v w)
= T(u),v+T(u),w
= u,T* (v)+u,T* (w)
u,T* (v+w)=u,T* (v)+u,T* (w) u,T* (v)+T* (w), uV
T* (v+w) =T* (v) + T* (w), v,w V
For any F ,
u,T* ( v) = T(u), v
= T(u),v
= u,T * (v)
= u, T* (v), u V
T* ( v)= T* (v), v V
Hence T is a linear transformation on V .
*
T* (v)=S(v), vV
T * =S
Hence T
*
is unique.
35
Theorem2: Let V (F ) be a finite dimensional inner product space and
B v1 ,v2 ,...vn be a orthonormal basis.
Proof:
If A (T )B ,C T * ( ) then
B
Cij T * (v j ), vi
vi ,T * (v j )
T (vi ), v j
Aij (A * )ij
Hence C A*
Thus if A (T )B thenA* T * ( ) B
1. (T ) T
* *
2. (S T )* S * T *
3. (T )* T *
4. (ST )* T * S*
Proof:
1.For any u, v V , consider
u, (T * ) v T * (u ), v
*
= v, T * (u )
= T (v ),u
u,T (v ) u V
(T * ) v T (v ),v V
*
(T * ) T
*
36
S (u ) T (u ), v
S (u ), v T (u ), v
u, S * (v ) u, T * (v )
u, S * (v ) T * (v )
u, (S * T * )(v ), u V
(S T ) v (S * T *)(v ),v V
*
(S T ) S * T *
*
3. u, ( T ) v ( T )u, v
*
T (u ), v
T (u ), v
u,T * (v )
u, ( T ) v u, T * (v ), u V
*
( T )* v T * (v ),v V
( T ) T *
*
S (T (u )), v
T (u ), S * (v )
u,T * (S * (v )) u, (T * S * (v )),u V
(ST )* v (T * S * )v,v V
(ST )* (T * S * )
Problem 1 :
Solution:
37
T (e1 ) T (1,0 ) (1,1) (1,0 ) (0,1)
T (e1 ) e1 e2 and
T (e2 ) T (0,1) (2, 1) (2,0 ) (0, 1) 2 (1,0 ) (1)(0,1)
T (e2 ) 2e1 e2
t
1 1 1 2
Matrix TB
2 1 1 1
*
1 2 1 1
Matrix T B
*
1 1 2 1
1 1 x x y
T* (x, y ) in the basis B is 1 y 2x y
2
T * (x, y ) (x y, 2x y )
Problem 2 :
A linear operator T on R2 (R) is defined by T (x, y) (2x y, x 3y) with standard
inner product. Find T* (x, y ) and T* (3, 5).
Solution:
T * (x, y ) (2x y, x 3y )
38
3.5 Least Square Approximation
Finding a vector �0 for which " �� − � " is as small as possible is called the
general least square.
Working Rule:
To find least square solution of AX b
( )
Step 3: If rankA n , the number of variables, then A* A is invertible. Find A*A .
1
( )
Step 4:Solution is X A* A
1
A*b
Minimal solution:
Suppose AX b is consistent with infinite number of solutions. Then it has only one
minimal solution s . If u is a solution of (AA )X bthen the minimal solution is
*
s A*u
Problem 1:
Find the least square solution and least square error of the equation
2 1 1
1 2 x 1 1
x
2 1
1 1
39
Solution:
Given equation is
2 1 1
1 x1
2 1
x
1 1 2 1
2 1 1
x1
Here A 1 2 , X ,and b 1
x2 1
1 1
t
2 1
2 1 1
Since A is a real matrix A 1
*
2
1 2 1
1 1
( )
The normal equations are A* A X A*b
2 1
2 1 1
A A A A
* t
1 2
1 2 1
1 1
6 5
5 6
1
2 1 1 4
And A b
*
1
1 2 1 1 4
But �(�) = 2 = ������ � ƒ ���i�����
A* A is invertible
( )
Normal equations are A* A X A*b
6 5 x1 4
5 6 x 4
2
6x1 5x2 4 …………(1)
5x1 6x2 4 …………(2)
40
4
∴the least square solution is X 0
11
4
11
12 1
11 11
1
AX 0 b 12 1 1
11 11
8 1 3
11 11
AX 0 b ( AX 0 b ), ( AX 0 b )
1 1 3 1 1 3
11 ,11 , 11 ,11 ,11 ,11
1 1
2 2 92 0.3
11 11 11
Solution:
x 5 y 3,
Given system is 2x 2 y 2,
x y 5
41
1 5 3
x
Therefore the coefficient matrix A 2 2 , X and b 2 So, the matrix
y 5
1 1
t
1 5
1 2 1
equation is AX b , A is real A A 2
* t
2
5 2 1
1 1
The normal equation is (A* A )X A*b …………………(1)
1 5
1 2 1
Now A A A A
* t
2 2
5 2 1
1 1
6 0
0 30
3
1 2 1 2 2
And A b
*
5 2 1 16
5
6 0 x 2
0 30 y 16
6x 2, 30 y 16
x 1, y 8
3 15
1
3
∴the least square solution is X0
8
15
42
3
1
1 5 1
3 2
Now AX 0 2 2
8 5
1 1 1
15
5
3 0
2 3
12
AX 0 b 2
5 5
1 5 24
5 5
AX 0 b ( AX 0 b ), ( AX 0 b )
Problem 3:
And
2 2 2t 1 4
A b Ab 2
* t
2 2 3 4
2 2 6 2 4
12 12 20 x1 4
12 12 20 x 2 4
20 20 44 x 4
3
x 2 y z 1, 2x 3y z 2, 4x 7 y z 4
Solution:
44
x 2 y z 1,
2 x 3 y z 2,
4x 7 y z 4
1 2 1 x 1
The coefficient matrix A 2 3 1 X y b 2
4 7 1 z 4
1 2 1
A 2 3 1 0
4 7 1
To find the minimal solution ,we must find one solution of (AA* )X b
t
1 2 1 1 2 4
Since A is real matrix, A* At 2 3 1 2 3 7
4 7 1 1 1 1
Now
1 2 1 1 2 4 6 7 19
AA AA 2
* t
3 1 2 3 7 7 14 28
4 7 1 1 1 1 19 28 66
and
6 7 19 x 1
(AA ) *
X b 7 14 28 y 2
19 28 66 z 4
6 x 7 y 19 z 1,
7 x 14 y 28 z 2,
19 x 28 y 66 z 4,
45
6 7 19
7 14 28 0
19 28 66
Put x 0, weget
7 y 19z 1,
14 y 28z 2
y 1,
Solving we get 7
z0
Therefore one solution is
x 0, y 1 , z 0
7
0
1
u
7
0
The minimal solution is s A*u
2
0 7
1 2 4
1 3
s 2 3 7
7 7
1 1 1
0 1
7
2 3 1
The minimal solution is x , y , z
7 7 7
46
Problem1
Find the least square line fitted to the data (1,1) ,(2,2),(3,2) ,(4,3) . Also find
the least square error.
Be the line fitted to the given data (1,1) ,(2,2),(3,2 ),(4, 3).
Substituting the points in (1), we get
a.1 b 1
a.2 b 2
a.3 b 2
a.4 b 3
11 1
21 2
a
The coefficient matrix A , X , B
31 b 2
41 3
the matrix equation is AX B
1 1
2 1 1 2 3 4
Since A is real , A* A'
3 1 1 1 1 1
4 1
1 1
1 2 3 4 2 1
Now A* A A' A
1 1 1 1 3 1
4 1
30 10
10 4
1
12 3 4 2
And A*B A'B
1111 2
3
23
8
47
30 10 a 23
the normal equations becomes
10 4 b 8
a 3 and b 1
5 2
3
5
Therefore X 0 is the least square solution
1
2
3 1
the line of best fit y x
5 2
And error B AX 0
11
10
1
2
1 3 17
1
AX 0 5 10
3 1 1 23
4 1 2 10
29
10
11
10
1 17
2
10
B AX 0
3 23
4 10
29
10
48
1
10
3
10
3
10
1
10
Error= B AX 0
1 32 (3)2 1
0.45
102 102 102 102
Problem:
Find the parabola of best fit to the data (3,9), (2, 6) , (0, 2), (1,1). Also find the error.
Solution:
The given points are (3, 9 ), (2, 6) , (0, 2 ), (1,1).
9 3 1 9
a 6
A 4 2 1
Where 0 , X b and B
0 1 2
c
1 1 1 1
A is real matrix
49
9 4 0 1
A A 3 2 0
* t
1
1 1 1 1
Now
9 4 0 1 9 3 1
A* A A t A 3 2 0 1 4 2 1
1 1 1 10 0 1
1 11
98 34 14
34 14 4
And
14 4 4
9 1 9 106
4 0
6
A*B At B 3 2 0 1 38
2
1 1 1
1
18
1
50
And the error is B AX 0
Now
1
9 3 1 3 9
4 6
AX 0 4 2 1
3 2
0 0 1 2
1 1 1 1
9 9
6 6
B AX 0 0
2 2
1 1
Therefore Error is B AX 0 0
51
3.7. Practice Quiz
1. Let be an inner product space, and let x and y be two vectors in V such
that " � " = 3 and " � " = 4 . What exactly can we say about " � + � " ?
a) " � + �" = 5
b) " � + � " is between 0 and 7 inclusive.
c) " � + � " is between 1 and 7 inclusive.
d) " � + � " = 7.
a) Is linearly independent
b) Is linearly dependent
c) Is an orthonormal basis for V
d) Span the vector space V
5. The cosine of the angle between the two vectors v = (1,-1,1) & u = (1,1,1)
1
a) is equal to 3
1
b) is equal to 9
c) is equal to √ 19
1
d) is equal to − 3
52
6. If the vectors v1, v2 and v3 form an orthonormal basis for R3
v1 = (1,0,0) ; v2 = (0, 1 , 1
); v3 = (0,- 1 1
, );
√2 √2 √2 √2
Then The vector w = (1,-1,1) can be expressed as a linear combination of the
vectors v1, v2 and v3 in the following way:
2 2
a. W = 1.v1 + v2 + v3
√2 √2
b. W = 1. v1 + √22 v2
c. W = 1.v1 + √22 v3
2 2
d. W = 1.v1 - v2 + v3
√2 √2
7. let u = (-1, 1/4) and v = (4, -1/8). Then find " � "
17
a) √ 16
b) √1025
16
c) √ 577
4
d) √ 174
17
a) √ 16
b) √1025
16
577
c) √ 4
d) √ 174
b) √1025
16
577
c) √ 4
d) √ 174
53
10. find the unit vector in the direction u.u = (3, 2, -5)
2
a) ( √338, , 5 )
√38
√38 2
b) ( √338, , − 5)
√38
√38
c) ( −3 , −2 −5
, )
√38 √38
d) ( √338, − 2 , − 5 )
√38 √38 √38
12. Find the dot product < u, v> where u = (3, 4) and v = (2, -3)
a) -30
b) -6
c) 13
d) 36
13. Find the dot product < u , v > v where u = (3, 4) and v = (2, -3)
a) (-12, 18)
b) (12, 18)
c) (-12, -18)
d) (12, -18)
14. Find < u + v , 2u – v > where <u , u> = 4 , <u , v> = −5 and <v , v> = 10
a) 7
b) 13
c) -13
d) -7
54
ANSWERS:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
b d d a a d c b c b a b a d
55
3.8 Assignment 1
2. Prove that every finite dimensional inner product space has an orthonormal
basis. (Or)
State and prove Gram-Schmidt orthogonalisation process.
3. Find an orthogonal basis of the inner product space R3(R) with standard inner
product, given the basis B={(1,1,0),(1,-1,1),(-1,1,2)} using Gram-
Schmidt orthogonalisation process.Also find the Fourier coefficients of the
vector (2,1,3) relative to orthonormal basis.
56
3.8 Assignment 2
57
3.8 Assignment 3
2. Use the least squares approximation to find the best fits with both
i)a linear function and ii) a Quadratic function. Also
Compute the error E in both cases for the following data
{(-3,9),(- 2,6),(0,2),(1,1)}
3.8 Assignment 4
2. Find the line of best fit to the data (2,1), (5, 2), (7, 3), (8, 3).. A lso find the
least square error.
3.8 Assignment 5
S.NO QUESTIONS K CO
Level
1. K1 CO4
Define an inner product space. Give an example.
Solution:
Let V be a real vector space. Suppose to each pair of
vectors, there is assigned a real number, denoted by u, v .
Symmetric property: u, v v, u
f (t), g ( t ) f ( t ) g ( t ) dt .
0
2. K2 CO4
Solution:
u 2 u, u u,u 2
u 2 u u .
61
3. If f (t ) 3t 5, g (t ) t 2 in the polynomial space p(t) with inner K2 CO4
1
Solution:
Given f ( t) 3t 5, g ( t) t 2
1 1 1
0 0 0
1
3t3 15t2 25t 3 15 25 13.
0
f (t) 13.
1
1
t5
1
1 1
g(t), g (t) g (t) g (t)dt t dt
2
g(t) g(t)
4
.
0 0 5 0 5 5
4. State Cauchy-Schwartz inequality. K1 CO4
Solution:
For any vectors u and v in an inner product space V , u, v u v
vectors.
Solution:
Given: u ( 2, 3, 5) and v (1, 4, 3)
u 22 32 52 4 9 25 38
v 12 ( 4 ) ( 3) 116 9 26
2 2
62
6. Define orthogonality. K1 CO4
Solution:
Let V be an inner product space over F and u, v V . The
Solution:
Let v be an inner product space. A set S ui of vectors in V is said
Solution:
of orthogonal vectors w, v1 0; w, v2 0
w, v1 x 2 y z w, v2 3 x y
Now, and
0 x 2 y z (1) 0 3x y (2)
any non-trivial solution to the above system gives a vector
orthogonal to both v1 and v2 .
w ( 1, 3, 5)
63
9 K2 CO4
Find k so that u (1, 2, k, 3) and v ( 3, k,7, 5) in R4 are
orthogonal.
Solution:
u is said to be orthogonal to v if u, v 0
Now, u, v (1, 2, k ,3 ) , ( 3, k , 7, 5 )
0 3 2k 7k 15
0 9k 12 9k 12 k 4
3
10. K2 CO4
Find the Fourier coefficient c of v (1,2, 3,4) and w (1,2,1, 2) in
R4
Solution:
w 2 w, w 12 22 12 22 1 4 1 4 10
v, w 8 4
c .
w 2
10 5
Solution:
"�"=√< �, � >
< �, � >=tr(A*A)
A*=[ 1 3]
2 − � −�
A*A=[ 10 2 + 4�]
2 − 4� 6
< �, � >=10+6=16
"�"= √ 16=4.
64
12. In the inner product space V(R), u,v ∈ � are such that u+v K2 CO4
and u-v are orthogonal prove that " � " = " � " .
Solution:
Given u+v and u-v are orthogonal
< � + �, � − � >= 0
< �, � > −< �, � > +< �, � > −< �, � >= 0
Solution:
i 2 3i 0
T 3 0 3 i . Since e is the standard orthonormal basis
0 e
2 5i i
for c3, T T
t
e e
i 2 3i 0 i 3 0
T 3 0 3 i T * 2 3i 0 2 5i
0 e
2 5i i
e
0 3i i
Solution:
A linear operator T on an inner product space V is said to
have an adjoint operator T * on V, if
T ( u) , v u,T * ( v ) u,v V .
65
12. In the inner product space V(R), u,v ∈ � are such that u+v K2 CO4
and u-v are orthogonal prove that " � " = " � " .
Solution:
Given u+v and u-v are orthogonal
< � + �, � − � >= 0
< �, � > −< �, � > +< �, � > −< �, � >= 0
Solution:
i 2 3i 0
T
3 0 3 i . Since e is the standard orthonormal basis
0 e
2 5i i
for c3, T T
t
e e
i 2 3i 0 i 3 0
T 3 0 3 i T * 2 3i 0 2 5i
0 e
i i
e
2 5i 0 3i
Solution:
A linear operator T on an inner product space V is said to
have an adjoint operator T * on V, if
T ( u) , v u,T * ( v ) u,v V .
66
15 Prove that if S and T are linear operators on V and is scalar K2 CO4
then ( ST )* T *S * .
Solution:
Given that S and T are linear operators
( ST ) u, v ( )
ST ( u ) , v T ( u ) , S * ( v ) u,T * S *v ( )
u, T *S * v
( ST )* T *S * .
Solution:
67
3.10 PART B
S.NO QUESTIONS K CO
Level
1. Apply Gram Schmidt process, find the orthogonal basis and K2 CO4
orthonormal basis for
the sub set S (1,0,1), (0,1,1), (1,3,3). Also find the Fourier
basis.
ANS:
(i) The orthogonal basis
( w1 , w2, w 3) 1
,0,
1 1 2 1 1 1 1
, , , , , , .
2 2 6 6 6 3 3 3
3 6
(iii) Fouriercoefficients: v, w , v, w , v, w 0 .
1
2 2
2 3
Schmidt method for the basis (sint, cost, 1, t ) using the inner
product f , g f ( t ) g ( t ) dt .
0
ANS:
(i) The orthogonal basis
68
3. Represent the polynomial f (x) 1 2x 3x2 as a linear K2 CO4
1 2 5
, x, 3x 2 1 ( ) for P (R). Where
2
2 2 8
1
f , g f (t) g (t)dt .
1
ANS:
2 6, 2 10
f (x ),u 1 2 2, f (x ),u 2 f (x ),u 3 .
3 5
1 2 6 3 2 10 5
f (x ) 2 2 x
3 2
5 8
3x 2
1 ( )
2
1 2 i and B 1 i 0 .
A
3 i
i i
Ans: || A || 4, || B || 3 and A, B i
Find the line of best fit to the data (2,1), (5, 2 ), (7, 3 ), (8, 3 )Also K2 CO5
find the least square error.
5.
Ans: y 5 x 2
14 7
6. Find the parabola of best fit to the data K2 CO5
(1,1),(0,1),(1,0),(2, 2)Also find the least square error.
Ans: y x 2 3 x 7
5 10
69
7. Find the minimal solution of the following system K2 CO 5
of inear equations
x 2 y z 4, x 2 y 2z 11, x 5y 19.
1
Ans: s 4
3
8. Prove that if S is an orthogonal set of non-zero vectors, then is K2 CO5
S linearly independent. (or)
T ( u) , v u,T * ( v ) u,v V .
Let V be the inner product space . Prove that the norm in V K2 CO4
satisfy the following properties
10
(i) || v || 0 and || v || 0 iff v 0
(ii) || kv || || k || || v ||
(iii) || u v || || u |||| v ||
70
SUPPORTIVE ONLINE CERTIFICATION COURSES
The following NPTEL and Coursera courses are the supportive online certification
courses for the subject Linear Algebra
Linear Algebra
https://swayam.gov.in/nd1_noc20_ma54/preview
Linear Algebra
https://www.coursera.org/learn/algebra-lineynaya
71
REAL TIME APPLICATIONS
Applications Of Matrices
https://youtu.be/rowWM-MijXU
https://youtu.be/fNk_zzaMoSs
https://youtu.be/k7RM-ot2NWY
https://youtu.be/jUIIm5C-xFs
72
Assessment Schedule (Proposed Date & Actual Date)
24/2/25 to
1 FIRST INTERNAL ASSESSMENT
25/2/25
01/04/25 to
2 SECOND INTERNAL ASSESSMENT
05/04/25
28/04/25 to
3 MODEL EXAMINATION
07/05/25
MINI PROJECT - 1
Project Title: Real-Time Hand Gesture Recognition Using Inner Products and
Norms
Objective: Build a real-time hand gesture recognition system that classifies different
gestures based on 3D hand pose data using norms and inner products.
Tasks:
Collect data on hand movements (e.g., using a camera or a depth sensor like
Kinect).
Preprocess the data into vectors representing different hand poses.
Apply inner product and norms to classify gestures based on the angle or distance
between gesture vectors.
Use the Gram-Schmidt process to create an orthogonal feature space for gesture
classification.
MINI PROJECT - 2
Project Title: Real-Time Video Compression with Least Squares for Efficient
Encoding
Objective: Implement a real-time video compression algorithm using least squares
approximation to reduce video size while maintaining quality.
Tasks:
Collect video data and represent frames as matrices.
Apply least squares approximation to reduce the dimensionality of the video
frames.
Implement the compression and decompression algorithms, and assess video
quality using metrics like PSNR (Peak Signal-to-Noise Ratio).
Develop a real-time interface to compress and display videos in near real-time with
minimal loss in quality.
MINI PROJECT - 3
REFERENCES :
G. Strang, “Linear Algebra and its applications”, Thomson
1
(Brooks / Cole), 4th Edition, New Delhi, 2005.
79
Thank you
Disclaimer:
This document is confidential and intended solely for the educational purpose of RMK Group of
Educational Institutions. If you have received this document through email in error, please notify the
system manager. This document contains proprietary information and is intended only to the
respective group / learning community as intended. If you are not the addressee you should not
disseminate, distribute or copy through e-mail. Please notify the sender immediately by e-mail if you
have received this document by mistake and delete this document from your system. If you are not
the intended recipient you are notified that disclosing, copying, distributing or taking any action in
reliance on the contents of this information is strictly prohibited.
80