Unit-4 - Inner Product Space
Unit-4 - Inner Product Space
UNIT 4
Structure
12.1 Introduction
'.
Ohjccllvc';
12.2 Inner Product
12.3 Norm of a Vector
12.4 Orthogonality
12.5 Summary
12.6 Solutions/Answers
12.1
4 INTRODUCTION
So f i r you have studied ma'ny interesting vector spaces over various fields. In this unit, and
the following ones, we will only consider real and complex vector spaces. In the Unitlower
2 you classes, you have
studied geometrical notions like the length of a vector, the angle between two vectors and
the dot product in R2 or R3.,In this unit we carry these concepts over to a more general
setting. We will define a certain special class of vector spaces which open up new and
interesting vistas for investigations in mathematics and physics. Hence their study is
extremely fruitful as far as the applications of the theory to problemdare concerned. This
fact will become clear in Units 14 ,and 15.
Before going further we suggest that you refer to Unit 2 for the definitions and properties of
the length and the scalar product of vectors of R' or R'.
Objectives
After reading this unit, you should be able to
define and give examples of inner product spaces;
define the norm of a vector and discuss its properties;
define orthogonal vectors and discuss some properties of sets of orthogonal vectors;
obtain an orthonormal basis from a given basis of a finite-dimensional inner product
space.
f'
12.2
4 INNER PRODUCT
a
In this section we start with defining concept which is the generalisation of the scalar
.
product that you came across in Unit 2. Recall that if (x,, x,, x,) and (y, y,, y, ) are two
.vectors in R', then their scalar produc? is
-(xI. xi. x,) . (Y]. Y2. Y,) = X I Y l + X'Y? + X,Y'.
We also remind you that given any complex number z = a + ib, where a, b E R,its
complex conjugate is 2 = a - ib.
7
Further. zZ = (zJ- 7
= a- + b2, and = z.
Now we are ready to define an inner product.
.
Definition: Let V be a vector space over the field F. A map ( ): V x V -,F: (; ) (x, y) =
( x. y ) is called an inner product (or scalar product) over V if it satisfies the following
conditions:
IP1) ( x . x ) 2 0 Y x E v.
IP 2) ( u. x ) = 0 iff x = 0.
IP4) ax.^)= a . ( x . y ) f o r a ~F a n d x . y ~V
- . -
IP 5 ) (y. x ) = ( x, y ) for all x, y E V. (Here ( x. y ) denotes the complex conjugate of the
number ( x, y ).)
Inner Prtducts and The scalar ( x. y ) is called the inner product (or scalar product) of the vector x with the
Quadratic Forms
vector y.
A vector space V over which an inner product has been defined is called an inner product
space, and is denoted by (V, ( , )).
We make a remark here.
Remark 1: Let a E F. Then a =aiff a E R. So 1P5 implies the following statements.
a) ( X . X ) ER V X EV , s i n c e ( x , x ) = ( x . x ) .
b) If F = R. then ( x, y ) = ( y, x ) V x , y E V.
I '
Solution: We need to define an inner product on R'. For this we define (u, v) = u . v Vu,
v E R- ('.' denoting the dot product). Then, for u = (x,, x,, x,) and v = (y ,, y,, y,).
3
.
(u, v) = x,y, + xzy, + x,y,. We want to check if ( ) satisfies IPI - IP5.
i) IPI is satisfied because (u, u) = xf + x: + x:, which is always non-negative.
ii) Now, (u, u) = 0 xf + xi + x: = 0 a x, = 0, x, = 0, x, = 0, since the sum of positive
real numbers is zero if and only if each of them is zero.
:., u = 0.
Also,ifu=O,thenx,=O=x,=x,~ (u,u)=O. :.
So, we have shown that 1P2 is satisfied by <, >.
i i i ) IP3 is satisfied because
+ V,W) = (xI + yl,)zl+ (x, + yz)z, + 0, + YJz,, where w = (z,, zz75 ) .
(U
= ( X , Z , + X2Zz + x,zJ + (Y . z , + YIZl + Y,z,) = (u. w) + (v, w) .
We suggest that you verify 1P4 and IP5. That's what El says!
E E l ) Check that the inner product on ~"atisfies IP4 and IP5.
The inner product that we have given in Example 1 can be generalised to the inner prdduct
.
( , ) on R: defined by ((x,, ... , xn),(yI. ... y,)) = xlyl+ xzyz+ ... + x,y, This is called the
standard inner product on Rn.
Let us consider another example now.
Example 2: Take F = C and, for x, y E C, define (x, y) = xy. Show that the map
( , ): C x C + C is an inner product.
Solution: IP I and IP2 are satisfied because. for any complex number x, xx 2 0. Also, x x = 0
if and only if x = 0.
To complete the solution you can try E2.
E2) Show t h i t IP3. IP4and IP5 are true for Example.2.
Inner Product Spaces '
In fact, Example 2 can be generalised to C", for any n > 0. We can define the inner product
of two arbitrary vectors
-
..y,)
n
x = ( x I,... ,x n ) and y = (y,,. E c" by (x, Y) = xi Yi.This inner product is called the
i=l
standard inner product on Cn.
The next example deals with a general complex vector space.
Example 3: Let V be a complex vector space of dimension n. Let B = ( e , , . . ., e n ]be a basis
of V. Given x, y E V 3 unique scalars a , , ..., an,b,, . . .. bn E C such that
n -
Define (x, y) = x a i bi .
i=l
n n n
Finally, m = x b , & = x b , a , =xa,b,=(x.y)
)=I ,=I 1=1
.
Thus, IPI - 1P5 are satisfied. This proves that ( ) is an inner product on V.
Note that, in Example 3, the inner product depended on the basis of V that we chose. This
suggests that an inner product can be defined on any finite-dimensional vector space. In fact,
many such products can be defined by choosing different bases in the same vector space.
You may like tu try the following exerci\e now.
E E3) Let X = x , , . . ., xn) be a set and V be the set of all functions from X to C. Then, with
respect to pointwise addition and scalar multiplication, V is a vector space over C. Now, for
any f. g E V, define
i I:
.
Show that (V. ( )) is an inner produce space.
rner Products and
Quadratic Form
We now state some properties of inner products that immediately follow from P I - IP5.
Theorem 1: Let (V, ( ,)) be an inner product space. Then, for any x. y, z E V and a, p E C.
a) ( a x + p y . z ) = a ( x . z ) + p ( y . z )
b) (x, a y + pz) = 6 (x, y) + ii(x, Z)
c) (0, x3 = (x, 0) = 0.
d) (x - y, z) = (x, z) - (y,z)
e) (x,z)=(y,z)+z~ V=x=y.
Prbol: We will prove (a) and (c). and leave the rest to you.
a) (ax + PY, < = (ax, 2) + (py,zl (by Ip3)
= a (x, Z) + p (Y.Z) C ~ 1Y ~ 4 )
C) The vector 0 E V can be written as 0 = 0 . y for some y E V.
-
Then, (x, 0) = (0, x) = 6 = 0.
The proof of this theorem will be complete once you solve E4.
E E4) Prove (b), (d) and (e) of Theorem 1. &
.
c
4 NORM OF A VECTOR
12.3
In Unit 2 we defined the length of a vector v in R~or R~to b e G . We will extend this
definition to the length of a vector in any inner product space.
.
Definition: If (V. ( )) is an inner product space and x E V . then the norm (or length) of
.
the vector x is defined to be d(x, x) It is denoted by 1x (( .
We make some pertinent remarks here. Inner Product Spaces
Remark2: a ) B y I P l , ( x , x ) l O + x ~ ~ . T h u s I l x ( ( 2 0 .
(1 (1
Also, by IP2, x =O iff x = 0.
b) F o r a n y a ~ c w
, eget((ax((=(al((x(l,
b e c a u s e ( a x l l = d w = ~-=dlal'(x,;)
= l a I r n =(a1(Ixll.
As in Unit 2, we call x E V a unit vector if ((XI( =l.
X
E E5) Show that for any x E V, x # 0, - is a unit vector.
x I1 1I
-
adding and subtracting (x, z)(x, z).
)I 1'
Now x - a z 2 0. This means that x - ((x, z)(' 1 (I2 + ((x, z) - a(' 2 Ova E F.
In particular, if we choose a = ( x, z ), we get
(1'
O < ( J x -\(x.z)12.
s + \..
(1
Fig. 1: x + Y I1 llx ll +((Y 1
4 ORTHOGONALITY
12.4
J(x.Y)I
In Theorem 2 we showed that 5 1 for any x, y E V. In Unit 2 (Theorem 2) we
I 1 I I1 y 1 I(x. Y ) (
have shown that, for non-zero vectors x and y (in o or R", is equal to the
magnitude of the cosine of the angle between them. We generalise this concept now. Y
For any inner product space V and for any non-zero x, y E V, we take I(X' to be the
1 I I Y /I
magnitude of the cosine of the angle between the two vectors x a n d y. I) x C
So what happens if x and y are perpendicular to each other? We find that (x. y) = 0. This Fig.2 11 x + y 11 + 11 x - y 11 2 = 2 (11 11 2 + 1 11 2 )
leads us to the following definition.
Definition: If (V, ( , )) is an inner product space and x, y E V, then x is said to be
orthogonal (or perpendicular) to y if (x, y) = 0. This is denoted by x ly .
For example, i = (1,O) is orthogonal to j = (0, I ) with respect to the standard inner product
in R ?
We now give some properties involving orthogonality. Their proof is left as an c\t,rcise for
you.
E9) Using the definitions of inner product and orthogonality, prove the following results for
an inner product space V.
a) O i x +XE V.
b) x l x iff x = 0, where x E V.
C) x I y - y i x , f o r x ~V.
d) x I y a x i y for any a E F. where x, y E V.
lnner Products end Let us consider some examples now.
QuadraticForms
Example 5: Consider V = Rn. If x = (x,, . .., x,) and y = (y, ,..., y,) are any two vecto? of
V, we define the inner product of x with y by
On the lines of Example 5, we can also show that the elements of the standard basis of Cn
are mutually orthogonal and of unit length with respect to the standard inner product.
Try the following exercises now.
A.
E El01 For x, y E (V,( , ) ) such that x I y, show that
Il x + Y 1' = 1I x 112 + 1 Y 1 2 .
(This is the Pythagoras Theorem when V = R~ (see Fig. 3).)
*
H X C
1)
Fig. 3: x+y y2 =I1x 112+[ y 1'
E El 1) 0btain.a vector v = (x, y, z) E R ~ S Othat v is perpendicular to (1, 0,O) as well as
(-1,2.0), with respect to the standard inner product.
In the next two theorems we present some properties of an orthogonal set. related to the
linear combination of its vectors.
.
Theorem 4: Let (V.( ) ) be an inner product space and x, y, ,. . .. yn E V such that
x I yl tt i = I, .. .., n. Then x is prthogonal to every linear combination of the vectors
y, ...., y,
n
Proof: Let = x a i y i where
, a i E F v i = I ,..., n.
i=I
Then, y E V and
(X.Y)=(X*kaiYi) = t ~ ( x * Y i ) =because(x,
, yl)=O+i.
i=l i=l
This shows that x I y.
.
Theorem 5: Let (V, ( )) be an inner product space and A = { x ,..... xn)C V be an
orthogonal set. Then, for any a, E F (i = 1 .. . .. n), we have
i=l
i=l
2
*\ail =Ofori=~....,n,sinceII~iII~t~foranyi.
* a , = O f o r i = l ,..., n.
Thus, ( x , ,..., xn) is linearly independent. Hence, A is linearly independent.
We have just proved that any orthogonal set is linearly independent. Therefore, any
orthogonal set in a vector space V of dimension n must have a maximum of n elements. So,
for example, any orthogonal subset of R\an have 3 elements, at the most.
We shall use Theorem 6 as a stepping stone towards showing that any inner product space
has an orthonormal set as a basis. But first, some definitions and remarks.
Definition: A basis of an inner product space is called an orthonormal basis if it\ elements
form an orthonormal set.
For example, the standard basis of R n is an orthonormal basis (Example 5).
Now, a small exercise.
E E l 3) Let { e l,..., e n )be an orthonormal basis for a real inner product space V. Let
n n n
x = x x , e i and y = x y i e i be elements of V. Show that (x, y) = x x i y i .
i=L i=l i=l
We now state the theorem that tells us of the existence of an orthonormal basis. Its proof .
consists of a method called the G r a m S c h m i d t orthogonalisation process.
Theorem 7: Let (V, ( , )) be a non-zero inner product space of dimension n. Then V has an
orthonormal basis.
Proof: We shall first show that it has an orthogonal basis, and then obtain an orthonormal
basis.
Let 1 v , ,. . ., v n J be a basis of V. From this basis we shall obtain an orthogonal basis
( w , ,w, ,. ... w n ) of V in the following way.
Inner Product Spaces
Take w , = v,. Define w, = v2 - w l ) w I . Then w 2 = v2 --
( ~ 1 . ~ 1 )
and
(w,. v I ) = ( v 2 . vI)--(vI. v I ) = O . T h a t is.(w,. w , ) = O . Further. v, = c , v , + w,.
( ~ 1 v. I )
where c , = \ V ? . W , )E F
(wI. wI)
is an orthonornial basi\ of V.
Solution: ( 1. t. t'] I!, a ba\i\ for P,. From this we will obtain an orthogonal basis
I
:..(w?. w , ) = l:(t-?)I dt = -1
13
.
7 ( t 2 . w,) (1'. w l )
w, = I - - W, - I
(wz. w,) ( ~ 1 w. I )
i =I i=l
This is true for any ai E F. Now choose ai = (y, xi) +Ai ,. .., n. Then we get
12.5 SUMMARY
In this unit we have discussed the following points. We have
1. defined and given examples of inner product spaces.
2. defined the norm of a vector.
3. proved the CauchySchwan, inequality.
4. defined an orthogonal and an orthonormal set of vectors.
5. shown that every finite-dimensional inner product space has an orthonomal basis, using
the Gram-Schmidt orthogonalisation process.
6. proved Bessel's inequality.
( f , f ) = O e f ( x i ) = O * = l ,..., n
w f is the zero function.
i=l i=l
n -
= Cg(xi )f (xi = (g, f ).
i=l
.'. (V, ( , )) is an inner product space.
E4) b) (x, a y + pz) = (ay + pz, x), by IP5
= a (y, x) + ~ ( zx),
, by Theorem I(a).
-- -
= a(y, x) +Cc(z, x)
-
: = a ( x , y) + E(x, z), by IP5.
.;. (b) is proved.
d) (x-y, z)= ( x + ( - l ) ~ , z ) = ( x , z ) + ( - l ) ( y , z), by Theorem I(a).
= (x, 2 ) - (y, z).
e) (x, Z) = (y, z)+z E V
*(x-y,z)=O Y z E V , by ( d ) above.
*
(x - y, x - y ) = 0, taking z = x - y, in particular.
* x - y = O , by IP2.
*x=y.
X 1
E5) Let u = -. Then (u, u) = - - = -(x, x)
x 11 1 I Hxlr
n n
where x a i e i , x b i e i are ekments of Vs
Ell) v l ( l , O , O ) ~ x ~ l + y ~ O + z ~ O = O ~ x = O
vl(-l,2,0)* X . ( - ~ ) + ~ . ~ + Z . O = -x+2y=O.
O*
So we get x = 0, y = 0. Thus, v is of the form (0,0, z) for z E R.
{11~:
We want the set - - where wl = v l and
NOW,(vZ,w I ) = ( v 2 ,v l ) = 2 + 0 + 3 = 5 .
,
Also (w w ) = (v,, v,) = 10, so that w, = 1 1 a.
5
:. W ? = (2,1,1) - - (I, 0.3) =
10