Lec 11 The Change of Basis Problem
Lec 11 The Change of Basis Problem
Vector Spaces
The Change-of-Basis Problem
Department of Mathematics
Email : mukeshn@goa.bits-pilani.ac.in
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 1 / 19
Recall Basis
Definition 1
If S = {v1 , v2 , . . . , vn } is a set of vectors in a finite-dimensional
vector space V , then S is called a basis for V if:
(a) S spans V .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 2 / 19
Recall Basis
Definition 1
If S = {v1 , v2 , . . . , vn } is a set of vectors in a finite-dimensional
vector space V , then S is called a basis for V if:
(a) S spans V .
(b) S is linearly independent.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 2 / 19
Recall Basis
Definition 1
If S = {v1 , v2 , . . . , vn } is a set of vectors in a finite-dimensional
vector space V , then S is called a basis for V if:
(a) S spans V .
(b) S is linearly independent.
Theorem 2
Let V be an finite dimensional vector space, and let
S = {v1 , v2 , · · · , vn } be any basis.
(a) If a set in V has more than n vectors then it is linearly dependent.
(b) If a set in V has fewer than n vectors, then it does not span V .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 2 / 19
ProofP(a): Let W = {w1 , w2 , . . . , wm } with m > n. Then
wi = nj=1 aij vj .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 3 / 19
ProofP(a): Let W = {w1 , w2 , . . . , wm } with m > n. Then
wi = nj=1 aij vj .
Considering m
P
i=1 αi wi = 0 gives us a linear system having more
unknowns
Pm than equation, thus, there is nontrivial solution of
i=1 i i = 0.
α w
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 3 / 19
ProofP(a): Let W = {w1 , w2 , . . . , wm } with m > n. Then
wi = nj=1 aij vj .
Considering m
P
i=1 αi wi = 0 gives us a linear system having more
unknowns
Pm than equation, thus, there is nontrivial solution of
i=1 i i = 0.
α w
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 3 / 19
ProofP(a): Let W = {w1 , w2 , . . . , wm } with m > n. Then
wi = nj=1 aij vj .
Considering m
P
i=1 αi wi = 0 gives us a linear system having more
unknowns
Pm than equation, thus, there is nontrivial solution of
i=1 i i = 0.
α w
Considering m
P
i=1 αi vi = 0 gives us a linear system having more
unknowns than equation, thus, there is nontrivial solution and hence
contradiction.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 3 / 19
Recall Dimension
Remark
Using the previous Theorem, all bases for a finite-dimensional vector
space have the same number of vectors.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 4 / 19
Recall Dimension
Remark
Using the previous Theorem, all bases for a finite-dimensional vector
space have the same number of vectors.
Definition 3
The dimension of a finite-dimensional vector space V is denoted by
dim(V ) and is defined to be the number of vectors in a basis for V .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 4 / 19
Recall Dimension
Remark
Using the previous Theorem, all bases for a finite-dimensional vector
space have the same number of vectors.
Definition 3
The dimension of a finite-dimensional vector space V is denoted by
dim(V ) and is defined to be the number of vectors in a basis for V .
In addition, the zero vector space is defined to have dimension zero.
Definition 3
The dimension of a finite-dimensional vector space V is denoted by
dim(V ) and is defined to be the number of vectors in a basis for V .
In addition, the zero vector space is defined to have dimension zero.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 5 / 19
Definition 4
If S = {v1 , v2 , . . . , vn } is a basis for a real vector space V , and
v = c1 v1 + c2 v2 + · · · + cn vn is the expression for a vector v in
terms of the basis S, then the scalars c1 , c2 , . . . , cn are called the
coordinates of v relative to the basis S.
The vector (c1 , c2 , . . . , cn ) in Rn constructed from these coordinates is
called the coordinate vector of v relative to S; it is denoted by
(v)S = (c1 , c2 , . . . , cn ).
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 5 / 19
Definition 4
If S = {v1 , v2 , . . . , vn } is a basis for a real vector space V , and
v = c1 v1 + c2 v2 + · · · + cn vn is the expression for a vector v in
terms of the basis S, then the scalars c1 , c2 , . . . , cn are called the
coordinates of v relative to the basis S.
The vector (c1 , c2 , . . . , cn ) in Rn constructed from these coordinates is
called the coordinate vector of v relative to S; it is denoted by
(v)S = (c1 , c2 , . . . , cn ).
Remark
Observe that (v)S is a vector in Rn , so that once an ordered basis S
is given for a real vector space V , Uniqueness of Basis Representation
establishes a one-to-one correspondence between vectors in V and
vectors in Rn .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 5 / 19
Definition 4
If S = {v1 , v2 , . . . , vn } is a basis for a real vector space V , and
v = c1 v1 + c2 v2 + · · · + cn vn is the expression for a vector v in
terms of the basis S, then the scalars c1 , c2 , . . . , cn are called the
coordinates of v relative to the basis S.
The vector (c1 , c2 , . . . , cn ) in Rn constructed from these coordinates is
called the coordinate vector of v relative to S; it is denoted by
(v)S = (c1 , c2 , . . . , cn ).
Remark
Observe that (v)S is a vector in Rn , so that once an ordered basis S
is given for a real vector space V , Uniqueness of Basis Representation
establishes a one-to-one correspondence between vectors in V and
vectors in Rn .
v ∈ V ←→ (v)S ∈ Rn
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 5 / 19
Examples:Coordinate Vectors Relative to Standard Bases
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 6 / 19
Examples:Coordinate Vectors Relative to Standard Bases
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 6 / 19
Examples:Coordinate Vectors Relative to Standard Bases
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 6 / 19
The Change-of-Basis Problem
The Change-of-Basis Problem
If v is a vector in a finite-dimensional vector space V , and if we
change the basis for V from a basis B = {u1 , u2 , . . . , un } to a basis
B 0 = {u01 , u02 , . . . , u0n }, how are the coordinate vectors [v]B and [v]B 0
related?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 7 / 19
The Change-of-Basis Problem
The Change-of-Basis Problem
If v is a vector in a finite-dimensional vector space V , and if we
change the basis for V from a basis B = {u1 , u2 , . . . , un } to a basis
B 0 = {u01 , u02 , . . . , u0n }, how are the coordinate vectors [v]B and [v]B 0
related?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 7 / 19
The Change-of-Basis Problem
The Change-of-Basis Problem
If v is a vector in a finite-dimensional vector space V , and if we
change the basis for V from a basis B = {u1 , u2 , . . . , un } to a basis
B 0 = {u01 , u02 , . . . , u0n }, how are the coordinate vectors [v]B and [v]B 0
related?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 7 / 19
The Change-of-Basis Matrix
A Procedure for Computing PB→B 0
• Step 1. Form the matrix [B 0 |B].
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 8 / 19
The Change-of-Basis Matrix
A Procedure for Computing PB→B 0
• Step 1. Form the matrix [B 0 |B].
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 8 / 19
The Change-of-Basis Matrix
A Procedure for Computing PB→B 0
• Step 1. Form the matrix [B 0 |B].
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 8 / 19
The Change-of-Basis Matrix
A Procedure for Computing PB→B 0
• Step 1. Form the matrix [B 0 |B].
• Step 4. Extract the matrix PB→B 0 from the right side of the
matrix in Step 3.
Theorem 5
Let B = {u1 , u2 , . . . , un } be any basis of Rn and let
S = {e1 , e2 , . . . , en } be the standard matrix basis for Rn . Then,
PB→S = [u1 | u2 | . . . | un ]
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 8 / 19
Example 6
Consider the bases B = {u1 , u2 , u3 } and B 0 = {u01 , u02 , u03 }, where
2 2 1
u1 = 1 , u2 = −1 , u3 = 2 ,
1 1 1
3 1 −1
0 0 0
u1 = 1 , u2 = 1 , u3 = 0 ,
−5 −3 2
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 10 / 19
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 11 / 19
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 12 / 19
Thus,
3 2 2.5
PB→B 0 = −2 −3 −0.5
5 1 6
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 13 / 19
−5
(2) When w = 8 , we have [w]B =
−5
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 14 / 19
−5 9
(2) When w = 8 , we have [w]B = −9, (Why?)
−5 −5
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 14 / 19
−5 9
(2) When w = 8 , we have [w]B = −9, (Why?)
−5 −5
since w = 9u1 − 9u2 − 5u1 .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 14 / 19
−5 9
(2) When w = 8 , we have [w]B = −9, (Why?)
−5 −5
since w = 9u1 − 9u2 − 5u1 .
Using transition matrix PB→B 0 [w]B , [w]B 0 =?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 14 / 19
−5 9
(2) When w = 8 , we have [w]B = −9, (Why?)
−5 −5
since w = 9u1 − 9u2 − 5u1 .
Using transition matrix PB→B 0 [w]B , [w]B 0 =?
3 2 2.5 9 −3.5
[w]B 0 = PB→B 0 [w]B = −2 −3 −0.5 −9 = 11.5
5 1 6 −5 6
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 14 / 19
−5 9
(2) When w = 8 , we have [w]B = −9, (Why?)
−5 −5
since w = 9u1 − 9u2 − 5u1 .
Using transition matrix PB→B 0 [w]B , [w]B 0 =?
3 2 2.5 9 −3.5
[w]B 0 = PB→B 0 [w]B = −2 −3 −0.5 −9 = 11.5
5 1 6 −5 6
(3) Direct: If
−5 x −3.5
xu01 + yu02 + zu03 = 8 =⇒ [w]B 0 = y = 11.5 ,
−5 z 6
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 14 / 19
−5 9
(2) When w = 8 , we have [w]B = −9, (Why?)
−5 −5
since w = 9u1 − 9u2 − 5u1 .
Using transition matrix PB→B 0 [w]B , [w]B 0 =?
3 2 2.5 9 −3.5
[w]B 0 = PB→B 0 [w]B = −2 −3 −0.5 −9 = 11.5
5 1 6 −5 6
(3) Direct: If
−5 x −3.5
xu01 + yu02 + zu03 = 8 =⇒ [w]B 0 = y = 11.5 ,
−5 z 6
3 1 −1 −5
7 23
OR − 1 + 1 + 6 0 = 8
2 −5 2 −3 2 −5
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 14 / 19
Recall: Row Space, Column Space, and Null Space
Definition 7
For a given m × n real matrix A
• Row Space ?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 15 / 19
Recall: Row Space, Column Space, and Null Space
Definition 7
For a given m × n real matrix A
• Row Space ?
• Column Space ?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 15 / 19
Recall: Row Space, Column Space, and Null Space
Definition 7
For a given m × n real matrix A
• Row Space ?
• Column Space ?
• Null Space?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 15 / 19
Recall: Row Space, Column Space, and Null Space
Definition 7
For a given m × n real matrix A
• Row Space ?
• Column Space ?
• Null Space?
Question 1
What relationships exist among the solutions of a linear system
Ax = b and the row space, column space, and null space of the
coefficient matrix A?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 15 / 19
Recall: Row Space, Column Space, and Null Space
Definition 7
For a given m × n real matrix A
• Row Space ?
• Column Space ?
• Null Space?
Question 1
What relationships exist among the solutions of a linear system
Ax = b and the row space, column space, and null space of the
coefficient matrix A?
Question 2
What relationships exist among the row space, column space, and null
space of a matrix?
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 15 / 19
Theorem 8
A system of linear equations Ax = b is consistent if and only if b is in
the column space of A.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 16 / 19
Theorem 8
A system of linear equations Ax = b is consistent if and only if b is in
the column space of A.
Theorem 9
If x0 is any solution of a consistent linear system Ax = b, and if
S = {v1 , v2 , . . . , vk } is a basis for the null space of A, then every
solution of Ax = b can be expressed in the form
x = x 0 + c1 v 1 + c2 v 2 + · · · + ck v k .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 16 / 19
Theorem 8
A system of linear equations Ax = b is consistent if and only if b is in
the column space of A.
Theorem 9
If x0 is any solution of a consistent linear system Ax = b, and if
S = {v1 , v2 , . . . , vk } is a basis for the null space of A, then every
solution of Ax = b can be expressed in the form
x = x 0 + c1 v 1 + c2 v 2 + · · · + ck v k .
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 16 / 19
Basis for row space/column space
Theorem 10
Elementary row operations do not change the null space of a matrix.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 17 / 19
Basis for row space/column space
Theorem 10
Elementary row operations do not change the null space of a matrix.
Theorem 11
Elementary row operations do not change the row space of a matrix.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 17 / 19
Basis for row space/column space
Theorem 10
Elementary row operations do not change the null space of a matrix.
Theorem 11
Elementary row operations do not change the row space of a matrix.
Theorem 12
If a matrix R is in row echelon form, then the row vectors with the
leading 1’ s (the nonzero row vectors) form a basis for the row space
of R,
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 17 / 19
Basis for row space/column space
Theorem 10
Elementary row operations do not change the null space of a matrix.
Theorem 11
Elementary row operations do not change the row space of a matrix.
Theorem 12
If a matrix R is in row echelon form, then the row vectors with the
leading 1’ s (the nonzero row vectors) form a basis for the row space
of R,
and the column vectors with the leading 1’ s of the row vectors form a
basis for the column space of R.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 17 / 19
Example
Example 13
Find bases for the row and column spaces of the matrix
1 −2 5 0 3
0 1 3 0 0
R= 0 0 0 1 0
0 0 0 0 0
Solution: r1 = [1 − 2 5 0 3]
r2 = [0 1 3 0 0]
r3 = [0 0 0 1 0]
form a basis of row space of R and the set {c1 , c2 , c4 } form a basis
for the column space R.
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 18 / 19
Example 14
Find basis for the row space of the matrix
1 −3 4 −2 5 4
2 −6 9 −1 8 2
R= −1 3 −4 2 −5 −4
2 −6 9 −1 9 7
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 19 / 19
Example 14
Find basis for the row space of the matrix
1 −3 4 −2 5 4
2 −6 9 −1 8 2
R= −1 3 −4 2 −5 −4
2 −6 9 −1 9 7
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 19 / 19
Example 14
Find basis for the row space of the matrix
1 −3 4 −2 5 4
2 −6 9 −1 8 2
R= −1 3 −4 2 −5 −4
2 −6 9 −1 9 7
Mukesh Kumar Nagar (BITS Pilani Goa) MATH F112 Vector Spaces January 31, 2025 19 / 19
Example 14
Find basis for the row space of the matrix
1 −3 4 −2 5 4
2 −6 9 −1 8 2
R= −1 3 −4 2 −5 −4
2 −6 9 −1 9 7