Mathematical Methods

in Electrical Engineering
Lecture 𝟎𝟏 – Vector Spaces

Dr. Elie Abou Diwan

 Vector Space
DEFINITION:
Let 𝑉 be a set on which two operations are defined: Addition (+), and multiplication by scalar
(. ). We say that the triplet (𝑉, +, . ) is a real vector space if the following 10 axioms are satisfied:

• AXIOM 𝟏: 𝑉, + is closed: If 𝒖, 𝒗 ∈ 𝑉 then 𝒖 + 𝒗 ∈ 𝑉 (if you add two elements of 𝑉 together,

you are guaranteed to get an element of 𝑉 as the sum)
• AXIOM 𝟐: + is commutative: ∀ 𝒖, 𝒗 ∈ 𝑉 then 𝒖 + 𝒗 = 𝒗 + 𝒖
• AXIOM 𝟑: + is associative: ∀ 𝒖, 𝒗, 𝒘 ∈ 𝑉 then 𝒖 + 𝒗 + 𝒘 = 𝒖 + 𝒗 + 𝒘
• AXIOM 𝟒: Zero vector (identity element): ∃ an element in 𝑉 called the zero vector and
denoted by 𝟎𝑽 such that 𝟎𝑽 + 𝒖 = 𝒖 + 𝟎𝑽 = 𝒖 ∀ 𝒖 ∈ 𝑉
• AXIOM 𝟓: Opposite vector: ∀ 𝒖 ∈ 𝑉, ∃ an element 𝒖′ ∈ 𝑉 such that 𝒖 + 𝒖′ = 𝒖′ + 𝒖 = 𝟎𝑽 ;
the vector 𝒖′ is denoted by −𝒖
 Vector Space (cont.)
• AXIOM 𝟔: 𝑉, . is closed: If 𝛼 ∈ R and 𝒖 ∈ 𝑉 then 𝛼. 𝒖 ∈ 𝑉
• AXIOM 𝟕: Distributivity of scalar multiplication with respect to vector addition: ∀ 𝛼 ∈ R and 𝒖,
𝒗 ∈ 𝑉 then 𝛼. 𝒖 + 𝒗 = 𝛼. 𝒖 + 𝛼. 𝒗
• AXIOM 𝟖: Distributivity of scalar multiplication with respect to field addition: ∀ 𝛼, 𝛽 ∈ R and
𝒖 ∈ 𝑉 then 𝛼 + 𝛽 . 𝒖 = 𝛼. 𝒖 + 𝛽. 𝒖
• AXIOM 𝟗: Compatibility of scalar multiplication with field multiplication: ∀ 𝛼, 𝛽 ∈ R and 𝒖 ∈ 𝑉
then 𝛼. 𝛽. 𝒖 = 𝛼. 𝛽 . 𝒖
• AXIOM 𝟏𝟎: Identity element of scalar multiplication: ∀ 𝒖 ∈ 𝑉 then 1. 𝒖 = 𝒖

REMARK:
If the scalars 𝛼 and 𝛽 are complex numbers, then the vector space is called a complex vector
space.
 Vector Space (cont.)
EXAMPLE 𝟏:
The simplest example of a vector space is 𝑉 = 𝟎 , called the zero vector space.

EXAMPLE 𝟐:
Let 𝑉 = R𝑛 = 𝑥1 , 𝑥2 , … , 𝑥𝑛 |𝑥𝑖 ∈ R , define + and . as follows:
If 𝒖 = 𝑥1 , 𝑥2 , … , 𝑥𝑛 , 𝒗 = 𝑦1 , 𝑦2 , … , 𝑦𝑛 and 𝛼 ∈ R, then:
𝒖 + 𝒗 = 𝑥1 + 𝑦1 , 𝑥2 + 𝑦2 , … , 𝑥𝑛 + 𝑦𝑛 and 𝛼. 𝒖 = 𝛼. 𝑥1 , 𝛼. 𝑥2 , … , 𝛼. 𝑥𝑛
(𝑉, +, . ) is a vector space, the zero vector is 𝟎𝑽 = 0,0, … , 0 and the opposite vector of 𝒖 is
𝒖′ = −𝒖 = −𝑥1 , −𝑥2 , … , −𝑥𝑛
 Subspace
DEFINITION:
Let 𝑉 be a vector space. A subset 𝑊 of 𝑉 is called a subspace if:

• CONDITION 𝟏: 𝑊 ≠ ∅ (can't be empty because it must include the zero vector)

• CONDITION 𝟐: 𝑊, + is closed: If 𝒖, 𝒗 ∈ 𝑊 then 𝒖 + 𝒗 ∈ 𝑊
• CONDITION 𝟑: 𝑊, . is closed: If 𝛼 ∈ R and 𝒖 ∈ 𝑊 then 𝛼. 𝒖 ∈ 𝑊

EXAMPLE 𝟏:
If 𝑉 is a vector space, then 𝟎𝑽 and 𝑉 are subspaces of 𝑉.
 Subspace (cont.)
EXAMPLE 𝟐:
𝑥 𝑦 𝑧
Let 𝐻 = 𝑥, 𝑦, 𝑧 |𝑎. 𝑥 + 𝑏. 𝑦 + 𝑐. 𝑧 = 0 and 𝐾 = 𝑥, 𝑦, 𝑧 | = = , then 𝐻 and 𝐾 are
𝑎 𝑏 𝑐
subspaces of R3 .

Proof:
• 𝐻 ≠ ∅ because 0,0,0 satisfies 𝑎. 0 + 𝑏. 0 + 𝑐. 0 = 0 (COND. 𝟏 verified)
• Let 𝒖𝟏 = 𝑥1 , 𝑦1 , 𝑧1 and 𝒖𝟐 = 𝑥2 , 𝑦2 , 𝑧2 ∈ 𝐻 and consider 𝒖𝟏 + 𝒖𝟐 .
𝒖𝟏 = 𝑥1 , 𝑦1 , 𝑧1 ∈ 𝐻 ⇒ 𝑎. 𝑥1 + 𝑏. 𝑦1 + 𝑐. 𝑧1 = 0
𝒖𝟐 = 𝑥2 , 𝑦2 , 𝑧2 ∈ 𝐻 ⇒ 𝑎. 𝑥2 + 𝑏. 𝑦2 + 𝑐. 𝑧2 = 0
𝑎. 𝑥1 + 𝑥2 + 𝑏. 𝑦1 + 𝑦2 + 𝑐. 𝑧1 + 𝑧2 = 0 + 0 = 0 ⇒ 𝒖𝟏 + 𝒖𝟐 ∈ 𝐻 (COND. 𝟐 verified)
• Let 𝒖 = 𝑥, 𝑦, 𝑧 ∈ 𝐻 and 𝛼 ∈ R and consider 𝛼. 𝒖.
𝒖 = 𝑥, 𝑦, 𝑧 ∈ 𝐻 ⇒ 𝑎. 𝑥 + 𝑏. 𝑦 + 𝑐. 𝑧 = 0
𝑎. 𝛼. 𝑥 + 𝑏. 𝛼. 𝑦 + 𝑐. 𝛼. 𝑧 = 𝛼. 𝑎. 𝑥 + 𝑏. 𝑦 + 𝑐. 𝑧 = 𝛼. 0 = 0 ⇒ 𝛼. 𝒖 ∈ 𝐻 (COND. 𝟑 verified)
CONCLUSION: 𝐻 is a subspace of R3 .
 Subspace (cont.)
Proof:
0 0 0
• 𝐾 ≠ ∅ because 0,0,0 satisfies = = (COND. 𝟏 verified)
𝑎 𝑏 𝑐
• Let 𝒖𝟏 = 𝑥1 , 𝑦1 , 𝑧1 and 𝒖𝟐 = 𝑥2 , 𝑦2 , 𝑧2 ∈ 𝐾 and consider 𝒖𝟏 + 𝒖𝟐 .
𝑥1 𝑦1 𝑧1
𝒖𝟏 = 𝑥1 , 𝑦1 , 𝑧1 ∈ 𝐾 ⇒ = =
𝑎 𝑏 𝑐
𝑥2 𝑦2 𝑧2
𝒖𝟐 = 𝑥2 , 𝑦2 , 𝑧2 ∈ 𝐾 ⇒ = =
𝑎 𝑏 𝑐
𝑥1 + 𝑥2 𝑦1 + 𝑦2 𝑧1 + 𝑧2 𝑥1 𝑥2 𝑦1 𝑦2 𝑧1 𝑧2
= = ⇒ + = + = + true
𝑎 𝑏 𝑐 𝑎 𝑎 𝑏 𝑏 𝑐 𝑐
⇒ 𝒖𝟏 + 𝒖𝟐 ∈ 𝐾 (COND. 𝟐 verified)

• Let 𝒖 = 𝑥, 𝑦, 𝑧 ∈ 𝐾 and 𝛼 ∈ R and consider 𝛼. 𝒖.

𝑥 𝑦 𝑧
𝒖 = 𝑥, 𝑦, 𝑧 ∈ 𝐾 ⇒ = =
𝑎 𝑏 𝑐
𝛼. 𝑥 𝛼. 𝑦 𝛼. 𝑧
= = true ⇒ 𝛼. 𝒖 ∈ 𝐾 (COND. 𝟑 verified) CONCLUSION: 𝐾 is a subspace of R3
𝑎 𝑏 𝑐
 Subspace (cont.)
APPLICATION:
Let 𝑆 = 𝑥1 , 𝑥2 , 𝑥3 |2𝑥1 + 3𝑥2 − 4𝑥3 = 0 .
a. Find three distinct elements of 𝑺.
0,0,0 , 3, −2,0 and 0,4,3 ∈ 𝑆.
b. Show that 𝑺 is a subspace of 𝐑𝟑 .
• 𝑆 ≠ ∅ because 0,0,0 satisfies 2 0 + 3 0 − 4 0 = 0 (COND. 𝟏 verified)
• Let 𝒖𝟏 = 𝑥1 , 𝑥2 , 𝑥3 and 𝒖𝟐 = 𝑦1 , 𝑦2 , 𝑦3 ∈ 𝑆 and consider 𝒖𝟏 + 𝒖𝟐 .
𝒖𝟏 = 𝑥1 , 𝑥2 , 𝑥3 ∈ 𝑆 ⇒ 2𝑥1 + 3𝑥2 − 4𝑥3 = 0
𝒖𝟐 = 𝑦1 , 𝑦2 , 𝑦3 ∈ 𝑆 ⇒ 2𝑦1 + 3𝑦2 − 4𝑦3 = 0
2 𝑥1 + 𝑦1 + 3 𝑥2 + 𝑦2 − 4 𝑥3 + 𝑦3 = 0 + 0 = 0 ⇒ 𝒖𝟏 + 𝒖𝟐 ∈ 𝑆 (COND. 𝟐 verified)
• Let 𝒖 = 𝑥1 , 𝑥2 , 𝑥3 ∈ 𝑆 and 𝛼 ∈ R and consider 𝛼. 𝒖.
𝒖 = 𝑥1 , 𝑥2 , 𝑥3 ∈ 𝑆 ⇒ 2𝑥1 + 3𝑥2 − 4𝑥3 = 0
2 𝛼𝑥1 + 3 𝛼𝑥2 − 4 𝛼𝑥3 ⇒ 𝛼 2𝑥1 + 3𝑥2 − 4𝑥3 = 𝛼(0) = 0 ⇒ 𝛼. 𝒖 ∈ 𝑆 (COND. 𝟑 verified)
Finally, by the subspace theorem, 𝑆 is a subspace of R3 .
 Linear Combination
DEFINITION:
A linear combination of the vectors 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 is of the form 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + ⋯ + 𝛼𝑛 𝒗𝒏 where
𝛼1 , 𝛼2 , … , 𝛼𝑛 are scalars.

REMARK: Any vector 𝒗 = 𝑥, 𝑦, 𝑧 ∈ R3 is a linear combination of the vectors 𝒆𝟏 = 1,0,0 ,

𝒆𝟐 = 0,1,0 and 𝒆𝟑 = 0,0,1 . Consequently, 𝒗 = 𝑥𝒆𝟏 + 𝑦𝒆𝟐 + 𝑧𝒆𝟑 .

THEOREM:
Let 𝑉 be a vector space and let 𝑊 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 be a subset of 𝑉. We say that 𝑊 spans 𝑉 (or
generates 𝑉) if every vector 𝒗 ∈ 𝑉 can be written as a linear combination of vectors in 𝑊. That is,
𝒗 = 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + ⋯ + 𝛼𝑛 𝒗𝒏 .

EXAMPLE:
𝒆𝟏 , 𝒆𝟐 and 𝒆𝟑 span R3 because we can construct any vector in R3 by some linear combination of
these three vectors. Furthermore, other vectors can be found to span R3 .
 Linear Combination (cont.)
APPLICATION 𝟏:
Determine whether the vectors 𝒗𝟏 = 1, −1,4 , 𝒗𝟐 = −2,1,3 and 𝒗𝟑 = 4, −3,5 span R3 .

By the span theorem, 𝒗𝟏 , 𝒗𝟐 and 𝒗𝟑 span R3 if we can construct any vector 𝒗 = 𝑎, 𝑏, 𝑐 in R3 by

some linear combination of these three vectors. Consequently, we must verify that ∀
𝛼1 , 𝛼2 and 𝛼3 , 𝒗 = 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + 𝛼3 𝒗𝟑 .

𝑎 1 −2 4
𝑏 = 𝛼1 −1 + 𝛼2 1 + 𝛼3 −3
𝑐 4 3 5
α1 − 2α2 + 4α3 = 𝑎 ①
⇒ −α1 + α2 − 3α3 = 𝑏 ②
4α1 + 3α2 + 5α3 = 𝑐 ③
 Linear Combination (cont.)
α1 − 2α2 + 4α3 = 𝑎 ① ① α1 − 2α2 + 4α3 = 𝑎 ①
⇒ −α1 + α2 − 3α3 = 𝑏 ② ⇒ ①+② −α2 + α3 = 𝑎 + 𝑏 ④
4α1 + 3α2 + 5α3 = 𝑐 ③ −4① + ③ 11α2 − 11α3 = −4𝑎 + 𝑐 ⑤

① α1 − 2α2 + 4α3 = 𝑎
⇒ ④ −α2 + α3 = 𝑎 + 𝑏
11④ + ⑤ 7𝑎 + 11𝑏 + 𝑐 = 0

This system has a solution only when 7𝑎 + 11𝑏 + 𝑐 = 0 . Consequently, 𝒗𝟏 = 1, −1,4 ,

𝒗𝟐 = −2,1,3 and 𝒗𝟑 = 4, −3,5 do not span R3 .
 Linear Combination (cont.)
APPLICATION 𝟐:
Determine whether the vectors 𝒗𝟏 = 1, −1,2 , 𝒗𝟐 = 2,1,3 and 𝒗𝟑 = −1,0,2 span R3 .

By the span theorem, 𝒗𝟏 , 𝒗𝟐 and 𝒗𝟑 span R3 if we can construct any vector 𝒗 = 𝑎, 𝑏, 𝑐 in R3 by

some linear combination of these three vectors. Consequently, we must verify that ∀
𝛼1 , 𝛼2 and 𝛼3 , 𝒗 = 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + 𝛼3 𝒗𝟑 .

𝑎 1 2 −1
𝑏 = 𝛼1 −1 + 𝛼2 1 + 𝛼3 0
𝑐 2 3 2
α1 + 2α2 − α3 = 𝑎 ①
⇒ −α1 + α2 = 𝑏 ②
2α1 + 3α2 + 2α3 = 𝑐 ③
 Linear Combination (cont.)
α1 + 2α2 − α3 = 𝑎 ① ① α1 + 2α2 − α3 = 𝑎 ①
⇒ −α1 + α2 = 𝑏 ② ⇒ ①+② 3α2 − α3 = 𝑎 + 𝑏 ④
2α1 + 3α2 + 2α3 = 𝑐 ③ −2① + ③ −α2 + 4α3 = −2𝑎 + 𝑐 ⑤

1
① α1 + 2α2 − α3 = 𝑎 ① ⑥ α3 = −5𝑎 + 𝑏 + 3𝑐
11
⇒ ④ 3α2 − α3 = 𝑎 + 𝑏 ④ ⇒ 1
④ α2 = 𝑎 + 𝑏 + α3
④ + 3⑤ 11α3 = −5𝑎 + 𝑏 + 3𝑐 ⑥ 3
① α1 = 𝑎 − 2α2 + α3

The three vectors 𝒗𝟏 = 1, −1,2 , 𝒗𝟐 = 2,1,3 and 𝒗𝟑 = −1,0,2 span R3 because no matter
what 𝑎, 𝑏 and 𝑐 you give me, I can give you an 𝛼1 , 𝛼2 and 𝛼3 .
 Linear Independence
DEFINITION:
• A set of vectors 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 is said to be linearly dependent if there exists 𝑛 scalars
𝛼1 , 𝛼2 , … , 𝛼𝑛 (not all zeros) such that 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + ⋯ + 𝛼𝑛 𝒗𝒏 = 𝟎
• A set of vectors 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 is said to be linearly independent if the only relation between
the vectors is the trivial relation 0𝒗𝟏 + 0𝒗𝟐 + ⋯ + 0𝒗𝒏 = 𝟎 i.e.
𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + ⋯ + 𝛼𝑛 𝒗𝒏 = 𝟎 ⇒ 𝛼1 = 𝛼2 = ⋯ = 𝛼𝑛 = 0

REMARKS:

REMARK 𝟏: The empty set is linearly independent.

REMARK 𝟐: For a set 𝑆 = 𝒗 , if 𝒗 ≠ 𝟎𝑽 then 𝒗 is linearly independent.

Proof: If 𝒗 ≠ 𝟎𝑽 , then the only scalar 𝛼 such that 𝛼𝒗 = 𝟎 is 𝛼 = 0. Consequently, 𝒗 is linearly
independent.
 Linear Independence (cont.)
REMARK 𝟑: If one of the vectors in the set 𝑆 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 in R𝑛 is the zero vector, then the
set is linearly dependent.
Proof: By renumbering the vectors, we may suppose that 𝒗𝟏 = 𝟎. Consequently, the equation
1𝒗𝟏 + 0𝒗𝟐 + ⋯ + 0𝒗𝒏 = 𝟎 shows that 𝑆 is linearly dependent.

APPLICATION:
Determine whether the vectors 𝒗𝟏 = 1, −1,2 , 𝒗𝟐 = 2,1,3 and 𝒗𝟑 = −1,0,2 are linearly
independent.

The vectors 𝒗𝟏 = 1, −1,2 , 𝒗𝟐 = 2,1,3 and 𝒗𝟑 = −1,0,2 are linearly independent if the
relation 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + 𝛼3 𝒗𝟑 = 𝟎 implies that 𝛼1 = 𝛼2 = 𝛼3 = 0.
 Linear Independence (cont.)
1 2 −1 0 ???
𝛼1 −1 + 𝛼2 1 + 𝛼3 0 = 0 ⇒ 𝛼1 = 𝛼2 = 𝛼3 = 0
2 3 2 0
1
α3 = −5𝑎 + 𝑏 + 3𝑐
11
From the previous application: 1
α2 = 𝑎 + 𝑏 + α3
3
α1 = 𝑎 − 2α2 + α3
α3 = 0
If we replace 𝑎, 𝑏 and 𝑐 by 0: α2 = 0
α1 = 0

The three vectors 𝒗𝟏 = 1, −1,2 , 𝒗𝟐 = 2,1,3 and 𝒗𝟑 = −1,0,2 are linearly independent
(none of the vectors can be represented as a linear combination of the other two).
 Basis
DEFINITION:
A set of vectors 𝐵 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 in a vector space 𝑉 is called a basis (plural bases) for 𝑉 if:
• CONDITION 𝟏: The vectors span 𝑉
• CONDITION 𝟐: The vectors are linearly independent

EXAMPLE:
𝐵 = 𝒆𝟏 , 𝒆𝟐 , 𝒆𝟑 where 𝒆𝟏 = 1,0,0 , 𝒆𝟐 = 0,1,0 and 𝒆𝟑 = 0,0,1 is a basis for R3 , since it
spans R3 and the vectors 𝒆𝟏 , 𝒆𝟐 and 𝒆𝟑 are linearly independent. This is called the standard basis
for R3 . In general, 𝒆𝟏 = 1,0,0, … , 0 , 𝒆𝟐 = 0,1,0, … , 0 , … , 𝒆𝒏 = 0,0,0, … , 1 is the standard
basis for R𝑛 .
 Basis (cont.)
APPLICATION 𝟏:
Show that the vectors 𝒗𝟏 = 1,2,1 , 𝒗𝟐 = 2,9,0 and 𝒗𝟑 = 3,3,4 form a basis for R3 .

STEP 𝟏: Determine whether the vectors 𝒗𝟏 = 1,2,1 , 𝒗𝟐 = 2,9,0 and 𝒗𝟑 = 3,3,4 span R3 .

𝑎 1 2 3
𝑏 = 𝛼1 2 + 𝛼2 9 + 𝛼3 3
𝑐 1 0 4

α1 + 2α2 + 3α3 = 𝑎 ①
⇒ 2α1 + 9α2 + 3α3 = 𝑏 ②
α1 + 4α3 = 𝑐 ③
 Basis (cont.)
α1 + 2α2 + 3α3 = 𝑎 ① ① α1 + 2α2 + 3α3 = 𝑎 ①
⇒ 2α1 + 9α2 + 3α3 = 𝑏 ② ⇒ −2① + ② 5α2 − 3α3 = −2𝑎 + 𝑏 ④
α1 + 4α3 = 𝑐 ③ −① + ③ −2α2 + α3 = −𝑎 + 𝑐 ⑤

① α1 + 2α2 + 3α3 = 𝑎 ① ⑥ α3 = 9𝑎 − 2𝑏 − 5𝑐
⇒ ④ 1
5α2 − 3α3 = −2𝑎 + 𝑏 ④ ⇒ ④ α2 = −2𝑎 + 𝑏 + 3α3
5
2④ + 5⑤ −α3 = −9𝑎 + 2𝑏 + 5𝑐 ⑥
① α1 = 𝑎 − 2α2 − 3α3

The three vectors 𝒗𝟏 = 1,2,1 , 𝒗𝟐 = 2,9,0 and 𝒗𝟑 = 3,3,4 span R3 because no matter what
𝑎, 𝑏 and 𝑐 you give me, I can give you an 𝛼1 , 𝛼2 and 𝛼3 .
 Basis (cont.)
STEP 𝟐: Determine whether the vectors 𝒗𝟏 = 1,2,1 , 𝒗𝟐 = 2,9,0 and 𝒗𝟑 = 3,3,4 are linearly
independent.
α3 = 9𝑎 − 2𝑏 − 5𝑐
1
From STEP 𝟏: α2 = −2𝑎 + 𝑏 + 3α3
5
α1 = 𝑎 − 2α2 − 3α3

α3 = 0
If we replace 𝑎, 𝑏 and 𝑐 by 0: α2 = 0
α1 = 0

The three vectors 𝒗𝟏 = 1,2,1 , 𝒗𝟐 = 2,9,0 and 𝒗𝟑 = 3,3,4 are linearly independent.

CONCLUSION: The vectors 𝒗𝟏 = 1,2,1 , 𝒗𝟐 = 2,9,0 and 𝒗𝟑 = 3,3,4 form a basis for R3 .
 Basis (cont.)
APPLICATION 𝟐:
Determine whether the vectors 𝒇𝟏 = 𝑥 2 − 1, 𝒇𝟐 = 𝑥 2 − 2 and 𝒇𝟑 = 𝑥 2 − 3 constitute a basis in
P2 .
???
α1 𝑥 2 − 1 + α2 𝑥 2 − 2 + α3 𝑥 2 − 3 = 0 ⇒ α1 = α2 = α3 = 0

𝑥 2 α1 + α2 + α3 − α1 − 2α2 − 3α3 = 0

α1 + α2 + α3 = 0 ① ① + ②: α2 = −2α3 ③
⇒
−α1 − 2α2 − 3α3 = 0 ② ① & ③: α1 = α3 ④
②, ③ & ④: 0α1 = 0 (Infinity of solutions)

CONCLUSION: The vectors 𝒇𝟏 , 𝒇𝟐 and 𝒇𝟑 are not linearly independent. Consequently, they do
not constitute a basis in P 2 .
 Basis (cont.)
DEFINITION:
A vector space 𝑉 is a finite dimensional if it has a finite basis.

THEOREM:
If 𝐵 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 and 𝐵′ = 𝒘𝟏 , 𝒘𝟐 , … , 𝒘𝒎 are two bases for the vector space 𝑉, then
𝑛 = 𝑚 (every basis for 𝑉 has the same number of vectors). The number of vectors in a basis for
𝑉 is called the dimension of 𝑉, denoted by 𝑑𝑖𝑚(𝑉). For example, the dimension of R𝑛 is 𝑛 since
the standard basis contains 𝑛 vectors.

REMARK:
If 𝑉 = 𝟎𝑽 , then 𝑉 has no basis because a set containing the zero vector is linearly dependent.
Consequently, 𝑑𝑖𝑚 𝑉 = 0.
 Basis (cont.)
THEOREM:
The span of a set of vectors is the set of all linear combinations of the vectors.
For example, 𝑠𝑝𝑎𝑛 𝒗𝟏 , 𝒗𝟐 = 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 |𝛼1 , 𝛼2 ∈ R .
In general, 𝑠𝑝𝑎𝑛 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 = 𝛼1 𝒗𝟏 + 𝛼2 𝒗𝟐 + ⋯ + 𝛼𝑛 𝒗𝒏 |𝛼1 , 𝛼2 , … , 𝛼𝑛 ∈ R .

THEOREM:
Let 𝑉 be a finite dimensional vector space and 𝐵 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 a set of vectors of dimension
𝑛 (𝑑𝑖𝑚(𝐵) = 𝑛).
• If 𝑑𝑖𝑚(𝑉) = 𝑛 (𝑉 has the same dimension as 𝐵) and if 𝐵 is linearly independent, then 𝐵 is a
basis for 𝑉.
• If 𝑑𝑖𝑚(𝑉) = 𝑛 (𝑉 has the same dimension as 𝐵) and if 𝐵 spans 𝑉 (we can construct any vector
of 𝑉 by some linear combination of 𝐵), then 𝐵 is a basis for 𝑉.
 Basis (cont.)
EXAMPLE:
The vectors 𝒗𝟏 = 1,1,0 , 𝒗𝟐 = 1,0,1 and 𝒗𝟑 = 0,1,1 form a basis for R3 .

Proof:
Let 𝐵 = 𝒗𝟏 , 𝒗𝟐 , 𝒗𝟑 .
𝑑𝑖𝑚 R3 = 𝑑𝑖𝑚 𝐵 = 3 (R3 has the same dimension as 𝐵).
1 1 0 0 ???
𝛼1 1 + 𝛼2 0 + 𝛼3 1 = 0 ⇒ 𝛼1 = 𝛼2 = 𝛼3 = 0
0 1 1 0
α1 + α2 = 0 α1 = 0
The vectors 𝒗𝟏 = 1,1,0 , 𝒗𝟐 = 1,0,1 and 𝒗𝟑 = 0,1,1 are
α1 + α3 = 0 ⇒ α2 = 0
α3 = 0 linearly independent.
α2 + α3 = 0
CONCLUSION: The vectors 𝒗𝟏 = 1,1,0 , 𝒗𝟐 = 1,0,1 and 𝑣3 = 0,1,1 form a basis for R3 .
 Orthogonal Bases
DEFINITION:
Consider the vector space R𝑛 , +, . .
The dot product (inner vector product) of two vectors 𝑿 = 𝑥1 , 𝑥2 , … , 𝑥𝑛 and
𝒀 = 𝑦1 , 𝑦2 , … , 𝑦𝑛 is:
𝑛

𝑿. 𝒀 = 𝑥𝑖 . 𝑦𝑖 = 𝑥1 . 𝑦1 + 𝑥2 . 𝑦2 + ⋯ + 𝑥𝑛 . 𝑦𝑛
𝑖=1

where denotes summation and 𝑛 is the dimension of the vector space.

 Orthogonal Bases (cont.)
REMARKS:

REMARK 𝟏: Two vectors 𝑿 and 𝒀 are said to be orthogonal if 𝑿. 𝒀 = 0

REMARK 𝟐: The length (or norm) of vector 𝑿 is 𝑿 = 𝑥12 + 𝑥22 + ⋯ + 𝑥𝑛2 = 𝑿. 𝑿

REMARK 𝟑: A vector 𝒖 is called a unit vector if its norm 𝒖 = 1

REMARK 𝟒: A set of vectors 𝑆 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 is said to be:

• Orthogonal: If and only if the vectors are pairwise orthogonal i.e. 𝒗𝒊 . 𝒗𝒋 = 0 ∀ 𝑖 ≠ 𝑗
• Orthonormal: If and only if it is orthogonal and the vectors are unit vectors 𝒗𝒊 = 1 ∀ 𝑖
 Orthogonal Bases (cont.)
THEOREM:
If 𝑆 is an orthogonal set of nonzero vectors (𝒗𝒊 ≠ 𝟎 ∀ 𝑖), then 𝑆 is linearly independent.

Proof:
Let 𝑆 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 be an orthogonal set of nonzero vectors.
We need to show that 𝐶1 𝒗𝟏 + 𝐶2 𝒗𝟐 + ⋯ + 𝐶𝑛 𝒗𝒏 = 𝟎 is only satisfied for 𝐶𝑖 = 0 for all 𝑖.
Let us manipulate the previous equation by multiplying it from both sides by 𝑣𝑖 :

𝒗𝒊 𝐶1 𝒗𝟏 + 𝐶2 𝒗𝟐 + ⋯ + 𝐶𝑖 𝒗𝒊 + ⋯ + 𝐶𝑛 𝒗𝒏 = 𝟎

Given that 𝑆 is an orthogonal set of nonzero vectors i.e. 𝒗𝒊 . 𝒗𝒋 = 0 ∀ 𝑖 ≠ 𝑗, we get: 𝒗𝒊 𝐶𝑖 𝒗𝒊 = 0

Since 𝒗𝒊 ≠ 𝟎 ∀ 𝑖 , then: 𝐶𝑖 = 0. This is true for 𝑖 = 1,2, … , 𝑛. Consequently, 𝑆 is linearly
independent.

COROLLARY: If 𝑆 is an orthonormal set, then 𝑆 is linearly independent.

 Orthogonal Bases (cont.)
THEOREM:
Let 𝑆 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 be a basis for a subspace 𝑉 of R𝑛 , then 𝑉 has an orthonormal basis.

(See APPENDIX 𝟏 for proof of this theorem and how to find the orthonormal basis)
 Orthogonal Functions
DEFINITION:
Let 𝑉 = 𝐶 𝑎, 𝑏 (set of continuous functions defined on the closed interval 𝑎, 𝑏 ). The inner
product of two functions 𝒇 and 𝒈 ∈ 𝑉 is defined by:

𝒇, 𝒈 = 𝒇 𝒙 𝒈 𝒙 𝑑𝑥
𝑎

EXAMPLE:
Let 𝒇 𝒙 = 1 + 𝑥 and 𝒈 𝒙 = 2 + 3𝑥 defined on 0,1 .
1 1 11
The inner product of 𝒇 and 𝒈 is: 𝒇, 𝒈 = 0 1 + 𝑥 2 + 3𝑥 𝑑𝑥 = 0
3𝑥 2 + 5𝑥 + 2 𝑑𝑥 =
2
1 2 𝑑𝑥 1 7
The length of 𝒇 is: 𝒇 = 𝒇, 𝒇 = 0
1+𝑥 = 0
𝑥2 + 2𝑥 + 1 𝑑𝑥 =
3
 Orthogonal Functions (cont.)
REMARK:
A set of functions 𝑆 = 𝒇𝟏 , 𝒇𝟐 , … , 𝒇𝒏 is said to be:
• Orthogonal over the interval 𝒂, 𝒃 : If and only if the functions are pairwise orthogonal i.e.
𝒇𝒊 , 𝒇 𝒋 = 0 ∀ 𝑖 ≠ 𝑗
• Orthonormal over the interval 𝒂, 𝒃 : If and only if it is orthogonal and the functions are
normalized i.e. 𝒇𝒊 = 𝒇𝒊 , 𝒇𝒊 = 1 ∀ 𝑖
 APPENDIX 𝟏
THEOREM:
Let 𝑆 = 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒏 be a basis for a subspace 𝑉 of R𝑛 , then 𝑉 has an orthonormal basis.

Proof:

STEP 𝟏: How to find the first unit vector of the orthonormal set
𝒗𝟏
Let 𝒗𝟏 ≠ 𝟎. The unit vector along the direction of 𝒗𝟏 is 𝒖𝟏 such that 𝒖𝟏 =
𝒗𝟏
 APPENDIX 𝟏 (cont.)
STEP 𝟐: How to find the second unit vector of the orthonormal set
Let 𝒗′𝟐 = 𝒗𝟐 − 𝑝𝑟𝑜𝑗𝒗𝟏 𝒗𝟐 (vector projection of 𝒗𝟐 onto 𝒗𝟏 )

𝒗′𝟐 = 𝒗𝟐 − 𝒗𝟐 cos 𝜃 𝒖𝟏 (vector projection = scalar projection × unit vector along the direction)

𝒗′𝟐 = 𝒗𝟐 − 𝒗𝟐 𝒖𝟏 cos 𝜃 𝒖𝟏

= 1 (length of unit vector 𝒖𝟏 )

𝒗′𝟐 = 𝒗𝟐 − (𝒗𝟐 . 𝒖𝟏 )𝒖𝟏

= 𝒗𝟐 𝒖𝟏 cos 𝜃 (dot product of two vectors)

𝒗′𝟐
The unit vector along the direction of 𝒗′𝟐 is 𝒖𝟐 such that 𝒖𝟐 =
𝒗′𝟐
 APPENDIX 𝟏 (cont.)
𝒗′𝟐 𝒖𝟏 .𝒗𝟐 −𝒖𝟏 .(𝒗𝟐 .𝒖𝟏 )𝒖𝟏 𝒖𝟏 .𝒗𝟐 −𝒗𝟐 .𝒖𝟏
Furthermore, 𝒖𝟏 . 𝒖𝟐 = 𝒖𝟏 . ′ = = =0
𝒗𝟐 𝒗′𝟐 𝒗′𝟐
Consequently, 𝒖𝟏 , 𝒖𝟐 is an orthonormal set (vectors are orthogonal and are unit vectors).

SKETCH:
𝒗′𝟐 𝒗𝟐

𝒗𝟐
𝒖𝟐
θ
𝒗𝟏
−𝑝𝑟𝑜𝑗𝒗𝟏 𝒗𝟐 𝒖𝟏 𝑝𝑟𝑜𝑗𝒗𝟏 𝒗𝟐
 APPENDIX 𝟏 (cont.)
STEP 𝟑: How to find the next unit vector of the orthonormal set (Induction step)
Assume that an orthonormal set 𝒖𝟏 , 𝒖𝟐 , … , 𝒖𝒌 was constructed, then how to find 𝒖𝒌+𝟏 ?

Let 𝒗′𝒌+𝟏 = 𝒗𝒌+𝟏 − 𝑝𝑟𝑜𝑗𝒗𝟏 𝒗𝒌+𝟏 − 𝑝𝑟𝑜𝑗𝒗𝟐 𝒗𝒌+𝟏 − ⋯ − 𝑝𝑟𝑜𝑗𝒗𝒌 𝒗𝒌+𝟏

Following the same procedure as in STEP 𝟐, we get:

𝒗′𝒌+𝟏 = 𝒗𝒌+𝟏 − 𝒗𝒌+𝟏 . 𝒖𝟏 𝒖𝟏 − 𝒗𝒌+𝟏 . 𝒖𝟐 𝒖𝟐 − ⋯ − 𝒗𝒌+𝟏 . 𝒖𝒌 𝒖𝒌

𝒗′𝒌+𝟏
The unit vector along the direction of 𝒗′𝒌+𝟏 is 𝒖𝒌+𝟏 such that 𝒖𝒌+𝟏 =
𝒗′𝒌+𝟏

Finally, 𝒖𝟏 , 𝒖𝟐 , … , 𝒖𝒌 , 𝒖𝒌+𝟏 is an orthonormal set.

 APPENDIX 𝟏 (cont.)
STEP 𝟒: How to find the orthonormal set of dimension 𝑛
Repeat STEP 𝟑 until we obtain an orthonormal set 𝒖𝟏 , 𝒖𝟐 , … , 𝒖𝒌 , 𝒖𝒌+𝟏 , … , 𝒖𝒏 of dimension 𝑛.

Since 𝒖𝟏 , 𝒖𝟐 , … , 𝒖𝒌 , 𝒖𝒌+𝟏 , … , 𝒖𝒏 is an orthonormal set, then 𝒖𝟏 , 𝒖𝟐 , … , 𝒖𝒌 , 𝒖𝒌+𝟏 , … , 𝒖𝒏 is

linearly independent. (see corollary slide 27)
Furthermore, 𝑑𝑖𝑚(𝑉) = 𝑛. Consequently, 𝒖𝟏 , 𝒖𝟐 , … , 𝒖𝒌 , 𝒖𝒌+𝟏 , … , 𝒖𝒏 is a basis of 𝑉. (see
theorem slide 23)