Lecture Plan

• Eigenvalues and Eigenvectors

• Solution of a linear system of algebraic equations

Intended Learning Outcomes

• Understanding of Eigenvalues and Eigenvectors

• Application of eigenvectors

• What does a solution of a system of algebraic equations

Eigenvectors
• Key topic in Linear Algebra

• Heavily used in datascience

• A special type of vectors to the considered matrix

1 - clustering
2 - Facial recognition
3 - Google page rank
4 - Eigen decomposition
5 - PCA / SVD

Recall Basis

The basis of a vector space is the set of all linearly

independent vectors that span the full space.
System of linear equations
• The key objective of linear algebra is to solve a system of
algebraic equations
• Let us consider two equations with two unknowns

Is
Here, x & y in equations (1) and (2) are the unknowns. (solution)
to be calculated.

• The above set of algebraic equations (1) & (2) can be written
as
VColumn
Row

Let

Then the system of linear equations can be written as

Here, A is a matrix, X is an unknown vector and b is a given vector.

Is 2Sinet Cosy 3
a diner
e logy systafega
 Recall linearly independent, Span, basis, etc from
vector space

Column space of a Matrix

The column space of A is all linear combination of columns of A, i.e.,

span of columns.

Let A be a 4 x 3 matrix:

Ti

The column C_i is a subspace of

Can the Columns of the matrix A form a Subspace?

Can the linear combinations of these Columns fill

the four-dimensional space?

Is there any relation between the column space and

the solution of the system of algebraic equations?
Basis of a Column space
The set of all linearly independent column vector forms the basis
for the column space

Column Rank of a matrix

The dimension of the column space of a matrix A is the rank of the
matrix A.

Row Rank of a matrix

The dimension of the row space of a matrix A is the rank of the
matrix A.

Properties • Let A be a real matrix of size m x n.

• The column rank and row rank are alway equal.

• rank (A) = rank (A^T) = rank (A^T A) = rank (A A^T)

• The rank of a matrix A is a nonnegative integer, and

n
rank A A
1min m

• a matrix that has rank min (m, n) is said to have full rank,

else the matrix is rank de cient

• Only zero matrix has rank zero

/

• a matrix that has rank min (m, n) is said to have full rank,

else the matrix is rank de cient

• Only zero matrix has rank zero

• Let B be an n x k matrix, then

rank(AB) min( rank(A), rank(B) )

what does the rank of a

data meant
Recall Matrix

Matrix is a linear transformation of (basis) vector.

Rotation 3 2 I oof
T.EE it

In general 1 8 6

Shear:

How about
Summary
matrix rotates and or streatchs

a vector when acts on it

Is there vector
any
that does not rotate when transformed

by a matrix

Ans eigenvectors
Eigenvalues & Eigenvectors

Let A be a square matrix. Multiply the matrix A by a vector

"X”, where AX is parallel to X.

it
i

Unknown
Such a vector x is an eigenvector, where lambda is an eigenvalue.

Let A

Ax 313 35 137 aka

let K Ax 3 3 326

x Y I A 2x

eigenvalues
own
its
on span
T.to
FBveoyn

eigenvectors
 Recall
0
LHS matrix vector multiplication

RHS scalar times a vector

looks awkward

Rewrite

A x
Ilse
Ax XIX D

A XI x o

. what does it mean

It will always be true when 2 0

how about
for a to
 The only possible
way
is dit CA XI O

Find an appropriate value

of X such that del A 211 0

Remark

A matrix need not have eigenvectors

always

Ey f rotation matrix

x i o
1
a Ii

no eigenvectors exist
 A matrix has one eigenvalues

with
many eigenvectors
x i
eg o

Howatthignenuatuereigintedons
diagonal
D
a
of
motif a EIR

X a vector
every
in the xy plane is an
eigenvector

all
basis vectors are eigenvectors
with diagonal entry being their eigenvalues
What is the significance of eigenvectors
in Data matrix
input
I
Let A xp
The matrix A push the

input vector x towards the dominant

eigenvector
target

A Age y
g
unseen data

Application
Pagerank axis
of rotation
Pagerank Algorithm

How to rank webpages

a that gets more referrals
page

IE

IFg

the transition matrix

0 0 I 42

We have 4 nodes have uniform

1
 ne ally V 14
14 1 14 4

New importance U AV

ftp.t

0 33

A've at
839
Alternatively let 2 741 2,71314

be the importance of the nodes

then the pagerank is

A x ax with 2 1

The eigenvector o 38 0.12 0.29 0.195

is the Pagerank