Lecture Plan

• Matrix in Data science, Linear Maps, Matrix,

Matrix-Vector and Matrix-Matrix multiplications
• Types of matrix, Determinant and inverse of a
matrix, Norm of a matrix
• Tensor, addition and subtraction of tensors
Kronecker product, Hardamard product

Intended Learning Outcomes

• Understanding of Matrices
• Representing data using matrices
• Matrix Operations
• Representing high-dimensional data
Major challenge in Data science

• How to represent and perform operation on

(high-)dimensional data?
• How to extract key features/ information from
high-dimensional data "ef ciently"?

use matrix algebra

Éf

e ca G
Functions and compositeI functions

Let Lex be a function with one

independent variable

2x KEIR
Eg Lex 5

Let had g of x where

Ge 506
g
x
fix hea g Lees
35
1 7
2 9 45
3 11 55
H 13 65

Are f L linear maps

Ice D Jay LCxty Yes fly

 linear
Let
f Go Dc 3 Is
f
Try L Canty I fastfly
WE Prove that
I
2x y Q LGD
fly LER
Actually
Jlaxty 2 2 0 3
I fed fly
Linear function in n-dimensional

Let see Rn then

if Cox e
fay
co
g o

Gets
f L GO
LG

l d fees moe MEIR

Scaling
 2 d Let 2 13
I Define L as

x fees f is 3
M
Eg
het se
2 y 2 L 3 Y

Xx fi's
Ex

It Can also be derived as

I 2 L 2 819

1819
2
f b
2 2 it
 4
It is enough to know
if
to find
I 9
f 9 a be R

Is there any Compact

form for
this mapping

can we write

1121 3 9 12
Is stretching a linear transformation

How about rotation 2

Matrix

Matrix is a linear transformation of (basis) vector.

É
Rotation

In general 3 313 a

Shear:

Dy
How about
De nition
Let m n EN A real valued
matrix A is an mn tuple of elements
ai ER I s it m is Jen which
is ordered as m rows d n columns

row
a a 9Heist
Am n nth
amn row

p p
nth Col
gsteal

men Square matrix

Matrix in Data science

Network

Adjacency matrix
2 3 4 5 6 7
I T I 0 0 0 1 T
2 I 1 O O O O O
3 0 0 1 0 1 I 0

A s y
5
I
i o o n o f

6 1 0 1 0 1 0

Most of the entries are zero

A is a sparse mat
can we store A efficiently
 Sparse matrix storage
Ex Compressed
Sparse Row CSR

Store only non zero entries

need two
arrays of
size N where n is
non zeros
not of
Data Matrix

Suppose we collect n number

of
features of a person we have n persons

istfeature
921
A
Amn
mthfeature
am Ama
i n person
1stperson nth
Matrix Addition

Amant Bmxn
Let A B E RMN
tb aint bin
At B
icinntion
an tbn

Can we add or subrart

matrices AERMM BEA
where men
C ATB

4th F
Matrix-vector multiplication

D
X
I

Ry 3 An 922 923

Rz 7 931 931 9333 3

3 zx

111 E D'D
that that multi

to L
1
11 11
M M
Shear rotation
 f

Ka
and
I T Ts
Ac Ae

where
a a
Gi D Ea
a an
scaleD

Note that
 EYE

WA Y Ca

use Column
form find C

AM
Ie II
11 I

I at al's 3
D
Identity matrix Cat He B
j t
The square matrix I is said to be an identity matrix
of
 Suppose A AT

A is symmetric

Multiplication of a matrix by a scalar.

Then where

Transpose of a matrix
ME
- writes Rows of A into Columns of A
Determinant of a matrix:

Al
ÉREY
3 2

Quanti es the change in the measure (area in 2d,

volume in 3d) due to linear transformation.

How about

How about

Same as above but the orientation got ipped

A EI
I Al y
LI
s
 5 2 51

Suppose
FIRE It

Suppose 1 A1 0
Singual matrix
Inverse of a matrix

Let A be a square matrix of order n. Suppose a

matrix B is said to be an inverse of A then

Further, B is denoted as A is unique

Suppose A is a ai clockwise

rotation then A is a go
Counter clockwise rotation

IL A is a leftward shear
then it is a highward shear
 Suppose we have

TT
known
Unknown
I known

We know the linear transformation (A), then question

is where should we start (x) to land at an expected
position (b)?

inverse transformation
Recall Norms of a vector:

Let X E IR ie x x sea
port

then I norm
of x is defined as

110011 is
71 boil

where 1 I is the absolute value

Hy la norm of se is defined as

1101122
Eh boil

Octoc
11041
Fix
How about the norms for matrix?
Column sum norm

Spectral norm

Where Imax is the largest eigenvalue

L_p norm

Spectral radius

Where a A denote the eigenvalues

A
of
Tensor

• How to represent three or more dimensional data?

xd
j Sy
yes
• How to represent in computer?

A JE JE
Tensor Addition / subtraction

Addition subsactin two tensors Lame

of of
size add subtract element wise
xd ax x dm
Let A B E Rd
die wthorder tensor then i ninth

element C At B will be
of
Ci ia in Ai ia in t bi ia in

NY
for D A B

di is in Ai iz in bi iz in
Tensor Product: Kronecker Product

Let A B E Rd edna
tensors then the right Kronecker
d x xant
product C AGB E IR

Example
A
I BE
I B 2B
C
B u B

I
D

E's an
Tensor Product: Hardamard Product

Let A B E pdixdax.edu
die withorder tensor then i ninth

element C A OB will be
of
Ci ia in Ai ia in bi ia in

Example

Remark:
Hadamard & Kronecker products are used in back
propagation implementation in Neural network
 Recap Poll:

Q1: A real m-by-n matrix is a linear transformation from

R^m to R^n. (True/False)

Q2: Identity matrix will always be a square matrix.

(True/False)

Q3: A data matrix representing n-unique features with

m(>n) data points will always be a full-rank matrix. (True/
False)

Q4: Suppose A and B are m-by-n matrices. The Hadamard

products AoB and BoA are equal. (True/False)

Q5: Let u be a n-by-1 vector, then uu^T will be an n-by-n

matrix with ‘n’ linearly independent rows. (True/False)