Lecture 2

Linear Algebra 1
Linear algebra reminder (and “expander”)
Vector space
• A vector space V is a collection of objects, called vectors, denoted by A,B,.. etc. for which
one defines operations of summation of 2 vectors and multiplication by a number which
obeys:
• Closure: also belongs to V
• Commutativity:
• Distributivity: ,
• Associativity: ,
• Multiplication by identity:
• Existence of the null vector:
• Existence of the “inverse” vector:
• A set of scalars a,b, etc are called a field over which a “vector space” is defined (Can be
either real, rational or complex). “Field” is a set with “+” and “*” operations defined,
which commute, have zeros and inverses, and “*” distributes over “+”. We will be mostly
interested in a field of complex numbers.
Vector space
• From definition it follows:
• element Is unique (= )
• Proof: Let there be 2 distinct zero elements, i.e. A+0=A, and A+0’=A, for all A in V. Then,
0=0’+0=0+0’=0’, thus 0=0’ and zero element is unique.
• (zero-number multiplied by any vector gives zero-vector)
• Proof: Consider A+0*A=(1+0)A=1A=A, thus 0*A=0, since for every A: A+0*A=A
• The inverse vector of every A is unique and
• Proof: Let assume that there are two distinct inverse elements: A+A’=0, A+A’’=0, then:
A’=A’+0=A’+(A+A’’)=(A’+A)+A’’=A’’, and inverse element is unique.
• A+(-1)*A=(1-1)A=0*A=0, thus (-1)*A=(-A) since the latter one is unique.
• Proofs of these are not too difficult, will do in the exercise portion of
the class.
Examples:
• Consider a set of real numbers (a), with the usual definition of summations and
multiplication by a number: it is a vector space.
• A set of entities of the form , with and , all being real numbers. This is a vector space. Is
a set a vector space? What about
• Consider 2x2 matrices with the usual rules of additions and number multiplication. Is
that a linear space?
• Consider 2x2 matrices with the usual rules of additions and number multiplication. Is
that a linear space? Is there a relationship of this space with the previous one?
• Consider all functions defined of the interval , with addition being pointwise addition,
and number multiplication as usual. Is it a vector space?
• What about functions f(x) defined on R, that also have a finite integral of f2(x) over the
whole axis?
• What about functions f(x) defined on R, that are infinitely differentiable and “quickly”
decay to zero as ? ( ).
Linear independence
• Consider a linear combination of n vectors from some linear space and consider an
equation:

If this equation is satisfied only by the trivial set of , then vectors are said to be
linearly independent.
• In other words, if are linearly independent, then I cannot express one of the vectors
in terms of the others, i.e. it is impossible to find a set of numbers , sum excludes .
• Definition: Maximum number of the linearly independent vectors in the given
vector space called the dimensionality (n) of the space.
Theorem: Any vector in the n-dimensional vector space can be written as a linear
combination of the n linearly independent vectors.
Definition: Basis: any n linearly independent vectors in the n-dimensional vector
space.
 Basis
• Let vectors are linearly independent and thus form a basis in our n-dimensional
vector space. Then for arbitrary vector a set of exist so that

A set of called components of a vector in the basis .

Theorem: A set of is unique for a given basis.
Proof: lets imagine that and are both components of the same vector. Then = ;

It is now trivial that if , , then

thus , but this is only possible if , since is a basis and thus is linearly independent.

• We can represent vectors in arbitrary space by an ordered column

of numbers, which are the components in particular basis.
Inner product spaces
• Definition: For any two vectors we define inner product, , a scalar (complex), satisfying the following
properties (we really defining a correspondence that takes two vectors and returns one number)

• with if and only if

• Definition: A vector space with inner product called an inner product space.
• Note (again): I did not say a word, how to calculate this dot product!
• A curios point: what if first vector is a linear combination?

In formal math, inner product is said to be a linear map in second vector and antilinear (conjugation) in
the first, i.e. order matters.
More definitions:
• If for two vectors , then we call these orthogonal.
• We will call the norm of the vector, normalized vector will have the . (not the only
choice for the definition of norm!)
• The basis which has all vectors orthogonal to each-other and normalized is
orthonormal basis.

• Theorem (Gram-Smidt): out of n linearly independent vectors one can construct n

orthogonal linear-independent vectors, i.e. one can always choose orthonormal basis….
• Swartz inequality:

• Triangle inequality:
Vector component through Inner
product
• Let be orthonormal basis, then If , and :

• Due to orthonormality of the basis. We also have introduced

Kroneker-delta .
• How to find the components of the vector?
Exercises:
• Prove uniqueness of zero.
• Prove zero-number times vector is zero-vector.
• Prove Gram-Schmidt theorem (by construction):
• A1’=A1 – just normalize. A2: A2’=A2-(A1,A2)A1, (and normalize) indeed, (A1,A2’)= (A1,A2)- (A1,A2) (A1,A1)=0. A2’<>0, since otherwise
A2=(A1,A2)A’1, and these two are not linearly independent, a contradiction. Continue to A3’=A3-(A1,A3)A’1- (A2,A3)A’2. and so on.
Every vector in that sequence is non-zero due to the linear independency of the original set.
• Prove Swartz inequality (non-trivial!):

Proof: consider the for vector :

• Prove triangle inequality:

Proof: Lets start with