Linear Analysis

Part II
1 Introduction: What is “linear analysis”?
The objects in this course are infinite dimensional vector spaces (hence the term
“linear”) over R or C, together with additional structure (a “norm” or “inner
product”) which “respects” in some way the linear structure. This additional
structure will allow us to do “analysis”. The most pedestrian way to understand
the last sentence is that it will allow us to “take limits”.
In fact, the extra structure allows much more than just “taking limits”.
Hidden in the notion of norm and inner product are notions of convexity, duality,
and orthogonality. In the most complicated structure we will discuss, that of
a Hilbert space, all these notions will be present simultaneously together with
completeness.
The point of view of this course, like most undergraduate mathematics
courses, will be axiomatic/revisionist. The objects will be defined, and the
basic theorems discussed without any “motivation”. The objects in this course
have been come to be viewed as so central for mathematics that it is presently
inaccurate to consider them as having any single “motivation”.
This being said, the subject arose and developed under specific mathematical
circumstances, and this may be useful for deeper understanding of the theory.
Nice notes can be found in [3, 2]. At the very least, however, everyone starting
out in this subject should know that the vector spaces that originally gave rise
to this theory were spaces of functions, and the linear operators were linear
partial differential operators, their inverses, and more generally so-called linear
integral operators. So the linear algebraic question “Under what conditions can
you invert an operator?”–and this represents the kind of question we will be
answering in this course–in practice meant “Under what conditions can you
solve a linear pde?”.
Because of this connection to spaces of functions, the subject is almost always
known by the name Functional Analysis, and it is under this name that you will
probably search the literature for more material, if you find these notes to be
confusing.

2 Normed vector spaces

2.1 Vector spaces
I assume that you are familiar with the notion of vector space, linear trans-
formation, subspace, quotient space, image, kernal, etc. In this course, vector
spaces will be over R or C.

1
 Recall that the span of a set S in V is the smallest subspacePm of V containing
S, or alternatively, the set of all finite linear combinations i=1 α Pi si . If this
set is V we say S spans V . A set S ⊂ V is linearly independent if m i=1 αi si =
0 =⇒ αi = 0 for all i. A set B ⊂ V is called a basis if it spans V and is linearly
independent. If S1 and S2 are subsets of V , and λ1 , λ2 are given scalars, we
will often use the notation λ1 S1 + λ2 S2 to denote the set of points1 of the form
{λ1 s1 + λ2 s2 }s1 ∈S1 ,s2 ∈S2 . Zorn’s lemma implies that all vector spaces have a
basis, and any two baseis have the same cardinality. We will go through this
proof later, as a sort of warm up for the use of Zorn’s lemma in the Hahn-Banach
Theorem. If the cardinality of B is finite, we say that V is finite dimensional,
otherwise, infinite dimensional.
Most of the Theorems in this course will be easy to prove in the finite
dimensional case by purely algebraic methods. Thus, the main emphasis of this
course will be the infinite dimensional case.

2.2 Normed vector spaces: the definition

We turn immediately to the central object of study in this course:
Definition 2.1. A normed vector space V is a vector space (over R or C)
together with2 a function V → R, the value of which at v will be denoted |v|,
such that | · | satisfies the following
1. |v| ≥ 0 for all v, and |v| = 0 iff v = 0 (positive definiteness)
2. |λv| = |λ||v|, for a scalar3 λ, and for v ∈ V
3. |v + w| ≤ |v| + |w| (triangle inequality)
I will give a whole list of examples soon enough. For the time being, let us
be content with Rn , with the Euclidean norm
.
q
|(x1 , . . . xn )| = x21 + . . . + x2n . (1)

(More generally, Let V be an n-dimensional vector space, let ei be a basis and

let (x1 , . . . xn ) denote the components of a vector x with respect to this basis.
Then the expression defined by the right hand side of (1) defines a norm on V .)
Thus, the notion just defined incorporates in particular one of the most familiar
mathematical structures.

2.3 The relation with the topology

Let’s think more carefully about the above definition to try to disect its struc-
ture. It should be clear that | · | defines a metric space structure on V , with
1 Elements of a vector space will often be referred to as “points”.
2I assume this use of language is familiar and unproblematic.
3 One can distinguish from the context whether | · | denotes norm of a vector or absolute

value of a scalar.

2
metric d given by d(x, y) = |x − y|, whose three defining properties are inherited
from those of the norm.
In particular, the metric space structure allows us to speak of a topology,
i.e. to identify open and closed sets and continuous mappings. We will see soon
that the latter, when required also to be linear, have an alternative characteri-
zation with respect to the norm.
In addition to open, closed, and continuous, we may talk about the notion
of basis for the topology, product topology, homeomorphism, dense, separable,
convergent sequence. Since the topology is metrizable, we may also talk about
Cauchy sequences and the notion of completeness. In the context of normed
vector spaces, these notions will be applied without further comment.
To understand the sense in which the topological structure is “married” to
the linear, we present the following
Proposition 2.1. Let V, | · | be a normed vector space. The vector space oper-
ations are continuous maps V × V → V , R × V → V .
Proof. We will do only +. Let U ⊂ V be open. Want to show that the inverse
image +−1 (U) is open. Let (v1 , v2 ) ∈ +−1 (U). That is to say, v1 + v2 = v
for some v ∈ U. Let B(ǫ) denote the open ball of radius ǫ around the origin.
v + B(ǫ) is the open ball around v of radius ǫ. Clearly v + B(ǫ) ⊂ U for some
ǫ > 0. By the triangle inequality, v1 + B(ǫ/2) + v2 + B(ǫ/2) = v + B(ǫ). But
(v1 + B(ǫ/2), v2 + B(ǫ/2)) is an open neighborhood around (v1 , v2 ). So +−1 (U)
is indeed open, as desired.
Corollary 2.1. Let V be as above. Translations and dilations are homeomor-
phisms.
Proof. Just consider, for any v0 ∈ V , the map V → V given by composing
the continuous map V → V × V defined by v 7→ (v0 , v) with the addition map
V × V → V . By the previous Proposition, this is continuous. On the other
hand, its (continuous again!) inverse is given by the analogous map defined by
−v0 . So the map is a homeomorphism.
Similarly for dilations. Let λ 6= 0 and consider the composition of the map
V → R × V defined by v 7→ (λ, v) with the map of the previous proposition,
etc.

2.4 A more abstract setting: topological vector spaces

To understand the message of the above proposition, let us formalise this concept
in a definition.
Definition 2.2. A topological vector space is a vector space V , together with a
topology, so that the vector space operations are continuous, and so that points
are closed sets.
The latter assumption is technical, and is to ensure that the space is Haus-
dorff.

3
2.5 Norms and convexity
At the most abstract level, the main object of this subject would be the topo-
logical vector space. Our previous proposition shows that normed vector spaces
are a special case.
To understand, at a slightly more abstract level, what is the extra structure
given by a norm, let us make the following
Definition 2.3. Let V be a vector space, and C ⊂ V a subset. C is said to be
convex if
tC + (1 − t)C ⊂ C
for all t ∈ [0, 1].
Proposition 2.2. Let V, | · | be a normed vector space. The unit ball B(1) ⊂ V
is convex.
Remark. Note that if C is convex, then translations of C, i.e. sets of the form
p + C, are also convex.
Definition 2.4. A locally convex topological vector space V is a topological
vector space with a basis of convex sets.
By the above, remark, it is clear that a sufficient condition for a topological
vector space to be locally convex is that every open set U ⊂ V containing the
origin contains a convex neighborhood of the origin. In any case, normed vector
spaces are clearly examples of locally convex topological vector spaces in view
of Proposition 2.2.
In the class of locally convex topological vector spaces, how special are metric
spaces? Let us make the following
Definition 2.5. Let V be a topological vector space. A subset B ⊂ V is said
to be bounded if for any open neighborhood U ⊂ V of 0, there exists an s > 0
such that B ⊂ tU for all t > s.
Remark. If V is a normed vector space, then B is bounded iff B ⊂ B(t) for
some t > 0, where B(t) denotes the open unit ball around the origin of radius t.
Proposition 2.3. Let V be a topological vector space, and assume C ⊂ V is a
bounded convex neighborhood of 0. Then V is normable, that is to say, a norm
| · | can be defined on V with the same induced topology.
Proof. For this we need
Lemma 2.1. C ⊂ C̃, where C̃ is a balanced bounded convex neighborhood of
the origin, that is to say, one for which in addition λC̃ ⊂ C̃ for all |λ| ≤ 1.
The proof is straightforward and omitted. Now define the function µC̃ by

µC̃ (v) = inf{t : v ∈ tC̃}.

This function is called the Minkowski functional of C̃. One checks explicitly
.
that |v| = µC̃ (v) defines the structure of a normed vector space on V .

4
Definition 2.6. A topological vector space V is said to be locally bounded if
there exists a bounded open neighborhood U of the origin.
Normed vector spaces are clearly locally bounded. Proposition 2.3 says that
a topological vector space is normable if it is locally bounded and locally convex.

2.6 Banach spaces

After the above diversion, we safely return to the world of normed vector spaces.
Fundamental for analysis is being able to take limits. For this, the metric space
notion of completeness is extremely useful. When this requirement is added to
a normed vector space, one arrives at the definition of a Banach space.
Definition 2.7. A normed vector space V , | · | is called a Banach space if it is
complete under the induced metric.

2.7 Examples
2.7.1 Finite dimensions
We have already seen the example of Rn , or Cn . As we shall see later on, these
are Banach spaces, as is any finite dimensional normed vector space.

2.7.2 The space C(X)

In general, let S be a set, and let FR (S) denote the set of real valued functions
on S, and let FC (S) denote the set of complex valued functions on B. These
are clearly vector spaces.
Let BR (S) denote the set of bounded real functions. This is a subspace of
FR (S), and thus a vector space. Define
|f | = sup |f (s)|. (2)
s∈S

It is easy to see that |f | defines a norm, making BR (S) into a normed vector
space.
On the other hand, let X be a compact Hausdorff space, and consider CR (X)
the set of all real valued continuous functions. We have
CR (X) ⊂ BR (S)
by well known properties of continuous functions. It inherits thus the norm.
(A subspace of a n.v.s is a n.v.s.!) It turns out that CR (X) is a Banach space.
Actually, you have shown this already in your analysis classes. For you have
probably encountered a theorem “Suppose a sequence of continuous fi on X
converge uniformly to f . Then f is continuous.” Exercise: How far away is
this theorem from the statement that CR (X) is a Banach space?
All the above considerations apply equally well to CC (X)
Understanding the topology of CR (X), in particular, idenitifying its compact
subsets, will be important later on.

5
2.7.3 C k (Ū )
Let U ⊂ Rn be open and bounded, and consider now the space of functions
f : U → R such that Dα f is continuous and uniformly bounded on U for all
multiindices |α| ≤ k.4 Denote this space by C k (Ū ). Consider the norm | · |k
defined by
|f |k = max |Dα f |
|α|≤k

where | · | denotes the supremum norm. This makes C k (Ū ) into a Banach space.

2.7.4 Lp
Let X = [0, 1] and let p > 1, and consider the set L̂p ([0, 1]) of all f : [0, 1] → R
such that f is continuous. Define a norm on L̂p by
Z 1 1/p
p
|f | = |f | (3)
0

This is indeed a norm! The triangle inequality holds in view of Minkowski’s

inequality. Exercise: Why is f positive definite?
For better or for worse, one of the most important facts of life in mathematics
is that the above space is not a Banach space. That is to say, it is not complete
under the induced norm.
From one point of view, this is not a problem. Every incomplete metric
space can be embedded into a larger one, which is complete. This larger one is
called the completion. For normed vector spaces, one easily shows that every
normed vector space V can be realised as a subspace of a larger one V ′ , such
that V inherits its norm from V ′ , V ′ is a Banach space, and V = V ′ . Moreover
V ′ is unique up to isometry (we’ll see what that means later). We call V ′ the
completion of V .
The problem is that the above construction is completely abstract. Elements
of the completion are equivalence classes of Cauchy sequences of V .
One of the miracles then of analysis is that the completion of L̂p can be
realised as equivalence classes of Lebesgue measurable functions f : [0, 1] →
R ∪ ±∞ where f ∼ g iff f = g a.e. Let us call this space Lp , without the hat.
The norm if just (3), where this integral is interpreted in the sense of Lebesgue.
Lp is thus a Banach space.

2.7.5 lp
The previous example is one of the most fruitful for the application of linear
analysis. Since we do not have the technology of measure theory at our disposal,
we will have to settle with a baby example, where functions are replaced by
sequences. (The latter of course are just functions on the natural numbers, and
4 Here αi , and D α f denotes
P
α = (α1 , . . . αn ) nonnegative integers, with |α| = i
∂ |α|
α α .
∂x1 1 ···∂xnn

6
this example can be thought of as a special case of the previous where [0, 1] is
replaced by N.) This is so-called “little” lp .
For 0 < p < ∞, we define
∞
X
lp (C) = {(x1 , x2 , x3 , . . .), xi ∈ C : |xi |p < ∞},
i=1

and for p = ∞, we define

l∞ (C) = {(x1 , x2 , x3 , . . .), xi ∈ C : sup |xi | < ∞}.
i

Alternatively, we may replace C with R. We may think of these as subsets of

FC (N) or FR (N). By the Minkowski inequality, lp is a subspace, and can be
made into a normed vector space with norm defined by
∞
!1/p
X
|x| = |xi |p (4)
i=1

for p ≥ 1, and
|x| = sup |xi |, (5)
i
for p = ∞. We will see later on that these are in fact Banach spaces. In the
case 0 < p < 1, lp is again a subspace of FC (N), but the expression (4) does not
define a norm.

2.7.6 C(Ω), C ∞ (Ω)

Let Ω ⊂ Rn be open, and let C(Ω) denote the set of continuous functions on
Ω. Continuous functions are no longer necessarily bounded, thus the expression
(2) is no longer well defined. We can still make C(Ω) into a topological vector
space as follows. Choose a sequence of compact Ki such that ∪Ki = Ω, and
Ki ⊂ Ki+1 . Let
V (i, n) = {f : |f |C(Ki ) < 1/n},
and consider the topology on C(Ω) generated by this family and all its translates.
This defines on C(Ω) the structure of a locally convex topological vector space.

2.8 Bounded linear maps

Now we know what the objects of interest are in this course. So what are the
morphisms, will ask the well-trained student in the language of category theory.
These are the so-called bounded linear maps.
Definition 2.8. Let V , W be topological vector spaces. A linear map T : V →
W is said to be bounded if E ⊂ V bounded implies T (E) ⊂ W is bounded.
Proposition 2.4. Let V , W be topological vector space, and T : V → W be
linear, and assume V and W are locally bounded. Then T is bounded iff T is
continuous.

7
Proof. First, I claim that T is continuous iff T is continuous at 0, i.e. iff for
any open subset U in W around 0, there exists an open neighborhood of 0 in V
contained in T −1 (U).
To see this, suppose then that indeed for any open neighborhood U of the
origin in W , there exists an open neighborhood Ũ of the origin in V , with
Ũ ⊂ T −1 (U). Let v0 and w0 be arbitrary points such that T (v0 ) = w0 . Then

T (v0 + Ũ ) = w0 + T (Ũ) ⊂ w0 + U.

This shows that the the inverse image of any open subset is open. (The other
direction of the implication is of course immediate.)
We now continue with the proof of the proposition. Suppose then T is
bounded. Let U ⊂ W be an arbitrary open neighborhood of 0, and let Ũ ⊂ V
be a bounded open neighborhood of 0. The latter exists since V is locally
bounded. We have that T (Ũ) is bounded, by assumption. Thus, by definition,
there exits an n > 0 such that T (Ũ) ⊂ nU. But then

n−1 Ũ ⊂ T −1 (U),

and this proves continuity, since U open implies n−1 U open by Corollary 2.1.
Suppose conversely that T is continuous, and let E ⊂ V be bounded. Let U
be an arbitrary open neighborhood of 0 in W . Since T −1 (U) is open, there exits
an open neighborhood of the origin Ũ ⊂ V such that T −1 (U) ⊃ Ũ, i.e. such that

U ⊃ T (Ũ).

On the other hand, by boundedness, it follows that

E ⊂ tŨ

for all t > s, but then

T (E) ⊂ T (tŨ) ⊂ tT (Ũ) = tU,

so T (E) is bounded. Thus T is bounded.

Specialising the definition to normed vector spaces we obtain
Definition 2.9. Let V , W be normed vector spaces. A linear map T : V → W
is said to be bounded if T (B(1)) ⊂ B(t) for some t > 0. We define ||T || to be
the infimum of all t > 0 such that the previous inclusion holds.
It is easy to see that ||T || is alternatively characterized by

||T || = sup |T v| = sup |T v| = sup |T v|.

|v|≤1 |v|=1 |v|<1

Let L(V, W ) denote the set of linear maps T : V → W . This is clearly a

vector space. Define B(V, W ) to be set of all bounded linear maps. We have

8
Proposition 2.5. B(V, W ) is a subspace of L(V, W ). ||T || defines a norm on
B(V, W ).
Proof. The proof of this is very easy. Clearly 0 ∈ B(V, W ) and ||0|| = 0. By
linearity, it is clear that if T ∈ B(V, W ), then λT ∈ B(V, W ) with ||T || = |λ|||T ||,
since
(λT )(B(1)) = T (λB(1)) = T (|λ|B(1)) = |λ|(T (B(1)))
so (if λ 6= 05 )
T (B(1)) ⊂ B(t)
iff
(λT )(B(1)) ⊂ |λ|B(t)
which is equivalent to
(λT (B(1)) ⊂ B(|λ|t).
Thus the set is closed under scalar multiplication, and the norm satisfies the sec-
ond property. Finally, suppose T1 , T2 ∈ B(V, W ), and the definition is satisfied
with t1 , t2 . We have

(T1 + T2 )(B(1)) = T1 (B(1)) + T2 (B(1))

⊂ B(t1 ) + B(t2 )
= B(t1 + t2 )

Taking the infimum over T , we obtain in particular

||T1 + T2 || ≤ ||T1 || + ||T2 ||.

2.9 The dual space V ∗

Definition 2.10. Let V be a normed vector space over R (C, respectively). The
space B(V, R) (respectively, B(V, C)) is known as the dual of V and is denoted
V ∗.
To distinguish, we will call L(V, R) (respectively, L(V, C)) the algebraic dual
of V .
Proposition 2.6. Let W be a Banach space, and V a normed vector space.
Then B(V, W ) is a Banach space.
Proof. Let Ti ∈ B(V, W ) be Cauchy. Let v ∈ V . Ti (v) is clearly Cauchy in W ,
because
|Ti (v) − Tj (v)| ≤ ||Ti − Tj || · |v|.
5 If λ = 0, again, there is nothing to show.

9
Thus Ti (v) converges by the completeness of W . Define T (v) = lim Ti (v).
Claim. T is linear, i.e. T ∈ L(V, W ). This follows immediately from the con-
tinuity of addition in W . On the other hand, T is clearly bounded. For given
any |v| ≤ 1,

|T (v)| ≤ |T (v) − Ti (v)| + |Ti (v)|

≤ ǫ + ||Ti || · |v|
≤ ǫ + ||Ti ||.

The above should be interpreted as follows. For any ǫ > 0, v > 0 there exists
an i depending on ǫ and v such that the second inequality holds.
Thus we have ||T || < ∞, so T ∈ B(V, W ). Finally we must show that
||T − Ti || → 0. We compute

|(T − Ti )(v)| ≤ |(T − Tj )(v)| + |(Tj − Ti )(v)|

≤ ǫ + ||Tj − Ti || · |v|
≤ ǫ + ||Tj − Ti ||,

where this is to be interpreted as follows: For any ǫ > 0, there exists a j

depending on v such that the second inequality holds for j > i. Now choose i
such that ||Tj − Ti || < ǫ and we’re done.
Note of course that ||T || = lim ||Ti ||.
In particular, since R (resp. C) are Banach spaces, it follows that
Corollary 2.2. Let V be a normed vector space. Then V ∗ is a Banach space.
It turns out that a good technique for proving that a n.v.s. is in fact Banach
is exhibiting as the dual space of another n.v.s.

2.10 Adjoint map

Let V , W , be normed vector spaces, T ∈ B(V, W ). Let us define a map T ∗ :
W ∗ → V ∗ as follows. For any w∗ ∈ W ∗ , let T ∗ (w∗ ) be the map V → R (or C
respectively) defined by
T ∗ (w∗ )(v) = w∗ (T v).
T ∗ (w∗ ) is clearly in the algebraic dual of V . Moreover, for |v| ≤ 1,

|T ∗ (w∗ )(v)| = |w∗ (T v)| ≤ ||w∗ || · |T v| ≤ ||w∗ ||||T ||

so in fact T ∗ (w∗ ) ∈ V ∗ . So T ∗ ∈ B(W ∗ , V ∗ ). On the other hand, the above

computation shows that

||T ∗ (w∗ )|| ≤ ||w∗ ||||T ||

and thus,
||T ∗ || ≤ ||T ||.
In fact, ||T ∗ || = ||T ||, but this will need Hahn-Banach.

10
2.11 V ∗∗
We define the double dual V ∗∗ of a normed vector space to be (V ∗ )∗ , i.e. the
dual of the dual.
Proposition 2.7. The map φ : V → V ∗∗ defined by

v 7→ v ∗∗

where v ∗∗ ∈ V ∗∗ is in turn defined by v ∗∗ (f ) = f (v), for all f ∈ V ∗ , is a bounded

linear map.
Proof. Linearity is clear. Moreover, the identity

|v ∗∗ (f )| = |f (v)| ≤ ||f |||v|

shows that ||φ|| ≤ 1.

Later on in the course, the Hahn-Banach theorem will show that this map
is an isometry, in particular the map is injective.
Note that, in contrast to the finite dimensional case, the map φ is not in
general surjective.

2.12 Examples!
2.12.1 Finite dimensions
Let V , W be finite dimensional normed vector spaces. Then any linear map
T : V → W is bounded. We will see this later on. Exercise: Let vi , wi be
baseis for V , W , respectively. What is the relation between the matrix of T and
T ∗?

2.12.2 Infinite dimensions

Consider lp , and define T : lp → lp by (x1 , x2 , . . .) 7→ (0, x1 , x2 , . . . ). Note that
this map is bounded, has ||T || = 1, and is injective but not surjective.
Consider the map
D : C 1 [0, 1] → C 0 [0, 1]
taking f 7→ f ′ , where ′ denotes the derivative. This is linear by the properties
of derivatives ((f + g)′ = f ′ + g ′ , etc.). Moreover, by defiinition of the relevant
norms, one sees trivially that ||D|| ≤ 1, and immediately that ||D|| = 1.
For a linear map which is not bounded, consider the space X, which as an
underlying vector space is C 1 [0, 1], but is made into a normed vector space with
the induced norm of C 0 [0, 1], i.e. the sup norm. Consider the map

id : X → C 1 [0, 1]

where the target is the usual C 1 defined as in Section 2.7.3, and the map is just
the identity. This map is clearly linear! But it is not bounded. For one can
consider a sequence fi of C 1 functions such that |fi | ≤ 1, yet |fi′ | → ∞.

11
3 Finite dimensional normed vector spaces
The point of the theory developed in this course is to treat the infinite dimen-
sional case. In this section, we shall see why the finite dimensional case is so
very different.
Let us begin with a definition.
Definition 3.1. We say that two norms | · |1 and | · |2 on a vector space V are
equivalent if there exits constants 0 < c < C < ∞ such that

c|v|1 ≤ |v|2 ≤ C|v|1 .

One easily sees that this defines an equivalence relation on the set of norms
on V . Clearly the induced topology defined by two equivalent norms is the
same. In particular, the notion of bounded operator with respect to the norms
coincides. Moreover, the notion of a Cauchy sequence with respect to the two
norms is the same. Thus, if | · |1 and | · |2 are equivalent, then vi is Cauchy with
respect to | · |1 iff it is Cauchy with respect to | · |2 .
In this section we will show that all finite dimensional normed vector spaces
have equivalent norms and are Banach. We will show that all linear maps are
bounded. Moreover, we will show the closed unit ball in a finite dimensional
n.v.s. is compact. Finally, we shall show that the latter fact is a characterization
of finite dimensionality for normed vector spaces, that is to say, if the closed
unit ball is compact, then the dimension is necessarily finite.
Given a finite n dimensional vector space V , we may think of it as Rn or Cn
after choice of basis. Let us define a norm on Rn , resp. Cn , by
n
X
|x|1 = |xi |.
i=1

We denote the induced normed vector space l1n .

Proposition 3.1. Let | · | be a norm on Rn . Then | · | is equivalent to | · |1 .
Note the immediate
Corollary 3.1. All norms on Rn are equivalent.
Proof. First, the easy direction. Let ei denote the standard basis. We have
X n

X
|x| ≤ |xi ||ei | = max |ei | |xi | ≤ C|x|1
1≤i≤n
i i=1

where C = max1≤i≤n |ei |.

For the other direction, we need some Lemmas.
Lemma 3.1. The function | · | is continuous on the unit circle S(1) with respect
to | · |1 .

12
Proof. This follows from the computation

| |x| − |y| | ≤ |x − y| ≤ C|x − y|1 .

Lemma 3.2. The closed unit ball (and thus the unit circle) are compact in the
topology of l1n .
Proof. Suppose xi is a sequence. Then xip is a sequence in R or C. Choose
j
a subsequence xj1k that converges to x̃1 , then a further subsequence x2km , etc.
Construct from the final choices a sequence Pnwe will again denote by xi . Clearly
x → x̃ = (x̃1 , . . . , x̃n ). Since |x − x̃|1 = j=1 |xj − x̃ij | → 0.
i i i

In fact, the above proof shows that any closed ball is compact in the topology
of l11 , and even more generally, any closed and bounded set.
From the above we know that | · | a continuous function on a compact set
attains its infimum. The latter then must be strictly positive. Thus we have

0 < c ≤ |x|

for all |x|1 = 1. But then, applying this to an arbitrary x̃ = λx, for 0 6= |λ| = |x̃|1
we obtain
0 < |x̃|1 c = |λ|c ≤ |λ||x| = |λx| = |x̃|.
This completes the proof.
Proposition 3.2. Let V be a finite dimensional normed vector space. Then the
closed unit ball is compact.
Proof. This follows from the statement proven about l11 in Lemma 3.2 and from
the previous proposition.
Proposition 3.3. Let V be a finite dimensional normed vector space. Then V
is a Banach space.
Proof. Let vi be Cauchy in V . It follows that vi is in particular bounded,
i.e. there exists an R such that vi ∈ B(R). But B(R) is compact (so in particular
complete!). So vi converges.
Corollary 3.2. Let V ⊂ W , where W is a normed vector space, and V is a
finite dimensional subspace. Then V is closed.

Proposition 3.4. Let T : V → W be linear, where V and W are normed vector

spaces and V is finite dimensional. Then T is bounded.

13
Proof. Since ImT is clearly finite dimensional, it suffices to consider the case
where W is also finite dimensional. It suffices to prove this when V = l1n ,
W = l1m . Consider the matrix Tij associated to T . We have
X X

T (x1 , . . . , xn ) = Ti1 xi , . . . , Tim xi

and thus
X
|T (x1 , . . . , xn )|1 = |Tij xi |
ij
X
= |Tij ||xi |
ij
X
≤ max |Tij | |xi |
ij
i
= C|(x1 , . . . , xn )|1 ,
where C = maxij |Tij |. This of course completes the proof.
Finally:
Proposition 3.5. Let V be a normed vector space with the property that the
closed unit ball is compact. Then V is finite dimensional.
Proof. Consider the open cover of the closed unit ball consisiting of all open
balls around arbitrary points, of radius 1/2. By compactness, there exits a
finite subcover, i.e.
B(1) ⊂ ∪ni=1 (xi + B(1/2)).
Now let W denote the subset of spanned by xi . This of course is finite dimen-
sional with dim W ≤ n. We have
B(1) ⊂ W + B(1/2).
Iterating once we obtain,
1
B(1) ⊂ W + (W + B(1/2)) = W + B(1/4).
2
Iterating arbitrarily many times, we obtain
B(1) ⊂ W + B(1/2m )
for all m ≥ 0, and thus
B(1) ⊂ ∩m (W + B(1/2m )) = W = W
where for the last equality we have used Corollary 3.2. But of course, this implies
that V ⊂ W , and thus V = W . So V is finite dimensional with dim V ≤ n.
The above proof can easily be generalised to a statement about locally com-
pact topological vector spaces, i.e. topological vector spaces with a neighborhood
of the origin with compact closure.

14
4 The Hahn-Banach Theorem
The Hahn-Banach Theorem is essentially a statement about the richness of V ∗ ,
i.e. it says (or rather, its corollaries show) that this space is sufficiently big.
At this point, we don’t even know whether in general V 6= 0 implies
V ∗ 6= 0!
The idea here is that one constructs elements of V ∗ be defining linear func-
tionals on subspaces of V (for instance finite dimensional ones, where this is
more or less trivial) and then extends them “little by little” to the whole of V .
The fact that one can extend a bounded linear functional from a codimension
1 subspace to the whole space is the content of Proposition 4.3. If
V = ∪Vi , (6)
where Vi are finite dimensional subspaces with Vi ⊂ Vi+1 of codimension 1, then
this allows one to obtain bounded linear functionals on V by induction.
The analogue of this induction procedure in the general case when (6) does
not hold is known as transfinite induction. We begin with a general discussion.

4.1 Introduction to transfinite induction

Everyone has their favourite method of transfinite induction. For us, it will be
via Zorn’s lemma.
Definition 4.1. A partially ordered set is a set S together with a binary relation
≤, i.e. a relation such that for all x, y in S either x ≤ y or x 6≤ y, satisfying
x ≤ x,
x ≤ y, y ≤ z =⇒ x ≤ z,
x ≤ y, y ≤ x =⇒ x = y.
Definition 4.2. Let S be a partially ordered set. A subset T ⊂ S is called
totally ordered if x 6≤ y =⇒ y ≤ x.
Definition 4.3. Let S be a partially ordered set, and S̃ a subset. An element
b ∈ S is said to be an upper bound for S̃ if x ≤ b for all x ∈ S̃. An element
l ∈ S is said to be a least upper bound for S̃ it is an upper bound, and moreover
l ≤ b for any upper bound b.
Definition 4.4. Let S be a partially ordered set. We say that S has the least
upper bound property if every non-empty totally ordered subset has a least upper
bound.
Definition 4.5. Let S be a partially ordered set. An element m ∈ S is called
maximal if m ≤ x =⇒ m = x.
Lemma 4.1. (Zorn) Let S be a non-empty partially ordered set with the least
upper bound property. There exists a maximal element m ∈ S.
The above “lemma” is equivalent to the axiom of choice.

15
4.2 Application: Every vector space has a basis
Proposition 4.1. Let V 6= {0} be a vector space. Then there exists a basis for
V.
In fact we will prove something stronger namely
Proposition 4.2. Let V 6= {0} be a vector space, and let S ⊂ V be linearly
independent. Then there exists a basis B of V with S ⊂ B ⊂ V .
Proof. Let S be the set of all linear independent subsets of V , containing B.
This set is non-empty since it contains S. We may partially order S as follows.
For S1 , S2 ∈ S, we say that S1 ≤ S2 if S1 ⊂ S2 . One checks easily that this
defines an order relation. Suppose now that T ⊂ S is a non-empty totally
ordered subset. Define S b = ∪S̃∈T S̃. The set S̃ contains S, and is linearly
independent, because if m
P
i=1 αi xi = 0, then there exists by total ordering a
S̃ ∈ T such that xi ∈ S̃ for all i = 1, . . . n, and one applies linear independence
of S̃ to deduce that αi = 0. Thus, Sb is clearly a least upper bound in S, so S
has the least upper bound property.
By Zorn’s lemma, it follows that S has a maximal element. Call this B. To
show B is a basis, we need only show that Span(B) = V . Suppose this is not
the case, i.e. suppose ∃v : v 6∈ SpanB. Consider the set

B̃ = {v} ∪ B.
Pn
Claim. B̃ is linearly independent. For if αv + i=1 αi vi = 0 for vi ∈ B̃, and
α 6= 0, one obtains X
v=− α−1 αi vi

and thus v ∈ Span(B), a contradiction. But now we have B ≤ B̃, but B 6= B̃,
and this is contradicts the maximality of B. So B is indeed a basis.
Note how it was important in the above that the scalars constitute a field.
The above result does not hold for modules.

Exercise. Show that any two baseis have the same cardinality.

4.3 The statement of the theorem

Let us restrict to real vector spaces V . We sometimes call linear maps f :
V → R linear functionals. The Hahn-Banach theorem is a statement about
the extendibility of bounded linear functionals defined on a subspace to linear
functionals on the whole space with the same norm.

4.3.1 The codimension 1 case

We begin with the case of codimension 1.
Although our primary application is to bounded linear functions, the follow-
ing somewhat more general setting is convenient.

16
Proposition 4.3. Let V be a real vector space, and let p : V → R be a function
p : V → R, with p(v1 + v2 ) ≤ p(v1 ) + p(v2 ), and p(λv) = λp(v) for all λ > 0. Let
W be a codimension 1 subspace of V , and suppose that f : W → R is a linear
functional such that f (w) ≤ p(w) for all w ∈ W . Then there exists a linear
functional f˜ : V → R such that f˜|W = f and f˜(v) ≤ p(v) for all v ∈ V .
The assumptions on p imply in particular that is is a convex function. Note
that, in the case of a normed vector space V , if ||f || < ∞, then the conditions of
the proposition are satisfied with p(x) = ||f |||x|, and one has that ||f˜|| = ||f ||.
Proof. Pick v0 ∈ V \ W . By codimensionality 1, it suffices to define f˜ so that
f˜(w+av0 ) ≤ p(w+av0 ) for all a ∈ R. That is to say f˜(w)+af (v0 ) ≤ p(w+av0 ).
This is equivalent to
af˜(v0 ) ≤ p(w + av0 ) − f (w)
and thus, with, a > 0, we obtain
f˜(v0 ) ≤ a−1 p(w + av0 ) − f (a−1 w),
and thus
f˜(v0 ) ≤ p(a−1 w + v0 ) − f (a−1 w),
for all w, a condition which, by redefining w, we may write as
f˜(v0 ) ≤ p(w + v0 ) − f (w) (7)
for all w.
On the other hand, for a < 0, we obtain the condition
f˜(v0 ) ≥ a−1 p(w + av0 ) − a−1 f (w),
or equivalently
f˜(v0 ) ≥ −(−a−1 )p(w + av0 ) + f (−a−1 w)
and thus, after redefining w as above, we obtain the condition
f˜(v0 ) ≥ −p(w − v0 ) + f (w). (8)
for all w.
So now, the condition that f˜(v0 ) can be chosen so that inequalities (7) and
(8) both hold is just that, for all w, w̃:
f (w̃) + f (w) ≤ p(w + v0 ) + p(w̃ − v0 ). (9)
But indeed, (9) holds. For we have
f (w̃) + f (w) = f (w̃ + w)
≤ p(w̃ + w)
= p(w + v0 + w̃ − v0 )
≤ p(w + v0 ) + p(w̃ − v0 ).

17
4.3.2 The general case
Theorem 4.1. Proposition 4.3 holds without the assumption that W has codi-
mension 1.
Proof. Consider the set S of all extensions f˜ : Ṽ → R of f , where W ⊂ Ṽ ⊂ V
is a subspace, where f˜|V = f , and where f˜(x) ≤ p(x) for all x ∈ Ṽ .
This set is nonempty, as it contains f itself. We may partially order the set
by setting f˜1 ≺ f˜2 if Ṽ1 ⊂ Ṽ2 , where Ṽi are the domains of f˜i , and f˜2 |Ṽ1 = f˜1 .
This clearly defines a partial ordering.
Claim. Under this partial ordering, S satisfies the least upper bound prop-
erty. For if T is a totally ordered subset, we can consider a f˜ defined on ∪Ṽ ∈T Ṽ
by f˜(x) = f˜α (x) for some f˜α such that its domain Ṽα contains x. One easily sees
that since T is totally ordered, this definition does not depend on the choice.
Finally, it is clear that f˜α ≺ f˜, for any f˜α ∈ T .
We may thus apply Zorn’s lemma to obtain a maximal f˜ ∈ S. We are left
with proving that the domain of f˜ is V . So let us suppose that this is not the
case. Let v0 ∈ V \ W̃ , where W̃ denotes the domain of f˜. Consider the set
Ṽ = Span(v0 , W̃ ). We have that W̃ ⊂ Ṽ is codimension 1. We may apply then
˜ ˜
Proposition 4.3 to extend f˜ to a linear f˜ : Ṽ → R, with f˜(x) ≤ p(x). But
˜ ˜
clearly, f˜ ≺ f˜ and f˜ 6= f˜. This contradicts maximality.
So the domain of f˜ is V , and the theorem is proven.
Corollary 4.1. Let V be a normed vector space, and W ⊂ V a subspace. Let
f ∈ W ∗ . Then there exits an f˜ ∈ V ∗ with f˜|W = f , and ||f˜|| = ||f ||.
We will refer to Theorem 4.1 or Corollary 4.1 as the Hahn-Banach theorem.

4.4 The dual space revisited

Armed with Hahn-Banach, we may now show that the dual space is big enough.
The main tool will be the following
Proposition 4.4. Let V be a normed vector space, and v ∈ V an arbitrary
element. There exists an fv ∈ V ∗ such that ||fv || = 1, and fv (v) = |v|.
Such an fv is called a support functional for v.
Proof. Consider the one-dimensional subspace W spanned by v, and define a
fv ∈ W ∗ by fv (v) = |v|. Clearly, ||fv || = 1. Now apply Hahn-Banach in the
form of Corollary 4.1.
Corollary 4.2. Let V be a normed vector space, and let v ∈ V . Then v = 0 iff
f (v) = 0 for all f ∈ V ∗ .
In particular,
Corollary 4.3. Let V 6= 0 be a normed vector space. Then V ∗ 6= 0.

18
 In fact
Corollary 4.4. Let V be a normed vector space, v, w ∈ V with v 6= w. Then
there exists an f ∈ V ∗ such that f (v) 6= f (w).
Proof. Just take f = fv−w .
For another manifestation of the richness of V ∗ , let us consider V ∗∗ . We
have
Proposition 4.5. The map φ : V → V ∗∗ is an isometry, i.e. ||φ(v)|| = |v|.
In particular, φ is injective.

Proof. We have already shown in Proposition 2.7 that ||φ(v)|| ≤ |v|. For the
other direction, just note that for |v| = 1, we can choose a support functional
fv with ||fv || = 1, and this gives

|φ(v)(fv )| = |fv (v)| = |v| = 1,

and thus ||φ(v)|| ≥ 1 for all |v| = 1. But this gives ||φ(v)|| ≥ |v| for all v.
Finally, in a similar spirit, we show
Proposition 4.6. Let V and W be normed vector spaces and let T : V → W
be a bounded linear map. Then T ∗ : W ∗ → V ∗ satisfies ||T ∗ || = ||T ||.
Proof. Again, we have already shown that ||T ∗ || ≤ ||T ||. Let v ∈ V be arbitrary
with |v| = 1, and choose a support functional fw for w = T v. We have

(T ∗ fw )(v) = fw (T (v)) = |w|.

Thus, ||T ∗ fw || ≥ |w|. Since ||fw || = 1, we have that

||T ∗ || ≥ |T v|.

We thus have that

||T ∗ || ≥ sup |T v| = ||T ||
|v|=1

and this completes the proof.

4.5 Remarks and Examples of V ∗

One is often quite weary in mathematics to take seriously anything proven with
the help of Zorn’s lemma. Perhaps a healthy attitude to take towards the Hahn-
Banach Theorem is that it is a satisfying and clarifying general statement that
in practice one rarely invokes.
Here, by “in practice”, one should read, when applying the theory to a
particular normed vector space. For most normed vector spaces appearing in
analysis, one has a good idea of the what their dual is, or at least, can easily

19
construct a large subset of the dual. One has no need for applying the axiom of
choice to construct such elements.

Example. Consider the space C([0, 1]). If g is a continuous function then we

can identify g with an element of (C([0, 1]))∗ defined by
Z 1
f 7→ f g.
0

One easily sees that the norm of g thought of as an element in (C([0, 1]))∗ is
|g|. Since (C([0, 1]))∗ is a Banach space, and the set C([0, 1]) is not complete
R
under the L1 norm, it is clear that (C([0, 1]))∗ contains elements not of the
above form. For instance, f 7→ f (x0 ) is an element of the dual not induced as
above. It turns out that the dual of C([0, 1]) can be identified with the space
of signed Borel measures. This is not a space we have the technology to work
with in this course.

Example. Consider the space lp for ∞ > p ≥ 1. Let x ∈ lq where q is defined

by
1 1
+ = 1, (10)
p q
with the convention that if p = 1, q = ∞. For s ≥ 1, let | · |s denote the ls norm.
We call q the conjugate exponent of p.
The element x defines an element of lp∗ by the action
X
y 7→ xi yi .

The convergence of this sum follows from the Hölder inequality, stating that for
x ∈ lq , y ∈ lp satisfiying (10)

|xy|1 ≤ |x|q |y|p .

It turns out (and you will show this on example sheets) that this identifica-
tion yields an isometric isomorphism of lp∗ and lq . What about l∞
∗
?

5 Completeness
In the last section, we essentially exploited the structure of convexity provided
by a norm defined on a vector space. (Our slightly more general formulation of
Hahn-Banach in terms of a convex functional p should make this clear.) In this
section, we will exploit completeness. Thus, Banach spaces will become here
important.
The considerations of the current section stem from the observation that
complete metric spaces are necessarily “big”. This notion of “big” is captured
by a concept known as Baire category.

20
5.1 Baire category
Definition 5.1. Let X be a topological space. We say that X is of first category
if
X = ∪i Ei
where Ei is nowhere dense6 . Otherwise, we say that X is of second category.
Our convention is that ∪i always denotes a countable union. We can al-
ternatively characterize the spaces of second category as follows: A nonempty
topological space X is of second category if either of the following equivalent
statements is true
1. For all Ui countable collection of open dense sets, ∩i Ui is nonempty.
2. If X = ∪Ci , where Ci is closed, then there exists an i such that Ci has
non-empty interior.
Theorem 5.1. Let X be a complete metric space. Then X is of second category.
Proof. Let Ui be a sequence of open dense sets in X. Choose x1 ∈ U1 . By
the density of U2 , and openness of U1 , there exists an open ball of radius ≤ 1,
such that Bx1 (ǫ1 ) ⊂ U1 , and x2 ∈ Bx1 (ǫ1 ) such that x2 ∈ U2 . Given now 0 <
ǫj ≤ j −1 , and Bxj (ǫj ) ⊂ Uj , and Bxj (ǫj ) ⊂ Bxj−1 (ǫj−1 ), for j = 1, . . . , i, we can
choose xi+1 ∈ Ui+1 ∩Bxi (ǫi ), and an ǫi+1 such that Bxi+1 (ǫi+1 ) ⊂ Bxi (ǫi )∩Ui+1 .
Clearly, xi is a Cauchy sequence, and converges thus to some x. Moreover,
Since xj ⊂ Bxi (ǫi ), for j ≥ i, and Bxi (ǫi ) is of course closed, then x = lim xj is
also contained in Bxi (ǫi ). Thus, x ∈ Ui for all i, so ∩Ui 6= ∅.

5.2 Applications
5.2.1 The existence of irrationals
Proposition 5.1. R \ Q 6= ∅.
Proof. Q is of first category, as it is countable and points are closed with empty
interior, yet R is of second category, as it is complete.
Note that this is a completely nonconstructive proof. Compare this with the
difficulty of constructing a particular number which is irrational.

5.2.2 Nowhere differentiable functions

Proposition 5.2. There exists a function f ∈ C 0 ([0, 1]) such that f is not
differentiable anywhere.
6A set is nowhere dense if its closure has empty interior

21
Proof. Consider the space C 0 ([0, 1]). This is a Banach space. For all integer
n ≥ 1, and rationals p ∈ Q ∩ [0, 1], define

En = {f ∈ C 0 ([0, 1]) : ∃p ∈ [0, 1] : |f ′ (p)| ≤ n}

Let Fn denote the closure of En . One easily shows that Fn is nowhere dense,
as any open set in C 0 ([0, 1]) contains functions such that the Lipschitz constant
is arbitrarily large at all p, and thus cannot be approximated by members of
En . Thus, ∪Fn is of the first category. Since C 0 is complete, we must have
C 0 6= ∪Fn , i.e. there exists an f which is differentiable nowhere.
Showing that Fn is nowhere dense may give you an idea for a constructive
proof of the statement of the Proposition. Try it.

5.3 Completeness meets linearity: Banach-Steinhaus

The following result is often called the Banach-Steinhaus theorem:
Theorem 5.2. Let V be a Banach spaces, W a normed vector space, and let
Tα be an arbitrary collection of bounded linear maps Tα : V → W , such that for
each x ∈ V ,
sup |Tα x| < ∞. (11)
α

Then
sup ||Tα || < ∞. (12)
α

Proof. For positive integers n, define Fn = {x : ∀α , |Tα x| ≤ n}. By the conti-

nuity of Tα (Remember Proposition 2.4!), we have that Fn is closed as it is an
intersection of closed sets. By our assumption (11),

∪Fn = V.

Since V is complete by assumption, and thus of second category by Theorem 5.1,

it must be that at least one of the Fn contains an open ball x0 + B(ǫ) for an
ǫ > 0, i.e. for all α,
Tα (x0 + B(ǫ)) ⊂ B(n).
But this means that for all α,

Tα (B(ǫ)) ⊂ Tα x0 + B(n)

Let n0 = supα |Tα x0 |. We have then

Tα (B(ǫ)) ⊂ B(n0 ) + B(n) = B(n0 + n)

and thus
Tα (B(1)) ⊂ ǫ−1 B(n0 + n) = B(ǫ−1 (n0 + n))
and thus ||Tα || ≤ ǫ−1 (n0 + n), i.e. we have shown (12).

22
5.4 Open mapping, inverse mapping, closed graph
Theorem 5.3. Let V and W be Banach spaces, and let T be a surjective
bounded linear map T : V → W . Then T is an open map, i.e. T (U) is open if
U is open.
Proof. First note the following
Lemma 5.1. T is open iff T (B(1)) ⊃ B(ǫ) for some ǫ > 0.
Proof. One direction is obvious. For the other, note that if U is an arbitrary
open set, and q = T (p) for p ∈ U, then there exists a δ such that p + B(δ) ⊂ U.
But then
T (U) ⊃ T (p + B(δ))
= T (p) + T (B(δ))
= q + δT (B(1))
⊃ q + B(δǫ),
and this shows that T (U) contains an open set around an arbitrary element q
of it.
So it suffices for us to show that under the assumptions of the Proposition,
T (B(1)) ⊃ B(ǫ) for some ǫ > 0. We certainly have V = ∪∞ n=1 n(B(1)). Thus,
by surjectivity,
W = T ∪∞


n=1 nB(1)
= ∪∞ n=1 nT (B(1)).

Thus we may certainly write

W = ∪∞
n=1 nT (B(1)).

Since we have exhibited W as a countable union of closed sets, and W is of

second category by assumption, it follows that there exists an n such that
nT (B(1)) ⊃ y0 + B(δ),
so
T (B(1)) ⊃ n−1 y0 + B(n−1 δ).
Now since B(1) = −B(1) (The unit ball is balanced!), we have
T (B(1)) = T (−B(1)) ⊃ −n−1 y0 + B(n−1 δ).
So
T (B(2)) = T (B(1)) + T (B(1))
= T (−B(1)) + T (B(1))
⊃ −n−1 y0 + B(n−1 δ) + n−1 y0 + B(n−1 δ)
= B(2n−1 δ).
By rescaling T , the proof is clearly complete by the following

23
Lemma 5.2. Let T be a bounded linear map T : V → W , where V is a Ba-
nach space and W a normed vector space, such that T (B(1)) ⊃ B(1). Then
T (B(1)) ⊃ B(1).
Proof. Let w ∈ B(1) ⊂ W . We have that w ∈ B(1 − δ) for some δ > 0. By
the density of T (B(ǫ)) in B(ǫ) for all ǫ > 0 (this follows by assumption, and
linearity), and the fact that, for any set X ⊂ V , X ⊂ X + B(ǫ̃), for any ǫ̃ > 0,
we have that for all i ≥ 1,
B(2−i (1 − 2−i δ)) = B(2−i−1 (1 − 2−i δ)) + B(2−i−1 (1 − 2−i δ)
= T (B(2−i−1 (1 − 2−i δ))) ∩ B(2−i−1 (1 − 2−i δ))
+ B(2−i−1 (1 − 2−i δ))
⊂ T (B(2−i−1 (1 − 2−i δ))) ∩ B(2−i−1 (1 − 2−i δ))
+ B(2−2i−2 δ) + B(2−i−1 (1 − 2−i δ))
= T (B(2−i−1 (1 − 2−i δ))) ∩ B(2−i−1 (1 − 2−i δ))
+ B(2−i−1 (1 − 2−i−1 δ)).
P∞
By induction, it follows that we may write w as w = i=1 wi , where
wi ∈ T (B(2−i (1 − 2−i+1 δ))) ∩ B(2−i (1 − 2−i+1 δ)).
If vi ∈ B(2−i (1 − 2−i+1 δ)) is thus such that T vi = wi , then
P
vi converges to a
v ∈ B(1) since V is a Banach space. By continuity and linearity, T v = w. Thus
w ∈ T (B(1)).

Note how the completeness assumptions for V and W enter in very different
ways. The completeness of W was only used to infer it is second category. In
fact, one can replace the assumptions that W is Banach and T is surjective with
the assumption that the image of T is of second category in W . Surjectivity
then follows after having shown that the map is open.
The following result is known as the Inverse Mapping Theorem. It is im-
portant in various applications of the theory. It is essentially an immediate
corollary of Theorem 5.3.
Theorem 5.4. Let V and W be Banach spaces, and let T : V → W be an
injective and surjective bounded linear map. Then T −1 is bounded.
Proof. The map T −1 exists and is linear. Since by Theorem 5.3, we have
T (B(1)) ⊃ B(δ),
for some δ > 0, it follows that
B(1) ⊃ T −1 (B(δ)),
i.e.
T −1 (B(1)) ⊂ B(δ −1 ),
i.e. T −1 is bounded with ||T −1 || ≤ δ −1 .

24
 Another celebrated Corollary of Theorem 5.3 is the so-called Closed graph
theorem.
Theorem 5.5. Let V and W be Banach spaces, and let T : V → W be linear.
Then T : V → W is bounded iff the graph Γ of V is closed as a subset of V × W .
By the graph of V , we mean the set Γ = {v, T v}v∈V ⊂ V × W .
Proof. Certainly, if T is bounded, then the graph is closed. So it suffices to
show the other implication.
Assume then that Γ is closed. As Γ is evidently a linear subspace of the
Banach space V → W , it follows that Γ is itself a Banach space.
Consider the map φ : Γ → V defined by

φ : (v, T v) 7→ v.

The map φ is clearly linear. Moreover, it is bounded, as |(v, T v)| = max{|v|, |T v|},
and thus ||φ|| ≤ 1. Finally, the map is manifestly both injective and surjective.
It follows from Theorem 5.4 that φ−1 is a bounded linear map. But this implies
that there exists a C such that for all v,

|T v| ≤ max{|v|, |T v|} ≤ C|v|,

i.e. T is bounded.
To see the gain apparent from the previous Theorem, consider the sequential
characterization of continuity. To show that T is continuous, one has to show
that if vi → v then T vi → T v. Armed with the above theorem, it suffices to
show that if vi → v and if T vi → w, then w = T v.

5.5 More applications

We have already seen some elementary applications of category arguments to
show the existence of “exotic” objects in analysis. (If R \ Q can be thought of
as exotic. . . )
Now that we have a surprising set of theorems which demonstrate the power
of mixing category arguments with linearity, we can show the existence of “ex-
otic” objects characeterized by properties of linear maps.
For instance, in the example sheets you will use the theorems of this section
to show that there exist (in fact many) continuous functions on S1 such that
their Fourier series diverges at some point.
A sketch of the argument is as follows: One obtains an explicit integral
representation
sin n + 21 (x − t)
Z π


Sn f (x) = f (t)
dt (13)
−π sin (x − t)/2
for the n’th partial sum operator

Sn : C(S1 ) → C(S1 ),

25
defined by
n
eikt fˆ(k)
X
Sn : f 7→
k=−n

where π
1
Z
fˆ(k) = f (t)e−ikt dt.
2π −π

Let φn denote the composition of Sn with the map “evaluation at 0.” Assuming
now that ∀f , supn |φn (f )| < ∞, one applies Banach-Steinhaus to obtain that
sup ||φn || < ∞. On the other hand, this is easily contradicted by using (13),
and choosing an appropriate f for each n.7 Thus there exists a continuous f
such that its Fourier series does not converge at the origin.
Another such argument, also left for the example sheets, is the following.
One can show that the Fourier series of an L1 function on S1 is an element
of c0 , the set of complex functions on Z that tend to 0 as |n| → ∞. Do all
sequences in c0 arise like this?
This would mean that the map Λ : L1 → c0 defined by taking a function to
its Fourier series is surjective. One first shows that this is an injective bounded
linear map. Were it surjective, its inverse would be bounded in view of the
inverse mapping theorem, Theorem 5.4. One obtains a contradiction by demon-
strating a sequence of L1 functions whose L1 norm goes to infinity, while the
sup norm of their Fourier coefficients remains bounded. Thus the answer is no.
One should make reference at this point to other results in analysis which are
beyond the scope of this class. If f ∈ C 1 (S1 ), then Sn f converges everywhere.
This is a 19th century theorem. On the other hand, for an Lp function on S1 ,
with p > 1 (in particular for continuous functions) the Fourier series converges
almost everywhere (i.e. the set where it doesn’t has measure 0). For p = 2, this
is the celebrated Carleson’s Theorem. This theorem is hard.
Finally, a classic result of Kolmogorov shows that there exists a function in
L1 (S1 ) such that the Fourier series diverges everywhere!

6 The topology of C(K)

The space of continuous functions on a compact Hausdorff space plays a fun-
damental role in functional analysis. In this section, we will try to understand
better how big this space is, and what is its topology. In particular, we will
be interested in constructing (Section 6.1) a rich class of functions in C(K),
identifying the compact subsets of C(K) (Section 6.2), as well as useful dense
subsets (Section 6.4).
An application of density and compactness results for solving an actual prob-
lem in analysis is given in Section 6.3.
7 Why is this less work than a constructive proof? Because the f you choose in showing

sup ||Sn || → ∞ can depend on n. The convergence of these f is irrelevant.

26
6.1 The extension theorem
Our first task will be to show that the space C(K) is sufficiently rich. Specif-
ically, we shall show that functions on C(K) can be constructed by extending
functions on closed subsets. Compare with the Hahn-Banach Theorem, Theo-
rem 4.1. In particular, C(K) separates points (Corollary 6.1).

6.1.1 compact Hausdorff =⇒ normal

Recall the following
Definition 6.1. A topological space X is said to be Hausdorff, when given p 6= q
in X, there exist open neighborhoods Up of p and Uq of q, such that Up ∩ Uq = ∅.
On the other hand, we have
Definition 6.2. A topological space X is said to be normal if given C1 , C2
closed with C1 ∩ C2 = ∅, then there exist open U1 ⊃ C1 , U2 ⊃ C2 such that
U1 ∩ U2 = ∅.
An alternative characterization of normality is: X is normal iff for all C1 ⊂
U2 , with C1 closed and U2 open, there exist U1 , C2 , open, closed, respectively,
such that C1 ⊂ U1 ⊂ C2 ⊂ U2 . This follows by considering the complement of
U2 , etc.
It is the normality condition that will be very useful for extending continuous
functions. Thus, it’s nice to know the following
Proposition 6.1. Let K be a compact Hausdorff space. Then K is normal.
Proof. Let C1 , C2 be closed subsets of K. As closed subsets of a compact set,
they are compact. Given any p ∈ C1 , q ∈ C2 , let p ∈ Up,q , q ∈ Vp,q be open
neighborhoods such that Up,q ∩ Vp,q = ∅. Fix q say. Then {Up.q }p∈C1 forms an
open cover of C1 , and thus there exists a finite subcover {Upi ,q }i=1...n . Let us
define
Uq = ∪ni=1 Upi ,q ,
Vq = ∩ni=1 Vpi ,q .
Clearly Uq ∩ Vq = ∅, and Uq ⊃ C1 .
Now repeat this construction for arbitrary q, and consider the resulting sets
{Uq }q∈C2 and {Vq }q∈C2 . The latter forms an open cover of C2 , and thus, by
compactness, there exits a finite subcover {Vqi }i=1...m . Define
V = ∪m
i=1 Vqi

U = ∩m
i=1 Uqi .
These sets are clearly open. By definition of a cover, V ⊃ C2 , while clearly
V ∩U = ∅. On the other hand, since Uqi ⊃ C1 for all i, it follows that U ⊃ C1 .
Note that the compactness of K was only used to assert the compactness of
the Ci . It follows thus from the proof that in any Hausdorff space X, one can
separate compact sets by open sets.

27
6.1.2 Urysohn’s lemma
We begin with the following
Lemma 6.1. Let X be a normal space, and let C0 , C1 be disjoint closed sets.
Then there exists an f ∈ C(X) with range [0, 1] such that f = 0 on C0 and
f = 1 on C1 .
Proof. Enumerate the rationals of Q ∩ [0, 1] as {qi }∞ i=0 , where q0 = 0, q1 = 1.
Define inductively a collection of open and closed sets Ui ⊂ Ci as follows: Let
U0 = ∅, F0 = C0 , let U1 be X \C1 and F1 = X. Given Ui , Fi , for 0 ≤ i ≤ n, with
the property that Ui ⊂ Fi , and, if qi < qj , then Fi ⊂ Uj , there exists a unique
interval (qi1 , qi2 ) containing qn+1 , with 0 ≤ i1 , i2 ≤ n, and no other qj ∈ (qi1 , qi2 )
for j = 0, . . . n. We have by normality that there exists Un+1 ⊂ Fn+1 such that
Fi1 ⊂ Un+1 ⊂ Fn+1 ⊂ Ui2 . It now follows that this holds for all qi < qj for
i = 0 . . . n + 1, and thus, by induction8 , one defines such a sequence for all
i = 0 . . . ∞.
Now define
f (x) = inf {qn : x ∈ Fn }.
n=0...∞

Certainly, if x ∈ C0 , then x ∈ F0 , so f (x) = 0. On the other hand, if x ∈ C1 ,

then x 6∈ U1 , so x 6∈ Fj for qj < 1, so f (x) = 1.
Claim: f is both lower and upper semicontinuous, and thus continuous. For,
given α, {x : f (x) > α} is clearly open. For if y is in this set, this implies that
there exists a qn > α such that y 6∈ Fn . Since the complement of Fn is open,
this implies that there exists a neighborhood of y which does not intersect Fn .
But this means that f ≥ qn > α on this neighborhood. Thus {x : f (x) > α} is
open.
On the other hand, {x : f (x) < α} is also open. For let y be in this set. This
means that there exist qn < qm < α such that y ∈ Fn . But this means that
y ∈ Um . Now since Um is open there is a neighborhood of y which is contained
in Um . But now, since Um ⊂ Fm , it follows that this neighborhood is contained
in Fm , i.e. f ≤ qm < α on this neighhorhood. Thus {x : f (x) < α} is open. We
have thus shown continuity, as desired.
In view of Proposition 6.1, the above result applies to compact Hausdorff
spaces. In particular,
Corollary 6.1. Let K be compact Hausdorff. Then C(K) separates points,
i.e. given p 6= q in K, there exists an f ∈ C(K) such that f (p) 6= f (q).
Proof. It suffices to remark that points are closed sets in a Hausdorff space.
8 Notethat to have an interval (qi1 , qi2 ), one needs n > 1. Thus the importance of defining
two base cases (0 and 1) for the induction.

28
6.1.3 The Tietze-Urysohn extension theorem
Finally,
Theorem 6.1. Let X be normal, and f : C → C a bounded continuous function
on a closed subset C. Then there exists a continuous extension f˜ : X → C, with
f˜|C = f , and |f˜| = |f |, where | · | denotes the sup norm.
Note that Lemma 6.1 is a special case of Theorem 6.1 where the range of f
consists of 2 points.
Proof. Clearly, by taking real and imaginary parts, translating and rescaling, it
suffices to consider the case where the range of f is [0, 1]. Also, one need not
worry about the condition |f˜| = |f |. For given fˆ : X → C any continuous exten-
ˆ
sion of f , we may define f˜ by f˜(x) = fˆ(x), if |fˆ(x)| ≤ |f |, f˜(x) = ei(arg f (x)) |f |
otherwise, and f˜ is again a continuous extension of f with |f˜| = |f |.
Define a sequence of closed sets and functions by induction as follows. Let
f0 = f , C0 = f −1 [0, 31 ] , F0 = f −1 [ 23 , 1] , and let g0 : X → [0, 13 ] be a

function taking the value 0 on C0 and 1/3 on F0 , obtained by applying the

previous Proposition, and let f1 = f − g0 |C be a function f1 : C → R. Note
that 0 ≤ f1 ≤ 32 , i.e.
 2
f1 : C → 0, .
3
i
Now, given fi : C → 0, 23 , define



1 2
i

Ci = fi−1 [0, ] ,
3 3
and
2 2
i 2
i

Fi = fi−1 [ , ] ,
3 3 3

i
let gi be a function retrieving 0 on Ci and 31 23 on Fi , given by Lemma 6.1.
Set
fi+1 = fi − gi |C . (14)
i+1
Clearly 0 ≤ fi+1 ≤ 23


. We thus have defined inductively functions
 1 2
i 
gi : X → 0, .
3 3
 2
i+1 
fi+1 : C → 0, .
3
Clearly, from (14), we obtain
∞
X
gi |C = f0 = f.
i=0

29
 P∞
Setting f˜ = i=0 gi , we have f˜ ∈ C(X), since
m
X
|gi | ≤ (2/3)n ;
i=n

we’re done.

6.2 Compactness: Arzela-Ascoli

Now we shall find a characterization for the compact subsets of C(K). The
following definition will be useful
Definition 6.3. Let X be a metric space and E a subset, and ǫ > 0. A set
N ⊂ X is said to be an ǫ-net for E if

E ⊂ ∪x∈N Bǫ (x).

Definition 6.4. Let X be a metric space, and E a subset. E is said to be

totally bounded if for all ǫ > 0 there exists a finite ǫ-net N for E in X.
Note that a subset S of a totally bounded set E is again totally bounded,
and an ǫ-net for E is an ǫ-net for S. If E is totally bounded than so is E.
Also note that totally bounded sets are certainly bounded. Finally, note that if
N ⊂ X is an ǫ-net for E, then there exists a 2ǫ net Ñ ⊂ E for E. Thus in the
above definition, we can equivalently require that N ⊂ E.
We have the following
Proposition 6.2. A set E is totally bounded iff for every sequence yi ∈ E,
there exists a subsequence which is Cauchy.
Proof. Suppose E is totally bounded and let yi be a sequence in E. Let Nj
denote an 1/j net for E, which exists by assumption. For j = 1, there is an
x1 ∈ N1 such that B1 (x1 ) that contains infinitely many of the yi . Let i1 be the
smallest such i. Now given yik ∈ ∩kj=1 B1/j (xj ) for all 1 ≤ k ≤ n, such that xj ∈
Nj and B1/j (xj ) contains infinitely many {yi }∩B1/m (xm ) for m <Tj, let xn+1 be
n
such that B1/(n+1) (xn+1 ) contains infinitely many of the {yi } ∩ j=1 B1/j (xj ),
and let in+1 be the first i ≥ ik such that yin+1 ∈ ∩n+1
j=1 B1/j (xj ). For n < m, we
have

d(yin , yim ) ≤ d(yin , xn ) + d(xn , yim )

≤ 2/n.

Thus yik is Cauchy.

Now suppose E is not totally bounded. Let ǫ > 0 be such that there does not
exist a finite ǫ-net. Choose y1 ∈ E at random. Having chosen y1 , . . . , yn ∈ E
with the property that d(yi , yj ) > ǫ for 1 ≤ i ≤ j ≤ n, since E 6⊂ ∪ni=1 Bǫ (yi ),
there exists a yi+1 ∈ E \ ∪ni=1 Bǫ (yi ). By induction one obtains a sequence {yi }
such that d(yi , yj ) > ǫ for all i, j. No subsequence of yi can be Cauchy.

30
Corollary 6.2. Let X be a complete metric space. A set E is totally bounded
iff E is compact.
Thus, since C(K) is a Banach space we are more than happy to classify
totally bounded sets.
We have
Definition 6.5. A subset F ⊂ C(K) is called equicontinuous at x ∈ K if for
every ǫ > 0 there exists a neighborhood U of x such that for y ∈ U, |f (y)−f (x)| <
ǫ for all f ∈ F. We say that F is equicontinuous if it is equicontinuous at x
for all x ∈ K.
Note than finite subsets of C(K) are clearly equicontinuous.
The Arzela-Ascoli theorem states
Theorem 6.2. Let K be compact Hausdorff. Then F ⊂ C(K) is totally bounded
iff it is bounded and equicontinuous.
Proof. Suppose F is totally bounded. It is certainly bounded. Given ǫ >
0, let {fi }ni=1 be an ǫ-net for F . Given x ∈ K, then Ui be subsets so that
|fi (y) − fi (x)| < ǫ. Define U = ∩i Ui . We have for y ∈ U,

|f (y) − f (x)| ≤ |f (y) − fi (y)| + |fi (y) − fi (x)| + |fi (x) − f (x)|.

Since {fi } forms an ǫ-net, there exists an i such that |f − fi | < ǫ. For this i, we
have then
|f (y) − f (x)| ≤ ǫ + |fi (y) − fi (x)| + ǫ ≤ 3ǫ.
We have shown that F is equicontinuous at x.
Conversely, suppose F is bounded and equicontinuous. Given ǫ > 0, x, let
Ux denote an open set around x such that |f (y) − f (x)| < ǫ for all y ∈ Ux .
The collection {Ux } forms an open cover for K, and thus there exists a finite
subcover {Uxi }ni=1 .
n
Consider F |{xi } as a subset of l∞ . By assumption, this is a bounded subset,
n
and thus, since we are in l∞ , totally bounded. There thus exists an ǫ-net
{fj |{xi } }m
j=1 ⊂ F |{xi } for this subset. For any j, we have

|f − fj | = max sup |f (y) − fj (y)|

i y∈Ui
≤ max sup |f (y) − f (xi )| + |f (xi ) − fj (xi )| + |fj (xi ) − fj (y)|
i y∈Ui
≤ max(ǫ + |f (xi ) − fj (xi )| + ǫ)
i
= (f − fj )|{xi } ∞
+ 2ǫ.

Since {fj |{xi } }m

j=1 ⊂ F |{xi } is an ǫ-net, there exists a j such that |(f −fj )|{xi } |∞ <
ǫ, thus, for this j,
|f − fj | < 3ǫ.
The functions fj thus constitute a 3ǫ-net for F in C(K). Thus F is totally
bounded.

31
 Note that examining the above proof, it is clearly enough to assume that
the family F is pointwise bounded, that is, for all x, supf ∈F |f (x)| < ∞.

Example. Consider K = U, where U is a bounded open subset in Rn . Let

B 1 (R) denote the open ball of radius R > 0 in C 1 (U). Then B 1 (R) ⊂ C 0 (U) is
equicontinuous.

6.3 Aside: The role of compactness results in analysis

What are compactness theorems in function spaces good for? We give some ap-
plications. One should think of these as at the heart of the theory of differential
equations, ordinary and partial.
A pedestrian way of thinking about the role of compactness is as follows:
Suppose you have a problem where the unknown is a function f , and you think
you have a good way of approximating solutions to this problem, call these
approximate solutions fi . Then if these fi ∈ B, where B is compact, it follows
that one can draw a convergent subsequence fik → f in the relevant space. The
function f is a good candidate for a solution to the problem.
We will proceed in this section to illustrate this idea in proving the existence
of solutions to o.d.e.’s with low regularity.
You are probably already familiar with the fundamental existence and unique-
ness theorem for o.d.e.’s, which states
Theorem 6.3. Let f : R → R be locally Lipschitz. Then for any t0 , x0 , there
exists a unique interval (T− , T+ ) such that −∞ ≤ T− < t0 < T+ ≤ ∞ and a
unique C 1 function x : (T− , T+ ) → R satisfying the initial value problem

x′ = f (x), x(t0 ) = x0 ,

such that x is not the restriction of an x̃ satisfying the above on a larger interval.
Moreover, let K ⊂ R be compact. Then if T± 6= ∞ there exist t+ < T+ , t− > T− ,
respectively, such that such that x([t+ , T+ )) ∩ K = ∅, x((T− , t− ]) ∩ K = ∅,
respectively.
This theorem can be proven with the so-called Banach fixed point theorem
or with Picard iteration. On the other hand, what happens when we weaken the
Lipschitz condition to continuity? In this section, we will prove the following
Theorem 6.4. Let f : R → R be continuous. Then for any t0 , x0 , there exists
an ǫ > 0 and a function x : (t0 − ǫ, t0 + ǫ) → R satisfying the initial value
problem
x′ = f (x), x(t0 ) = x0 . (15)
That is to say, we still have existence, but we have lost uniqueness.

Example. Consider the IVP: x′ = 2|x|1/2 , x(0) = 0. One solution is x = 0,

and an other is x = 0 for t ≤ 0, and x = t2 for t ≥ 0. What are all solutions?

32
Proof. We shall prove Theorem 6.4 from Theorem 6.3. We shall make funda-
mental use of Arzela-Ascoli.
Choose δ > 0, and let F = sup|x−x0 |≤δ |f (x)|. Let ǫ > 0 be such that
ǫ(F + 1) ≤ δ/2. Let B denote the set {f + g̃} where g̃ ranges over functions
R → R with supremum less than or equal to 1.
Lemma 6.2. If x : [t0 − ǫ− , t0 + ǫ+ ] → R solves (15), for 0 < ǫ± ≤ ǫ, with
f˜ ∈ B in place of f , then
|x − x0 | < δ, (16)
|x′ | ≤ F + 1. (17)
The above lemma is an example of an a priori estimate.
Proof. For such an x, we have

x′ = f˜(x)

for a f˜ in B. Thus, in view of the definition of B and F , (17) would follow

immediately from (16).
Consider the subset t ∈ [t0 , t0 + ǫ] such that (16) holds for all t̄ ∈ [2t0 − t, t] ∩
[t0 − ǫ− , t0 + ǫ+ ]. Call this subset T . The set T is clearly open in [t0 , t0 + ǫ],
by continuity of x. Moreover, t0 ∈ T , and thus T = 6 ∅. On the other hand, for
t ∈ T , we have

sup |f˜(x(t̄))| ≤ sup |f (x)| + 1

[2t0 −t,t]∩[t0 −ǫ− ,t0 +ǫ+ ] |x−x0 |≤δ
≤ F + 1.

Thus, for t̄ ∈ [2t0 − t, t] ∩ [t0 − ǫ− , t0 + ǫ+ ],

Z t̄
|x(t̄) − x0 | = x′ (t̂)dt̂
t0
Z t̄
= f (x(t̂))dt̂
t0
Z t̄
≤ (F + 1) dt̂
t0

≤ (F + 1)ǫ
≤ δ/2.

By continuity of x, this implies that T ⊂ [t0 , t0 + ǫ] is closed. By connectedness

of [t0 , t0 + ǫ], it follows that T = [t0 , t0 + ǫ]. That is to say, we have shown that
(16) holds for all t ∈ [t0 − ǫ− , t0 + ǫ+ ].
Now consider, f˜ ∈ B ∩ C 1 , and let x : (T− , T+ ) → R be the solution of
Theorem 6.3. In view of Lemma 6.2, we have that T− < t0 − ǫ < t0 + ǫ < T+ .
Thus, we may restrict x to a function x : [t0 − ǫ, t0 + ǫ].

33
 Consider the subset S ⊂ C 1 ([t0 − ǫ, t0 + ǫ]) which consists of solutions to
(15) on the interval [t0 − ǫ, t0 + ǫ], with f replaced by f˜ ∈ B, restricted to
[x0 − δ, x0 + δ]. We have just shown that this set is nonempty, and it contains
a unique function corresponding to any f˜ ∈ B ∩ C 1 ([x0 − δ, x0 + δ]).
Now we show
Lemma 6.3. S ⊂ C([t0 − ǫ, t0 + ǫ]) is totally bounded.
Proof. In view of Arzela-Ascoli, it suffices to show the uniform boundedness and
equicontinuity of S. This is immediate from (16) and (17).
On the other hand, we have
Lemma 6.4. C 1 ([x0 − δ, x0 + δ]) ⊂ C([x0 − δ, x0 + δ]) is dense.
Proof. Omitted. This will follow in particular from the next section. In the
meantime, see if you can come up with a direct proof.
In particular, consider a sequence fi → f with fi ∈ B ∩ C 1 ([x0 − δ, x0 + δ]).
By density, such a sequence exists. Let xi ∈ S denote the corresponding solution
to (15) with fi . By Lemma 6.3, there exists a subsequence xik such that xik → x
for some x ∈ C 0 ([t0 −ǫ, t0 +ǫ]). The function x is a good candidate for a solution
of (15)! But we have to be careful; we’re not done yet. We still must deduce
that x actually solves (15). (At this point, we don’t even know yet that x is
differentiable!)
This doesn’t turn out to be so difficult. Since fik → f uniformly, and
xik → x uniformly, it follows that fik (xik )(t) → f (x(t)) uniformly in t. Since
x′ik = fik (xik ), this implies by an elementary result in real analysis that x′ exists
and equals lim x′ik , i.e. f (x).
The importance of these ideas is even greater when one passes from ordinary
differential equations to partial. Such applications would take us, however, too
far afield.

6.4 Dense subsets in C(K): Stone-Weierstrass

We have already seen in the last section the importance of having good dense
subsets of a relevant function space. See Lemma 6.4. In this section, we shall
prove a general theorem that will allow us to come up with many useful dense
subsets of C(K). Here two algebraic notions will play a crucial role, that of a
lattice and that of an algebra.
Definition 6.6. Let (V, +) be a vector space over R or over C, and let · :
V × V → V be a binary operation. We say that (V, +, ·) is an algebra (over R
of C) if (V, +, ·) is a rng9 , and λ(v · w) = v · (λw) = (λv) · w, for scalars λ.
If V is a normed vector space and |v · w| ≤ |v||w|, we say that V is a normed
algebra. If V is in addition Banach, we say that V is a Banach algebra. If V
9 ring without necessarily multiplicative identity

34
is commutative as a ring, we say that V is a commutative algebra. If V has a
multiplicative identity, we say V is unitial.
Examples. The space C(K) is certainly a unitial commutative Banach alge-
bra under the usual multiplication of functions. L(V, V ) is an algebra under
composition of linear transformations. If V is a normed vector space, then
B(V, V ) ⊂ L(V, V ) is a unitial subalgebra which is a normed algebra, which if
V is a Banach space, is a unitial Banach algebra.
On the other hand
Definition 6.7. A lattice is a partially ordered set L such that any two element
subset has a least upper bound and a greatest lower bound.
If p and q are two elements of L, we may denote the least upper bound and
greatest lower bound by p ∨ q and p ∧ q, respectively. We may give equivalently
an algebraic characterization of the notion of lattice in terms of a set L and
two binary operations ∨ and ∧ satisfying a collection of axioms. What are the
axioms?
The vector space FR (X), the real valued functions on a topological space
X, is a lattice where FR (X) is given the obvious partial ordering f ≤ g ⇐⇒
∀x f (x) ≤ g(x). We have then

(f ∨ g)(x) = max{f (x), g(x)}, (f ∧ g)(x) = min{f (x), g(x)}.

The space of continuous functions CR (X) ⊂ FR (X) is easily seen to be closed

under the lattice operations, i.e. it is a sublattice of FR (X) and thus itself a
lattice.
In what follows for now, we will be considering CR (K), where K is compact
Hausdorff. The main theorem of this section, known as the Stone-Weierstrass
Theorem, is the following
Theorem 6.5. If A ⊂ CR (K) is a subalgebra such that A separates points, then
A = C(K), or there exists an x ∈ K such that A = {f ∈ C(K) : f (x) = 0}.
Example. Let A denote the set of polynomials in x1 . . . xn , thought of as
functions on U, where U is a bounded open subset of Rn . The set A is easily
seen to be an algebra separating points, and since 1 ∈ A, the second possibility
of the theorem is excluded. Thus A = C(U). This is the classical Weierstrass
theorem. Note that since A ⊂ C k (U) for any k ≥ 0, we have in particular
C k (U) = C(U ).
Proof. The importance of real-valued functions is precisely that we may exploit
the lattice property. We have the following
Lemma 6.5. Suppose L ⊂ CR (K) is a sublattice, and suppose that g ∈ CR (K)
is such that for each x, y ∈ K, there exists an f ∈ L such that |f (x) − g(x)| < ǫ,
|f (y) − g(y)| < ǫ. Then g ∈ L. In particular, if this assumption is true for all
g ∈ CR (K), then L = CR (K).

35
Proof. Let g(x) ∈ CR (K) be as in the assumption of the lemma. Given ǫ > 0,
we will produce an f ∈ L such that |f − g| < ǫ.
Pick an x ∈ K, and for each y ∈ K, let fx,y be a function with

|fx,y (x) − g(x)| < ǫ, |fx,y (y) − g(y)| < ǫ.

By continuity of fx,y , g, the inequality |fx,y − g| < ǫ must hold in open sets
around x, y. Call these open sets Vx,y , Ux,y . We have that {Ux,y } is an open
cover for K, and thus there exists a finite subcover Ux,yi . If we consider Vx =
∩ni=1 Vx,yi , and
fx = fx,y1 ∧ · · · ∧ fx,yn ,
we have
fx (y) < ǫ + g(y) (18)
for all y, and
g − ǫ < fx (19)
in Vx . By the defintion of a sublattice, fx ∈ L. Now, consider the open cover
Vx for K. By compactness of K there exists a finite subcover Vxj . Define

f = fx1 ∨ · · · ∨ fxm .

Again, since L is closed under the lattice operations, f ∈ L. We have

f (y) < ǫ + g(y)

on account of (18), and

g(y) − ǫ < f (y)
on account of (19), by definition of ∨ and the fact that Vx is a cover. That is
to say, we have, for all y,
|f (y) − g(y)| < ǫ,
as required.
What has all this to do with subalgebras, you will say. The answer is given
by the following lemma.
Lemma 6.6. Let A ⊂ CR (K) be a subalgebra, closed in the topology of CR (K).
Then A is a sublattice of CR (K).
Proof. We note that max{f (x), g(x)} = 12 (f (x) + g(x) + |f (x) − g(x)|), and
similarly for min. Thus, it suffices to show that if f ∈ A, so is x 7→ |f (x)|.
Moreover, it suffices to show this for f satisfying,
√ say, |f | ≤ 1. (Why?)
Now, consider the function gǫ (x) = ǫ2 + x. For −1 ≤ x ≤ 1, we have that
|gǫ (x2 ) − |x|| ≤ ǫ.
The function gǫ is analytic in a neighborhood of t = 12 and equal to its power
series in [0, 1], which moreover, converges uniformly. Write
   2
1 1
gǫ (x) = c0 + c1 x − + c2 x − + ...
2 2

36
 Pn
and let Sn denote the partial sum i=0 ci (x − 21 )i , and let S̃n = Sn − Sn (0).
We have that S̃n is a polynomial without constant term. On the other hand,
Sn (0) → ǫ, as n → ∞, thus, for n ≥ N , we have |Sn (0)| < 2ǫ.
We have that S̃n ◦ f 2 ∈ A, as A is an algebra, and S̃n has no constant term.
Since |f | ≤ 1 implies that 0 ≤ f 2 ≤ 1, we have that, given ǫ > 0, there exists
an Ñ such that, for n ≥ Ñ ,
|S̃n ◦ f 2 − |f || ≤ |S̃n ◦ f 2 − gǫ ◦ f 2 | + |gǫ ◦ f 2 − |f ||
≤ |S̃n ◦ f 2 − Sn ◦ f 2 | + |Sn ◦ f 2 − gǫ ◦ f 2 |
+ |gǫ ◦ f 2 − |f ||
< 2ǫ + ǫ + ǫ = 4ǫ.
(In the above centred formula, |f | denotes the function x 7→ |f (x)|, not the sup
of f .) Thus, since A is assumed closed, and ǫ > 0 is arbitrary, we are done.
Aside. One can remark that in the above lemma we do not use continuity. We
could replace CR (K) in the statement with the subspace FRb (K) ⊂ FR (K) of
bounded functions, given the supremum norm. Compare with Lemma 6.5 where
continuity is used in a fundamental way.
To complete the proof of Theorem 6.5, there is very little to say. Consider
A. Note that this is again an algebra by the continuity of the multiplication,
addition, etc. Thus, by Lemma 6.6, A is a sublattice of C(K).
Let us suppose that for all x ∈ K, there exists an f ∈ A such that f (x) 6= 0.
For x 6= y, let fx and be a function such that fx (x) 6= 0, let fy be a function
such that fy (y) 6= 0, and let fx,y be a function such that fx,y (x) 6= fx,y (y).
By defining f˜ = fx + αfx,y + βfy , for some α, β ∈ R, we obtain a function
such that f˜(x) 6= 0, f˜(y) 6= 0, and f˜(x) 6= f˜(y). It follows that (f˜(x), f˜(y)),
(f˜2 (x), f˜2 (y)) are linearly independent, and thus the assumptions of Lemma 6.5
hold for arbitrary g ∈ C(K).
Applying thus Lemma 6.5, we have shown thus that if for all x ∈ K there
exists an f ∈ A such that f (x) 6= 0, it follows that A = C(K).
On the other hand, suppose now that there exists a point such that for all
f ∈ A, f (x) = 0. Let us consider the algebra A′ which is spanned by A and the
constants. This is easily seen to equal {A + λ1}λ∈R . We have that A′ satisfies
the property enunciated in the previous paragraph, and in addition, separates
points. Thus, we have A′ = C(K). But now let g ∈ C(K) such that g(x) = 0,
and let ǫ > 0 be arbitrary. This means we may write
|g − (f + λ)| < ǫ
for some f ∈ A, λ ∈ R. Evaluating at x, since f (x) = g(x) = 0, we obtain
|λ| < ǫ. Thus, |g − f | < 2ǫ. It follows that g ∈ A.
We state the complex version of Stone-Weierstrass
Theorem 6.6. Let A ⊂ CC (K) be a subalgebra over C separating points, and
moreover, suppose that A is closed under complex conjugation. Then A =
CC (K), or there exists an x ∈ K such that A = {f ∈ CC (K) : f (x) = 0}.

37
Proof. Recall that ℜf = 12 (f + f¯), ℑf = 2i (f¯ − f ). Thus, by assumption,
f ∈ A =⇒ ℜf ∈ A, ℑf ∈ A. Consider the subalgebra A′ (over R) of CR (K)
generated by ℜf , ℑf , for all f ∈ A. We have A′ ⊂ A, and moreover, it is easily
seen to separate points.
Suppose that for all x ∈ K, there exists an f ∈ A′ such that f (x) 6= 0.
Then by Theorem 6.5, we have that A′ = CR (K). But then for any u ∈ CR (K),
v ∈ CR (K) there exists fj ∈ A′ → u, gj ∈ A′ → v, and thus fj + igj → u + iv.
But fj + igj ∈ A. Thus, we have shown in this case A = CC (K).
If on the other hand, there exists an x such that f (x) = 0 for all x ∈ A′ ,
then argue as in the last part of the proof of Theorem 6.5.

6.5 Applications to Fourier series and the discovery of or-

thogonality
Theorem 6.6 is useful in the context of classical Fourier series.
Let K = S1 , parametrized, say, by (−π, π], and consider the vector space
A generated by {einx }n∈Z . The elements of A are called the trigonometric
polynomials. A is clearly an algebra. Moreover, A separates points, and contains
the constants. Finally, since einx = e−inx , it follows that A is closed under
complex conjugation. It follows that any f ∈ C(S1 ) can be approximated by an
element of A.
This fact is crucial in the proof of:
PN ˆ
Proposition 6.3. Let f ∈ C(S1 ), and let SN (f ) = n=−N f (n)e
inx
be the
N ’th partial sum of f ’s Fourier series, i.e. where
Z π
1
fˆ(n) = f (t)e−int .
2π −π

We have that Z
lim |f − SN (f )|2 → 0.
N →∞

Proof. If P is a trigonometric polynomial and N ≥ deg P , then it follows that

SN (P ) = P . By Stone-Weierstrass applied to trigonometric polynomials, given
ǫ, there exists a P such that |P − f | < ǫ. For such N we have

|f − SN (f )|2 ≤ |f − P |2 + |P − SN (P )|2 + |SN (P ) − SN (f )|2

= |f − P |2 + |P − SN (P )|2 + |SN (P − f )|2
= |f − P |2 + |SN (P − f )|2

and thus
Z π Z π Z π
2 2
|f − SN (f )| ≤ |f − P | + |SN (P − f )|2
−π −π −π
≤ 2πǫ + 2πǫ2 .
2

38
 Where did the last inequality come from? The claim is that for any function
g ∈ C(S1 ), we have Z π Z π
|SN (g)|2 ≤ |g|2 . (20)
−π −π
Try showing this directly at this stage.
The geometric structure of (20) may not be immediately apparent. The einx
form what is known as an orthonormal set in C(S1 ), with respect to the “inner
product” Z π
1
f ·g = f ḡ.
2π −π
Orthonormal means einx · eimx = δnm , where δnm = 1 if n = m and 0 otherwise.
Modulo the convenient factor of (2π)−1 , this inner product is related to the L2
norm by |f |2 = f · f .
In this language, we can understand SN as an “orthogonal projection”, and
(20) will just follow from the general statement that orthogonal projections do
not increase the norm.
We embark in the next section on a general study of normed spaces whose
norm arises from an inner product in the sense described above. These are
called Euclidean spaces. A Euclidean space which is also complete is called a
Hilbert space. We will return to thus to (20) later. . .

7 Hilbert space
In this section, we shall introduce the concept of a Hilbert space, that is to say
a Banach space whose norm arises from an inner product. We have already
seen to a certain extent at the end of the previous section how the notion of
orthogonality may be useful. It is hard to give a sense of just how important
this notion is in analysis.
The inner product structure will also allow us to revisit the notion of duality,
defined for general Banach space earlier. As we shall see, for Hilbert spaces, the
dual and the adjoint map have very concrete realisations.

7.1 Defintions: inner product space, Euclidean space, Hil-

bert space
Definition 7.1. Let V be a vector space over R or C. Let p : V × V → R, C,
respectively, be a map such that
p(v, w) = p(w, v) (21)
p(λ1 v1 + λ2 v2 , w) = λ1 p(v1 , w) + λ2 p(v2 , w) (22)
p(v, v) ≥ 0, p(v, v) = 0 iff v = 0. (23)
We call p an inner product on V ; we often denote it by < v, w > instead of
p(v, w). A vector space V together with such an inner product <, > is known as
an inner product space.

39
 The properties (21), (22) together are sometimes called sesquilinearity and
define the notion of a Hermitian form. In the real case, then this just means
that p is a bilinear form.
If < v, w >= 0 we say that v and w are orthogonal.
The most fundamental perhaps property of inner product spaces is the so-
called Schwarz inequality.
Proposition 7.1. Let (V, <, >) be an inner product space. Then
√
| < v, w > | ≤ < v, v >< w, w >, (24)

with equality iff v = λw.

Proof. If v = λw, we clearly have equality. By replacing w by w̃ = αw with
|α| = 1, we can arrange so that < v, w̃ > is real. The validity of the equality for
w̃ implies that for w. So in what follows we may assume < v, w > is real, and,
moreover, that v 6= λw
Consider < v + tw, v + tw >, for t ∈ R. By (23) and our assumption, this is
strictly positive for all t. Expanding, we obtain

< v + tw, v + tw > = < v, v > + < v, tw > + < tw, v > +t2 < w, w >
= < v, v) + 2t < v, w > +t2 < w, w > .

Since this polynomial is strictly positive its roots are not real, that is to say
4 < v, w >2 −4 < v, v >< w, w >< 0. But this is the desired ineqality.
Aside: In general, we call a vector v isotropic with respect to a Hermitian
form if p(v, v) = 0; we call a Hermitian form positive if p(w, w) ≥ 0 for all w,
and we call it non-degenerate if p(v, w) = 0 for all w implies v = 0. The above
proposition has thus shown that a positive hermitian form is non-degenerate iff
there are no isotropic vectors, i.e. iff it is positive definite.
Proposition 7.2. Let (V, <, >) be an inner product space. Define | · | on V by
√
|v| = < v, v >. (25)

Then | · | defines a norm on V .

Proof. It is enough to show the triangle inequality. We have

|v + w|2 = < v + w, v + w >

= < v, v > + < v, w > + < w, v > + < w, w >
≤ < v, v > +2| < v, w > |+ < w, w >
√
≤ < v, v > +2 < v, v >< w, w >+ < w, w >
√ √
= ( < v, v > + < w, w >)2

from which the triangle inequality follows immediately.

40
Definition 7.2. A normed vector space (E, | · |) where | · | is defined by (25) for
some inner product <, > on E, is called a Euclidean space.
Proposition 7.3. Let (E, | · |) be a Euclidean space. Then there is a unique
<, > such that (25) holds.
Proof. Let <, > be such that (25) holds. In the case of a real inner product, we
have
1
< v, w > = (< v + w, v + w > − < v, v > − < w, w >)
2
1
= (|v + w|2 − |v|2 − |w|2 )
2
whereas in the case of a complex inner product, we have

< v, w > + < w, v >=< v + w, v + w > − < v, v > − < w, w >,

while

−i < v, w > +i < w, v >=< v + iw, v + iw > − < v, v > + < w, w >

from which we obtain

< v, w >= . . . ,
an expression the reader is invited to fill in.
The above identities are sometimes known as the polarization identities.
In what follows then, given a Euclidean space V , then <, > will denote this
uniquely defined inner product giving | · | by (25).
The following parallelogram law may be familiar.
Proposition 7.4. Let (E, | · |) be a Euclidean space. Then

|v − w|2 + |v + w|2 = 2|v|2 + 2|w|2 (26)

Proof. We expand using the inner product.

|v − w|2 + |v + w|2 = < v − w, v − w > + < v + w, v + w >

= < v, v > − < v, w > − < w, v > + < w, v >
+ < v, v > + < v, w > + < w, v > + < w, w >
= 2 < v, v > +2 < w, w >
= 2|v|2 + 2|w|2 .

Also,
Proposition 7.5. Let (E, | · |) be a Euclidean space. Suppose v and w are
orthogonal. Then
|v + w|2 = |v|2 + |w|2 (27)

41
Proof. Expand using the inner product. . .
Iterating, we have that if vi are pairwise orthogonal than
n 2 n
X X
vi = |vi |2 .
i=1 i=1

Finally, we have
Definition 7.3. Let (H, | · |) be a Euclidean space which is Banach, i.e. such
that | · | defines a complete metric. We say H is a Hilbert space.
It turns out that any Euclidean space can be embedded into a larger Hilbert
space by taking the completion. For this, let us first note the following easy
proposition:
Proposition 7.6. Let E be a Euclidean space. Then <, >: E × E → C is
continuous.
Proof. This follows from the computation

| < v, w > − < ṽ, w̃ > | ≤ | < v, w > − < v, w̃ > | + | < v, w̃ > − < ṽ, w̃ > |
= | < v, w − w̃ > | + | < v − ṽ, w̃ > |
≤ |v||w − w̃| + |v − ṽ||w̃|.

From this follows easily

Proposition 7.7. Let E denote a Euclidean space, and Ē its completion. Then
the inner product extends to an inner product on Ē, making the latter into a
Hilbert space.
Proof. This also follows easily from the computation of the previous proposition.
The uniqueness follows in particular directly from the continuity of the inner
product.
In view of the above Proposition, Euclidean spaces are sometimes called
pre-Hilbert spaces. We shall not use such awful terminology here.10

7.1.1 Examples
For us, this whole story began with C(S1 ) with
Z π
.
< f, g >= f ḡ.
−π
10 I will also resist the temptation to call Hilbert spaces post-Euclidean spaces.

42
We could also equally well have taken C([a, b]) for some a < b. This is clearly a
Euclidean space with the “L2 norm”
s
p Z b
|f |2 = < f, f > = |f |2
a

Alas, this is not a Hilbert space, as we know already from our discussion of
Banach spaces. Its completion is, by Proposition 7.7. It is a miraculous fact
that this completion can be realised as a set of (equivalence classes of) Lebesgue
measurable functions, where < f, g > is defined with respect to the Lebesgue
integral. This is the space L2 , which in any case, we have discussed before in the
context of Banach spaces. For this, however, you will need more mathematical
technology than you have.
So we have to settle for little l2 . This space is easily seen to be a Euclidean
space with inner product X
< a, b >= ai b i .
√
Since we know it to be a Banach space with the norm |a|2 = < a, a >, it
follows that l2 is a Hilbert space.
The space l2 is clearly separable, i.e. there exists a countable dense subset.
We will see later on that all infinite dimensional separable Hilbert spaces are
isometrically isomorphic to l2 . So l2 is not a bad example to have. . .
This being said, the whole point about setting up the theory of Hilbert spaces
is applying it to solve problems in analysis. Knowing, thus, that a particular
space is indeed a Hilbert space, is fundamental. This is why it really is a shame
that we cannot talk about L2 .

7.2 Orthogonal complements

7.2.1 Basic definitions
Let E be Euclidean. We have already defined what it means for v and w to be
orthogonal, namely < v, w >= 0.
Definition 7.4. Let S ⊂ E. We define the orthogonal space of S, denoted S ⊥ ,
.
to be the set S ⊥ = {v ∈ E : ∀w∈S , < v, w >= 0}.
A trivial application of continuity and linearity of the inner product yields
Proposition 7.8. Let S ⊂ E. Then S ⊥ is a closed subspace of E, and S ⊥ =
(Span S)⊥ .
Finally, if V is itself a subspace, since we have that V ∩ V ⊥ = 0, it follows
that V + V ⊥ in our usual notation is equal to the direct sum V ⊕ V ⊥ .

7.2.2 Existence of projection maps

Experience from finite dimensional euclidean space may lead you to believe
that V ⊕ V ⊥ = E, for any E, at least if V is closed. This isn’t always true in
Euclidean spaces! However, we have

43
Theorem 7.1. If F ⊂ E be a subspace, where F is assumed complete and E is
Euclidean, then F ⊕ F ⊥ = E. Moreover, for arbitrary x ∈ E, writing uniquely
x = x1 + x2 , where x1 ∈ F , x2 ∈ F ⊥ , then x1 is characterized uniquely by

|x2 | = |x1 − x| = inf |y − x|.

y∈F

Note that, of course, the assumptions of the theorem are satisfied if F is finite
dimensional, or alternatively, if E is Hilbert and F is closed. In particular, in a
Hilbert space, they are satisfied if F = S ⊥ .
Proof. The statement of the theorem should tip you off to the nature of the
proof. Take a minimising sequence yi for |y − x|, that is to say, a sequence
.
yi ∈ F such that lim |yi − x| = inf y∈F |y − x| = d.
First, I claim that yi is Cauchy. For this, apply (26) for v = x−yi , w = x−yj
to obtain

|yj − yi |2 + |2x − yi − yj |2 = 2|yi − x|2 + 2|yj − x|2 ,

and thus

|yj − yi |2 = 2|yi − x|2 + 2|yj − x|2 − |2x − yi − yj |2

= 2|yi − x|2 + 2|yj − x|2 − 4|x − (yi − yj )/2|2
≤ 2(d + ǫ) + 2(d + ǫ) − 4d
≤ 4ǫ

if i, j are such that |yi − x|2 ≤ d + ǫ, |yj − x|2 ≤ d + ǫ.

So yi is indeed Cauchy, and thus, by the assumption of completeness, con-
verges to some y. By continuity of | · |, it follows that |y − x| = d.
So the claim is now that y is the unique element of F that achieves d, and
that we can set x1 = y, x2 = y − x.
So define x2 as above and suppose x2 6∈ F ⊥ . Let ỹ ∈ F such that < x2 , ỹ >6=
0. By multiplying ỹ by a scalar, we can arrange so that < x2 , ỹ >< 0. Now
consider for t > 0,

< y + tỹ − x, y + tỹ − x > = < y − x, y − x > + < tỹ, y − x >

+ < y − x, tỹ > +t2 < ỹ, ỹ >
= d2 + 2t < ỹ, x2 > +t2 |ỹ|2 .

Now, for small enough t the second term is negative and dominates the last
term. For such a t then we have

< y + tỹ − x, y + tỹ − x >< d2

and this contradicts the definition of d in view of the fact that y + tỹ ∈ F .
When F ⊕ F ⊥ = E, we say that F ⊥ is an orthogonal complement of F .
We may interpret the above Theorem in terms of projection operators.

44
Corollary 7.1. Let F , E be as in Theorem 7.1. There exists a unique operator
P : E → E such that P (E) = F , P (F ⊥ ) = 0, P 2 = P , (I − P )(E) = F ⊥ ,
(I − P )(F ) = 0, (I − P )2 = (I − P ), and ||P || ≤ 1, ||I − P || ≤ 1, with equality
if F 6= 0, F 6= E, respectively.

7.2.3 The Riesz representation theorem

Let E be a Euclidean space. Clearly, by the Schwarz inequality, an element v
defines an element of the Banach space E ∗ by

w 7→< w, v > .

Denote this map φv ∈ E ∗ , and let φ : E → E ∗ take v to φv . We have |φv | = |v|.

Now suppose H is Hilbert. We have
Proposition 7.9. Let H be Hilbert, and φ be defined as above. Then φ : H →
H ∗ is an isometric anti-isomorphism. In particular, H ∗ is a Hilbert space.
Proof. Let 0 6= ξ ∈ H ∗ . Consider ker ξ. By continuity of ξ, ker ξ is a closed
subspace, and thus, by Theorem 7.1, there exists an orthogonal complement.
By linear algebra, this is 1-dimensional and thus spanned by some ṽ. Define
v = aṽ, so that
< v, v >= ξ(v).
By linearity, now < bv, v >= ξ(bv). Thus φv and ξ coincide on ker ξ and on
(ker ξ)⊥ . We have shown surjectivity for φ. The anti-isomorphism part follows
easily.
Orthogonality and completeness allow us to realise the abstract dual in a
very explict way!

7.2.4 Aside: Return to Fourier series

Consider C(S1 ) with the L2 norm and the problem considered in Section 6.5.
The claim is that the map SN is precisely the map P : E → F of the Corol-
lary 7.1, where E is C(S1 ), and F is the finite dimensional, and thus complete,
subspace spanned by {einθ }|n|≤N . Given this, the statement ||SN || ≤ 1 follows
from Corollary 7.1.
So why is SN a projection operator? We know F ⊕ F ⊥ = E. We know
already that f ∈ F implies SN f = f . On the other hand, f ∈ F ⊥ means that
< f, einθ >= 0, for |n| ≤ N , and thus SN f = 0. But now this implies that SN
satisfies the conditions for P .

7.3 Orthonormal systems and baseis

We just saw thus in the previous section the importance of having an orthonor-
mal set of vectors, i.e. unit vectors which are pairwise orthogonal.
We now turn to a general discussion. First the definitions, then a return to
the examples.

45
7.3.1 Definitions and existence
Definition 7.5. Let E be Euclidean. A set {eα } of unit vectors is called an
orthonormal system if < eα , eβ >= 0 for α 6= β.
Definition 7.6. Let E be Euclidean. An orthonormal system is called maximal
if it cannot be extended to a strictly larger orthonormal system.
By Zorn’s lemma, there always exists a maximal orthonormal system in any
Euclidean space E.
Proposition 7.10. Let H be Hilbert, and S a maximal orthonormal system.
Then Span S = H.
⊥
Proof. Consider S ⊥ . By Proposition 7.8, we have that S ⊥ = Span S . Setting
F = Span S, we have by Theorem 7.1 that H = F ⊕ F ⊥ . If F ⊥ = 0, we have
that H = F = Span S, as desired. Otherwise, we have that S ⊥ = F ⊥ 6= 0, in
which case there exists a unit x ∈ S ⊥ . The system {x} ∪ S is then orthonormal
and contains strictly S. This contradicts the maximality of S.
Note also the easy converse, which holds in any Euclidean space:
Proposition 7.11. Let E be Euclidean, and S be an orthonormal system such
that Span S = E. Then S is maximal.
Definition 7.7. Let H be Hilbert. A maximal orthonormal system is called a
Hilbert space basis.
Propostion 7.11 thus provides an alternative characterization of a Hilbert
space basis.
We shall see in the examples that either characterization may be easier to
check in showing that a given orthonormal system is indeed a basis.
As discussed before, Zorn’s lemma is not something one should use lightly. It
turns out that in the separable case we can avoid it completely in constructing
Hilbert space baseis. First we note the following
Proposition 7.12. Let {xi }N i=1 be linearly independent for some N ≤ ∞. Then
there exist {ei }N
i=1 such that for all n = 1 . . . N , we have Span (ej )j=1...n =
Span (xj )j=1...n .
The above Proposition, is nothing but the celebrated Gram-Schmidt orthog-
onalization procedure.
Proof. By induction. Let e1 = x1 /|x1 |. Having constructed e1 . . . en , for n ≥ 1,
define
n −1 n
X X

en+1 = xn+1 − < xn+1 , ei > ei xn+1 − < xn+1 , ei > ei .
i=1 i=1

(Note that en+1 = |Pn xn+1 |−1 (Pn xn+1 ), where Pn is the orthogonal projection
to the subspace Span (xj )j=1...n .)

46
 From this, we obtain the following
Proposition 7.13. Let H be separable, and let {yi } be countable set such that
Span (yi ) = H. Then there exists a countable Hilbert space basis for H in the
span of {yi }.
Proof. Go down the list of {yi }, and discard yj in the span of the previous. One
arrives at a {yik } which are linearly independent, and such that Span {yik } =
Span {yi }. Now, apply Gram-Schmidt to {yik }.

7.3.2 Examples
Take l2 , and let ei = (0, . . . , 0, 1, 0, . . .) where the 1 is in the i’th place. This is
clearly maximal, and thus a basis.
Take the completion H of C(S1 ) with respect to the L2 norm. (This space is
also known as L2 .) We know by Stone-Weierstrass that the algebra of trigono-
metric polynomials is dense in C(S1 ) with respect to the sup norm. This means
that it is also dense with respect to the L2 norm. (Why?) Thus A is dense in
H as well. By orthogonality, it follows that A is a basis.
As abstract Hilbert spaces, the above two examples are actually the same. A
countable, non-finite basis for a Hilbert space can be thought of as an isometric
isomorphism with l2 . We turn to this now.

7.4 The isomorphism with l2

For warm up, try proving for yourself the following statements: Let H be a finite
dimensional Hilbert space. Then there exists a Hilbert space basis {ei }ni=1 . H
is isometrically isomorphic to l2n via the map

x 7→ (< x, e1 >, . . . < x, en >).

We have
x =< x, e1 > e1 + . . . + < x, en > en ,
|x|2 = | < x, e1 > |2 + · · · + | < x, en > |2 .
We now move on to the general, separable case. First we show the following
Lemma 7.1. (Bessel’s inequality) Let E be Euclidean, and {ei }N
i=1 be a count-
able orthonormal system, for some 1 ≤ N ≤ ∞. For x ∈ E, define xi =<
x, ei >. Then
XN
|xi |2 ≤ |x|2 .
i=1

If N = ∞, (xi ) ∈ l2 .
Pn
Proof. Let Fn the span of {ei }ni=1 , note that i=1 |xi |2 = |PFn x|2 ≤ |x|2 . If
N = ∞, take the limit.

47
Proposition 7.14. Let H be a separable Hilbert space, and let {ei }N
i=1 be a
countable basis for some 1 ≤ N ≤ ∞. Let x, y ∈ H, and define xi =< x, ei >,
yi =< y, ei >. Then
N
X N
X
x= xi ei , y= yi ei , (28)
i=1 i=1

N
X
< x, y >= xi yi , (29)
i=1

where if N = ∞, the latter series converges absolutely.

Pn
Proof. Consider sn = i=1 xi ei . In the case N = ∞, by Bessel’s inequality, sn
is Cauchy, Thus sn converges to some s by completeness. In the case N < ∞,
set s = sN .
Consider s − x, and sn − x. Since < sn − x, em >= 0 for n ≥ m, it follows
(by continuity of the inner product in the case N = ∞, and trivially if N < ∞)
⊥
that < s − x, em >= 0 for all m < N + 1. Thus, s − x ∈ Span {ei } = 0. So
s = x.
To show (29), Consider < sn , s̃n >, where s̃n is defined for y replacing x.
By the properties of inner products, we have
n
X
< sn , s̃n >= xi y i .
i=1

In the case N < ∞, this immediately gives (29) for n = N . Otherwise, by

the continuity of the inner product, the left hand side converges to < x, y >.
On
Pn the other hand,pPn the rightPnhand side is an absolutely convergent series since
|x y | ≤ |x |2 |y |2 ≤ |x||y|.
i=1 i i i=1 i i=1 i

Formula (29), and its specialisation to x = y, are classically known as Par-

seval’s identities.
Essentially, we have shown that H isometrically embeds to l2 or l2N by the
map x → (xi ). In the finite dimensional case, by dimensionality considerations,
the above immediately proves the statement claimed at the beginning of this
section, that is to say, H is isometrically isomorphic to l2N .
In the countably infinite case, to complete the isometric isomorphism with
l2 , we need the surjectivity of this map. This is provided by the so-called Riesz-
Fisher theorem, which here is demoted to a

Proposition 7.15. Let H be a separable Hilbert space with countably infinite

basis {ei }. Let (xi ) ∈ l2 (C). Then there exists an x ∈ H such that < x, ei >= xi ,
namely
∞
. X
x= xi ei .
i=1

48
 Pn
Proof. Consider the partial sum sn = i=1 xi ei . Since (xi ) ∈ l2 , it follows by
the Pythagorean theorem that the sequence sn is Cauchy. Thus by completeness
sn → x. We show as before now that < x, ei >= xi by noting that < sn , ei >=
xi for n ≥ i, and the continuity of the inner product.

8 Compact self-adjoint operators and Spectral

Theory
8.1 The spectrum and resolvent: definition
Definition 8.1. Let X be a Banach space, and T ∈ B(X, X). We define the
spectrum of T , denoted σ(T ), by

σ(T ) = {λ ∈ C : (T − λI)−1 does not exist}.

The resolvent set ρ(T ) is defined to be complement of the spectrum, i.e.

.
ρ(T ) = C \ σ(T ).

Note that if (T − λI)−1 exists, then it is bounded by the inverse mapping

theorem. For this, the completeness of X is essential.
Definition 8.2. The resolvent of T is a map R : ρ(T ) → B(X, X) defined by
λ 7→ (T − λI)−1 .
Definition 8.3. Let X, T be as above. If Ker(T − λI) 6= 0, we say that λ is an
eigenvalue for T , and Ker(T −λI) is the eigenspace of λ. The set of eigenvalues
is called the point spectrum and denoted σp (T ). Clearly σp (T ) ⊂ σ(T ).
In finite dimensions, for (T − λI)−1 not to exist, it must be by the dimension
theorem that Ker(T − λI) 6= 0, that is to say, in finite dimensions

σp (T ) = σ(T ).

In infinite dimensions, in general σp (T ) 6= σ(T ).

8.2 Structure of the spectrum

In this section, we show the closedness, boundedness, and non-emptyness of the
spectrum.
Theorem 8.1. Let X, T be as above. Then σ(T ) is a closed, nonempty subset
of {|λ| ≤ ||T ||}.
Proof. To prove that σ(T ) is closed: This is equivalent to showing that the
resolvent set ρ(T ) is open.
First we note the following. Let U ⊂ B(X, X) denote the subset of invertible
operators.

49
Lemma 8.1. U is open.
Proof. Let S1 ∈ U. Let S2 be such that ||S1 − S2 || < ǫ. Note that

S2 = S1 (I − S1−1 (S1 − S2 )).

Define Q = S1−1 (S1 − S2 ). If ||Q|| < 1, then

(I − Q)−1 = 1 + Q + Q2 + Q3 + . . . (30)

and
1
||(I − Q)−1 || ≤ .
1 − ||Q||
We have thus that S2 is invertible for ǫ < ||S1−1 ||−1 , with S2−1 given by S2−1 =
(I − Q)−1 S1−1 and moreover,

||S1−1 ||
||S2−1 || ≤ .
1 − ||S1−1 ||||S1 − S2 ||

Now suppose that λ ∈ ρ(T ). This means that (T − λI) ∈ U. But then
T − µI ∈ U, for µ sufficiently close to λ, since

||(T − λI) − (T − µI)|| = |λ − µ|.

We have shown thus openness of the resolvent set , and thus, closedness of the
spectrum.
To remark that σ(T ) ⊂ {|λ| ≤ ||T ||}, or equivalently ρ(T ) ⊃ {|λ| > ||T ||},
just note that for |λ| > ||T ||, T − λI = λ(λ−1 T − I), and λ−1 T − I is invertible
by the previous, since ||λ−1 T || < 1. Thus (T − λI)−1 exists, i.e., λ ∈ ρ(T ). Note
moreover, that for such λ, we have

||R(λ)|| = ||(T − λI)−1 || ≤ |λ|−1 (1 − |λ−1 |||T ||)−1 . (31)

We turn to showing non-emptyness. Choose a point λ0 in the resolvent, and

let λ be sufficiently close to λ0 . The formula (30) says that

T − λI = (T − λ0 I)(I − (T − λ0 I)−1 (T − λ0 I − (T − λI)))

= (T − λ0 I)(I − (T − λ0 I)−1 ((λ − λ0 )I))

Thus, for small enough |λ − λ0 |,

∞
X
R(λ) = (T − λI)−1 = (T − λ0 I)−i−1 (λ − λ0 )−i I.
i=0

That is to say, R is an operator valued holomorphic function on ρ(T ).

So suppose that σ(T ) = ∅. That is to say, ρ(T ) = C. In the language of
complex analysis, this means that R is entire. On the other hand, by (31), we

50
have that R is bounded. Liouville’s theorem from complex analysis says that R
must be constant.
But of course, R cannot be constant, because (T − λI)−1 6= (T − µI)−1 if
λ 6= µ. The contradiction proves ρ(T ) 6= C, and thus, σ(T ) 6= ∅.
One might want to compare with the finite dimensional case. Specialising
to that case, in view of σp (T ) = σ(T ), we have shown that every linear trans-
formation has an eigenvalue.
The usual proof of this latter fact goes through the characteristic polynomial.
Any root of the characteristic polynomial is an eigenvalue. All polynomials over
C have a root, by the fundamental theorem of algebra. Thus, any T has an
eigenvalue.
The algebraic device of the characteristic polynomial is not available to us in
infinite dimensions. But one must remember, that even in finite dimensions, this
does not render the proof completely algebraic. For the fundamental theorem
of algebra requires an analytic argument. In fact, one classic proof proceeds
precisely via Liouville’s theorem!

8.3 Compact operators

8.3.1 Definitions
Definition 8.4. Let X, Y be Banach spaces. An operator T ∈ L(X, Y ) is said
to be compact if E ⊂ X bounded implies T (E) is totally bounded.
Note that a compact operator is in particular, bounded.
Proposition 8.1. Let X, Y be Banach spaces. Then T ∈ L(X, Y ) is compact
iff T (B(1)) is totally bounded iff T (B(1)) is compact.
The proof of the above is immediate.

8.3.2 Examples and non-examples

If X and Y are finite dimensions, then every T ∈ L(X, Y ) is compact. In infinite
dimensions, “most” operators are not compact. In particular we easily see that
the identity map I : X → X is compact iff X is finite dimension.
The zero map is certainly compact. A non-trivial example of a compact
operator is again the identity map, considered though as a map φ : C 1 (U) →
C 0 (U ). The compactness of this map is a corollary of Arzela-Ascoli.
A more interesting example of a compact operator is the map L : C 2 (S1 ) →
0
C (D), where D denotes the closed unit ball, and L takes a function f to the
unique solution ψ ∈ C 0 (D) ∩ C 2 (B(1)) of 2ψ = 0 with boundary values f .
The compactness of this operator follows follows from estimates you proved on
example sheets.

51
8.3.3 The spectrum of compact operators
We shall not give a general discussion of the theory of compact operators on
general Banach spaces, in particular, their spectrum. Let us just quote the
following theorem:
Theorem 8.2. Let T : X → X be compact. Then the point spectrum of T is a
countable set {λi }. If X is infinite dimensional, then σ(T ) = {0} ∪ {λi }, and
λi → 0 if there are infinitely many λi . Moreover, the eigenspace corresponding
to λi 6= 0 is finite dimensional.

8.4 Self-adjoint operators on Hilbert space

We have already defined the adjoint

T ∗ : Y ∗ → X∗

of an operator
T : X → Y.
Suppose now that X = Y = H a Hilbert space. We know that H can be identi-
fied with H ∗ via the Riesz Representation Theorem. Thus, we can “compare”
T and T ∗ . Let φ : H → H ∗ be the antilinear isometry.
Definition 8.5. We say that T : H → H is self-adjoint if φ ◦ T ◦ φ−1 = T ∗ .
We have
Proposition 8.2. Let T : H → H be bounded, and let T ∗ : H ∗ → H ∗ be the
adjoint, and let φ be the map of the Riesz Representation theorem. Then

< T x, y >=< x, φ−1 ◦ T ∗ ◦ φ(y) > . (32)

In particular T : H → H is self adjoint iff < T x, y >=< x, T y > for all x, y.

Proof. Formula (32) is just obtained by chasing arrows.

< x, φ−1 ◦ T ∗ ◦ φ(y) > = (T ∗ ◦ φ(y))(x)

= (T ∗ (φ(y)))(x)
= φ(y)(T (x))
= < T x, y > .

Note that < x, φ−1 ◦ T ∗ ◦ φ(y) > for all x, y, determines T ∗ . Thus the iff
statement.

8.4.1 Eigenvalues and eigenspaces of compact self-adjoint operators

First some notation. Let T : H → H. We have defined already the eigenspace
corresponding to an eigenvalue λ. Let us denote this by Eλ . And Let PEλ
denote the orthogonal projection to Eλ .

52
Proposition 8.3. Let T : H → H be self-adjoint. Then σp (T ) ⊂ R.
Proof. Let λ ∈ σp (T ) with 0 6= v ∈ Eλ . Then

λ|v|2 =< λv, v >=< T v, v >=< v, T v >=< v, λv >= λ̄|v|2

from which we obtain λ = λ̄.

Proposition 8.4. Let T : H → H be self-adjoint, and let λ, ν ∈ σp (T ) be
distinct. Then Eλ ⊥ Eν .
Proof. Let 0 6= v ∈ Eλ , 0 6= w ∈ Eν . We have

λ < v, w >=< λv, w >=< T v, w >=< v, T w >= ν < v, w >

from which we obtain, since λ 6= ν, that < v, w >= 0.

Proposition 8.5. Let T : H → H be self-adjoint, and λα ∈ σp (T ). Then

T (⊕α Eλα ) ⊂ ⊕α Eλα

T (⊕α Eλα ) ⊂ ⊕α Eλα

T ((⊕α Eλα )⊥ ) ⊂ (⊕α Eλα )⊥
Finally, σp (T |(⊕α Eλα )⊥ ) = σp (T ) \ ∪α {λα }.
Proof. The first inclusion is clear, the second follows from the first by continuity.
The third follows from the following. Let w ∈ (⊕α Eλα )⊥ . We have that for all
v ∈ Eλα , < w, v >= 0. By then < T w, v >=< w, T v >= λ < w, v >= 0. Thus
T w ∈ (⊕α Eλα )⊥ .
Proposition 8.6. Let T : H → H be compact, self-adjoint, and let λ0 > 0.
Then
dim ⊕λ0 ≤|λα |,λα ∈σp (T ) Eλα < ∞
Proof. Suppose not. Then there exists an infinite sequence of unit vectors {vi }
with T vi ⊥ T vj for i =
6 j, and ||T vi || ≥ λ0 . It follows that

||T vi − T vj ||2 =< T vi , T vi > + < T vj , T vj >≥ 2λ20 ,

thus no subsequence of T vi converges. Thus T is not compact.

Corollary 8.1. Let T be compact, self-adjoint. If λ 6= 0, then Eλ is finite
dimensional.

Corollary 8.2. Let T be compact, self-adjoint. Given, ǫ, here are only finitely
many λα with |λα | > ǫ. Thus σp (T ) is either finite or countably infinite with
λi → 0.

53
8.4.2 The spectral theorem
Theorem 8.3. Let T be a self-adjoint, compact operator T : H → H. Then the
point spectrum of T is a real countable set {λi }N
i=1 for some 1 ≤ N ≤ ∞. The
eienspaces Eλi are finite dimensional for λi 6= 0, and Eλi ⊥ Eλj for i 6= j. If
their are infinitely many λi , then λi → 0. Moreover, if H is infinite dimensional
then σ(T ) = σp (T ) ∪ {0}. Finally, we may write T as
N
X
T = λi PEλi .
i=1

Proof. Like other results in Hilbert space theory, this is proven by exploiting
variational methods. In particular, we have a variational characterization of the
eigenvalues of T . For this, the following Lemma will be useful:
Lemma 8.2. Let T : H → H be self-adjoint. Then

||T || = sup | < T x, x > |. (33)

|x|=1

√
Proof. This is not immediately obvious because ||T || is defined as sup < T x, T x >.
First we note that
||T || = sup | < T x, y > |. (34)
|x|=1,|y|=1

Let xi be a maximizing sequence for |T xi |, with |xi | = 1. This means that

||T || = ||T ||−1 lim < T xi , T xi >

= ||T ||−1 lim < xi , T (T xi ) >
= lim < xi , T (|T xi |−1 T xi >

So, setting yi = |T xi |−1 T xi , we have that < xi , T yi >→ ||T ||. On the other
hand, | < x, T y > | ≤ ||T || for any |x| = 1, |y| = 1.
Let λ denote the right hand side of (33). We compute that
1
| < T x, y > | ≤ | < T (x + y), (x + y) > − < T (x − y), (x − y) > |
4
1
≤ (λ||x + y||2 + λ||x − y||2 )
4
λ
≤ (||x + y||2 + ||x − y||2 )
4
λ
= (2||x||2 + 2||y||2 )
4
= λ,

where we have used the parallelogram law. Thus, (34) implies (33).
The notation λ was meant to be suggestive! Without loss of generality, let
us assume that λ = sup|x|=1 < T x, x >. We will show that λ is an eigenvalue.

54
 For this, let xi be a maximising sequence for < T x, x >, i.e., let < T xi , xi >→
λ with |xi | ≤ 1. By compactness of T , there exists a subsequence xik such that
T xik → y.
Now we have

< T xi − λxi , T xi − λxi >= ||T (xi )||2 − 2λ < T xi , xi > +λ2 < xi , xi >2 .

Thus ||T xi − λxi ||2 → 0. Since T xik → 0, then xik → 0 and T y = λy. Note
that y 6= 0. We have produced an eigenvector of eigenvalue λ.
Consider now the eigenspace Eλ . By compactness, it is necessarily finite
dimensional. Writing H = Eλ ⊕ Eλ⊥ , we have that T (Eλ ) = Eλ , T (Eλ )⊥ = Eλ⊥ .
The problem is thus reduced to understanding an operator on a smaller space.
Iterating the above argument with Eλ⊥ , in place of H, etc., we obtain a
sequence of distinct eigenvalues |λ1 | ≥ |λ2 | ≥ · · · . If this sequence is finite, we
must have
H = ⊕ni=1 Eλi
from which one easily deduces
n
X
T = λi PEλi .
i=1

If the λi are infinite in number, then |λi | → 0. For otherwise, there is an

infinite dimensional subspace H∞ ⊂ H such that for |x| = 1, < T x, T x >≥
λ0 < x, x >≥ λ0 , from which one contradicts compactness.
It follows thus that we must have that T is the 0 map on
⊥
(⊕Eλi ) = (⊕Eλi )⊥ ,

for T P(⊕Eλi )⊥ cannot have a nonzero eigenvalue. That is to say

H = E0 ⊕ ⊕∞
i=1 Eλi ,

from whichP it is clear that σp ⊂ {0} ∪ λi .

m
From || i=n λi P EλI || ≤ max |λi |, we obtain immediately that from the fact
that |λi | → 0 and previous considerations that
∞
X
λi PEλi
i=1

exists and equals T .

The only thing left is to show that, in the infinite dimensional case σ ⊂
{0}∩σp , as the other inclusion is immediate from the closedness of the spectrum.
For this, let ν 6∈ {0} ∪ σp (T ). Let Tn denote the partial sum of the series
representing T . We have
n
X
Tn − νI = (λi − ν)PEλi − νP(⊕ni=1 Eλi )⊥
i=1

55
From which is is clear that (Tn −νI)−1 exists, and ||(Tn −νI)−1 || ≤ max{|ν|−1 , |λi −
ν|−1 }. We know that for every ǫ there exists an n such that ||T − Tn || < ǫ, and
thus, by the above computation

||(Tn − νI)−1 ||||T − νI − (Tn − νI)|| < ǫ′

thus, we have the invertibility of T − νI. It follows that ν 6∈ σ(T ).

9 Thanks
Thanks to Susan Thomas and Paul Jefferys for comments and corrections.

References
[1] B. Bollobas Linear Analysis: an introductory course second edition, Cam-
bridge University Press, 1999
[2] J. Dieudonné History of Functional Analysis North-Holland Mathematics
Studies, 49, Amsterdam, 1981
[3] P. Lax Functional Analysis Wiley-Interscience, New York, 2002
[4] W. Rudin Functional Analysis second edition, McGraw-Hill, New York,
1991

56