Metric
Metric
The second part of this course is about metric geometry. We will study metric spaces,
low distortion metric embeddings, dimension reduction transforms, and other topics. We
will discuss numerous applications of metric techniques in computer science.
1. Non-negativity: d(x, y) ≥ 0.
Elements of X are called points of the metric space, and d is called a metric or distance
function on X.
Occasionally, spaces that we consider will not satisfy condition 4. We will call such spaces
semi-metric spaces.
Definition 1.2. A space (X, d) is a semi-metric space if it satisfies conditions 1-3 and 4 ′ :
4 ′ . if x = y then d(x, y) = 0.
1. Euclidean Space. Space Rd equipped with the Euclidean distance d(x, y) = ∥x−y∥2 .
1
2. Uniform Metric. Let X be an arbitrary non-empty set. Define a distance function
d(x, y) on X by d(x, y) = 1 if x ̸= y and d(x, x) = 0. The space (X, d) is called a
uniform or discrete metric space.
3. Shortest Path Metric on Graphs. Let G = (V, E, l) be a graph with positive edge
lengths l(e). Let d(u, v) be the length of the shortest path between u and v. Then
(V, d) is the shortest path metric on G.
4. Tree Metrics. A very important family of graph metrics is the family of tree metrics.
A tree metric is the shortest path metric on a tree T .
Definition 1.3. A normed space is a pair (V, ∥ · ∥), where V is a linear space (vector space)
and ∥ · ∥ : V → R is a norm on V such that the following conditions hold for every x, y ∈ V .
1. ∥x∥ > 0 if x ̸= 0.
Definition 1.4. We say that a sequence of points xi in a metric space is a Cauchy sequence
if
lim sup d(xi , xj ) = 0.
i→∞ j≥i
A metric space is complete if every Cauchy sequence has a limit. A Banach space is a
complete normed space.
Remark 1.5. Every finite dimensional normed space is a Banach space. However, an in-
finite dimensional normed space may or may not be a Banach space. That said, all spaces
we discuss in this course will be Banach spaces. Further, for every normed (metric) space
V there exists a Banach (complete) space V ′ that contains it such that V is dense in V ′ .
Here is an example of a non-complete normed space. Let V be the space of infinite se-
quences a(1), a(2),
P∞ . . . , a(n), . . . in which only a finite number of terms a(i) are non-zero.
Define ∥a∥ = i=1 |a(i)|. Then (V, ∥ · ∥) is a normed space but it is not complete, and thus
(V, ∥ · ∥) is not a Banach space. To see that, define a sequence ai of elements in V as follows:
ai (n) = 1/2n if n ≤ i and ai (n) = 0, otherwise. Then ai is a Cauchy sequence but it has no
limit in V . Space ℓ1 , which we will define in the next section, is the completion of (V, ∥ · ∥).
2
1.2 Lebesgue Spaces Lp (X, µ)
In this section, we define Lebesgue spaces, a very important class of Banach spaces. Let
(X, µ) be a measure space. We consider the set of measurable real valued functions on X.
For p ≥ 1, we define the the p–norm of a function f by
Z 1/p
p
∥f ∥p = |f (x)| dµ(x) .
X
∥f ∥∞ = sup |f (x)|.
Caveat: The norm ∥f ∥p can be equal to 0 for a function f ∈ Lp (X, µ), which is not
identically equal to 0. So formally Lp (X, µ) (as defined above) is not a normed space. The
standard way to resolve this problem is to identify functions that differ only on a set of
measure 0. The norm ∥ · ∥∞ is usually defined as
2. Space ℓdp . Let X = {1, . . . , d}, and µ be again the counting measure; i.e. µ(S) = |S|
for S ⊂ N. The elements of ℓdp are d-tuples of real numbers a = (a1 , a2 , . . . , ad ) ∈ Rd .
The p−norm of a vector a = (a1 , a2 , . . . , ad ) equals
d
!1/p
X
∥a∥p = |a|p .
i=1
3. Space Lp [a, b]. Let X = [a, b], and µ be the standard measure on R. The elements
of Lp [a, b] are measurable functions f : [a, b] → R with ∥f ∥p < ∞. The p–norm of a
function f equals
Z b 1/p
p
∥f ∥p = |f (x)| dx .
a
3
Lemma 1.6. For every 1 ≤ p < q ≤ ∞, we have ℓp ⊂ ℓq and Lq [0, 1] ⊂ Lp [0, 1]. Both
inclusions are proper.
Proof. We consider the case when q < ∞. Let a ∈ ℓp . P Let I = {i : |ai | ≥ 1}. Note that I
is a finite set, as otherwise we would have that ∥a∥pp ≥ i∈I |ai |p = ∞. For every i ∈ / I, we
q p
have |ai | < |ai | . Therefore,
X X X X X
∥a∥qq = |ai |q + |ai |q ≤ |ai |q + |ai |p ≤ |ai |q + ∥a∥pp < ∞.
i∈I i∈I
/ i∈I i∈I
/ i∈I
We conclude that a ∈ ℓq .
p q
R Now qlet f ∈R Lq [0, 1]. Let I = {x : |f (x)| ≤ 1}. Note that |f | < |f | when x ∈ / I, and
I
|f (x)| dx ≤ I 1 dx ≤ 1. Therefore,
Z 1 Z Z Z
p p p p
∥f ∥p = |f (x)| dx = |f (x)| dx+ |f (x)| dx ≤ 1+ |f (x)|q dx ≤ 1+∥f ∥qq < ∞.
0 I [0,1]\I [0,1]\I
(V ∗ , ∥ · ∥∗ ) is a Banach space.
Let p, q ∈ (1, ∞) s.t. 1/p + 1/q = 1. Later in this course we will show that the dual of
ℓp is ℓq and vice versa. Similarly the dual of Lp (X, µ) is Lq (X, µ) and vice versa. The duals
of ℓ1 and L1 (X, µ) are ℓ∞ and L∞ (X, µ). However, ℓ1 is not the dual of ℓ∞ , and, in general,
L1 (X, µ) is not the dual of L∞ (X, µ). That said, ℓd1 and ℓd∞ are duals of each other.
We say that V is reflexive if V = V ∗∗ (or more precisely V ∗∗ is isometrically isomorphic to
V ). As we pointed out above, ℓp and Lp (X, µ) are reflexive spaces for p ∈ (1, ∞). However,
ℓ1 , ℓ∞ , L1 (R), and L∞ (R) are not. Importantly, all finite dimensional spaces are reflexive.
4
Let V be a finite dimensional space and S be a centrally symmetric closed convex body.
Further, assume that some neighborhood of 0 lies in S. Define ∥ · ∥ as follows: ∥u∥ =
min{α > 0 : u/α ∈ S} for u ̸= 0 and ∥0∥ = 0. Then (V, ∥ · ∥) is a normed space; further, S
is the unit ball of V .
We see that in finite dimensions, there is a one-to-one correspondence between norms
and their unit balls.
Exercise 4. Consider Euclidean space V . We identify the dual space V ∗ with V in the
standard way: for u ∈ V , u(v) = ⟨u, v⟩. Let ∥ · ∥ be an arbitrary norm on V and ∥ · ∥∗ be
the dual norm. Prove that the unit balls B and B∗ of ∥ · ∥ and ∥ · ∥∗ , respectively, are polar
sets of each other.
b = grad f is colinear with grad g = (p|a1 |p−1 sgn a1 , . . . , pap−1 d sgn ad ). That is, for some
p−1 p
t > 0, |bi | = t|ai | for all i. Therefore, |ai ||bi | = t|ai | . Then
X X X
ai b i ≤ |ai ||bi | = t |ai |p = t.
i i i
5
We conclude that t = ∥b∥q and ⟨a, b⟩ ≤ t = ∥a∥p ∥b∥q . We proved Hölder’s inequality for
vectors a with ∥a∥p = 1. The general case follows for the homogeneity of the inequality.
Exercise 5. For a given a ∈ Rd , define b as follows: bi = |ai |p/q sgn ai . Show that ⟨a, b⟩ =
∥a∥p ∥b∥q . Conclude that
⟨a, b⟩
∥b∥q = ∥b∥∗p ≡ sup .
b̸=0 ∥a∥p
Theorem 2.3 (Interpolation Inequality). Let 1 ≤ p < r < q ≤ ∞. Define p̂ = 1/p, q̂ = 1/q,
r̂ = 1/r,
r̂ − q̂ p̂ − r̂
α= and β= .
p̂ − q̂ p̂ − q̂
∥a∥r ≤ ∥a∥αp · ∥a∥βq
for every a ∈ Rd .
Proof. Note that α + β = 1 and r̂ = αp̂ + β q̂ (that is, r̂ is a convex combination of p̂ and q̂
p q
with weights α and β). Let p′ = αr and q ′ = βr . Then 1/p′ +1/q ′ = r ·(αp̂)+r ·(β q̂) = rr̂ = 1
d d d
!1/p′ d
!1/q′
Hölder q′
p′
X X X X
∥a∥rr = |ai |r = |ai |αr · |ai |βr ≤ (|ai |αr ) · |ai |βr
i=1 i=1 i=1 i=1
d
!1/p′ d
!1/q′
′ ′
X X
= |ai |p · |ai |q = ∥a∥p/p
p · ∥a∥q/q
q = ∥a∥αr βr
p · ∥a∥q
i=1 i=1
Therefore,
∥a∥r ≤ ∥a∥αp · ∥a∥βq
Proof. We apply the interpolation inequality with q = ∞. Then q̂ = 0 and thus α = p/r,
β = 1 − p/r. We have
On the other hand, let ξ be a random coordinate of a chosen uniformly at random. Then,
d
!1/p d
!1/r
∥a∥p X |ai |p Lyapunov’s Ineq. X |ai |r ∥a∥r
1/p
= ≡ ∥ξ∥p ≥ ∥ξ∥r = = 1/r .
d i=1
d i=1
d d
6
3 Metric Embeddings
Consider two metric spaces (X, dX ), (Y, dY ) and a map f : X → Y . We say that f : X → Y
is a Lipschitz map if there is a number C such that
The Lipschitz constant ∥f ∥Lip of f is the minimum C such that this inequality holds.
We say that a bijective map φ : X → Y is an isometry if for every x1 , x2 ∈ X,
dY (φ(x1 ), φ(x2 )) = dX (x1 , x2 ). We say that an injective map φ : X → Y is an isomet-
ric embedding if φ is an isometry between X and φ(X) (the image of X under φ).
The distortion of a map f : X → Y equals ∥f ∥Lip · ∥f −1 ∥Lip where f −1 is the inverse map
from f (X) to X.
Exercise 6. Prove that f has distortion at most D if and only if there is a number c > 0
such that
First, we prove that ∥φ(xi ) − φ(xj )∥∞ ≤ d(xi , xj ). We need to show that all coordinates
of the vector φ(xi ) − φ(xj ) are bounded by d(xi , xj ) in the absolute value. Indeed, the
k-th coordinate of φ(xi ) − φ(xj ) equals d(xk , xi ) − d(xk , xj ). From the triangle inequalities
for xi , xj and xk , it follows that |d(xk , xi ) − d(xk , xj )| ≤ d(xi , xj ). Now, we verify that
7
∥φ(xi ) − φ(xj )∥∞ ≥ d(xi , xj ). Note that ∥φ(xi ) − φ(xj )∥∞ ≥ |d(xk , xi ) − d(xk , xj )| (the
absolute value of the k-th coordinate) for every k. In particular, this inequality holds for
k = i. That is,
∥φ(xi ) − φ(xj )∥∞ ≥ |d(xi , xi ) − d(xi , xj )| = d(xi , xj ).
Theorem 4.2. Let p ∈ [1, ∞). Metric space ℓd2 (Euclidean d-dimensional space) embeds
isometrically into Lp (X, µ) for some space X.
⟨v, x⟩
fv (x) = .
M
We prove that the embedding is an isometry. Consider two vectors u and v. Let w = u − v,
and e = w/∥w∥2 . We have,
p
⟨u, x⟩ ⟨v, x⟩
Z Z
1
p
∥φ(u) − φ(v)∥p = − dγ(x) = p |⟨u − v, x⟩|p dγ(x)
ℓ2d M M M d
ℓ2
Z Z
1 p 1 p
= p |⟨∥w∥ e, x⟩| dγ(x) = p ∥w∥ |⟨e, x⟩|p dγ(x)
M ℓd2 M ℓd2
Consider a coordinate frame in which the x1 -axis is parallel to the vector e (i.e. vector e has
coordinates (1, 0, . . . , 0)). Then |⟨e, x⟩| = |x1 |. We get
!1/p
∥w∥2
Z
∥φ(u) − φ(v)∥p = |x1 |p dµ(x) = ∥w∥2 = ∥u − v∥2 .
M ℓd2
We showed that every finite subset S of ℓd2 embeds isometrically into space Lp (X, µ). Can
we embed S into a “simpler” space ℓN p (for some N )? We will see that all spaces Lp (X, µ)
(of sufficiently large dimension) have essentially the same finite metric subspaces. Therefore,
if a metric space embeds into some Lp (X, µ), then it also embeds into ℓN p for some N .
Theorem 4.3. Let S be a finite subset of Lp (Z, µ), n = |S|, and N = n2 + 1. Then S
8
Proof. Consider the linear space M of all symmetric n ×nn matrices with zeros on the
n
diagonal. The dimension of M is 2 . Consider a map f : R → M defined as follows. The
map f sends a vector u ∈ Rn to the matrix A = (aij ) with entries aij = |ui − uj |p . Clearly,
f (v) ∈ M for every v ∈ Rn . Let B = f (Rn ) ≡ {f (u) : u ∈ Rn } and C = conv(B).
For every metric space (S, d) on a set S = {s1 , s2 , . . . , sn }, we define a matrix F S by
Fij = d(si , sj )p . The theorem follows from the following two lemmas.
S
for some b(1) , . . . , b(N ) ∈ B and some x(k) ∈ f −1 (b(k) ) (xi is a preimage of b(k) ). By the
definition of F S , we have
N
X N
X
p (k) (k)
d(si , sj ) = FijS = fij (x ) = (k)
|xi − xj |p . (1)
k=1 k=1
(1) (2) (N )
φ(si ) = (xi , xi , . . . , xi ).
Note that equation (1) says that d(si , sj )p = ∥φ(si ) − φ(sj )∥pp , and therefore d(si , sj ) =
∥φ(si ) − φ(sj )∥p . We conclude that φ is an isometric embedding.
Corollary 4.6. Suppose that S is a subset of ℓd2 . Then S isometrically embeds into ℓN
p ,
|S|
where N = 2 + 1.
9
Exercise 8. In our proof, we used the Carathéodory theorem for arbitrary convex sets: every
point in the convex hull of S ⊂ Rd is a convex combination of at most d + 1 points from
S. Show that if the convex hull conv(S) of S is a cone, then every point conv(S) is a
linear combination, with positive coefficients, of at most d points inS. Conclude that in the
statement of Theorem 4.3 we can replace N = n2 + 1 with N = n2 .
Definition 4.7. Let cp (X) be the least distortion1 with which a finite metric space (X, d)
embeds into ℓp .
Theorem 4.8. For every finite metric space X and every p ∈ [1, ∞], we have 1 = c∞ (X) ≤
cp (X) ≤ c2 (X).
Proof. The inequality 1 = c∞ (X) ≤ cp (X) follows from Theorem 4.1. Let f be an embedding
of X into ℓ2 (X) with distortion c2 (X). By Corollary 4.6, there is an isometric embedding g
of f (X) ⊂ ℓ2 into ℓp . Then map g ◦ f is an embedding of X into ℓp with distortion at most
c2 (X). We conclude that cp (X) ≤ c2 (X).
1
A simple compactness argument shows that there is an embedding with the least possible distortion.
10