Introduction

Records
Asymptotic laws
Bibliography
An Introduction to Extreme Value Statistics
Loc Turban
mailto:turban@lpm.u-nancy.fr
January 12, 2012
M Frechet R A Fisher L H C Tippett E J Gumbel W Weibull
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
1
Introduction
2
Records
3
Asymptotic laws
4
Bibliography
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
100m olympic records Usain Bolt Beijing 2008
Market Crash 1929
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Solar activity Solar spots
Earthquakes Lisbon 1755
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
X is a continuous random variable taking its values in R
Probability density
p(x)dx = Pr {x < X x + dx}
(Cumulative) Distribution function
P(x) = Pr {X x}=
_
x

p(x

)dx

, P() = 0, P(+) = 1
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Maximum and minimum records
Consider a sequence of t independent and identically
distributed (iid) random variables {X
1
, X
2
, . . . X
t
} with the
probability density p(x).
A maximum record R(t) at time t is such that
R(t) = max {X
1
, X
2
, . . . X
t
}
In the same way a minimum record r (t) is dened as
r (t) = min {X
1
, X
2
, . . . X
t
}
and, by convention
R(1) = r (1) = X
1
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Records probabilities
The cumulative probabilty distribution of R(t) is given by
P(R, t) = Pr {X
1
, X
2
, . . . X
t
R} = [P(R)]
t
The corresponding probability density p(R, t) is such that
p(R, t)dR = Pr{R max {X
1
, X
2
, . . . X
t
} < R + dR} = dP(R, t)
so that
p(R, t)dR = t
..
t choices
t1 values R
..
[P(R)]
t1
p(R)dR
. .
1 value [R,R+dR]
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
In the same way we have
p(r , t)dr = t
..
t choices
t1 values >r
..
[1 P(r )]
t1
p(r )dr
. .
1 value [r dr ,r ]
where
1 P(r ) =
_
+
r
p(x)dx
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Universal properties
Some properties of records, depending only on the ordering of the
X
i
, are independant of the probability density and thus universal
for iid random variables.
Number of records
The probability that X
t
is a maximum (minimum) record is 1/t
since the maximum (minimum) value can be realized by any of the
t variables with the same probability.
Thus the mean number of records at t is
N(t) =
t

k=1
1/k = H
t
= ln t + + O(1/t), = 0.5772 . . .
where H
t
is a harmonic number and is the Euler constant.
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Universal properties
To a sequence of random variables, for example
{0.2, 0.4, 0.3, 0.1, 0.5}
one associates a binary string
{R, R, L, L, R}
where R stands for records and L for lower.
At time t, there are 2
t1
possible binary strings which can be
vizualized as a tree of records.
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Tree of records
From R. Schmittmann and R. K. P. Zia
Am. J. Phys. 67 (1999) 1269
Tree of records at t = 4
(a) The 2
t1
binary strings.
(b) The lifetimes of records.
(c) The string probabilities.
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Lifetime
From the record lifetimes (b) and the string probabilities (c) we
deduce the lifetime histogram N
t
() at time t = 4
N
4
(4) =
1
4
N
4
(3) =
1
12
+
1
4
=
1
3
N
4
(2) = 2
1
8
+
1
24
+
1
12
+
1
8
=
1
2
N
4
(1) =
1
12
+ 2
1
24
+
1
4
+ 2
1
12
+ 2
1
8
+ 4
1
24
= 1
which suggests the simple universal form
N
t
() =
1

Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Lifetime
Let us rst consider the probability that a record is established at
t

and still survives at t

+ 1
p
t
() = Pr{

..
R
..
t

L L
..
t

+1
}
=
1
t

_
1
1
t

+ 1
_

_
1
1
t

+ 1
_
=
1
t

+ 1
Thus the probability that a record is established at t

and beaten
at t

+ is given by

t
() = Pr{

..
R
..
t

L L R
..
t

+
} = p
t
()
1
t

+
=
1
t

+ 1

1
t

+
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Universal string probability
Let us now consider the string probabilities involved in the tree of
records
p(
1
,
2
, ,
k
) = Pr{

1
..
R
..
1
L L

2
..
RL L
..

1
+
2

k
..
RL L
..

k
i =1

i
}
where

k
i =1

i
= t.
It is given by a product of probabilties p
t

i
(
i
) = 1/t
i
where t
i
is
the nal time of each sub-sequence so that
p(
1
,
2
, ,
k
) =
1

1
(
1
+
2
) (

k
i =1

i
)
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Universal lifetime histogram
The mean number of records with lifetime at time t is obtained
by summing the appropriate record probabilities over the initial
time t

N
t
() =
t

=1

t
()
. .
t

+t
+ p
t
()
. .
t

+1=t
=
1

1
+ 1
+
1
+ 1
+
1
t 1

1
t
+
1
t
=
1

The mean number of records and mean lifetime are given by

N
t
=
t

=1
N
t
() = H
t
,
t
=

t
=1
N
t
()

t
=1
N
t
()
=
t
H
t
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Nonuniversal properties
Any quantity involving the values (R or r ) of the records, like the
moments, the dierent record correlation functions, has a
nonuniversal value.
We shall illustrate this point on the example of the moments of the
records at time t given by
R
n
(t) =
_
+

dR tp(R)[P(R)]
t1
. .
p(R,t)
R
n
r
n
(t) =
_
+

dr tp(r )[1 P(r )]

t1
. .
p(r ,t)
r
n
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Uniform probability density
Then
p(x) =
_
1 if x [0, 1]
0 otherwise
, P(x) = x
so that
R
n
(t) = t
_
1
0
dR R
t+n1
=
t
t + n
, R
n
(t)
tn
1
n
t
r
n
(t) = t
_
1
0
dr r
n
(1 r )
t1
= t B(n + 1, t)
= t
(n + 1)(t)
(n + t + 1)
=
_
n + t
t
_
1
=
1
(1 + t)(1 + t/2) (1 + t/n)
, r
n
(t)
tn

n!
t
n
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Uniform probability density
Mean record values for the uniform probability density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Exponential probability density
Then
p(x) =
_
e
x
if x 0
0 otherwise
, P(x) = 1 e
x
so that
R
n
(t) = t
_

0
dR R
n
_
1 e
R
_
t1
e
R
= t
t1

k=0
_
t 1
k
_
(1)
k
_

0
dR R
n
e
(k+1)R
= n!t
t1

k=0
_
t 1
k
_
(1)
k
(k + 1)
n+1
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Exponential probability density
A saddle-point evaluation of the rst integral leads to
R
n
(t)
ln tn
ln
n
(t)
For the minimum record
r
n
(t) = t
_

0
dr r
n
_
e
r
_
t1
e
r
= t
_

0
dr r
n
e
tr
=
n!
t
n
which is equal to the asymtotic value obtained before with the
uniform density.
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Exponential probability density
Mean record values for the exponential probability density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Exponential probability density
It may be easier to evaluate the integrals by using P [0, 1]
instead of R or r as the integration variable
R
n
(t) =
_
1
0
dP tP
t1
R
n
(P)
r
n
(t) =
_
1
0
dP t[1 P]
t1
r
n
(P)
With the uniform density, for which P(x) = x, the integration
variable is not changed.
For the exponential density, we have
P(x) = 1 e
x
, x(P) = ln(1 P)
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Exponential probability density
so that
R
n
(t) =
_
1
0
dP tP
t1
[ln(1 P)]
n
r
n
(t) =
_
1
0
dP t[1 P]
t1
[ln(1 P)]
n
Making use of the exponential generating function of the Stirling
numbers of the rst kind

k=n
_
k
n
_
x
k
k!
=
1
n!
[ln(1 x)]
n
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Exponential probability density
we obtain
R
n
(t) =

k=n
n!
k!
_
k
n
_
t
_
1
0
dP P
t+k1
=

k=n
t
t + k
n!
k!
_
k
n
_
r
n
(t) =

k=n
n!
k!
_
k
n
_
t
_
1
0
dP P
k
[1 P]
t1
=

k=n
n!t!
(t + k)!
_
k
n
_
which can be used to write the following identities

k=n
_
k
n
_
(t + k)k!
=
t1

k=0
_
t 1
k
_
(1)
k
(k + 1)
n+1
,

k=n
_
k
n
_
(t + k)!
=
1
t
n
t!
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Characteristic largest value
The characteristic largest value u(t) is such that the mean
number of random variables exceeding u is equal to one at time t.
Thus, since
Pr{X
i
> x} = 1 P(x)
this largest value is given by
t[1 P(u)] = 1, P(u) = 1
1
t
As t , P(u) 1 and the characteristic largest value tends to
the maximum value of x.
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Characteristic largest value: examples
Exponential density:
p(x) = e
x
, (x > 0)
1 P(u) = e
u
=
1
t
u =
ln t

Gumbel
Exponential density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Characteristic largest value: examples
Pareto density:
p(x) =
k
x(x)
k
, (k > 0, x > 1/)
1 P(u) =
1
(u)
k
=
1
t
u =
t
1/k

Frechet
Pareto density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Characteristic largest value: examples
Generalized Pareto density:
p(x)=k(1x)
k1
, (k >0, 0<x <1/)
1 P(u) = (1 u)
k
=
1
t
u =
1 t
1/k

Weibull
Generalized Pareto
density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 1: Gumbel distribution
Emil Julius Gumbel (1891-1966),
German mathematician, pacist and
anti-Nazi campaigner.
He was Professor of Statistics at
Heidelberg from 1923 to 1932, when he
lost his position. He ed to France
where, supported by Frechet, he
became Professor in Lyon. Threatened
by extradition in 1940, he emigrated to
the United States where he worked at
Columbia University.
With M. Frechet, L. H. C. Tippett and
R. A. Fisher, he pioneered the
mathematical eld of extreme value
theory.
(Wikipedia)
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 1: Gumbel distribution
The Gumbel (or type 1) distribution is the asymptotic extreme
value distribution which is obtained for iid random variable of the
exponential type.
In this case we have
lim
x
p(x) = lim
x
[1 P(x)] = 0
and according to de lHospitals rule
p(x)
1 P(x)
=
p

(x)
p(x)
when x ,
whereas the quotient of the two sides tends to one.
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 1: Gumbel distribution
We determine the form of the Gumbel distribution on the example
of the exponential initial distribution.
Then we have
e
u
= t, P(x) = 1
e
(xu)
t
, P(y) = 1
e
y
t
(y = x u)
so that the asymtotic distribution (Gumbel or type 1) is given by

G
(y) = lim
t
[P(y)]
t
= lim
t
_
1
e
y
t
_
t
= exp
_
e
y
_
(< y < )
Exponential
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 1: Gumbel distribution
Gumbel probability density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 2: Frechet distribution
Maurice Rene Frechet (1878-1973) was
a French mathematician.
He made major contributions to the
topology of point sets and introduced
the entire concept of metric spaces. He
also made several important
contributions to the eld of statistics
and probability, as well as calculus.
His dissertation opened the entire eld
of functionals on metric spaces and
introduced the notion of compactness.
Independently of Riesz, he discovered
the representation theorem in the space
of Lebesgue square integrable functions.
(Wikipedia)
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 2: Frechet distribution
The Frechet (or type 2) distribution holds for initial distributions of
the Pareto type such that
lim
x
x
k
[1 P(x)] =
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 2: Frechet distribution
We determine the form of the asymptotic distribution using the
Pareto distribution for the initial distribution. Then we have
(u)
k
= t, P(x) = 1
1
t
_
x
u
_
k
= 1
1
t
_
x
u
_
k
P(y) = 1
y
k
t
(y = x/u)
so that the asymtotic distribution (Frechet or type 2) is given by

F
(y) = lim
t
[P(y)]
t
= lim
t
_
1
y
k
t
_
t
= exp
_
y
k
_
(0 < y < )
Pareto
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 1: Frechet distribution
Frechet probability density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 3: Weibull distribution
Waloddi Weibull (1887-1979) was a
Swedish engineer, scientist, and
mathematician.
He obtained his doctorate from the
University of Uppsala in 1932. In 1939
he published his paper on Weibull
distribution in probability theory and
statistics. In 1941 he received a
personal research professorship in
Technical Physics at the Royal Institute
of Technology in Stockholm.
Weibull published many papers on
strength of materials, fatigue, rupture
in solids, bearings, and of course, the
Weibull distribution, as well as one
book on fatigue analysis in 1961.
(Wikipedia)
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 3: Weibull distribution
The Weibull distribution attracts initial distributions with a limited
range, P() = 1 for nite, such that the rst k 1 derivatives of
P(x) vanish at x = whereas
P
(k)
() = (1)
k1

We determine the asymptotic extreme value distribution on the

example of the generalized Pareto distribution
P(x) = 1 (1 x)
k
, (k > 0, 0 < x < 1/)
which gives back the uniform distribution when k = 1.
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 3: Weibull distribution
With the generalized Pareto initial distribution, we have
t(1 u)
k
= 1, P(x) = 1
1
t
_
1 x
1 u
_
k
P(y) = 1
(y)
k
t
_
y =
1 x
1 u
_
so that the Weibull (type 3) distribution is given by

W
(y) = lim
t
[P(y)]
t
= lim
t
_
1
(y)
k
t
_
t
= exp
_
(y)
k
_
(< y < 0)
Generalized Pareto
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Type 1: Weibull distribution
Weibull probability density
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics
Introduction
Records
Asymptotic laws
Bibliography
Elementary:
B. Schmittmann and R. K. P. Zia, Am. J. Phys. 67 (1999)
1269
Historical:
M. Frechet, Ann. Soc. Pol. Math. 6 (1927) 93
R. A. Fisher and L. H. C. Tippett, Proc. Cambridge Phil.
Soc. 24 (1928) 180
E. J. Gumbel, Ann. Inst. H. Poincare 5 (1935) 115
B. Gnedenko, Ann. Math. 44 (1943) 423
Books:
E. J. Gumbel, Statistics of extremes (Dover, 2004)
J. Galambos, The asymptotic theory of extreme order
statistics (Robert Krieger Publishing Co, 1987)
Loc Turban Groupe de Physique Statistique Nancy An Introduction to Extreme Value Statistics