4.

Interval estimation

The goal for interval estimation is to specify the accurary of an

estimate.

A 1 − α confidence set for a parameter θ is a set C(X) in the

parameter space Θ, depending only on X, such that

Pθ (θ ∈ C(X)) = 1 − α.

Note: it is not θ that is random, but the set C(X).

1
 For a scalar θ we would usually like to find an interval

C(X) = [l(X), u(X)]

so that Pθ (θ ∈ [l(X), u(X)]) = 1 − α. Then [l(X), u(X)] is an

interval estimator or confidence interval for θ; and the observed

interval [l(x), u(x)] is an interval estimate. If l is −∞ or if u is

+∞, then we have a one-sided estimator/estimate. If l is −∞, we

have an upper confidence interval, if u is +∞, we have an lower

confidence interval.

2
 Example: Normal, unknown mean and variance. Let

X1, . . . , Xn be a random sample from N (µ, σ 2), where both µ and

√
σ 2 are unknown. Then (X − µ)/(S/ n) ∼ tn−1 and so

1−α
 
X −µ
= Pµ,σ2 
 √ ≤ tn−1,1−α/2

S/ n
√ √
= Pµ,σ2 (X − tn−1,1−α/2S/ n ≤ µ ≤ X + tn−1,1−α/2S/ n),

and so the (familiar) interval with end points

√
X ± tn−1,1−α/2S/ n

is a 1 − α confidence interval for µ.

3
 Construction of confidence sets

Pivotal quantities

A pivotal quantity (or pivot) is a random variable t(X, θ) whose

distribution is independent of all parameters, and so it has the same

distribution for all θ.

√
Example: (X−µ)/(S/ n) in the example above has tn−1-distribution

if the random sample comes from N (µ, σ 2).

4
 We use pivotal quantities to construct confidence sets, as follows.

Suppose θ is a scalar. Choose a, b such that

Pθ (a ≤ t(X, θ) ≤ b) = 1 − α.

Manipulate this equation to give Pθ (l(X) ≤ θ ≤ u(X)) = 1 − α (if t

is a monotonic function of θ); then [l(X), u(X)] is a 1 − α confidence

interval for θ.

5
 Example: Exponential random sample.

Let X1, . . . , Xn be a random sample from an exponential dis-

tribution with unknown mean µ. Then we know that nX/µ ∼

Gamma(n, 1).

If the α-quantile of Gamma(n, 1) is denoted by gn,α then

1 − α = Pµ(nX/µ ≥ gn,α )

= Pµ(µ ≤ nX/gn,α ).

Hence [0, nX/gn,α ] is a 1−α confidence interval for µ. Alternatively,

we say that nX/gn,α is the upper 1 − α confidence limit for µ.

6
 Confidence sets derived from point estimators

Suppose θ̂(X) is an estimator for a scalar θ, from a known distri-

bution. Then we can take our confidence interval as

[θ̂ − a1−α , θ̂ + b1−α ]

where a1−α and b1−α are chosen suitably.

If θ̂ ∼ N (θ, v), perhaps approximately, then for a symmetric in-

terval choose

√
a1−α = b1−α = z1−α/2 v.

Note: [θ̂ − a1−α , θ̂ + b1−α ] is not immediately a confidence interval

for θ if v depends on θ: in that case replace v(θ) by v(θ̂), which is a

further approximation.

7
 Approximate confidence intervals

Sometimes we do not have an exact distribution available, but

normal approximation is known to hold.

Example: asymptotic normality of m.l.e.

We have seen that, under regularity,

θ̂ ≈ N (θ, I −1(θ)).

If θ is scalar, then (under regularity)

r
θ̂ ± z1−α/2/ I(θ̂)

is an approximate 1 − α confidence interval for θ.

8
 Sometimes we can improve the accuracy by applying (monotone)

transformation of the estimator, using the delta method, and invert-

ing the transformation to get the final result.

As a guide line for transformations, in general a normal approxima-

tion should be used on a scale where a quantity ranges over (−∞, ∞).

9
 Example: Bivariate normal distribution. Let (Xi, Yi),

i = 1, . . . , n, be a random sample from a bivariate normal distribu-

tion, with unknown mean vector and covariance matrix. The param-

eter of interest is ρ, the bivariate normal correlation. The MLE for

ρ is the sample correlation

Pn
i=1 (Xi − X)(Yi − Y )
R= r
Pn
,
i=1 (Xi − X)2 ni=1(Yi −Y )2
P

whose range is [−1, 1]. For large n,

R ≈ N (ρ, (1 − ρ2)2/n),

using the expected Fisher information matrix to obtain an approxi-

mate variance (see the section on asymptotic theory).

But the distribution of R is very skewed, the approximation is

poor unless n is very large.

10
 For a variable whose range is (−∞, ∞), we use the tranformation

1
Z= log[(1 + R)/(1 − R)];
2

this transformation is called the Fisher z transformation. By the

delta method,

Z ≈ N (ζ, 1/n)

where ζ = 12 log[(1 + ρ)/(1 − ρ)].

So a 1−α confidence interval for ρ can be calculated as follows: for

√
ζ compute the interval limits Z ± z1−α/2/ n, then transform these

to the ρ scale using the inverse transformation ρ = (e2ζ −1)/(e2ζ +1).

11
 Confidence intervals derived from hypothesis tests

Define C(X) to be the set of values of θ0 for which H0 would not

be rejected in size-α tests of H0 : θ = θ0. Here the form of the 1 − α

confidence set obtained depends on the alternative hypotheses.

Example: to produce an interval with finite upper and lower limits

use H1 : θ 6= θ0; to find an upper confidence limit use H1− : θ < θ0.

12
 Example: Normal, known variance, unknown mean.

Let X1, . . . , Xn ∼ N (µ, σ 2) be i.i.d., where σ 2 known.

For H0 : µ = µ0 versus H1 : µ 6= µ0 the usual test has an

acceptance region of the form

X − µ0
√ ≤ z1−α/2.
σ/ n

So the values of µ0 for which H0 is accepted are those in the interval

√ √
[X − z1−α/2σ/ n, X + z1−α/2σ/ n];

this interval is a 100(1 − α)% confidence interval for µ.

13
 For H0 : µ = µ0 versus H1− : µ < µ0 the UMP test accepts H0 if

√
X ≥ µ0 − z1−α/2σ/ n

i.e., if

√
µ0 ≤ X + z1−α/2σ/ n.

√
So an upper 1 − α confidence limit for µ is X + z1−α/2σ/ n.

14
 Hypothesis test from confidence regions

Conversely, we can also construct tests based on confidence inter-

val:

For H0 : θ = θ0 against H1 : θ 6= θ0, if C(X) is 100(1 − α)%

two-sided confidence region for θ, then for a size α test reject H0 if

θ0 6= C(X): The confidence region is the acceptance region for the

test.

If θ is a scalar: For H0 : θ = θ0 against H1− : θ < θ0, if C(X)

is 100(1 − α)% upper confidence region for θ, then for a size α test

reject H0 if θ0 6= C(X).

15
 Example: Normal, known variance. Let X1, . . . , Xn ∼

N (µ, σ 2) be i.i.d., where σ 2 is known.

For H0 : µ = µ0 versus H1 : µ 6= µ0 the usual 100(1 − α)%

confidence region is

√ √
[X − z1−α/2σ/ n, X + z1−α/2σ/ n],

so reject H0 if

X − µ0
√ > z1−α/2.
σ/ n

16
 To test H0 : µ = µ0 versus H1− : µ < µ0: an upper 100(1 − α)%

√
confidence region is X + z1−α/2σ/ n, so reject H0 if

√
µ0 > X + z1−α/2σ/ n

i.e. if

√
X < µ0 − z1−α/2σ/ n.

17
 We can also construct approximate hypothesis test based on ap-

proximate confidence intervals.

Example: Use asymptotic normality of m.l.e. to derive a Wald

test.

18
 Prediction Sets

What is a set of plausible values for a future data value?

A 1 − α prediction set for an unobserved random variable Xn+1

based on the observed data X = (X1, . . . , Xn) is a random set P (X)

for which

P(Xn+1 ∈ P (X)) = 1 − α.

Sometimes such a set can be derived by finding a prediction pivot

t(X, Xn+1) whose distribution does not depend on θ.

If a set R is such that P(t(X, Xn+1) ∈ R) = 1 − α, then a 1 − α

prediction set is

P (X) = {Xn+1 : t(X, Xn+1) ∈ R}.

19
 Example: Normal, unknown mean and variance. Let

X1, . . . , Xn ∼ N (µ, σ 2) be i.i.d., where both µ and σ 2 are unknown.

A possible prediction pivot is

Xn+1 − X
t(X, Xn+1) = r .
S 1 + n1

2
As X ∼ N (µ, σn ) and Xn+1 ∼ N (µ, σ 2) is independent of X, it

follows that Xn+1 − X ∼ N (0, σ 2(1 + 1/n)), and so t(X, Xn+1) has

tn−1 distribution.

Hence a 1 − α prediction interval is

{Xn+1 : |t(X, Xn+1)| ≤ tn−1,1−α/2}

 v
1
 u

 u
= X : X − S 1 + tn−1,1−α/2 ≤ Xn+1
u
n+1
t


 n
v 
1
u 
u 

≤ X + S 1 + tn−1,1−α/2 .
u
t
n 


20