G14FOS-E1

The University of Nottingham

SCHOOL OF MATHEMATICAL SCIENCES

A LEVEL 4 MODULE, AUTUMN SEMESTER 2014–2015

FUNDAMENTALS OF STATISTICS

Time allowed THREE Hours

Candidates may complete the front cover of their answer book and sign their desk card but must
NOT write anything else until the start of the examination period is announced.

This paper contains FIVE questions which carry equal marks.

Full marks may be obtained for FOUR complete answers.
Credit will be given for the best FOUR answers.

An indication is given of the approximate weighting of each section of a question by means of a figure
enclosed by square brackets, eg [12], immediately following that section.

Only silent, self-contained calculators with a Single-line Display

or Dual-line Display are permitted in this examination.

Dictionaries are not allowed with one exception. Those whose first language is not
English may use a standard translation dictionary to translate between that language and
English provided that neither language is the subject of this examination. Subject specific
translation dictionaries are not permitted.
No electronic devices capable of storing and retrieving text, including electronic
dictionaries, may be used.

Do NOT turn examination paper over until instructed to do so.

ADDITIONAL MATERIAL: Neave Statistics Tables

G14FOS-E1 Turn over

 2 G14FOS-E1

1 (a) (i) Explain what is meant if we say that ĝ(X) is an unbiased estimator of g(θ).

(ii) Let pX (x|θ) denote a probability mass function or probability density function. State
the Neyman factorization theorem.
(iii) Suppose that T (x) is a sufficient statistic. Use the Neyman factorization theorem to
prove the following:
pX (x|θ)
if T (x) = T (x′ ) then is independent of θ. [9]
pX (x′ |θ)

(b) Let X1 , . . . , Xn denote an independent and identically distributed sample from the
distribution with probability density function
 −1
θ if x ∈ [0, θ],
pX (x|θ) =
0 otherwise.

(i) Determine the likelihood function L(θ|x).

(ii) Find the maximum likelihood estimator (MLE) of θ, giving brief justification.

(iii) Prove that the MLE, θ̂, is sufficient for θ in this example.
(iv) By using the identity
  Yn
Prob max Xi ≤ x|θ = Prob(Xi ≤ x|θ),
1≤i≤n
i=1

find the cumulative distribution function and hence the probability density function of
θ̂. [12]

(c) (i) State the Lehmann–Scheffé theorem.

(ii) Using part (b)(iv), show that
 
n
E[θ̂] = θ.
n+1

(iii) Find the unique minimum variance unbiased estimator of θ.

[Note: You may assume without proof that the relevant statistic is complete.] [9]

G14FOS-E1
 3 G14FOS-E1

2 (a) Let pX (x|θ) denote a probability mass function or probability density function.
(i) Define the score statistic U(X).
(ii) Prove that U(X) has zero expectation.
(iii) State a necessary and sufficient condition for an unbiased estimator ĝ(X) of g(θ) to
attain the Cramér–Rao lower bound. [8]

(b) Let X1 , . . . , Xn be an independent and identically distributed sample from the distribution
with probability mass function
(1 − θ)3 x
pX (x|θ) = θ (x + 1)(x + 2), . . . x = 0, 1, 2, . . . 0 < θ < 1.
2

(i) Calculate the log-likelihood for θ and determine the score statistic.
θ
(ii) Find an unbiased estimator of .
1−θ
(iii) Explain, giving brief justification, whether this unbiased estimator attains the Cramér–
Rao lower bound.
(iv) Calculate the Cramér–Rao lower bound. [11]

(c) Suppose X ∼ Binomial(n, θ1 ) and Y ∼ Binomial(n, θ2 ), where X and Y are independent.

We wish to calculate an approximate 95% confidence interval for the quantity
   
θ1 θ2
h(θ1 , θ2 ) = log − log
1 − θ1 1 − θ2
using the data X and Y .
(i) Find the maximum likelihood estimators θ̂1 of θ1 and θ̂2 of θ2 .
√ √
(ii) Determine approximate large-sample distributions of n(θ̂1 − θ1 ) and n(θ̂2 − θ2 ).
(iii) Using the delta method, find the approximate large-sample distribution of
√ 
n h(θ̂1 , θ̂2 ) − h(θ1 , θ2 ) .
(iv) Hence find an approximate 95% confidence interval for h(θ1 , θ2 ). [11]
 
n! x n−x
Note: If X ∼ Binomial(n, θ) then P (X = x|θ) = θ (1 − θ) .
x!(n − x)!

G14FOS-E1 Turn over

 4 G14FOS-E1

3 (a) (i) Define the Jeffrey’s prior.

(ii) Explain what is meant by a conjugate prior.

(iii) Determine the form of a conjugate prior when

pX (x|θ) = exp A(θ)B(x) + C(θ) + D(x) . (3.1)

(iv) Given a sample x1 , . . . , xn from (3.1), find the posterior distribution for θ using the
conjugate prior you obtained in part (a)(iii). [10]

(b) Suppose X ∼ Binomial(n, θ), so that

n!
Prob(X = x|θ) = θx (1 − θ)n−x , x = 0, 1, . . . , n, 0 < θ < 1.
x!(n − x)!
Consider two models M0 and M1 :

M0 : X ∼ Binomial(n, θ), θ ∼ Uniform[0, 1];

M1 : X ∼ Binomial(n, θ), θ ∼ Beta(2, 1).

1
The prior model probabilities are P [M0 ] = 3
and P [M1 ] = 23 .
(i) Calculate the Bayes factor in favour of model M1 versus model M0 .
(ii) By considering the posterior odds of model M1 versus model M0 , show that model
M1 is to be preferred if and only if

x > 41 (n − 2). [12]

(c) Suppose that X ∼ Binomial(n, θ) with prior for θ the uniform distribution on [0, 1]. Let
Y ∼ Binomial(m, θ) denote a new observation.
(i) Calculate the predictive distribution of Y given that X = x.
(ii) Identify this distribution in the case m = 1. [8]

[Note: The Beta(α, β) distribution has probability density function

1
θα−1 (1 − θ)β−1 , θ ∈ [0, 1], α > 0, β > 0,
B(α, β)
where B(α, β) = Γ(α)Γ(β)/Γ(α + β) and the gamma function Γ satisfies
Γ(α + 1) = αΓ(α) for all α > 0.]

G14FOS-E1
 5 G14FOS-E1

4 (a) Let θ = (θ1 , . . . , θk ) denote a vector of probabilities which sum to 1, i.e. θi ≥ 0 for
i = 1, . . . , k and ki=1 θi = 1. We say that N = (N1 , . . . , Nk ) ∼ Multinomial(n+ , θ)
P
when
k
(n+ )! Y ni
P (N = n|θ) = Qk θi ,
i=1 ni ! i=1

where n = (n1 , . . . , nk ), the ni are non-negative integers and n+ = ki=1 ni . We say that
P
a probability vector θ ∼ Dir(α) if θ has the probability density function
k
Y
D(α)−1 θiαi −1
i=1

on the set θi > 0 for all i = 1, . . . , k and ki=1 θi = 1, where α = (α1 , . . . , αk ) is a vector
P
of prior parameters with αi > 0 for i = 1, . . . , k and
Qk
Γ(αi )
D(α) = i=1 ,
Γ(α+ )
Pk
where α+ = i=1 αi and Γ is the gamma function. Recall that for all α > 0,
Γ(α + 1) = αΓ(α).

(i) If θ ∼ Dir(α), find the prior mean vector

µprior = E[θ] = (E(θ1 ), . . . , E(θk )) .
(ii) Find the posterior π(θ|N = n) where N ∼ Multinomial(n+ , θ) and θ has a Dir(α)
prior.
(iii) Find the posterior mean


µpost = E[θ|n] = E(θ1 |n), . . . , E(θk |n) .

(iv) Let n̄ = n−1

+ n denote the sample mean. Show that

µpost = γµprior + (1 − γ)n̄,

where γ is a scalar quantity to be determined.
(v) Briefly explain the role of γ. [17]

(b) (i) Define what is meant by a Bayesian Credible Region.

(ii) Define what is meant by a Highest Posterior Density Region (HPDR).
(iii) By considering the case where X1 , . . . , Xn are independent and identically distributed
N(µ, σ 2 ) random variables, where σ 2 is known and µ has an (improper) uniform prior
on (−∞, ∞), find the 95% HPDR region for µ.
(iv) Find the corresponding 95% classical confidence interval for µ when σ 2 is still assumed
known.
(v) Discuss briefly how what you found in (b)(iii) relates to what you found in (b)(iv). In
what sense are they similar? In what sense are they different? [13]

G14FOS-E1 Turn over

 6 G14FOS-E1

5 (a) Let pX (x|θ) denote a probability model, let π(θ) denote a prior for θ and let L(d, θ) denote
a loss function.
(i) What is a decision rule δ?
(ii) Define the risk R(δ, θ) of a decision rule δ with respect to the loss function L(d, θ).

(iii) Define what is meant by an admissible decision rule.

(iv) Define the Bayes risk R̄(δ, π) of δ when prior π is used.
(v) What is the Bayes (decision) rule? How should we calculate the Bayes rule in practice? [11]

(b) Consider a single observation x from the probability model

θ3 2 −θx
pX (x|θ) = xe (x > 0, θ > 0) (5.1)
2
and consider the improper prior π(θ) ∝ θ−1 .
(i) Find the posterior density π(θ|x).
(ii) Given that the loss function L(d, θ) = (d − θ)2 is used, find the corresponding Bayes
rule. [8]

(c) Continuing with model pX (x|θ) in (5.1), consider the decision rule δc (x) = c/x, where
c > 0 is a constant.
(i) Show that the risk R(δc , θ) of δc is given by

θ2 2
R(δc , θ) = (c − 2c + 2).
2
[Hint: If X ∼ Gamma(α, β) and α > 2 then E[X −1 ] = β/(α − 1) and
Var(X −1 ) = β 2 /{(α − 1)2 (α − 2)}.]
(ii) What choice of c minimises the risk?

(iii) What choice of c corresponds to the Bayes rule found in part (b)?
(iv) What is the main conclusion? How does your conclusion relate to what you know
about the admissibility or otherwise of Bayes rules? [11]

[Note: Recall that the Gamma(α, β) distribution has probability density function
β α α−1 −βx
x e (x > 0, α > 0, β > 0),
Γ(α)
where Γ(α) is the gamma function that satisfies Γ(α + 1) = αΓ(α) for all α > 0.]

G14FOS-E1 End