ECON0019: Quantitative Economics and

Econometrics
Lecture 8: Heteroskedasticity, proxy variables, and
measurement errors

Professor Dennis Kristensen

UCL

November 29, 2018

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 1 / 35

Today’s Lecture

At today’s lecture, we will:

1 Relax the assumption of regression errors being homoskedastic
2 Introduce the idea of proxy variables and measurement errors: When
you don’t have "exact" data on relevant variables.
The lecture will be based on Wooldridge’s Chapter 8, Section 9.2 and
Section 9.4.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 2 / 35

Key results under homoskedasticity

1 Under Assumptions MLR.1-MLR.4, OLS is unbiased (and consistent)

2 If we add the assumption of homoskedasticity,

MLR.5 : Var (y jx ) = Var (u jx ) = σ2 is constant,

we furthermore have that:

OLS is BLUE (Best Linear Unbiased Estimator)
the proposed t-stat’s and F -stat’s asymptotically follow N (0, 1) and
F -dist’s, and so can be used to test hypotheses.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 3 / 35

But homoskedasticity rarely holds: variance of wages rises
with years of schooling

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 4 / 35

Implications of heteroskedasticity

Under MLR.1-4, but in the presence of heteroscedasticity:

OLS estimates are still unbiased (and consistent) but no longer BLUE
Our estimator of std( β̂) (that is, our standard errors) are, however,
biased
As a consequence: t-statistics, F -statistics and con…dence intervals
that we have used so far are invalid under heteroskedasticity.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 5 / 35

Correcting for heteroskedasticity

How to do inference in the presence of heteroscedasticity?

Robust standard errors: We will develop new standard errors which are
consistent, even in the presence of heteroscedasticity.
Using these robust standard errors in our t-statistics, we can draw
correct inference.
Similarly, we can adjust F statistics and con…dence intervals so that
they are robust to heteroskedasticity.
Can we get estimators that are BLUE under heteroscedasticity?
Yes, but these estimators rest on strong assumptions and don’t always
perform well in …nite samples.
So generally it is better to stick to OLS (even if it’s asymptotically
ine¢ cient)

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 6 / 35

Heteroskedastic simple regression

Consider …rst a simple linear regression model

yi = β0 + β1 xi + ui

Assume that SLR.1-SLR.4 hold, but not SLR.5. Rather, we have

Var(ui jxi ) = σ2 (xi ) varies as a function of xi .

We will use the following short-hand notation: σ2i = σ2 (xi ). If

σ2i = σ2 , data is homoskedastic.
Recall that
1
n ∑ni=1 (xi x̄ )ui
β̂1 = β1 +
σ̂2x
where σ̂2x = ∑ni=1 (xi x̄ )2 /n. What is the variance of β̂1 when
SLR.5 no longer holds?

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 7 / 35

Conditional variance of OLS with heteroskedasticity

!
1
n ∑ni=1 (xi x̄ )ui
Var( β̂1 jXn ) = Var β1 + Xn
σ̂2x
!
n
1
= 4 2
σ̂x n
Var ∑ (xi x̄ )ui Xn
i =1
n
1
= 4 2
σ̂x n
∑ (xi x̄ )2 Var ( ui j Xn )
i =1
n
1
= 4 2
σ̂x n
∑ (xi x̄ )2 σ2i .
i =1

If homoskedasticity, ∑ni=1 (xi x̄ )2 σ2i = σ2 ∑ni=1 (xi x̄ )2 , and the above

collapses to the "usual" expression.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 8 / 35

Large-sample distribution of OLS with heteroskedasticity

1
n ∑ni=1 (xi x̄ )ui
β̂1 = β1 + ,
σ̂2x
where, by LLN and CLT as n ! ∞, σ̂2x !p σ2x and

1 n 1 n Var ((xi µx )ui )

n i∑ n i∑
a
(xi x̄ )ui ' (xi µx )ui N 0, .
=1 =1 n

Here, Var((xi µx )ui ) = E (xi µx )2 ui2 and so

E [(xi µx )2 u i2 ] !
N 0,
a
n E (xi µx )2 ui2
β̂1 β1 + =N β1 , .
σ2x σ4x n

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 9 / 35

Non-robust ("usual") standard errors lead to incorrect
inference
If σ2i 6= σ2 , OLS is still consistent and normally distributed, but its
variance is more complicated (6= σ2 /σ2x n)
In particular, the standard errors derived under homoskedasticity
(HOM)
σ̂
seHOM ( β̂1 ) = p is not a consistent estimator of std( β̂1 ).
σ̂x n
As a consequence,
β̂1 β1
tβ̂ = is not N (0, 1) ,
1 seHOM ( β̂1 )
F -stat’s do not follow F -dist.
... and so you are prone to draw incorrect inference (accept false
hypotheses, reject true ones)
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 10 / 35
Robust standard errors - a correction
So we need to come up with another estimator of Var( β̂1 ) under
heteroskedasticity. White (1980) showed how to do it:
As part of our asymptotic analysis, we found that in large samples
E (xi µx )2 ui2
VarHR ( β̂1 ) ' .
σ4x n
But that’s just the ratio of two population moments (albeit more
complicated than before). These can be estimated as follows:
1 Run OLS and compute ûi = yi β̂0 β̂1 xi , i = 1, ..., n.
2 Compute the following heteroskedasticity-robust (HR) sthandard
errors,
q
∑ni=1 (xi x̄ )2 ûi2
1
n
seHR ( β̂1 ) = p
σ̂2x n
This is a consistent estimator for heteroscedasticity of any form, incl the
case where σ2i = σ2 .
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 11 / 35
Robust standard errors in Multiple regressions

The results for the SLR generalise to the MLR,

y = β0 + β1 x1 + . . . + βk xk + u

Assume that MLR.1-MLR.4 hold, but not MLR.5 so that

Var(ui jxi 1 , . . . , xik ) = σ2 (xi 1 , . . . , xik ) = σ2i .

Our "usual" estimator of Var( β̂j ) is now inconsistent, and instead we

use the following heteroskedasticity-robust version,
q
∑ni=1 r̂ij2 ûi2
seHR ( β̂j ) = ,
SSRj
where r̂ij is the …rst-step regression residual and SSRj is the residual sum
of squares from this regression.
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 12 / 35
Heteroscedasticity-robust inference

Once we have obtained correct (consistent) standard errors, we can

use these to obtain a robust t-stat,

β̂j βj a
tβ̂ = N (0, 1) .
j seHR ( β̂j )

The usual critical values are used – only di¤erence is that we are
using a di¤erent estimator of std( β̂j ).
Robust con…dence intervals are obtained by using seHR ( β̂j ) in place
of seHOM ( β̂j ).
Also possible to derive robust F -tests (although we will not go into
derivations here)
Note that robust inference is justi…ed ONLY when sample size is
su¢ ciently large (all results are asymptotic)

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 13 / 35

log-wage regression, with and without robust S.E.’s

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 14 / 35

House price regression, with and without robust S.E.’s

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 15 / 35

Eyeballing heteroskedasticity: Look at the residuals...

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 16 / 35

... Does dispersion of residuals appear to vary with
regressors?

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 17 / 35

In some cases, transforming data can remove
heteroskedasticity....

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 18 / 35

... For example, using log-tranforms

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 19 / 35

Log-house price regression: Less evidence of
heteroskedasticity

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 20 / 35

Generalised Least Squares (GLS) I
OLS is no longer e¢ cient (smallest variance) when errors are
heteroskedastic. A better procedure is GLS, here presented for SLR,
yi = β0 + β1 xi + ui .
Suppose that the functional form of σ2 (x ) =Var(u jx ) is known to us.
For example, that σ2 (x ) = α0 + α1 x 2 for known values of α0 and α1 .
q
We can then divide through with σi = α0 + α1 xi2 in our MLR to
remove heteroskedasticity:
yi 1 xi ui
= β0 + β1 + .
σi σi σi σi
This can be rewritten as
yi = β0 xi 0 + β1 xi 1 + ui ,
where, yi = yi /σi , xi 0 = 1/σi ,..., and ui = ui /σi . The rescaled
regression error ui is homoskedastic...
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 21 / 35
Generalised Least Squares II

yi = β0 xi 0 + β1 xi 1 + ui ,
ui 1 σ2i
Var (ui jxi ) = Var j xi = Var ( ui j xi ) = = 1.
σi σ2i σ2i
Infeasible GLS estimator of β0 , β1 : Run OLS in this transformed SLR.
Since ui in this transformed version is homoskedastic, the GLS
estimators are BLUE and we don’t need to adjust standard errors.
The above assumed we know σ2 (x ) – wholly unrealistic. Instead use
feasible GLS estimator:
1 Use OLS to obtain ŷ = β̂ + β̂ x
i 0 1 i 1 + . . . + β̂k xik + ûi .
2 Estimate σ2 (x ). E.g., if σ2 (x ) = α + α x 2 , α and α can be
0 1 0 1
estimated by OLS, ûi2 = α̂0 + α̂1 xi2 + êi , where ei regression error.
q
3 Estimate σ̂i = α̂0 + α̂1 xi2 and run OLS in
ŷi = β0 x̂i 0 + β1 x̂i 1 + ui ,
where, ŷi = yi /σ̂i , x̂i 0 = 1/σ̂i , and x̂i 1 = xi 1 /σ̂i .
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 22 / 35
Generalised Least Squares - summary

Unfortunately, this feasible version of GLS requires:

correct speci…cation of the functional form of σ2 (x ).
multiple estimation steps (and so more estimation errors)
As a consequence, in small and moderate sample sizes, feasible GLS
is rather ill-behaved. In particular, very far from normally distributed.
So only recommended to use GLS if you have a large sample and/or
you are interested in the conditional variance
If you like to bet on the stock market, variance is very important!
And so big banks spent a lot of time modelling and estimating
conditional variance.
So getting the variance right is often important in itself – just not so
much in micro applications.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 23 / 35

Use of proxy variables - a part of our own lives

In many situations, we are forced to employ “proxy variables”

(proxies).
For instance, admission to the BSc Econ programme here at UCL is
based on A levels marks (possibly together with language test scores).
Similar, admission at US universities is based on SAT scores and high
school GPAs
These function as proxies for applicants’abilities to do well in the
programme, including their intelligence.
No one argues that standardized tests or grade point averages are
actually measuring aptitude, or intelligence;
but there are reasons to believe that the observable variable is well
correlated with the unobservable, or latent, variable.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 24 / 35

Wage regression: An example of the use of proxy variables
Similarly, econometricians routinely use proxy variables in their
models. We have actually done it in this course:
Remember this wage regression?
[ = β̂ + β̂ years_of _school + β̂ math11
lnpay 0 1 2

Here, math11 = score on math test at age 11, it was included to

control for ability.
Ideally we would like to run the following regression:
lnpay = β0 + β1 years_of _school + β2 ability + u
But ability is not available to us in our data set. In fact, we’ll
probably have a hard time reaching an agreement on what ability
really is....
So what should we make of the …rst regression? Can we still interpret
the OLS estimator β̂1 as a measure of returns-to-schooling (having
controlled for ability)?
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 25 / 35
General approach to using proxy variables

Ideal (correct) regression

y = β0 + β1 x1 + β2 x2 + u

where x1 is observed, while x2 is unobserved/latent.

Actual regression with x2 being a proxy for x2 :

ŷ = β̂0 + β̂1 x1 + β̂2 x2 .

Will the OLS estimators β̂1 and β̂2 provide meaningful estimates of
β1 and β2 ?

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 26 / 35

Model for latent and proxy variable
Assume that MLR.1-MLR.4 are satis…ed for the "ideal" regression.
In addition, suppose that

x2 = δ0 + δ1 x2 + v , E [v jx2 ] = 0 (1)

where v contains factors of x2 that are excluded in x2 (e.g., ability is

more than just your performance in a math test)
We then need the following additional assumptions in order for the
"proxy" regression to be (partially) consistent:
P.1 E [u jx2 ] = 0 – The proxy is „just a proxy“ for the omitted
variable, it does not belong into the population regression,
i.e. it is uncorrelated with its error.
P.2 E [x2 jx1 , x2 ] = E [x2 jx2 ] – The proxy variable is a „good“
proxy for the omitted variable, i.e. using other variables in
addition will not help to predict the omitted variable.
P.3 The model (1) is correct – less important, can be generalised.
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 27 / 35
(Partial) consistency of "proxy" regression

Substituting in x2 = δ0 + δ1 x2 + v ,

y = β0 + β1 x1 + β2 x2 + u
= ( β0 + β2 δ0 ) + β1 x1 + β2 δ1 x2 + ũ, ũ = β2 v + u.

If we can verify that E [ũ jx1 , x2 ] = 0 then MLR.1-MLR.4 will hold for
the above regression, and we can conclude that

β̂1 !p β1 , β̂2 !p β2 δ1 .

But
E [u jx1 , x2 ] = β2 E [v jx1 , x2 ] + E [u jx1 , x2 ]
where
E [v jx1 , x2 ] = E [v jx2 ] = 0,
P.2 P.3

E [u jx1 , x2 ] = 0.
P.1+MLR.4

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 28 / 35

Discussion of proxy assumptions in the wage example

[ = β̂ + β̂ years_of _school + β̂ math11

lnpay 0 1 2

If x2 = math11 and x2 = ability satisfy P.1-P.2:

β̂1 is an unbiased/consistent estimate of the returns to education
our estimate of returns to ability ( β̂2 ) will generally be biased
(β2 δ1 6= β2 unless δ1 = 1).
Discussion of P.1: Should be full…lled as math11 is not a direct wage
determinant; what matters is how able the person proves at work
Discussion of P.2: Most of the variation in ability should be
explainable by variation in math11, leaving only a small rest to
years_of _school.
So probably not exactly satis…ed, but close enough such that the bias
in β̂1 is relatively small.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 29 / 35

Measurement errors (or errors-in-variables)

Measurement error in variables: the variable that theory tells us

belongs in the relationship has been incorrectly measured in the
available data.
Examples:
Survey data: self-reported age, income, alcohol consumption, cigarette
consumption, etc.
Polls: (Some) people are embarrased about their political beliefs
Fat …ngers and other technical issues with data handling.
Measurement errors are related to proxies – in both cases, the "ideal"
variable is unavailable and we use another one instead.
But the interpretation of the error linking the two together (latent
and actual) is di¤erent.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 30 / 35

Measurement error in dependent variable
Let y denote the variable that we would like to explain through a
MLR,
y = β0 + β1 x1 + + βk xk + u
Let y be the mismeasured observation of y with
e0 = y y
being the measurement error.
Regression with errors in dependent variable,
y = β0 + β1 x1 + + βk xk + u + e0 .
If (in addition to MLR.1-MLR.4), E [e0 jx1 , ..., xk ] = 0, then OLS in
last regression will be consistent.
But variances of OLS estimates will generally be larger than the ones
using y . For example, if u and e0 are uncorrelated
Var(u + e0 ) =Var(u ) +Var(e0 ).
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 31 / 35
Reasonable to assume measurement error is
mean-independent?

The requirement E [e0 jx1 , ..., xk ] = 0 has to be analysed on a

case-by-case basis: What are the (potential) sources of measurement
errors, and how do they relate to the regressors.
Survey data example: y =cigarette consumption and x = income.
Low income people are generally more like to smoke, and its only
smokers who will lie about their consumption (you’d think...). So
E [e0 jincome ] 6= 0 most likely.
Fat …ngers example: Assuming that nobody has been tampering
with data on purpose, then fat …ngers are random mistakes in data
and so indpendent of regressors. So E [e0 jx1 , ..., xk ] = 0 seems
reasonable.

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 32 / 35

Measurement error in regressor(s)
Measurement error in an explanatory variable is, on the other hand, a
far more serious problem.
Suppose true model is (just one regressor for simplicity),
y = β0 + β1 x + u,
but instead of x we observe
x = x + e, where e is the measurement error.
We then estimate β1 by OLS in the following regression,
y = β0 + β1 x + ũ, ũ = u β1 e.
"Classical" errors-in-variables (CEV) assumption:
E [e jx ] = 0 ( case of errors in y )
This implies that
Cov (e, x ) = Cov (e, x + e ) = Cov (e, x ) + Cov (e, e ) = Var (e ) ,
and so....
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 33 / 35
OLS is biased/inconsistent due to CEV in regressors

y = β0 + β1 x + ũ, ũ = u β1 e,
where, using Cov(u, x1 ) = 0,

Cov (ũ, x ) = Cov (u β1 e, x ) = Cov (u, x ) β1 Cov (e, x1 ) = β1 Var (e )

This implies that MLR.4 fails in the above regression and so OLS will
be biased and inconsistent:
1 n
n ∑i =1 (xi x̄ ) ũi Cov (ũ, x1 ) Var (e )
β̂1 = β1 + 1 n 2
!p β1 + = β1 1
Var (x ) Var (x )
n ∑i =1 (xi x̄ )
So OLS will su¤er from a negative bias – we will tend to
underestimate the actual e¤ect of x. This is referred to as
attenuation bias.
This result can be generalised to MLR, but size and sign of biases are
less clear-cut.
D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 34 / 35
Summary of today’s lecture

Heteroskedasticity-robust inference: Unless you’re certain that errors

are homoskedastic, always use heteroskedastic-robust standard errors.
Otherwise, inference will be invalid.
Proxy variables: Can be helpful (unless you proxy the variable of
interest): Under reasonable assumptions you can obtain unbiased
estimates of the remaining coe¢ cients.
Measurement errors: Not a big problem if in response variable, but
will generate biases in estimates if in regressors.
Coming soon... In next term, you will be introduced to an alternative
to OLS that can remove these biases

D Kristensen (UCL) ECON0019: Lecture 8 November 29, 2018 35 / 35