Part IIA Paper 3 Econometrics

Lecture 8:
Heteroskedasticity

Oleg I. Kitov
oik22@cam.ac.uk

Faculty of Economics and Selwyn College

Michaelmas Term 2021

1/20
Lecture outline

I Heteroskedasticity in linear models.

I Consequences of heteroskedasticity on OLS estimators:
- still unbiased and consistent;
- no longer efficient [best].
I Testing for heteroskedasticity:
- Goldfeld-Quandt test;
- Breusch-Pagan test;
- White’s test.
I Dealing with heteroskedasticity:
- correct potential model misspecification;
- use robust standard errors;
- generalized least squares (GLS).

2/20
Heteroskedasticity, OLS and Gauss-Markov

I Consider Yi = β0 + β1 Xi1 + · · · + βk Xik + ui = X|i β + ui , i = 1, . . . , n.

I (A4) constant conditional variance of errors Var (ui |Xi ) = σ 2 , for all i.
I Constant conditional variance of dependent variable Var (Yi |Xi ) = σ 2 .
I If (A4) violated, we have heteroskedasticity, Var (ui |Xi ) 6= σ 2 .
I Heteroskedasticity could be that Var (ui |Xi ) = σi2 , with σi2 6= σj2 , for i 6= j.
I Typically variance of errors depends on regressors, Var (ui |Xi ) = f (Xi ).
I When errors are heteroskedastic, OLS estimators β̂j , for j = 0, . . . , k, are:
- unbiased; consistent; asymptotically normal [with robust errors];
- not efficient [OLS formula for Var(β̂j ) assumes iid errors, so is wrong];
- (A4) violated, so Gauss-Markov theorem does not apply;
- there are linear unbiased estimators that have lower variance than OLS.
I If use usual OLS formula to estimate Var(β̂j ) under heteroskedasticity:
- standard errors will be biased and t-statistics will be wrong;
- will reach wrong inference on H0 : βj = 0 [too low or too high].
3/20
Example: wage, IQ and education
I Consider: wage
[ i = β̂0 + β̂1 educi + β̂2 IQi + β̂3 experi , i = 1, . . . , n.
[ . As û¯ = 0, Var
I OLS residuals: ûi = wage − wage c (ui ) = 1 P û 2 .
i i n−k−1 i
I Check if ûi2 vary with educi or experi [evidence of heteroskedasticity].
I If so, then we may suspect that Var (ui ) = f (educi , experi ).
c (ûi |educi = 9) < Var
I Clearly, from Figure below: Var c (ûi |educi = 12).

5000000 5000000

4000000 4000000

3000000 3000000
res_wage2

res_wage2

2000000 2000000

1000000 1000000

0 0
8 10 12 14 16 18 0 5 10 15 20 25
years of education years of work experience 4/20
Unbiasedness of OLS
I Consider Yi = β0 + β1 Xi + ui , for a random sample (Yi , Xi ), i = 1, . . . , n.
I Take expectation of β̂1 conditional on all observations X = (X1 , . . . , Xn ).
Pn
" Pn
#
i=1 Xi − X̄ ui h i
i=1 Xi − X̄ ui
β̂1 = β1 + Pn
2 =⇒ E β̂1 |X = β1 +E Pn
2 |X .
i=1 Xi − X̄ i=1 Xi − X̄

I Note E [Xi |X] = E [Xi |Xi ] = Xi since Xi are independent, hence:

! " n
#
h i 1 X

E β̂1 |X = β1 + Pn
2 E Xi − X̄ ui |X
i=1 Xi − X̄ i=1
! n
1 X

= β1 + Pn
2 Xi − X̄ E [ui |X] .
i=1 Xi − X̄ i=1

I Clearly, E[β̂1 |X] = β1 if exogeneity is assumed, E[ui |X], for all i.

I Apply the law of iterated expectations: E[β̂1 ] = E[E[β̂1 |X]] = E[β1 ] = β1 .
I OLS unbiased without any assumptions about the variance of ui ! 5/20
OLS estimator of error variance

6/20
Conditional variance of the OLS estimator

I Conditional variance is simplified only if

- random sample, or no serial correlation Cov (ui , uj |X) = 0 all i 6= j;
- errors are homoskedastic, Var (ui |X) = σ 2 :

Su2 Su2
Sβ2ˆ1 = Var


c β̂1 |X = P
2 =
n
Xi − X̄ SSTX
i=1

I Under heteroskedasticity, Var (ui |X) = σi2 and σi2 6= σj2 some i 6= j:
- error variance estimator Su2 is wrong [σi2 vary across individuals i];
- OLS estimator Sβ̂2 is wrong for Var(β̂1 );
1
a
- asymptotic distribution β̂1 ∼ N(β1 , Sβ̂2 ) has the wrong variance;
1
- testing significance H0 : β1 = 0 will give wrong inference.
I OLS is not optimal in presence of heteroskedasticity since
- OLS gives equal weight to all observations, but...
- observations with larger/smaller error variance contain less/more information;
7/20
Causes of heteroskedasticity

I Heteroskedasticity is often a bye-product of model misspecification:

- omitted variables [e.g. nonlinear effects are missing from model];
- sub-population differences [e.g. different models for men and women];
- wrong functional form of Xi [e.g. effects of variables may be nonlinear];
- wrong functional form of Yi [e.g. could be that ln Yi is more appropriate];
- not “genuine” heteroskedasticity, but an artifact of other problems;
- need to address model misspecification, heteroskedasticity will likely go away.
I Instrumental variable (IV) estimation [will consider later this term]:
- errors may increase as IV becomes more extreme in either direction;
- plot of residuals against IV has hourglass shape.
I Measurement error:
- e.g. some respondents in surveys provide more accurate data than others.
I Genuine heterogeneity of errors conditional on observed characteristics:
- e.g. sales of larger firms might be more volatile than sales of smaller firms.
8/20
Heteroskedasticity due to sub-population differences

9/20
Goldfeld-Quandt test for heteroskedasticity
I Consider: wagei = β0 + β1 educi + β2 IQi + β3 experi + ui .
I Test if error variance is monotonic [increasing/decreasing] in educi .
I Split sample into two sub-samples [with/without higher education]:
(1) (1) (1) (1)
educi ≤ 12 : wagei = β0 + β1 educi + β2 IQi + β3 experi + ui , n1 , SSR1
(2) (2) (2) (2)
educi > 12 : wagei = β0 + β1 educi + β2 IQi + β3 experi + vi , n2 , SSR2

I H0 : σu2 = σv2 [homoskedasticity] vs H1 : σu2 < σv2 [heteroskedasticity].

I F -test for equality of variances, with Su2 and Sv2 estimators of σu2 and σv2 :
Sv2 SSR2 / (n2 − k − 1)
W = 2
= ∼ Fn2 −k−1,n1 −k−1 .
Su SSR1 / (n1 − k − 1)

I Major disadvantages of Goldfeld-Quandt test [archaic, not really used]:

- depends on arbitrary criteria for splitting data into sub-samples;
- doesn’t provide guidance for adjusting the model for heteroskedasticity;
- can only detect heteroskedasticity that is monotonic in the regressor;
- will falsely not reject H0 when error variance is quadratic in regressor. 10/20
Goldfeld-Quandt test in wage regression

I [wage2.dta] STATA estimation:

- reg wage educ IQ exper if educ <= 12; SSR1 = 45312537, n1 = 481,
- reg wage educ IQ exper if educ > 12; SSR2 = 81856465, n2 = 454.

I F -test statistic [random variable]:

SSR2 / (n2 − k − 1)
W = ∼ Fn2 −k−1,n1 −k−1 .
SSR1 / (n1 − k − 1)

I Observed sample test statistic:

81856465/451
w= = 1.91
45312537/478

I One-sided critical value with significance α = 0.05 is F0.95,451,478 = 1.16.

I Reject H0 with α = 0.05, since 1.91 > 1.16, evidence of heteroskedasticity.
I Conclude error variance is monotonically increasing in education.
11/20
Breusch-Pagan test for heteroskedasticity
I Breusch-Pagan tests for linear form of heteroskedasticity .
I Crucial assumption: normality of errors [if violated, test is invalidated].
I Version 1: variance of errors is linear in a individual regressor:
- auxiliary regression: ûi2 = γ0 + γ1 Xi1 + εi ;
- H0 : γ1 = 0, homoskedasticity Var (ui ) = σ 2 .
- H1 : γ1 6= 0, heteroskedasticity Var (ui ) = f (Xi1 ).
I Version 2: variance of errors is linear in all regressors:
- auxiliary regression: ûi2 = γ0 + γ1 Xi1 + · · · + γk Xik + εi ;
- H0 : (γ1 = 0) ∩ · · · ∩ (γk = 0), homoskedasticity Var (ui ) = σ 2 .
- H1 : (γ1 6= 0) ∪ · · · ∪ (γk 6= 0), heteroskedasticity Var (ui ) = f (Xi1 , . . . , Xik ).
I Version 3: use predicted values of the dependent variable:
- auxiliary regression: ûi2 = γ0 + γ1 Ŷi + εi ;
- H0 : γ1 = 0, homoskedasticity Var (ui ) = σ 2 .


- H1 : γ1 6= 0, heteroskedasticity Var (ui ) = f Ŷi .
I In all cases use test statistic nR 2 ∼ χ2q , q is the number of restrictions.
12/20
White’s test for heteroskedasticity

I Auxiliary regression for ûi2 : include Xij , their squares and cross-terms.
I Auxiliary regression [simpler]: ûi2 = δ0 + δ1 Ŷi + δ2 Ŷi2 + εi .
I H0 : (δ1 = 0) ∩ (δ2 = 0), homoskedasticity Var (ui ) = σ 2 .
 
I H1 : (δ1 6= 0) ∪ (δ2 6= 0), heteroskedasticity Var (ui ) = f Ŷi .
I LM = nR 2 ∼ χ22 and reject H0 if test-statistic is greater than χ21−α,2 .
I Could also include higher powers of Ŷi , e.g. cubed Ŷi3 .
I STATA adds squares and cross-terms explicitly to auxiliary regression.
I White’s advantages compared with Breusch-Pagan:
- relaxes assumption of normally distributed errors;
- flexible functional form, identifying nearly any pattern of heteroskedasticity.
I White’s disadvantages compared with Breusch-Pagan:
- cannot determine explicit functional form of heteroskedasticity;
- loses power quickly when the number of regressors goes up.
13/20
Breusch-Pagan and White’s tests in wage regression

I Consider: wagei = β0 + β1 educi + β2 IQi + β3 experi + ui .

I [wage2.dta] STATA: reg wage educ IQ exper
I Breusch-Pagan 1: ûi2 = γ0 + γ1 educi + εi :
- hettest educ
- χ2 (1) = 41.25 and p = 0, reject H0 , Var (ui ) depends on educi .
I Breusch-Pagan 2: ûi2 = γ0 + γ1 educi + γ2 IQi + γ3 experi + εi :
- hettest educ IQ exper
- χ2 (3) = 45.52 and p = 0, reject H0 , Var (ui ) depends on regressors.
I Breusch-Pagan 3: ûi2 = γ0 + γ1 wage
[ i + εi :
- hettest
- χ2 (1) = 38.89 and p = 0, reject H0 , Var (ui ) depends on educi .
I White’s test: ûi2 = γ0 + γ1 educi + · · · + γ9 IQi ×experi + εi :
- imtest, white
- χ2 (9) = 19.34 and p = 0.0224, reject H0 , Var (ui ) depends on regressors.
14/20
Dealing with heteroskedasticity: model specification [1/2]
I First, try to correct model misspecification.
I Respecify the model or transform the variables, e.g. consider ln wagei .
I Predicted values: lwage
\ i = β̂0 + β̂1 educi + β̂2 IQi + β̂3 experi .
I OLS residuals: ûi = lwagei − lwage
\i .
I ûi2 does not vary with educi and experi [as much as in model for wagei ].

4 4

3 3
res_lwage2

res_lwage2
2 2

1 1

0 0
8 10 12 14 16 18 0 5 10 15 20 25
years of education years of work experience 15/20
Dealing with heteroskedasticity: model specification [2/2]

I Consider: ln wagei = β0 + β1 educi + β2 IQi + β3 experi + ui .

I [wage2.dta] STATA: reg lwage educ IQ exper
I Breusch-Pagan 1: ûi2 = γ0 + γ1 educi + εi :
- hettest educ
- χ2 (1) = 0.89 and p = 0.3453, do not reject H0 , Var (ui ) = σ 2 .
I Breusch-Pagan 2: ûi2 = γ0 + γ1 educi + γ2 IQi + γ3 experi + εi :
- hettest educ IQ exper
- χ2 (3) = 4.54 and p = 0.2089, do not reject H0 , Var (ui ) = σ 2 .
I Breusch-Pagan 3: ûi2 = γ0 + γ1 lwage
\ i + εi :
- hettest
- χ2 (1) = 0.05 and p = 0.8197, do not reject H0 , Var (ui ) = σ 2 .
I White’s test: ûi2 = γ0 + γ1 educi + · · · + γ9 IQi ×experi + εi :
- imtest, white
- χ2 (9) = 9.15 and p = 0.4231, do not reject H0 , Var (ui ) = σ 2 .
16/20
Dealing with heteroskedasticity: robust errors [1/3]

I Heteroskedasticity causes OLS standard errors to be biased.

I OLS assumes that errors are independent and identically distributed.
I Robust standard errors relax either or both of those assumptions.
I Robust standard errors do not change OLS coefficient estimates.
I Robust standard errors do change standard errors and significance tests.
I Test statistics will robust errors give reasonably accurate p values.
I STATA option robust relaxes the assumption that the errors are iid.
I [wage2.dta] STATA: estimate and produce output summary table:
- eststo clear
- eststo: quietly reg wage educ IQ exper
- eststo: quietly reg wage educ IQ exper, vce(robust)
- esttab, se r2 scalars(F)

17/20
Dealing with heteroskedasticity: robust errors [2/3]
(1) iid errors (2) robust errors
wage wage
∗∗∗
educ 58.10 58.10∗∗∗
(7.056) (7.427)
∗∗∗
IQ 5.069 5.069∗∗∗
(0.941) (0.897)
exper 17.42∗∗∗ 17.42∗∗∗
(3.116) (3.104)
const −539.4∗∗∗ −539.4∗∗∗
(116.7) (115.0)
n 935 935
2
R 0.162 0.162
Standard errors in parentheses
∗p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001
18/20
Dealing with heteroskedasticity: robust errors [3/3]

I Heteroskedasticity robust [White-Huber-Eicker] standard errors.

I Do not impose any assumptions on the structure of heteroskedasticity.
I Robust standard errors are appropriate even under homoskedasticity.
I Estimate Yi = β0 + β1 Xi1 + · · · + βk Xik + ui , get residual ûi .
I Regress Xij on all other explanatory variables, get residual ε̂ij , j = 1, . . . , k.
Pn
I Compute SSRj = i=1 ε̂2ij , sum of squared residuals from regression of Xij .
I Heteroskedasticity robust standard errors is then

  Pn ε̂2 û 2
ij ij
Var β̂j = i=1 2
SSRj

I This only works asymptotically [in large samples].

q
I Heteroskedasticity-robust t-statistics valid: divide β̂j by Var(β̂j ).
I The usual F -statistic is invalid, need heteroskedasticity-robust statistic.
19/20
Dealing with heteroskedasticity: GLS
I Consider Yi = β0 Xi0 + β1 Xi1 + · · · + βk Xik + ui for i = 1, . . . , n.
I If the heteroskedasticity form is known, can use Generalized Least Squares.
I Suppose Var (ui |Xi ) = σ 2 h (Xi ) = σ 2 hi for some hi [e.g. hi = Xi1 ].
√
I Multiply errors ui by 1/ hi , for i = 1, . . . , n, to make them homoskedastic:
 
ui 1 1
Var √ = Var (ui ) = σ 2 hi = σ 2 .
hi hi hi
√
I Transform the regression model by multiplying through by 1/ hi :
       
Yi Xi0 Xi1 Xik ui
√ = β0 √ + β1 √ + · · · + βk √ + √
hi hi hi hi hi
Ỹi = β0 X̃i0 + β1 X̃i1 + · · · + βk X̃ik + ũi

I In GLS, observations with higher error variance have less weight.

I This model has homoskedastic errors ũi and OLS is BLUE.
I In reality, we never know the form of heteroskedasticity hi . GLS infeasible.
I Estimate heteroskedasticity form and apply feasible GLS (FGLS). 20/20