Structural Equation Modeling

An Econometrician’s Introduction

PD Dr. Stefan Klößner

Winter Term 2018/19

E R SIT
IV
UN

A
S
SA

IS

R
S

A VIE N
 Overview
NOT: easy to digest introduction for practitioners
instead: theoretical foundations of SEM
many formulas, using a lot of the ’Greeks’, i.e. Greek
symbols used to denote quantities appearing in SEM
models
aims to make the econometrically trained learn what SEM
’theoretically’ is
aims to explain why (one branch of) SEM is called
’covariance-based’
tries to gradually consider increasingly complex models,
moving from simple linear regression models to SEM
models
NOT: detailed discussion of all aspects of SEM

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 2

 Recap: Simple Linear Regression I

the simple linear regression models looks as follows:

y = α + βx + u,

where y, x, u are random (!) variables which are called,

respectively,
I regressand, dependent variable, or endogeneous variable: y
I regressor, independent variable, exogeneous variable: x
I error term, disturbance: u
α and β are unknown parameters which describe,
respectively,
I the intercept of the regression line: α
I the slope of the regression line: β

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 3

 Recap: Simple Linear Regression II
given data (y1 , x1 ), . . . , (yT , xT ) (with T denoting the sample
size), α and β are usually estimated by OLS, with formulas
T
P
(xt −x)(yt −y)
t=1 sxy
I β̂ = T
= sx2
−x)2
P
(xt
t=1
I α̂ = y − β̂x
notice that β̂ can be calculated using demeaned data
(xt − x, yt − y) only, while α̂ depends on the means y, x of
y, x (as well as β̂)
therefore, if one is only interested in β, the parameter
describing the relation between y and x, then one may
calculate the estimate β̂ using the demeaned versions of x
and y

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 4

 Recap: Multiple Linear Regression I

the multiple linear regression model looks as follows:

y = α + β1 x1 + . . . + βn xn + u,

where y, x1 , . . . , xn are random variables which are called,

respectively,
I regressand, dependent variable, or endogeneous variable: y
I regressors, independent variables, exogeneous variables:
x1 , . . . , xn
I error term, disturbance: u
βi (i = 1, . . . , n) determines by how much y is expected to
change if xi changes by one unit and all other regressors
stay unchanged

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 5

 Recap: Multiple Linear Regression II
given data (y1 , x11 , . . . , xn1 ), . . . , (yT , x1T , . . . , xnT ) (with T
denoting the sample size), α, β1 , . . . , βn are usually
estimated by OLS, satisfying the following formulas:
   −1  
β
c1 s11 s12 · · · s1n s1y
 ..   .. .
.. . .. .
..   ... 
 . = . ,
I
 

β
cn sn1 sn2 · · · snn sny

I b = y − (β
α c1 x1 + . . . + β
cn xn ), with
T
1 P
I sij := T (xit − xi )(xjt − xj ) (i, j = 1, . . . , n),
t=1
T
1 P
I siy := T (xit − xi )(yt − y) (i = 1, . . . , n).
t=1

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 6

 Recap: Multiple Linear Regression III

notice that βb1 , . . . , βbn can be calculated using demeaned

data (xit − xi (i = 1, . . . , n), yt − y) only, while α
b depends on
the means y, xi (i = 1, . . . , n) of the variables (as well as
βb1 , . . . , βbn )
therefore, if one is only interested in regression coefficients
β1 , . . . , βn , the parameters describing the relations between
the regressand y and the regressors x1 , . . . , xn , then one
may calculate the estimates βb1 , . . . , βbn using the demeaned
versions of regressand and regressors

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 7

 Simultaneous Equations I
we can generalize the multiple linear regression model with
respect to the number of dependent variables by allowing
more than one dependent variable
such models are called ’simultaneous equation models’,
they look as follows (with m denoting the number of
dependent variables):

y1 = α1 + β12 y2 + . . . + β1m ym + γ11 x1 + . . . + γ1n xn + u1

y2 = α2 + β21 y1 + β23 y3 + . . . + β2m ym + γ21 x1 + . . . + γ2n xn + u2
..
.
ym = αm + βm1 y1 + . . . + βm,m−1 ym−1 + γm1 x1 + . . . + γmn xn + um

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 8

 Simultaneous Equations II

denoting y := (y1 , . . . , ym )0 , x := (x1 , . . . , xn )0 ,

u := (u1 , . . . , um )0 , α := (α1 , . . . , αm )0 ,

0
 
β12 . . . ... β1m
 β21 0 β23 ... β2m 
 .. .. .. .. .. 
B :=  . . . . .  ∈ Rm×m ,
 
 ... .. 
βm−1,1 . βm−1,m 
βm1 ... . . . βm,m−1 0

Γ := (γij )i=1,...,m,j=1,...,n ∈ Rm×n , this can be written in compact

form:
y = α + By + Γx + u

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 9

 Simultaneous Equations III

estimating simultaneous equation models, though typically

not done using OLS, usually results in αb=y −B by −bΓ x,
while the estimates B and Γ can be calculated using the
b b
demeaned data y − y and x − x
y = α + B y + Γ x + u is called the structural form, while

y = (I − B)−1 α + (I − B)−1 Γ x + (I − B)−1 u

| {z } | {z } | {z }
α
e Γ
e u
e

is called the reduced form

y1 , . . . , ym are called endogeneous, x1 , . . . , xn are called
exogeneous

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 10

 SEM or Simultaneous Equations with Latent
Variables I
an SEM model is ’simply’ a simultaneous equations model
where, unfortunately, the endogeneuos and exogeneous
variables are latent, i.e. not observable
in SEM models,
I the m (latent) endogeneous variables are usually called η
I the n (latent) exogeneous variables are usually called ξ
I the m-dimensional error term is usually called ζ
I the relations between the latent variables are modeled by

η = Bη + Γξ + ζ or η = αη + B η + Γ ξ + ζ,

with the assumptions E(ζ) = 0, I − B non-singular, and ξ

uncorrelated to ζ. The intercept term αη is only included
when means of the latent variables are to be considered,
too.
Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 11
 SEM or Simultaneous Equations with Latent
Variables II
the so-called structural equation η = (αη +)B η + Γ ξ + ζ
must be accompanied by so-called measurement equations
which relate the latent variables ξ and η to their observable
counterparts x and y:

x = Λx ξ+δ, y = Λy η+, or x = αx +Λx ξ+δ, y = αy +Λy η+,

where
I x and y , often called indicators (for ξ and η), consist of q and
p observable variables, respectively,
I Λx ∈ Rq×n and Λy ∈ Rp×m are matrices which contain the
so-called factor loadings
I δ and  are q- and p-dimensional error terms.
I the intercept terms αx and αy are included only if means are
to be considered, too.
Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 12
 SEM or Simultaneous Equations with Latent
Variables III
all error terms are assumed to have zero mean
usually, ξ, x, η, y are also assumed to have zero mean: in
this case, no intercept terms appear in the above equations,
and demeaned data are used (this does not hold, though,
for instance for so-called multi-group or latent curve models)
it is assumed that both  and δ are uncorrelated with ζ and
ξ, and that the two error terms  and δ are uncorrelated
often, the covariance matrices are called Φ (for ξ), Ψ (for ζ),
Θ (for ), and Θδ (for δ)
however, the notations Σξ , Σζ , Σ , and Σδ are much more
intuitive and will be used in the sequel

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 13

 Summary of SEM Equations, Quantities, and
Assumptions
structural equation: η = (αη +)B η + Γ ξ + ζ
measurement equations:
x = (αx +)Λx ξ + δ, y = (αy +)Λy η + 
ξ: n latent exogeneous random variables with covariance
matrix Σξ
η: m latent endogeneous random variables
x, y: q- and p-dimensional observable random variables,
indicators of ξ and η
ζ, δ, : m-, q-, p-dimensional error terms with covariance
matrices Σζ , Σδ , Σ
it is assumed that Σξζ = 0, Σδ = 0, Σξ = 0, Σξδ = 0,
Σζ = 0, Σζδ = 0
fixed, but unknown quantities (model parameters): B, Γ, Λx ,
Λy ; when means are considered, additionally αη , αx , αy
Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 14
 Fundamental Theorem of SEM I
under the assumptions stated above,
I the covariance matrix Σx of the latent exogeneous variables’
observed indicators x is given by:

Σx = Λx Σξ Λ0x + Σδ

I the covariance matrix Σxy between the latent exogeneous

variables’ obeserved indicators x and the latent
endogeneous variables’ observed indicators y is given by:

Σxy = Λx Σξ Γ0 (I − B 0 )−1 Λ0y

I the covariance matrix Σx of the latent endogeneous

variables’ observed indicators y is given by:

Σy = Λy (I − B)−1 ΓΣξ Γ0 + Σζ (I − B 0 )−1 Λ0y + Σ

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 15

 Fundamental Theorem of SEM II
the unknown parameters, i.e. the structural parameters
contained in B and Γ, the factor loadings contained in Λx
and Λy , as well as the variances and covariances contained
in Σξ , Σζ , Σδ , and Σ are determined such that the
differences between the model-implied covariances
delivered by the fundamental theorem and the empirical
estimates Σ bx , Σb xy , and Σ
b y are as small as possible
estimating the parameters by matching empirical and
model-implied covariances can be done by using the
empirical covariances of the observed variables only,
therefore software packages often do not only accept raw
data, but can also simply be given the empirical covariances
for these reasons, one speaks of ’covariance-based’
estimation of SEM models

Structural Equation Modeling (WS 18/19) An Econometrician’s Introduction Slide 16