Introduction to Economic Analysis and Quantitative Methods

ECON1151-Lecture 4 Part III

Dr. Rafael Wildauer

1 / 49
Outline

References

Chapter 2: Normal, χ-Squared, t and F Distributions

Chapter 2: Random Sampling and the Distribution of the Sample Average

Chapter 2: Weak Law of Large Numbers and the Central Limit Theorem

Further Thoughts on Sampling

2 / 49
Textbooks

Larsen, R., & Marx, M. (2012). Introduction to mathematical statistics and its
applications. Pearson.
Stock, J., & Watson, M. (2015). Introduction to econometrics. Pearson.

3 / 49
Textbook Clarification

I Stock and Watson (2015) is the primary and required text.

I All chapter numbers refer to this text.

I Larsen and Marx (2012) is a more advanced auxiliary text if you want to go
beyond the material here.

4 / 49
Chapter 2: Normal, χ-Squared, t and F Distributions

5 / 49
The Normal Distribution

6 / 49
The Normal Distribution

I If a (continous) random variable X with mean µX and variance σX2 is normally

distributed it can be described by the following probability function
(x−µX )2
1 −
2σ 2
pdf (x) = q e X

2πσX2

I The normal distribution is symmetric around its mean

I and 95% of its probability fall into the interval [µX − 1.96σX ; µX + 1.96σX ]

I It is expressed as X ∼ N[µX , σX2 ]

7 / 49
8 / 49
The Standard Normal Distribution

I A special case is the Standard Normal Distribution with µ = 0 and variance

σ 2 = 1 (red curve in the previous picture)

I We can transform any random variable Y into a "normalized" random variable

Z by:
Z = (Y − µY )/σY
And this standardized variable Z has a remarkable property: We can use it to
analyse standard distributions in general

I The CDF of the normal distribution is complicated. However we can rely on

tables / computers to calculate the CDF of the standard normal distribution:

CDF (c) = Pr (Z ≤ c) = Φ(c)

9 / 49
10 / 49
The Standard Normal Distribution
I What does that mean?

I First, we introduced a symbol for the cumulative distribution function (cdf)

of the standard normal distribution:
Φ(c) = Pr (Z ≤ c)

I Second, it means that if we are interested in Pr (Y ≤ c) of any normally

distributed random variable Y, we can compute that probability by computing
Pr (Z ≤ d) instead where:
Y −µY
I Z is the standardized random variable: Z = σY

c−µY
I and d is the standardized value of c: d = σY

I We know that Pr (Z ≤ d) = Pr (Y ≤ c) and thus can answer the initial question

about Y!
11 / 49
12 / 49
The χ-Squared Distribution

I The chi-Squared (pronounced ki:) Distribution is used to test certain

hypotheses

I It is defined as the distribution "of the sum of m squared independent

standard normal random variables"

I For example, let (Z1 , Z2 , and Z3 ) be independent standard normal random

variables. Then (Z12 + Z22 + Z3 ) has a chi-squared distribution with 3 degrees
of freedom.

I It is denotes as χ2k for k degrees of freedom.

13 / 49
Probability Density Functions of χ2 Distributions

14 / 49
The Student t Distribution

I The Student t distribution with m degrees of freedom is defined to be the

distribution of the ratio of a standard normal random variable, divided by the
square root of an independently distributed chi-squared random variable with
m degrees of freedom divided by m.

I It is denoted tm for m degrees of freedom.

I The key property to remember is that as m increases it becomes more and

more similar to the standard normal distribution

15 / 49
Probability Density Functions of t Distributions

16 / 49
The F Distribution

I The F distribution with m and n degrees of freedom, denoted Fm,n, is defined

to be the distribution of the ratio of a chi-squared random variable with
degrees of freedom m, divided by m, to an independently distributed
chi-squared random variable with degrees of freedom n, divided by n.

I It is denoted as Fm,n for m and n degrees of freedom.

17 / 49
Probability Density Functions of F Distributions

18 / 49
Chapter 2: Random Sampling and the Distribution of the
Sample Average

19 / 49
Sample vs Population

The key idea in statistics is that we use information from a sample in order to
obtain insights about the population!

This is why it is important to distinguish between the two!!!

20 / 49
I Almost all the techniques discussed later on involve averages or weighted
averages of data from a sample.

I This is why understanding the distribution of sample averages is important

I This section first talks about random sampling and then the distribution of the
sample mean

21 / 49
Random Sampling
I The simplest form of random sampling is to randomly select n units from the
population and each unit is selected with the same probability

I This is called simple random sampling

I Example:
I If we randomly select days on which we record our commuting time, ...
I then we obtain a series of (y1 , . . . , yn ) records of commuting time.
I Each yi is a realization of random variable Yi which is the commuting time on
day i.
I and since we chose the days at random (in advance), the commuting time on
one day provides no information about commuting time on another day.
I Which means that the sequence of random variables (Y1 , . . . , Yn ) are
independent.
22 / 49
Independently and identically distributed (i.i.d)

I The previous example explains why simple random sampling yields a sample
which observations are independently and identically distributed

23 / 49
The Sampling Distribution of the Sample Average

I If we have a sequence of (y1 , . . . , yn ) observations we can compute the

sample mean or sample average as:
n
1X
y= yi
n
i=1

I Since each yi is a realization of a random variable it means in a different

situation we could have obtained a different sequence of yi0 s

I Which means that y is itself a random variable

24 / 49
Mean and Variance of y

I Since y is a random variable we can calculate the expected value (mean) and
variance:
n
1X 1
E[y ] = E[yi ] = nµY = µY
n n
i=1

I Here we use the fact that each yi was drawn from the same distribution and
thus has the same mean µY

25 / 49
For the variance we have:
n
" #
1X
Var [y ] = Var yi =
n
i=1

n n n n
1 X 1 X X 1 X
= Var [y i ] + Cov [y ,
i jy ] = Var [yi ]
n2 n2 n2
i=1 i=1 j=1,j6=i i=1

1 2 σY2
= nσ =
n2 Y n
σY
For the standard deviation we take the square root: σy = √
n

26 / 49
Mean and Variance of y
Let’s state the results again:
I The expected value of the sample mean (y ) is given by:

E[y ] = µY

I the variance and standard deviation of the sample mean (y ) are given by:
σY2
Var [y ] = σy2 =
n
σY
Std.Dev [y ] = σy = √
n

I These results hold whatever the common distribution of the random variables
Y1 , ..., Yn is! If it where a normal distribution we could even say that
" #
σY2
y ∼ N µY ,
n
27 / 49
Mean and Variance of y

I Do these results make sense?

I Let’s think about them!

28 / 49
Chapter 2: Weak Law of Large Numbers and the Central
Limit Theorem

29 / 49
The sampling distribution of y
I We said that only if we know that Y1 , ..., Yn are each normally distributed that
we can say that we know the distribution of y . Namely:
" #
σY2
y ∼ N µY ,
n

I If we don’t know the exact distribution for each Y1 , ..., Yn we can only derive
two characteristics of the distribution of y , namely its mean and its variance:
E[y ] = µY
σY2
Var [y ] = σy2 =
n

I But if we want to make statements about a likely range within y falls we know
more than E[y ] and Var [y ]. We need to know the (sampling) distribution of y
30 / 49
The asymptotic distribution of y

I There are two remarkable result in probability theory which allow us to learn
more about how y if we assume that the sample becomes very large: n → ∞

31 / 49
The Law of Large Numbers

I The first of these results is the (weak) law of large numbers which says that
if the sample size is large, y will be very close to µY with high probability.

I In a more general sense it is the justification that if we have a decently large

sample, we can actually learn about the population (means).

32 / 49
Central Limit Theorm

I The second of these results is the central limit theorm (CLT). It says that ,
when the samplesize is large, the sampling distribution of the standardized
sample average y −µ
σ
Y
is approximately normal.
y

I Remarkably the CLT holds independent of how the random variables Y are
distributed.

I This result will allow us to perform hypothesis tests and make statements
about how certain we are about results obtained from our sample!

33 / 49
The CLT in Action
I Let’s carry out a small Monte Carlo experiment to see whether this is all true!

I The setting:
I Draw a random sample of size n from a uniform distribution over the interval
[1, 100] (for that type of distribution we have µ = 50.5 and σ 2 = 816.75) and
compute the standardized sample average.
I generate 10,000 draws (samples) and see how the standardized sample
averages are distributed

I What does the CLT predict?

I With increasing sample size the distribution of the standardized sammple

averages should converge to a standard normal distribution

I So a practical question is how large should each of these 10,000 samples be?
What should we choose for n? Let’s start with n=5.
34 / 49
10,000 samples, each of size n = 5

35 / 49
10,000 samples, each of size n = 20

36 / 49
10,000 samples, each of size n = 50

37 / 49
What happens if we sample from a very uneven distribution

I Let’s assume the population we are interested in is described by a

ParetoDistribution[100,000; 2.5]
α
αxmin
I The pdf of such a distribution is defined as pdf (x) = x α+1

I Key features of this Pareto Distributions

I Minimum value is 100,000
I Population units with values substantially larger than 100,000 are rare
I However extreme values are possible

38 / 49
Richest 100 observations from a Pareto vs Normal Distribution

39 / 49
Richest 100 observations from a Pareto vs Normal Distribution

40 / 49
10,000 samples, each of size n = 5

41 / 49
10,000 samples, each of size n = 20

42 / 49
10,000 samples, each of size n = 100

43 / 49
10,000 samples, each of size n = 2000

44 / 49
Central Limit Theorem

I even when we sample from a highly unevenly distributed population the

distribution of the sample average approaches a normal distribution as long
as we have a large enough sample

I However it also means that we need to be careful with highly skewed

(uneven) distributions!!

45 / 49
Further Thoughts on Sampling

46 / 49
Sampling Problems
I Selection bias results when a subset of the units in the population is
less/more likely to be included in the sample (and the research design does
not or cannot take this into account)
I pure internet surveys
I survivorship bias (in firm data)
I differential non-response bias results for example when the probability of
participating in a survey is systematically linked to some characteristics of the
units in the population. (wealthy households less likely to participate in surveys
on household finances)

I Non-observation bias: studyin characteristics which are very unevenly

distributed (wealth; heavy tailed distributions) requires unfeasibly large
sample sizes in order to obtain enough of these individuals in the sample;
otherwise most samples would underestimate the population’s characteristic
(e.g. total wealth)
47 / 49
48 / 49
Most important takeaway
sample is not the population!
I will have different notation for sample and population characteristics!

I population characteristics
I e.g.: the true proportion intending to vote for a certain party in the next election
(can be more complicated like an average treatment effect)
I it is unknown and we try to estimate it from the population

I sample characteristics
I values we obtain from the sample (like the proportion of people intending to vote
for X)
I Use it to make statements about the population (statistical significance of an
effect, margin of error around voting intention we observe in the sample)
If your sample is biased (see above) your conclusions about the population will be
biased!
49 / 49