Introductory Econometrics: A Modern Approach (7e)

Chapter 4
Multiple Regression Analysis:
Inference

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 1
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (1 of 37)

• Statistical inference in the regression model
• Hypothesis tests about population parameters
• Construction of confidence intervals

• Sampling distributions of the OLS estimators

• The OLS estimators are random variables
• We already know their expected values and their variances
• However, for hypothesis tests we need to know their distribution
• In order to derive their distribution we need additional assumptions
• Assumption about distribution of errors: normal distribution

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 2
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (2 of 37)

• Assumption MLR.6 (Normality of error terms)

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 3
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (3 of 37)

• Discussion of the normality assumption
• The error term is the sum of “many” different unobserved factors.
• Sums of independent factors are normally distributed (CLT).
• Problems:
• How many different factors? Number large enough?
• Possibly very heterogenuous distributions of individual factors
• How independent are the different factors?

• The normality of the error term is an empirical question.

• At least, the error distribution should be “close” to normal.
• In many cases, normality is questionable or impossible by definition.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 4
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (4 of 37)

• Discussion of the normality assumption (cont.)
• Examples where normality cannot hold:
• Wages (nonnegative; also: minimum wage)
• Number of arrests (takes on a small number of integer values)
• Unemployment (indicator variable, takes on only 1 or 0)

• In some cases, normality can be achieved through transformations

of the dependent variable (e.g. use log(wage) instead of wage).
• Under normality, OLS is the best (even nonlinear) unbiased estimator
• Important: For the purposes of statistical inference, the assumption of
normality can be replaced by a large sample size.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 5
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (5 of 37)

• Terminology

• Theorem 4.1 (Normal sampling distributions)

• Under assumptions MLR.1 – MLR.6:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 6
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (6 of 37)

• Testing hypotheses about a single population parameter
• Theorem 4.2 (t-distribution for the standardized estimators)
• Under assumptions MLR.1 – MLR.6

• Null hypothesis (for more general hypotheses, see below)

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 7
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (7 of 37)

• t-statistic (or t-ratio)

• Distribution of the t-statistic if the null hypothesis is true

• Goal: Define a rejection rule so that, if it is true, H0 is rejected only

with a small probability (= significance level, e.g. 5%)
© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 8
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (8 of 37)

• Testing against one-sided alternatives (greater than zero)
• Reject the null hypothesis in favour of the
alternative hypothesis if the estimated
coefficient is “too large” (i.e. larger than a
critical value).

• Construct the critical value so that, if the null

hypothesis is true, it is rejected in, for example,
5% of the cases.

• In the given example, this is the point of the t-

distribution with 28 degrees of freedom that is
exceeded in 5% of the cases.

• Reject if t-statistic is greater than 1.701

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 9
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (9 of 37)

• Example: Wage equation
• Test whether, after controlling for education and tenure, higher work
experience leads to higher hourly wages.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 10
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (10 of 37)

• Example: Wage equation (cont.)

• The effect of experience on hourly wage is statistically greater than

zero at the 5% (and even at the 1%) significance level.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 11
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (11 of 37)

• Testing against one-sided alternatives (less than zero)
• Reject the null hypothesis in favour of the
alternative hypothesis if the estimated
coefficient is “too small” (i.e. smaller than a
critical value).

• Construct the critical value so that, if the null

hypothesis is true, it is rejected in, for example,
5% of the cases.

• In the given example, this is the point of the t-

distribution with 18 degrees of freedom so that
5% of the cases are below the point.

• Reject if t-statistic is less than -1.734

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 12
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (12 of 37)

• Example: Student performance and school size
• Test whether smaller school size leads to better student performance

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 13
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (13 of 37)

• Example: Student performance and school size (cont.)

• One cannot reject the hypothesis that there is no effect of school size
on student performance (not even for a lax significance level of 15%).

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 14
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (14 of 37)

• Example: Student performance and school size (cont.)
• Alternative specification of functional form:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 15
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (15 of 37)

• Example: Student performance and school size (cont.)

• The hypothesis that there is no effect of school size on student performance

can be rejected in favor of the hypothesis that the effect is negative.
• How large is the effect?

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 16
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (16 of 37)

• Testing against two-sided alternatives
• Reject the null hypothesis in favour of the
alternative hypothesis if the absolute value of
the estimated coefficient is too large.

• Construct the critical value so that, if the null

hypothesis is true, it is rejected in, for example,
5% of the cases.

• In the given example, these are the points of

the t-distribution so that 5% of the cases lie in
the two tails.

• Reject if absolute value of t-statistic is less than

-2.06 or greater than 2.06

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 17
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (17 of 37)

• Example: Determinants of college GPA

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 18
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (18 of 37)

• “Statistically significant” variables in a regression
• If a regression coefficient is different from zero in a two-sided test, the
corresponding variable is said to be “statistically significant”.
• If the number of degrees of freedom is large enough so that the normal
approximation applies, the following rules of thumb apply:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 19
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (19 of 37)

• Testing more general hypotheses about a regression coefficient
• Null hypothesis

• t-statistic

• The test works exactly as before, except that the hypothesized value is
substracted from the estimate when forming the statistic.
© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 20
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (20 of 37)

• Example: Campus crime and enrollment
• An interesting hypothesis is whether crime increases by one percent if
enrollment is increased by one percent.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 21
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (21 of 37)

• Computing p-values for t-tests
• If the significance level is made smaller and smaller, there will be a point where
the null hypothesis cannot be rejected anymore.
• The reason is that, by lowering the significance level, one wants to avoid more
and more to make the error of rejecting a correct H0.
• The smallest significance level at which the null hypothesis is still rejected, is
called the p-value of the hypothesis test.
• A small p-value is evidence against the null hypothesis because one would
reject the null hypothesis even at small significance levels.
• A large p-value is evidence in favor of the null hypothesis.
• P-values are more informative than tests at fixed significance levels.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 22
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (22 of 37)

• How the p-value is computed (here: two-sided test)?
• The p-value is the significance level at which one is
indifferent between rejecting and not rejecting the null
hypothesis.

• In the two-sided case, the p-value is thus the

probability that the t-distributed variable takes on a
larger absolute value than the realized value of the test
statistic, e.g.
P(|t|>1.85) = 2*(0.0359) = .0718

• From this, it is clear that a null hypothesis is rejected if

and only if the corresponding p-value is smaller than
the significance level.

• For example, for a significance level of 5% the t-statistic

would not lie in the rejection region.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 23
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (23 of 37)

• Guidelines for discussing economic and statistical significance
• If a variable is statistically significant, discuss the magnitude of the coefficient to
get an idea of its economic or practical importance.
• The fact that a coefficient is statistically significant does not necessarily mean it
is economically or practically significant!
• If a variable is statistically and economically important but has the “wrong”
sign, the regression model might be misspecified.
• If a variable is statistically insignificant at the usual levels (10%, 5%, or 1%), one
may think of dropping it from the regression.
• If the sample size is small, effects might be imprecisely estimated so that the
case for dropping insignificant variables is less strong.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 24
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (24 of 37)

• Confidence intervals
• Simple manipulation of the result in Theorem 4.2 implies that

• Interpretation of the confidence interval

• The bounds of the interval are random.
• In repeated samples, the interval that is constructed in the above way will cover
the population regression coefficient in 95% of the cases.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 25
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (25 of 37)

• Confidence intervals for typical confidence levels

• Relationship between confidence intervals and hypotheses tests

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 26
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (26 of 37)

• Example: Model of firms‘ R&D expenditures

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 27
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (27 of 37)

• Testing hypotheses about a linear combination of the parameters
• Example: Return to education at two-year vs. at four-year colleges

• A possible test statistic would be:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 28
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (28 of 37)

• The standard error of the difference in parameters is impossible to
with standard regression output

• An alternative method is to make a substitution in variables.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 29
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (29 of 37)

• Estimation results

• This method works always for single linear hypotheses

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 30
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (30 of 37)

• Testing multiple linear restrictions: The F-test
• Testing exclusion restrictions

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 31
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (31 of 37)

• Estimation of the unrestricted model

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 32
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (32 of 37)

• Estimation of the restricted model

• Test statistic:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 33
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (33 of 37)

• Rejection rule
• A F-distributed variable only takes
on positive values. This corresponds
to the fact that the sum of squared
residuals can only increase if one
moves from H1 to H0.

• Choose the critical value so that we

incorrectly reject the null
hypothesis in, for example, only 5%
of the cases.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 34
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (34 of 37)

• Test decision in example

• Discussion
• The three variables are “jointly significant”
• They were not significant when tested individually
• The likely reason is multicollinearity between them

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 35
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (35 of 37)

• Test of overall significance of a regression

• Discussion
• The three variables are “jointly significant”
• They were not significant when tested individually
• The likely reason is multicollinearity between them

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 36
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (36 of 37)

• Testing general linear restrictions with the F-test
• Example: Test whether house price assessments are rational

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 37
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (37 of 37)

• Unrestricted regression

• Restricted regression

• Test statistic

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 38