Department Of Computer Engineering

Class: TE Question Answer Sub: QA

4. Introduction to Multiple Linear Regression

1. What do you mean by Partial correlation coefficients? Explain in detail.(D-24)

Ans:

2. Write short note on Multiple Regression.

Ans:
Multiple linear regression is used to estimate the relationship between two or more
independent variables and one dependent variable. You can use multiple linear regression when
you want to know:

• The strong relationship between two or more independent variables and one dependent
variable (e.g. how rainfall, temperature, and amount of fertilizer added affect crop
 Department Of Computer Engineering
Class: TE Question Answer Sub: QA

growth).
• The value of the dependent variable at a certain value of the independent variables (e.g.
the expected yield of a crop at certain levels of rainfall, temperature, and fertilizer
addition).
Key Concepts:
• Dependent Variable:

The variable being predicted or explained (e.g., house price, salary).

• Independent Variables:
The variables used to predict the dependent variable (e.g., house size, location, number
of bedrooms).
• Linear Relationship:
Assumes a straight-line relationship between the variables, meaning a change in an
independent variable leads to a predictable change in the dependent variable.
• Prediction:
Uses the regression equation to estimate the value of the dependent variable based on
the values of the independent variables.
• Explanation:
Provides insights into how much each independent variable contributes to the overall
variance in the dependent variable.

The formula for a multiple linear regression is:

• = the predicted value of the dependent variable

• = the y-intercept (value of y when all other parameters are set to 0)
• = the regression coefficient ( ) of the first independent variable ( ) (a.k.a. the
effect that increasing the value of the independent variable has on the predicted y value)
• … = do the same for however many independent variables you are testing
• = the regression coefficient of the last independent variable

• = model error (a.k.a. how much variation there is in our estimate of )

To find the best-fit line for each independent variable, multiple linear regression calculates three
things:

• The regression coefficients that lead to the smallest overall model error.
• The t test of the overall model.
• The associated p value (how likely it is that the t statistic would have occurred by chance if
the null hypothesis of no relationship between the independent and dependent variables was
true).
 Department Of Computer Engineering
Class: TE Question Answer Sub: QA

3. Write short note on Significance of Overall fit of regression model (D-24)

Ans:
There is a relationship between the dependent variable and the set of independent variables.

• Invalid Model. There is no relationship between the dependent variable and the set of
independent variables. In this case, all of the regression coefficients βi in the population
model are zero. This is the claim for the null hypothesis in the overall model
test: H0:β1=β2=⋅s=βk=0.
• Valid Model. There is a relationship between the dependent variable and the set of
independent variables. In this case, at least one of the regression coefficients βi in the
population model is not zero. This is the claim for the alternative hypothesis in the overall
model test: Ha:at least one βi≠0.

The logic behind the overall model test is based on two independent estimates of the variance
of the errors:

• One estimate of the variance of the errors, MSR, is based on the mean amount of explained
variation in the dependent variable y.
• One estimate of the variance of the errors, MSE, is based on the mean amount of unexplained
variation in the dependent variable y.

The overall model test compares these two estimates of the variance of the errors to determine if
there is a relationship between the dependent variable and the set of independent
variables. Because the overall model test involves the comparison of two estimates of variance,
an F-distribution is used to conduct the overall model test, where the test statistic is the ratio of
the two estimates of the variance of the errors.

The mean square due to regression, MSR, is one of the estimates of the variance of the
errors. The MSR is the estimate of the variance of the errors determined by the variance of the
predicted y^-values from the regression model and the mean of the y-values in the sample, y―. If
there is no relationship between the dependent variable and the set of independent variables, then
the MSR provides an unbiased estimate of the variance of the errors. If there is a relationship
between the dependent variable and the set of independent variables, then the MSR provides an
overestimate of the variance of the errors.

The overall model test depends on the fact that the MSR is influenced by the explained variation in
 Department Of Computer Engineering
Class: TE Question Answer Sub: QA

the dependent variable, which results in the MSR being either an unbiased or overestimate of the
variance of the errors. Because the MSE is based on the unexplained variation in the dependent
variable, the MSE is not affected by the relationship between the dependent variable and the set of
independent variables, and is always an unbiased estimate of the variance of the errors.

The null hypothesis in the overall model test is that there is no relationship between the dependent
variable and the set of independent variables. The alternative hypothesis is that there is a relationship
between the dependent variable and the set of independent variables. The F-score for the overall
model test is the ratio of the two estimates of the variance of the
errors, F=MSRMSE with df1=k and df2=n−k−1. The p-value for the test is the area in the right tail
of the F-distribution to the right of the F-score.

4. What are assumptions of Multiple Linear Regression (Dec.23)

Ans:
Multiple linear regression analysis is predicated on several fundamental assumptions that
ensure the validity and reliability of its results. Understanding and verifying these assumptions is
crucial for accurate model interpretation and prediction.
MLR requires at least two independent variables.
Multiple Linear Regression Assumptions

First, multiple linear regression requires the relationship between the independent and dependent
variables to be linear. The linearity assumption can best be tested with scatterplots. The following
two examples depict a curvilinear relationship (left) and a linear relationship (right).

Second, the multiple linear regression analysis requires that the errors between observed and
 Department Of Computer Engineering
Class: TE Question Answer Sub: QA

predicted values (i.e., the residuals of the regression) should be normally distributed. This
assumption may be checked by looking at a histogram or a Q-Q-Plot. Normality can also be
checked with a goodness of fit test (e.g., the Kolmogorov-Smirnov test), though this test must be
conducted on the residuals themselves.

Third, multiple linear regression assumes that there is no multicollinearity in the data.
Multicollinearity occurs when the independent variables are too highly correlated with each other.

The last assumption of multiple linear regression is homoscedasticity. A scatterplot of residuals

versus predicted values is good way to check for homoscedasticity. There should be no clear
pattern in the distribution; if there is a cone-shaped pattern the data is heteroscedastic.

Note: All the problems were solved in lectures.