Scribd
Upload
244 views13 pages

L-6 Multiple Regression Analysis

Multiple linear regression allows modeling of a dependent variable as a function of several independent variables. It generalizes simple linear regression, which involves only one independent variable. Stepwise regression selects a subset of independent variables that best explain the dependent variable through forward selection, backward elimination, or bi-directional elimination of variables based on statistical criteria. The goal is to identify the most important predictors and obtain a parsimonious model for the dependent variable.

Uploaded by

BHAVESH JOSHI
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
244 views13 pages

L-6 Multiple Regression Analysis

Multiple linear regression allows modeling of a dependent variable as a function of several independent variables. It generalizes simple linear regression, which involves only one independent variable. Stepwise regression selects a subset of independent variables that best explain the dependent variable through forward selection, backward elimination, or bi-directional elimination of variables based on statistical criteria. The goal is to identify the most important predictors and obtain a parsimonious model for the dependent variable.

Uploaded by

BHAVESH JOSHI
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

Multiple Regression

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 1


Regression analysis
• Regression analysis is a technique which is used to find a causal
relationship between dependent variable and independent variables.
• For example

• Relationship between effective time of study and marks scored

• Relationship between average food intake and weight

• Relationship between investment on marketing, product cost and demand of


product

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 2


• A brief classification of regression models
• Regression models can be classified based on –
➢No. of independent variables-
• If one independent variable, then it is known as simple regression model
• If two or more independent variable, then it is known as multiple regression model

➢Shape of regression curve-


• regression models can also be classified based on the shape of the curve; i.e. polynomial,
exponential, logarithmic etc.

➢Type of dependent (output) variable (y)-


• If dependent variable is continuous (output is a numeric value)- simple linear regression,
multiple linear regression, polynomial regression etc.
• If dependent variable is categorial (only binary output will come as 0 or 1, i.e. pass or fail,
Yes or No, Married or Unmarried, Profit or Loss, Win or Defeat etc.)- logistic regression,
Probit regression etc.
Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 3
Linear Regression Model
• A regression model is called “Linear” if
• The power of the regression coefficients (also called as Parameter) should be
maximum one.
• The first derivative w.r.t. any parameter should not have
• Each term in model is one of the following-
• A constant
• A parameter multiplied by an independent variable
• Equation of linear regression model is

• 𝑫𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆 = 𝒄𝒐𝒏𝒔𝒕𝒂𝒏𝒕 + 𝒑𝒂𝒓𝒂𝒎𝒆𝒕𝒆𝒓𝟏 ∗ 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆𝟏 + ⋯ +


𝒑𝒂𝒓𝒂𝒎𝒆𝒕𝒆𝒓𝒏 ∗ 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆𝒏

• Mathematically; 𝑦 = 0 + 1 𝑥1 + 2 𝑥2 +⋯+ 𝑚 𝑥𝑚

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 4


Non-linear Regression Model
• Simply, it doesn’t follow the rules of linear model
• A regression model is called “non-linear” if the derivative w.r.t. parameter
depends on one or more parameters.
𝑫𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆
𝒌
= 𝒄𝒐𝒏𝒔𝒕𝒂𝒏𝒕 + 𝒑𝒂𝒓𝒂𝒎𝒆𝒕𝒆𝒓𝟏 ∗ 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆𝟏 + ⋯ + 𝒑𝒂𝒓𝒂𝒎𝒆𝒕𝒆𝒓𝒏
∗ 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆𝒏

𝝏𝒀
Linear model: 𝑌 = 𝑎 + 𝑏𝑋 = 𝑋;
𝝏𝒃

𝝏𝒀 𝝏𝒀
Linear model: 𝑌 = 𝑎 + 𝑏𝑋 + 𝑐𝑋 2 = 𝑋; = 𝑋2
𝝏𝒃 𝝏𝒄

𝝏𝒀 𝝏𝒀
Non-Linear model: 𝑌 = 𝑎 + 𝑏 2 𝑋 + 𝑐𝑋 = 2𝒃𝑋; =𝑋
𝝏𝒃 𝝏𝒄

𝝏𝒀
Non-Linear model: 𝑌 = 𝑎 + (ln 𝑏) 𝑋 ; it is non-linear because 𝝏𝒃 will depend on b.

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 5


Linear regression: dependent/ output variable (y) is continuous in nature;
i.e. it takes numerical values.

Simple linear regression equation Multiple linear regression equation


• Linear with Only one independent • Linear with More than one
variable (x) and one dependent independent variables (x) and one
variable (y) dependent variable (y)
• Equation 𝑦= 0 + 1𝑥 +𝜀 •𝑦= 0+ 1 𝑥1 + 2 𝑥2 + ⋯+
𝑚 𝑥𝑚
where:
𝒊 is coefficient of regression (also known as parameter) model
𝜺 is a random variable called the error term

NOTE- For better explanation in Hindi, I recommend you to watch this 15 minute video on Youtube
https://www.youtube.com/watch?v=lzGKRSvs5HM copy this link to your browser

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 6


• Multiple Linear regression eq. 𝑦 = 0 + 1 𝑥1 + 2 𝑥2 + ⋯+ 𝑚 𝑥𝑚
• Example :-
weight = 𝑎 + 1 ∗ food intake + 2 ∗ 𝑒𝑥𝑒𝑟𝑐𝑖𝑠𝑒 𝑡𝑖𝑚𝑒 + 3 ∗ 𝑖𝑛𝑐𝑜𝑚𝑒
• Since, weight of a person is majorly depends upon food taken by person and time
spent in physical exercise. It doesn't or very less likely to depend on the income of
a person.

• So, effect of factors of food intake and exercise time is much higher than effect of
income on body weight of a person.
• i.e. independent variables food intake and exercise time best explains the
dependent variable weight
• So, parameters 1 , 2 must be much higher than 3
Mathematically ( 1 , 2 >>> 3)

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 7


Stepwise Regression
Suppose a situation where an Dependent Variable (y) depends upon
various Independent Variables. Some of them significantly affect the
output (D.V.) whereas some I.V. doesn’t affect as much as others.
• 𝑦 = 0 + 1 𝑥1 + 2 𝑥2 + 3 𝑥3 + 4 𝑥4 + 5 𝑥5 + 6 𝑥6 + 7 𝑥7 +
8 𝑥8 + 9 𝑥9 + 10 𝑥10 + 11 𝑥11 + 12 𝑥12 +
• In stepwise regression, we try to make/fit a equation (regression
model) which contains only those independent variables who affects
the output significantly.
• In stepwise regression, a variable is selected for either addition or
subtraction from set of variables
Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 8
Stepwise Regression cont…
So, the main objective of Stepwise Regression is to choose a subset of
the independent variables which “best” explains the dependent
variable.
➢ There are 3 main approaches to carry out a stepwise regression

• Forward Selection

• Backward Elimination

• Bi-directional Elimination

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 9


Forward selection
• Starting with no variables in the model
• Perform a testing of addition of each variable one by one using pre-
defined criteria
• Add the variable whose inclusion/addition gives the most significant
improvement to the model
• Repeat the process until no significant improvement will obtain to the
model
• Now model contains only those variable who have significant impact
on the output.

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 10


Backward Elimination
• Start with all the variables in the model
• Perform a testing of deletion of each variable one by one using pre-
defined criteria
• Drop the “least significant” variable one at a time
• Repeat the process until deletion of variable doesn’t affect the model
• Now model contains only those variable who have significant impact
on the output.

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 11


Bi-directional elimination

• It is a combination of both forward selection and backward


elimination
• Perform the testing at each step for variables to be included or
dropped.
• Perform a forward selection, but drop the variable which become no
longer significant after introduction of new variables.

Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 12


Lecture-06, Course- 4ME2-01, IV Sem (Mech. Engg.) 13

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy