REGRESSION ANALYSIS

[ And it’s application in Business ]

M.Ravishankar
 Introduction. . .

• Father of Regression Analysis

Carl F. Gauss (1777-1855).

• contributions to physics, Mathematics &

astronomy.

• The term “Regression” was first used in

1877 by Francis Galton.
Regression Analysis. . .

• It is the study of the

relationship between
variables.

• It is one of the most

commonly used tools for
business analysis.

• It is easy to use and

applies to many
situations.
 Regression types. . .

• Simple Regression: single explanatory

variable

• Multiple Regression: includes any number of

explanatory variables.
• Dependant variable: the single variable being explained/
predicted by the regression model

• Independent variable: The explanatory variable(s) used to predict

the dependant variable.

• Coefficients (β): values, computed by the regression tool,

reflecting explanatory to dependent variable relationships.

• Residuals (ε): the portion of the dependent variable that isn’t

explained by the model; the model under and over predictions.
 Regression Analysis. . .

• Linear Regression: straight-line relationship

– Form: y=mx+b

• Non-linear: implies curved relationships

– logarithmic relationships
 Regression Analysis. . .

• Cross Sectional: data gathered from the

same time period

• Time Series: Involves data observed over

equally spaced points in time.
Simple Linear Regression Model. . .

• Only one
independent
variable, x
• Relationship
between x and y
is described by a
linear function
• Changes in y are
assumed to be
caused by changes
in x
Types of Regression Models. . .
Estimated Regression Model. . .

The sample regression line provides an estimate of

the population regression line

Estimated Estimate of Estimate of the

(or predicted) the regression regression slope
y value
intercept
Independent

ŷ i = b0 + b1x variable

The individual random error terms ei have a mean of zero

Simple Linear Regression Example. . .

• A real estate agent wishes to examine

the relationship between the selling price
of a home and its size (measured in
square feet)
• A random sample of 10 houses is
selected
– Dependent variable (y) = house price in
$1000s
– Independent variable (x) = square feet
 House Price in
Square Feet
$1000s
(x)
(y)
245 1400
312 1600
279 1700
Sample Data 308 1875
199 1100
219 1550
405 2350
324 2450
319 1425
255 1700
 Regression Statistics
Multiple R
R Square
0.76211
0.58082
Output. . .
Adjusted R
Square 0.52842
Standard Error 41.33032
The regression equation is:
Observations 10
house price = 98.24833 + 0.10977 (square feet)
ANOVA Significance
df SS MS F F

Regression 1 18934.9348 18934.9348 11.0848 0.01039

Residual 8 13665.5652 1708.1957
Total 9 32600.5000

Coefficient Upper
s Standard Error t Stat P-value Lower 95% 95%

Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386

Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

 Graphical Presentation . . .
• House price model: scatter plot and
regression line
450
400
House Price ($1000s)

350
Slope
300
250
= 0.10977
200
150
100
50
Intercept 0
= 98.248 0 500 1000 1500 2000 2500 3000
Square Feet

house price = 98.24833 + 0.10977 (square feet)

 Interpretation of the Intercept, b0

house price = 98.24833 + 0.10977 (square feet)

• b0 is the estimated average value of Y when the

value of X is zero (if x = 0 is in the range of
observed x values)
– Here, no houses had 0 square feet, so b0 = 98.24833
just indicates that, for houses within the range of sizes
observed, $98,248.33 is the portion of the house price
not explained by square feet
Interpretation of the Slope Coefficient, b1

house price = 98.24833 + 0.10977 (square feet)

• b1 measures the estimated change in the

average value of Y as a result of a one-
unit change in X
– Here, b1 = .10977 tells us that the average value of a
house increases by .10977($1000) = $109.77, on
average, for each additional one square foot of size
 Example: House Prices

House Price Estimated Regression Equation:

Square Feet
in $1000s
(x)
(y) house price = 98.25 + 0.1098 (sq.ft.)
245 1400
312 1600
279 1700 Predict the price for a house
308 1875
with 2000 square feet
199 1100
219 1550
405 2350
324 2450
319 1425
255 1700
 Example: House Prices

Predict the price for a house

with 2000 square feet:

house price = 98.25 + 0.1098 (sq.ft.)

= 98.25 + 0.1098(200 0)

= 317.85
The predicted price for a house with 2000
square feet is 317.85($1,000s) = $317,850
 Coefficient of Determination, R2
Coefficient of determination
SSR sum of squares explained by regression
R =
2
=
SST total sum of squares

Note: In the single independent variable case, the coefficient

of determination is

R =r2 2

where:
R2 = Coefficient of determination
r = Simple correlation coefficient
Examples of Approximate R2 Values

y
R2 = 1

Perfect linear relationship

between x and y:
x
R2 = 1
y 100% of the variation in y is
explained by variation in x

x
R = +1
2
Examples of Approximate R2 Values

y
0 < R2 < 1

Weaker linear relationship

between x and y:
x
Some but not all of the
y
variation in y is explained
by variation in x

x
 Examples of Approximate R2 Values

R2 = 0
y
No linear relationship
between x and y:

The value of Y does not

x depend on x. (None of the
R2 = 0
variation in y is explained
by variation in x)
 Output. . .
SSR 18934.9348
Regression Statistics R =2
= = 0.58082
Multiple R 0.76211 SST 32600.5000
R Square 0.58082
Adjusted R 58.08% of the variation in
Square 0.52842 house prices is explained by
Standard Error 41.33032 variation in square feet
Observations 10

ANOVA Significance
df SS MS F F
18934.934 11.084
Regression 1 18934.9348 8 8 0.01039
Residual 8 13665.5652 1708.1957
Total 9 32600.5000

Coefficien P- Upper
ts Standard Error t Stat value Lower 95% 95%
0.1289 232.0738
 Standard Error of Estimate. . .

• The standard deviation of the variation of

observations around the regression line is
estimated by
SSE
sε =
n − k −1
Where
SSE = Sum of squares error
n = Sample size
k = number of independent variables in the
model
 The Standard Deviation of the
Regression Slope
• The standard error of the regression slope
coefficient (b1) is estimated by
sε sε
sb1 = =
∑ (x − x) 2
( ∑ x)
∑x − n 2
2

where:
sb1 = Estimate of the standard error of the least squares slope
SSE
sε = = Sample standard error of the estimate
n−2
 Output. . .
Regression Statistics sε = 41.33032
Multiple R 0.76211
R Square 0.58082
Adjusted R
Square 0.52842 sb1 = 0.03297
Standard Error 41.33032
Observations 10

ANOVA Significance
df SS MS F F
18934.934 11.084
Regression 1 18934.9348 8 8 0.01039
Residual 8 13665.5652 1708.1957
Total 9 32600.5000

Coefficien P- Upper
ts Standard Error t Stat value Lower 95% 95%
0.1289 232.0738
 Reference. . .

• Business statistics by S.P.Gupta & M.P.Gupta

Sources retrieved from Internet…

• www.humboldt.edu
• www.cs.usask.ca
• www.cab.latech.edu
• www.quickmba.com
• www.wikipedia.com
• www.youtube.com
 M.RAVISHANKAR
MBA(AB) 2008-2010 Batch
NIFTTEA KNITWEAR FASHION
INSTITUTE
TIRUPUR

OXYGEN024@GMAIL.COM