Regression analysis

Regression analysis is concerned with the study of the dependence of one variable (the
dependent variable) on one or more other variables (the independent variables). In
regression analysis we use independent variables to estimating or predicting the average
changes of the dependent variables.

Examples:
 The marketing director of a company may want to know how the demand for the
company’s product is related, to say advertising expenditure.

 Suppose the sales manager of a company say X, wants to determine how the
number of credit cards sell is related to the number of call.

Some terms related to regression analysis:

A Scatter plot: is a chart that portrays the relationship between two variables.

Dependent variable: The dependent variable is the variable being predicted or

estimated. It is denoted by y.

Independent variable: The Independent Variable provides the basis for estimation. It is
the predictor variable. It is denoted by x.

Example: If we want to know the expected weekly production of a company then

production will be the dependent variable and the predictor/independent variables could
be the capital, number of labours engaged, supply of raw materials etc.

Linear Regression Model: The equation that describes how y is related to x and an error
term is called the regression model.

The linear regression equation is:

The estimated linear regression equation is give as:

Where,
is the average predicted value of for any .

and are called parameters of the model.

is called intercept or it is the estimated value when X=0
is called slope or the average change in estimated value of for each change
of one unit in x.

1 TM
 is a random variable called the error term.

Slope: A slope of 2- means that every 1-unit change in X yields a 2-unit change in Y.

Error term: The deviation of a particular value of from is called error term or the
stochastic disturbance term. It is written as .

Assumption of Linear Regression Model:

1. The ’s are linear functions of plus a random error term.

2. The dependent values ( ) are independent of each other, i.e. if we obtain

a large value of on the first observation, the result of the second and
subsequent observations will not necessarily provide a large value.
3. The error terms are random variables with mean, and variance, .i.e.
and for i=1,2,….n.
4. The random error terms, are not correlated with one another,

So that for all .

Least Squares Method: To estimate the regression parameters we used least square
method. This regression technique that calculates the  so as to minimize the sum of the
squared errors. That is

Where:
= observed value of the dependent variable for the ith observation
= estimated value of the dependent variable for the ith observation

Slope for the estimated regression equation:

Intercept for the estimated regression equation:

= value of independent variable for ith observation

= value of dependent variable for ith observation
= mean value for independent variable
= mean value for dependent variable

2 TM
 n = total number of observations

Problem: Suppose a company’s owner wants to forecast sales on the basis of

advertising expenses. The owner would like to review the relationship between sales
and the amount spent on advertising. Below is the information on sales and
advertising expense for the last four months:
Advertising expense(x) Sales revenue(y)
Month ($ million) ($ million)
July 2 7
August 1 3
September 3 8
October 4 10

a) Draw scatter plot.

b) Determine the estimated regression equation.
c) Interpret the value and .
d) Estimate sales when $3 million is spent on advertising.

Solution:
a) If we plot the data and draw scatter plot we get as below plot

12

10

8
Sales.rev

0
0 1 2 3 4 5
Adv.exp

b) We know,

3 TM
 Advertisin Sales
g revenue(y
expense(x) )
2 7 -0.5 0 0.25 0
1 3 -1.5 -4 2.25 6
3 8 0.5 1 0.25 0.5
4 10 1.5 3 2.25 4.5

= =

5 11

Now,

=2.2

=7-2.2*2.5
=1.5

Now if we put the values of estimated regression equation is

c) Interpretation:
Here slope is, =2.2.This means that an increase of $1million in advertising cost, the
sales revenue will increase $2.2 million.
And =1.5 means that, that is if there is no advertisement cost, then sales revenue
would be $1.5 million.

4 TM
d) Now if x= 3, then

So, when advertisement cost is $3 million, the sales revenue would be 8.1 million.

Standard Error: The Standard Error of Estimate measures the scatter, or dispersion, of
the observed values around the line of regression .The formula that is used to compute the
standard error:

Example: From the previous example we calculate the value of .As we know

Standard Error,

7 5.9 1.1 1.21

3 3.7 -0.7 0.49
8 8.1 -0.1 0.01
10 10.3 -0.3 0.09

=1.8

Here, n=4.

Now, by putting values we get,

5 TM
So, the standard error is 0.95.

Coefficient of Determination ( ): The coefficient of determination tells the percent of

the variation in the dependent variable that is explained (determined) by the model and
the explanatory variable.

Interpretation of : Suppose =92.7%.

Interpretation: Almost 93% of the variability of the dependent variables explained by
the independent variables.

Relationship between regression and correlation:

Regression versus correlation:

• In Correlation the primary objective is to measure the strength or degree of
linear association between two variables.

In regression analysis, we are not primarily interested in such a measure. Instead,

we try to estimate or predict the average value of one variable on the basis of the
fixed values of other variables

• In correlation analysis, there is no distinction between the dependent and

explanatory variables.

In regression analysis there is an asymmetry in the dependent and explanatory

variables are treated. The dependent variable is assumed to be statistical, random,
or stochastic (that is, to have a probability distribution). The explanatory
variables, on the other hand, are assumed to have fixed values (in repeated
sampling),

6 TM