UNIT-1

Machine Learning
TEXT BOOKS:

• Stephen Marsland, ―Machine Learning – An Algorithmic Perspective‖, Second Edition,

Chapman and Hall/CRC Machine Learning and Pattern Recognition Series, 2014.

• 2. Tom M Mitchell, ―Machine Learning‖, First Edition, McGraw Hill Education, 2013.

REFERENCES:

• Peter Flach, ―Machine Learning: The Art and Science of Algorithms that
Make Sense of Data‖, First Edition, Cambridge University Press, 2012.

• Jason Bell, ―Machine learning – Hands on for Developers and Technical

Professionals‖, First Edition, Wiley, 2014

• 3. Ethem Alpaydin, ―Introduction to Machine Learning 3e (Adaptive

Computation and Machine Learning Series), Third Edition, MIT Press, 2014
 Machine Learning
UNIT I: INTRODUCTION:

• Learning(Book-1) – Types of Machine Learning –

Supervised Learning – The Brain and the Neuron –
Design a Learning System(Book-2) – Perspectives and
Issues in Machine Learning – Concept Learning Task –
Concept Learning as Search – Finding a Maximally
Specific Hypothesis – Version Spaces and the Candidate
Elimination Algorithm – Linear Discriminants: (Book-1)
– Perceptron – Linear Separability – Linear Regression.
 Learning
• Learn from data (TB)
• It gives us flexibility
Defn : A computer program is said to ‘Learn’ from
experience ‘E’ wrt class of tasks ‘T and performance
measure ‘P’ if the performance at tasks improves
with ‘E’.
 Examples
Checkers game Handwriting Robot driving
recognition learning
T Playing checkers Recognizing and Driving on public 4
classifying lane highway using
handwritten words sensors
in images

P Performance measure % of words Avg. distance

correctly classified travelled before
error
E Playing practice games Database of Sequence of images
handwritten words and steering
and classification commands
 Machine Learning
• It makes computers modify or adapt to their actions
• It merges ideas from :
• Neuroscience \biology\stats\maths\physics
Data mining emerged which extracts information from
massive data sets
Large datasets have algorithms with high complexity
Complexity broken into two types :
a) Complexity of Training
b) Complexity of applying trained algorithm
 Applications of ML
• Learning to recognize spoken words of speech
recognition systems
• Learning to drive autonomous vehicles which
are computer controlled
• Learning to construct astronomical structures
• Learning to play world class games like
backgammon etc..
Applications of ML
 Types of ML
1)Supervised learning : training with correct responses
based on the training set. Algorithm responds to all
inputs
2) Unsupervised learning : correct responses are not
provided .i/p should have some thing in common.
Statistical approaches are used called as density
estimators .
3)Reinforcement learning : b/n supervised and
unsupervised .Algorithm is told it is wrong .Different
possibilities have to be tried .does not suggest
improvement .
4)Evolutionary Learning : biological evolution is seen
as learning process .It improves survival rate . It also
suggests how good the solution is .
1.Supervised Learning
 1.Supervised Learning
• There are two types :a) Regression b) Classification
• A set of data ( training) consists of set of input –target
data
• Set of data (xi, ti) , i = 1 to N
Training Output
data (Ti) data (Ti)

• ML is used for learning process

• Generalization: which solution is better to test and how
we pick the values b/n data points.
a) Regression : suppose we take X ,Y,t st. X->Y, be the
scalars
 1.Supervised Learning
Regression is a statistical approach to find the correlations
between variables ( dependent and independent )

• Regression algorithms gives you a continuous output. That

means if you are asking to build model that predict the future
outcome where the output will be continuous. Then you must
choose one of the regression algorithms to build model
• Ex: Weather forecasting, Market Trends, etc.
• Below are some popular Regression algorithms which come
under supervised learning:
• Linear Regression
• Regression Trees
• Non-Linear Regression
• Bayesian Linear Regression
• Polynomial Regression
 1.Supervised Learning
Applications of Regression:

Predicting prices: For example, a regression model could be

used to predict the price of a house based on its size, location,
and other features.
Forecasting trends: For example, a regression model could
be used to forecast the sales of a product based on historical
sales data and economic indicators.
Identifying risk factors: For example, a regression model
could be used to identify risk factors for heart disease based
on patient data.
Making decisions: For example, a regression model could be
used to recommend which investment to buy based on market
data.
 b)classification
• A decision points separated by decision boundaries .
• Curse of dimensionality : as no of inputs
increases ,training of classifier takes longer as the no.
of i/p dim grows .
• Classification algorithms are used when the
output variable is categorical, which means
there are two classes such as Yes-No, Male-
Female, True-false, etc.
• Classification algorithms:
• Random Forest
• Decision Trees
• Logistic Regression
• Support vector Machines
 2.Unsupervised Learning
❑Unsupervised learning is a machine learning technique in
which models are not supervised using training dataset. Instead,
models itself find the hidden patterns and insights from the given
data. It can be compared to learning which takes place in the
human brain while learning new things

❑The goal of unsupervised learning is to find the underlying

structure of dataset, group that data according to similarities,
and represent that dataset in a compressed format.

❑An unlabeled input data, which means it is not categorized and

corresponding outputs are also not given.

❑ This unlabeled input data is fed to the machine learning model

in order to train it.
 2.Unsupervised Learning

❑ Firstly, it will interpret the raw data to find the hidden patterns
from the data and then will apply suitable algorithms such as k-
means clustering, Decision tree, etc.

❑ Once it applies the suitable algorithm, the algorithm divides

the data objects into groups according to the similarities and
difference between the objects.
2.Unsupervised Learning
• Types of Unsupervised Learning Algorithm:

Clustering: Clustering is a method of grouping the objects into

clusters such that objects with most similarities remains into a
group and has less or no similarities with the objects of another
group. Cluster analysis finds the commonalities between the data
objects and categorizes them as per the presence and absence of
those commonalities.
• Association: An association rule is an unsupervised learning
method which is used for finding the relationships between
variables in the large database. It determines the set of items
that occurs together in the dataset. Association rule makes
marketing strategy more effective. Such as people who buy X
item (suppose a bread) are also tend to purchase Y
(Butter/Jam) item. A typical example of Association rule is
Market Basket Analysis.
Unsupervised Learning algorithms:
• K-means clustering
• KNN (k-nearest neighbors)
• Hierarchal clustering
• Anomaly detection
• Neural Networks
• Principle Component Analysis
• Independent Component Analysis
 3.Reinforcement Learning
❑ Reinforcement Learning is a feedback-based Machine learning
technique in which an agent learns to behave in an environment
by performing the actions and seeing the results of actions. For
each good action, the agent gets positive feedback, and for each
bad action, the agent gets negative feedback or penalty.

❑ In Reinforcement Learning, the agent learns automatically

using feedbacks without any labeled data, unlike supervised
learning

❑ Since there is no labeled data, so the agent is bound to learn by

its experience only
 3.Reinforcement Learning
❑ Ex: Automatic car driving , robotic dog etc..
3.Reinforcement Learning
4. Evolutionary Learning
 BRAIN AND THE NEURON
The brain deals with
1. What we want noise
2. Inconsistent data
3. Produces answers
Neurons : die as we grow in age without degrade in performance
• Processing units of the brain are neurons (nerve cells) (100 billions = 10 11 )
• Transmitters : contain chemicals within the fluid of the brain (↑) or (↓) in electrical
potential within neuron.
• If this membrane potential reaches some threshold –neuron spikes or fires.
• A pulse of fixed duration is sent down the axon.
Axon : divides (arborise) into connections and connecting to each of these neurons in a
synapse.
• Each neuron is connected to 100 trillion (=1014) synapses
• Neuron must wait for some time to recover its energy (refractory period) before it can
fire again .
Learning : occurs in the brain using the principal concept as plasticity.
Plasticity : modify the strength of the synaptic connection between neurons thus creating
new connection.
 see a flower from
eyes

Brain
Neuron and myelinated axon, with signal flow from inputs at dendrites to outputs
at axon terminals
 Methods
a)Hebb’s Rule :
• Introduced by Donald Hebb
• Change in the strength of the synaptic connections are ‘α’(correlation in firing
of 2 connecting neurons )else the neurons will die. It is a learning rule that
describes how the neuronal activities influence the connection between neurons

Hebbian Learning Rule Algorithm :

1. Set all weights to zero, wi = 0 for i=1 to n, and bias to zero.

2. For each input vector, S(input vector) : t(target output pair), repeat steps 3-5.

3. Set activations for input units with the input vector Xi = Si for i = 1 to n.

4. Set the corresponding output value to the output neuron, i.e. y = t.

5. Update weight and bias by applying Hebb rule for all i = 1 to n:

Implementing AND Gate :
Step 1 :
Set weight and bias to zero, w = [ 0 0 0 ] T and
b = 0.
Step 2 :
Set input vector Xi = Si for i = 1 to 4.
X1 = [ -1 -1 1 ]T
X2 = [ -1 1 1 ]T
X3 = [ 1 -1 1 ]T
X4 = [ 1 1 1 ] T
Step 3 :
Output value is set to y = t.
Step 4 :
Modifying weights using Hebbian Rule:
First iteration –
w(new) = w(old) + x1y1 = [ 0 0 0 ]T + [ -1 -1 1 ]T . [ -1 ] =
[ 1 1 -1 ]T
For the second iteration, the final weight of the first one will
be used and so on.
Second iteration –
w(new) = [ 1 1 -1 ]T + [ -1 1 1 ]T . [ -1 ] = [ 2 0 -2 ]T
Third iteration –
w(new) = [ 2 0 -2]T + [ 1 -1 1 ]T . [ -1 ] = [ 1 1 -3 ]T
Fourth iteration –
w(new) = [ 1 1 -3]T + [ 1 1 1 ]T . [ 1 ] = [ 2 2 -2 ]T
So, the final weight matrix is [ 2 2 -2 ]T
Testing the Network:

For x1 = -1, x2 = -1, b = 1, Y = (-1)(2) + (-1)(2) + (1)(-2) = -6

For x1 = -1, x2 = 1, b = 1, Y = (-1)(2) + (1)(2) + (1)(-2) = -2
For x1 = 1, x2 = -1, b = 1, Y = (1)(2) + (-1)(2) + (1)(-2) = -2
For x1 = 1, x2 = 1, b = 1, Y = (1)(2) + (1)(2) + (1)(-2) = 2
The results are all compatible with the original table.
b)Pavlov : “grandmother gives a chocolate to her dogs”
called classical conditioning
• Dogs are trained with food by ringing a bell
• The strength of synapse b/n hearing bell and saliva reflex
is fired.

Stimulus- something that exists in the surroundings or that is

presented

Response- How the animal responds to the stimulus

1. The unconditioned stimulus (US)
2. The Unconditioned Response (UR)
3. The neutral stimulus (NS)
4. The Conditioned Stimulus (CS)
5. The Conditioned Response (CR)

Ex:
When a dog presented with a meat naturally salivates to the
meat, naturally salivates to the meat especially when hungry
c) Mc Culloch and Pits Neurons : (1943)

● Mc culloch -pitts neuron was the earliest neuran n/w

discovered in 1943
● Also called M-P neuron
● A set of weighted inputs (wi) coresp to synapse
● An adder sums the i/p signals (=membrane of cell
collecting electrical changes)
● It has two layers i) input layer ii) output layer
● since the firing of the o/p neuron is based on the
threshold, the activation function here is defined as
f(yin) = 1 if yin >= θ
0 if yin < θ
Activation Function (threshold function)
• To decide whether the neurons fire (spikes) for
the inputs given
• Let the inputs be (x1,x2 …. ,xm) where
if x1 = 1(fired)
x2 = 0(didn’t fire)
x3 = 0.5(no biological significance)

• Strength of the synapse = ( x1w1+x2w2+ … ,xmwm)

• The formula is h = Σ ( wixi)

The threshold value should satisfy the following condition:

θ >nw-P
n → no. of neurons in input
w → positive weight
p → negative weight
Hence , Mc culloch and pitts is a binary threshold device .
Yin - x1w1+x2w2+ … ,xmwm

(1,1) = 1*1 + 1*1 =2

(1,0) = 1*1 + 0*1 =1
(0,1) = 0*1 + 1*1 =1
(0,0) = 0*1 + 0*1 =0

Now need to find out the threshold value in such a way the y
neuron should fire as per the threshold value.

Threshold value is set equal to 2 ( θ =2)

θ >= nw -p

n → no. of neurons in input (2)

w → positive weight (1)
p → negative weight (0)
n= 2, w =1 and p=0
Substituting the above values in the
mentioned equation, we get

θ >= 2*1-0 = 2
θ >= 2

thus, the output of neuron Y can be written as

Y= f (yin) = 1 if yin >= 2
0 if yin < 2
 Limitations of Mac Culloch and Pitts
Neural Model
• Neurons are more complicated
• Inputs need not be linear
• Neuron gives a spike train of output ( 2 outputs)
• They give graded output in continuous way
• Firing the changes over time
• They are biochemical devices hence the amt of charge
give out can vary.
• Neurons are updated randomly(asynchronous).
• Weights are (+ve (excitatory) & –ve (inhibitory))
• Weights change from +ve to –ve
• Links (synapse) are in a feedback loop but not possible in
the network of loops.
 Designing A Learning System
1. Choosing the training system
1.1) whether the training experience provides –direct / indirect
feedback regarding the choices made by performance of the

system
1.2) Degree to which the learner controls the sequence of training
examples
1.3) How well it represents distribution of examples
2. Choosing the target function
3. Choosing a representation for the target function
4. Choosing a Function Approximation Algorithm
4.1) Estimating training values
4.2) Adjusting the weights
5. The final design
 Designing A Learning System
1) Choosing the training system:
Type of training experience available :It has impact
on success or failure of the learner .
Key attributes :
i) whether the training experience provides –direct /
indirect feedback regarding the choices made by
performance of the system.
Eg: Checkers game : system learns from direct training
(board states ,correct move sequence , final outcome )
Credit assignment : degree to which each move in the
sequence deserves credit/blame for final outcome .
Learning from direct training is easier than indirect
training .
 ii) Degree to which the learner controls the sequence of training
examples :
• Relies on the teacher (board states and correct moves)
• Learner has control over (board states and indirect training classification)

iii) How well it represents distribution of examples

P-performance must be measured (%)of win
If E: training experience consists of (games played against itself)
• Distribution of training examples = testing examples

Learning Problem
Task T : playing checkers
P : % of games won in the tournament
E : games played against itself
Choice :
1. Exact type of knowledge to be learned
2. Representation for target knowledge
2:Choosing Target Function
• Determine exactly what type of knowledge will
be learned
• How it will be used by performance program?
Answer: choose best legal moves in large search
space Function ChooseMove

Notation :ChooseMove : B->M

B : legal board states
M : moves
Improve the ‘P’ of Task ‘T’
 Evaluation Function
• Assigns a numerical score to any given board state
• Target Function = V
• Notation V: B->R ,
where B = legal board state
R = set of real nos.
Target value V(b) for an arbitrary state b in B
1) If b is final board state = won ,V(b) = 100
2) If b is a final board state =lost ,V(b) = -100
3) If b= drawn state , v(b) = 0 , V(b) = drawn
4) If b!=final state ,V(b) = V(b’)
One must know the need to search ahead all the way to the end of
the game according to (4).
The process of learning the target function is called as function
approximation( ) .
 Step:3 Choosing representation for target
function
• Choose a value (V) which is very close to
= linear combination of the following board features

x1: the number of black pieces on board

x2: no. of Red pieces
X3: black kings
X4: red kings
X5: no. of black pieces threatened by Red (captured on red in next turn)
X6: no. of red pieces threatened by black.
Learning Program: will represent (b) as linear function of the form :-

w0…….w6 = numerical coefficients /weights

 4. Choosing a function Approximation
Algorithm
• To learn target function set of training
examples are required .
• It specifies board state b as training value
• Each training example is an ordered pair
Eg : black has won the game
(x2 = 0 , no red is left)
• Target function value = +100
(black has won hence +ve)
 4.1Estimating the training values

• In Every step, We consider the successor

(Depending on the next step of opponent)

= represents Approximation
successor(b) = next board state following ‘b’
(Estimating that this move will help/destroy
opponent)
 4.2 Adjusting the weights

• Choosing the weights (wi) to best fit training

examples
• Solution : best define the hypothesis /weights
minimizing squared error ‘E’ between training
values and values predicted over hypothesis
There are some algorithms to find the weights of
linear functions, Here we are using Lease mean
square (Used to minimize the error)
if error =0 → No need to change weights
error = positive → each weight is increased
error = negative → each weigh is decreased
 5.THE FINAL DESIGN
• It has four modules
1)Performance system: solves given performance
task .Next move is determined by .The
performance becomes increasingly accurate.
2) Critic: Takes as the input the history or trace of
the game and produces as output a set of training
examples of the target function .It operates on
train rule ie
3)Generalizer : takes as input the training
examples and produces an output hypothesis that
is its estimation of target function.
• Experiment generator: It picks new practice
problems that maximize learning rate (η) of
over all system
 Issues in machine learning
• What algorithm exists for learning general target
function from training examples.
• How much training data is sufficient?
• When and how prior knowledge held by learner
guide the process
• What is the best strategy for choosing next
training experience?
• How to reduce learning task ?
• How learner alter its representation in target
function
 Concept learning
• Defn : Every concept is a Boolean-valued
function defined over larger set.
• Learning involves acquiring general concepts
from specific training examples
• Each such concepts can be viewed as describing
some subset of objects or events defined over a
larger set.
• A task of acquiring potential hypothesis
(solution) that best fits the given training
examples
 Concept learning

Days on which my friend Aldo enjoyed his favorite

water sports?
 Hypothesis
• Consists of conjunction of constraints on instance
attributes
• It has 6 constraints according to the table
• Indicated by ‘?’ (any value is acceptable for this
attribute)
• Indicated by single value ‘warm’
• ‘Φ’ = no values accepted (null)
• Defn : Inference : set of items over which the concept
is defined
• Let ‘x’ be the instance
• Let ‘h’ be the hypothesis
• ‘x’ satisfies all constraints of ‘h’ ie (h(x) = 1) , x is +ve
Q)Aldo enjoys his favorite sport only on cold
days with high humidity.
General hypothesis :
Representation of expression :
No. of days it is positive:
Instance is rep. by x =
{sky,Airtemp,Humidity,Wind,water,forecast}
Target Concept(c)= concept /function to be
learned .C is a Boolean valued function over x
• Target Concept corresponds to the value of the
attribute.
• Ordered pair (x, c(x)) describes training
example
Given : target concept ( c )EnjoySport :X {0,1}
D = set of available training examples
H = set of all possible hypothesis regarding the
identity of target concept .
X= possible days described by the attributes
Determine : ‘h’ in ‘H’ that represents Boolean-
valued functions defined over X
h=
Goal : find ‘h’ such that h(x) = c(x) for all x in X
 Inductive Learning Hypothesis
• It can guarantee that the output hypothesis
fits the target concept over the training data
• Hypothesis that best fits the observed training
data
• Defn : any hypothesis found to approximate
target function over large set of training
examples .
 Concept learning as a search
• Defn : searching through large space of hypothesis
implicitly defined by hypothesis representation
• Goal : find the hypothesis that best fits the training
examples .
• Instance space X : contains 3.2.2.2.2.2 = 96 distinct
instances
• Hypothesis (H) : contains 5.4.4.4.4.4=5120
(syntactically distinct hypothesis )
• If one of the move ‘Φ’ is present in the (semantically)
distinct hypothesis then 1+(4.3.3.3.3.3) = 973(if it
considers every instance as negative )
General to specific ordering of hypothesis

• Defn : we can design learning algorithm that

exhaustively searches even infinite hypothesis
space without explicitly enumerating every
hypothesis .
• Consider 2 hypothesis

Consider h1 and h2 are +ve(h2 imposes fewer

instances and classifies more instances as +ve
For any instance : x in X
Hypothesis: h in H
X satisfies h iff: h(x) = 1

Special case : one hypothesis is more general than others

hj ( more general _ than) hk indicated as iff

Defines partial order over hypothesis space H if it is (reflexivity(

) , anti-symmetric (if (a,b)€ R and (b,a) € R then a = b ) and transitivity
 Finding Maximally Specific Hypothesis
[FIND-S]
• Hypothesis consists of training examples (+ve
and –ve)
 Step 1
• Initialize h to more specific hypothesis in H
• None of the ‘Φ’ constraints are satisfied
• according to the example

• h is specific(–ve)
• Place’?’ to any
position (3rd ).
• This is –ve(no
revision is
required for h
4th training is a positive example
Search moves from h to h from most specific to most general hypothesis
along partial ordering
•No explicit enumerated members
•IT uses more _general_then partial ordering to main representation of set of H .
•The subset of all hypothesis is called as version space with respect to H and D.
Version spaces types
 1.List _Then _eliminate
• List all of its members
• 1st initialize version space containing all H
• Eliminate any H focused inconsistent
• Version space shrinks as training examples are observed
• Only one H remains that is consistent with all examples
giving Target Concept( c )
• If insufficient data = FOUND then output entire set of H
consistent with observed data
• It can be applied when H is finite
• It output all H
Example:
 2.More compact Representation
• Represented by most general and least general
members
• They form general and specific boundary sets with
partially ordered hypothesis space.
Boundary sets G and S
Version space representation theorem
• Let X be any arbitrary set of instances
• Let H be set of Boolean values hypothesis
defined over X
• Let c:X->[0,1] be arbitrary target defined over
X
• Let D be an arbitrary set of training examples
{<x, c(x)>} for all values of X,H,c,D such that S
and G are well defined .
 Perceptron
• Type of ANN is called as perceptron, is the first neural
network model designed in 1958
• Perceptron is linear binary classifier used for supervised
learning
• Perceptron learning model is a combination of McCulloch-
pitts model and Hebbian learning rule of adjusting weights
• It takes a value of real valued inputs and
calculates a linear combination of these
inputs .
• Given <x1,x2, ….. , xn> as inputs and a
(x1,x2,…. , xn) is computed by perceptron

• Wi = real valued constant

• -w0 is a threshold on weighted combinations
of inputs .
• It gets the outputs as 1
• If then
• vector form is
• Perceptron function is

• The space H of the candidate hypothesis

considered in perceptron learning = set of ( all
possible real valued weight vectors)
•
 Linearly separable
• +ve and –ve examples are separated by a
hyperplane and it is called as linearly
separable .
• If we assume boolean values of 1(true) and 0
( false )
• Perceptron can be represented as

• Cannot be represented as its value is 1

Perceptron
• Learning rate is : 0.1 < η < 0.4
• Bias: fix the value of the threshold for the
neuron at zero. The input to the weight is ±1.
• The weight is -1 when the inputs are al;l
zeros .This is called bias(w0j) .
complexity is
O(mn).
For T iterations,
costs is
O(Tmn).
 example
• w0 = −0.05, w1 = −0.02, w2 = 0.02.
• inputs are 0: (0, 0)
• Input to the bias weight is always −1,\
• value that reaches the neuron is −0.05 × −1 +−0.02 × 0 + 0.02 × 0 = 0.05
Result : This value is above 0, so the neuron fires and the output is 1, which is incorrect according to the
target.
Apply the equation : wij ← wij − η(yj − tj ) · xi for η = 0.25
w0 : −0.05 − 0.25 × (1 − 0) × −1 = 0.2, (3.7)
w1 : −0.02 − 0.25 × (1 − 0) × 0 = −0.02, (3.8)
w2 : 0.02 − 0.25 × (1 − 0) × 0 = 0.02. (3.9)
feed in the next input (0, 1)
apply the learning rule again:
w0 : 0.2 − 0.25 × (0 − 1) × −1 = −0.05, (3.10)
w1 : −0.02 − 0.25 × (0 − 1) × 0 = −0.02, (3.11)
w2 : 0.02 − 0.25 × (0 − 1) × 1 = 0.27. (3.12)
(1, 0) input the answer is already correct
.
 Linear Regression
• Linear regression is a basic and commonly used type of
predictive analysis.
• The overall idea of regression is to examine two things:

(1) does a set of predictor variables do a good job in

predicting an outcome (dependent) variable?

(2) Which variables in particular are significant predictors

of the outcome variable, and in what way do they–
indicated by the magnitude and sign of the beta estimates–
impact the outcome variable?
• These regression estimates are used to explain the relationship
between one dependent variable and one or more independent
variables.

• The simplest form of the regression equation with one

dependent and one independent variable is defined by the
formula

y = c + b*x
where y = estimated dependent variable score,
c = constant,
b = regression coefficient,
x = score on the independent variable.
• Naming the Variables. There are many names for a
regression’s dependent variable. It may be called an outcome
variable, criterion variable, endogenous variable, or
regressand.

• The independent variables can be called exogenous variables,

predictor variables, or regressors.

• Three major uses for regression analysis are

(1)determining the strength of predictors,

(2)forecasting an effect, and
(3) trend forecasting.
• First, the regression might be used to identify the strength
of the effect that the independent variable(s) have on a
dependent variable. Typical questions are what is the
strength of relationship between dose and effect, sales and
marketing spending, or age and income.

• Second, it can be used to forecast effects or impact of

changes. That is, the regression analysis helps us to
understand how much the dependent variable changes with
a change in one or more independent variables.
• A typical question is, “how much additional sales income
do I get for each additional $1000 spent on marketing?”

• Third, regression analysis predicts trends and future

values. The regression analysis can be used to get point
estimates. A typical question is, “what will the price of gold
be in 6 months?”
Linear Regression
• Linear Regression