Scribd
Upload
10 views15 pages

Logistic Regression

logistic

Uploaded by

Aman Desai
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views15 pages

Logistic Regression

logistic

Uploaded by

Aman Desai
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Logistic Regression

1
Problem Setting

● is the input space

○ is a random variable describing the input

● is the output space with K number of classes.

○ is a random variable describing the output.

● is the data distribution

○ is the predicted output probabilities given an input

2
Problem Setting

● We are interested in the probability of a class given the data point:

● If the above probability is known for all K classes, we can predict the label as:

3
Assumption

Logistic Regression (LR) assume that the log-odds are linear.

4
Assumption

Logistic Regression (LR) assume that the log-odds are linear.

5
Fitting LR models

● We maximize the likelihood of betas given the dataset.

● We assume that all the N data pairs are observed independently.

● Maximizing likelihood is equivalent tp maximizing log-likelihood which is


defined as:

● We can optimize the above equation using any gradient descent based
algorithm.

6
Fitting LR model for 2 class setting

● Assume there are 2 classes: Class 1 and Class 0.

● We want to maximize the log-likelihood which is defined as:

7
Fitting LR model for 2 class setting

Rearranging log-likelihood

8
Fitting LR model for 2 class setting

● To maximize log-likelihood, we have to differentiate w.r.t. beta and set it to


zero.

for notational simplicity we will refer

9
Fitting LR model for 2 class setting

● To maximize log-likelihood, we have to differentiate w.r.t. beta and set it to


zero.

● It is not easy to find beta, that satisfies the above equation. We can use an
iterative algorithm to find beta. One such algorithm is called Newton-Raphson
algorithm.

10
Fitting LR model for 2 class setting

Newton-Raphson method:

Rewriting the notations in vector/matrix form, say

11
Fitting LR model for 2 class setting

Newton-Raphson method:

12
Fitting LR model for 2 class setting

Therefore, Newton-Raphson method:

13
Fitting LR model for 2 class setting

Therefore, Newton-Raphson method:

The above solution can also be obtained from solving weighted least square:

14
Summary
• Logistic Regression is a classification approach!

• Assumes that the class probabilities are given by a logit or sigmoid function.

• Directly models the separating surface as a linear function.

• Especially popular in binary classification.

• Can be combined with Variable Selection techniques to yield a sparse


classifier.

15

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy