Scribd
Upload
35 views18 pages

L25 - Naïve Bayes

Hang

Uploaded by

directadmission465
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
35 views18 pages

L25 - Naïve Bayes

Hang

Uploaded by

directadmission465
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Naïve Bayes Classifier

Background
• There are three methods to establish a classifier
a) Model a classification rule directly
Examples: k-NN, decision trees, perceptron, SVM
b) Model the probability of class memberships given input data
Example: multi-layered perceptron with the cross-entropy cost

c) Make a probabilistic model of data within each class


Examples: naive Bayes, model based classifiers

• a) and b) are examples of discriminative classification


• c) is an example of generative classification
• b) and c) are both examples of probabilistic
classification
2
Probability Basics
• Prior, conditional and joint probability
– Prior probability:
– Conditional probability:
– Joint probability:
– Relationship:
– Independence:
• Bayesian Rule

3
Example by Dieter Fox
Probabilistic Classification
• Establishing a probabilistic model for classification
– Discriminative model

– Generative model

• MAP classification rule


– MAP: Maximum A Posterior
• Generative classification with the MAP rule
– Apply Bayesian rule to convert:

7
Naïve Bayes
• Bayes classification

Difficulty: learning the joint probability


• Naïve Bayes classification
– Making the assumption that all input attributes are independent

8
To create a classifier model, find the probability of given
set of inputs for all possible values of the class
variable C and pick up the output with maximum
probability. This can be expressed mathematically as:

P(C) is class probability and P(xi | C) is conditional probability.


Example
• Objective: Classify whether players will play or not
based on weather condition.

10
• Step 1: Convert the data set into a frequency
table
• Step 2: Create Likelihood table by finding the
probabilities.
• Step 3: Now, use Naive Bayesian equation to
calculate the posterior probability for each
class.
– The class with the highest posterior probability is
the outcome of prediction.
• Problem: Players will play if weather is sunny. Is this
statement is correct?

• P(Yes | Sunny) = P( Sunny | Yes) * P(Yes) / P (Sunny)

• Here we have
– P (Sunny |Yes) = 3/9 = 0.33,
– P(Sunny) = 5/14 = 0.36,
– P( Yes)= 9/14 = 0.64

• Now, P (Yes | Sunny) = 0.33 * 0.64 / 0.36 = 0.60, which has


higher probability.
Pros:
• It is easy and fast to predict class of test data
set.
• It also perform well in multi class prediction
• When assumption of independence holds, a
Naive Bayes classifier performs better
compare to other models like logistic
regression and you need less training data.
• It perform well in case of categorical input
variables compared to numerical variable(s).
Cons:
• Zero Frequency Problem:
– If categorical variable has a category (in test data set),
which was not observed in training data set,
then model will assign a 0 (zero) probability and will
be unable to make a prediction.
– To solve this, we can use the smoothing technique.
One of the simplest smoothing techniques is called
Laplace estimation.
• Naive Bayes is also known as a bad estimator.
• Assumption of independent predictors
Applications of Naive Bayes Algorithms
• Real time Prediction
• Multi class Prediction
• Text classification/ Spam Filtering/ Sentiment
Analysis
• Recommendation System
Conclusions
• Naïve Bayes based on the independence assumption
– Training is very easy and fast; just requiring considering each
attribute in each class separately
– Test is straightforward; just looking up tables or calculating
conditional probabilities with normal distributions
• A popular generative model
– Performance competitive to most of state-of-the-art
classifiers even in presence of violating independence
assumption
– Many successful applications, e.g., spam mail filtering
– Apart from classification, naïve Bayes can do more…

18

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy