Scribd
Upload
46 views18 pages

AIML Ritesh

The document discusses classification tasks and evaluating classification models using confusion matrices. It defines binary and multi-class classification and explains how to calculate accuracy, precision, recall and F1 scores from confusion matrices. It also describes micro averaging, macro averaging and weighted averaging of these metrics.

Uploaded by

surya332ygl
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
46 views18 pages

AIML Ritesh

The document discusses classification tasks and evaluating classification models using confusion matrices. It defines binary and multi-class classification and explains how to calculate accuracy, precision, recall and F1 scores from confusion matrices. It also describes micro averaging, macro averaging and weighted averaging of these metrics.

Uploaded by

surya332ygl
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Classification Task:

Binary classification: Classify an RGB image of cropped patches


of skin as healthy or having psoriasis lesion (i.e. unhealthy).
Class Name Class Label Categorical label
Healthy 0 [10]
Unhealthy 1 [01]

Numerical
MultiClass classification: label 2 0 1

class more than 2 0

1 : Categorical

2
Confusion Matrix

Confusion Matrix for Binary Classes


A binary class dataset is one that consists of just two
distinct categories of data. These two categories can be named
the “positive” and “negative” for the sake of simplicity.
Recall = Sensitivity = TPR (true positive rate)
Precision = PPV (Positive predictive value)
Specificity = TNR (true negative rate)
Confusion Matrix for Multi-Class Classification

𝐴𝑙𝑙 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛 (𝑖. 𝑒. 𝑑𝑖𝑎𝑔𝑜𝑛𝑎𝑙 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠) 𝑇𝑃1 + 𝑇𝑃2 + 𝑇𝑃3


𝑂𝑣𝑒𝑟𝑎𝑙𝑙 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = =
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛 𝑜𝑟 𝑑𝑎𝑡𝑎 (𝑖. 𝑒. 𝑠𝑢𝑚 𝑜𝑓 𝑎𝑙𝑙 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠) σ 𝑐𝑒𝑙𝑙
Another example
calculate the class-wise accuracy, precision, recall, and f1-
scores

Converting a multi-class confusion matrix to a one-vs-all


Converting a multi-class confusion matrix to a one-vs-all (for class-2) matrix
(for class-1) matrix
Micro Averaging
The micro-averaged f1-score is a global metric that is calculated by considering the net TP, i.e., the sum of the
class-wise TP (from the respective one-vs-all matrices), net FP, and net FN. These are obtained to be the
following:

Net TP = 52+28+25+40 = 145


Net FP = (3+7+2)+(2+2+0)+(5+2+12)+(1+1+9) = 46
Net FN = (2+5+1)+(3+2+1)+(7+2+9)+(2+0+12) = 46

Note that for every confusion matrix, the net FP and net FN will have the same value.

Thus, the micro precision and micro recall can be calculated as:

Micro Precision = Net TP/(Net TP+Net FP) = 145/(145+46) = 75.92%


Micro Recall = Net TP/(Net TP+Net FN) = 75.92%
Thus, Micro F-1 = Harmonic Mean of Micro Precision and Micro Recall = 75.92%.
= (Net TP)/(Net TP+(0.5/(net FP+ net FN)))

Since all the measures are global, we get:


Micro Precision = Micro Recall = Micro F1-Score = Accuracy = 75.92%

In this example only all coming equal. But not equal in general
Macro Averaging
The macro-averaged scores are calculated for each class individually, and then the unweighted (or
normal) mean of the measures is calculated to calculate the net global score. For the example we have
been using, the scores are obtained as the following:

The unweighted means of the measures are obtained to be:


Macro Precision = 76.00% = {(81.25+87.5+56.82+78.43)/4} = (P1+P2+P3+P4)/4
Macro Recall = 75.31% = (R1+R2+R3+R4)/4
Macro F1-Score = 75.60% = (F1_1+F1_2+F1_3+F1_4)/4 = Harmonic Mean of Macro Precision and Macro
Recall
Weighted Averaging
The weighted-average scores take a sample-weighted mean of the class-wise scores obtained. So, the
weighted scores obtained are:

= Harmonic Mean of weighted


Precision and weighted Recall
Another example

Think about … what will be the weightage precision


Normalized Confusion Matrix
from sklearn.metrics import confusion_matrix
confusion_matrix(y_true, y_pred, normalize=‘all/pred/true/None')

The parameter normalize allows to report ratios


instead of counts. The confusion matrix can be
normalized in 3 different ways: 'pred', 'true',
and 'all' which will divide the counts by the sum
of each columns, rows, or the entire matrix,
respectively.
If None, confusion matrix will not be normalized.

When A row represents an instance of the actual class, and a column represents an instance of the predicted class
THEN:
The “normalized” term means that each of these groupings is represented as having 1.00 samples. Thus, the sum of
each row in a balanced and normalized confusion matrix is 1.00, because each row sum represents 100% of the
elements in a particular topic, cluster, or class.
confusion matrix is used to evaluate the quality of the output of a classifier. The diagonal elements represent the
number of points for which the predicted label is equal to the true label, while off-diagonal elements are those
that are mislabeled by the classifier. The higher the diagonal values of the confusion matrix the better, indicating
many correct predictions.
The figures show the confusion matrix with and without normalization by class support size (number of elements
in each class). This kind of normalization can be interesting in case of class imbalance to have a more visual
interpretation of which class is being misclassified.
from sklearn.metrics import confusion_matrix
confusion_matrix(y_true, y_pred, normalize=‘true')
from sklearn.metrics import

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy