Classification Metrics

Scenario 1: Email Spam Detection

Predicted
Spam Not Spam
Spam 80 40
Actual
Not Spam 10 870

TP+TN 80+ 870 950

 Accuracy = TP+ TN + FP+ FN = 80+870+10+ 40 = 1000 = 0.95

TP 80 80
 Precision = TP+ FP = 80+10 = 90 = 0.89

TP 80 80
 Recall = TP+ FN = 80+40 = 120 = 0.67

Precision∗Recall 0.89∗0.67 0.5963

 F1-score = 2 * Precision+ Recall = 2 * 0.89+0.67 = 2 * 1.56 = 2 * 0.38 = 0.76

60 20 TP 60
15 105 TN 105
FP 15
FN 20

Accuracy (TP + TN) / (TP + TN + FP + FN) 165

Precision TP / (TP + FP) 60
Recall TP / (TP + FN) 60
F1-Score 2 * (Precision * Recall) / (Precision + Recall) 0.60

Scenario 2: Disease Diagnosis

Predicted
Disease No disease

Disease 60 20
Actual
No disease 15 105

TP+TN 60+105 165

 Accuracy = TP+ TN + FP+ FN = 60+105+15+20 = 200 = 0.825

TP 60 60
 Precision = TP+ FP = 60+15 = 75 = 0.8

TP 60 60
 Recall = TP+ FN = 60+20 = 80 = 0.75
 Precision∗Recall 0.8∗0.75 0.6
 F1-score = 2 * Precision+ Recall = 2 * 0.8+0.75 = 2 * 1.55 = 2 * 0.388 = 0.774

Scenario 3: Image Classification (Cat vs. Dog)

Predicted
Cat Not cat

Cat 90 30
Actual
Not cat 10 170

TP+TN 90+170 260

 Accuracy = TP+ TN + FP+ FN = 90+170+10+30 = 300 = 0.867

TP 90 90
 Precision = TP+ FP = 90+10 = 100 = 0.9

TP 90 90
 Recall = TP+ FN = 90+30 = 120 = 0.75

Precision∗Recall 0.9∗0.75 0.675

 F1-score = 2 * Precision+ Recall = 2 * 0.9+0.75 = 2 * 1.65 = 2 * 0.4091 = 0.82

100 0 TP 100
50 850 TN 850
FP 50
FN 0

Accuracy (TP + TN) / (TP + TN + FP + FN) 950

Precision TP / (TP + FP) 100
Recall TP / (TP + FN) 100
F1-Score 2 * (Precision * Recall) / (Precision + Recall) 0.67

Scenario 4: Fraud Detection

Predicted
Fraud Not fraud

Fraud 100 0
Actual
Not Fraud 50 850

TP+TN 100+850 950

 Accuracy = TP+ TN + FP+ FN = 100+850+50+ 0 = 1000 = 0.95
 TP 100 100
 Precision = TP+ FP = 100+50 = 150 = 0.667

TP 100 100
 Recall = TP+ FN = 100+0 = 100 = 1.0

Precision∗Recall 0.67∗1.0 0.67

 F1-score = 2 * Precision+ Recall = 2 * 0.67+1.0 = 2 * 1.67 = 2 * 0.401 = 0.802

Scenario 5: Product Recommendation System

An AI model predicted that of the 250 visitors to a website, 100 would buy, but only 80 actually bought. Of the 150
they predicted would not buy, 30 bought. Draw a confusion matrix & calculate Accuracy, Precision, Recall & F1-
Score.

Predicted
Recommend Don’t recommend

Recommend 80 30
Actual
Don’t recommend 20 120

TP+TN 80+120 200

 Accuracy = TP+ TN + FP+ FN = 80+120+20+30 = 250 = 0.8

TP 80 80
 Precision = TP+ FP = 80+20 = 100 = 0.8

TP 80 80
 Recall = TP+ FN = 80+30 = 110 = 0.72727

Precision∗Recall 0.8∗0.72727 0.58182

 F1-score = 2 * Precision+ Recall = 2 * 0.8+0.72727 = 2 * 1.52727 = 2 * 0.380954 = 0.762

80 30 TP 80
20 120 TN 120
FP 20
FN 30

Accuracy (TP + TN) / (TP + TN + FP + FN) 200

Precision TP / (TP + FP) 80
Recall TP / (TP + FN) 80
F1-Score 2 * (Precision * Recall) / (Precision + Recall) 0.58

Scenario 6: Sentiment Analysis (Positive/Negative)

Predicted
Positive Negative

Positive 150 15
Actual
Negative 25 180
 TP+TN 150+180 330
 Accuracy = TP+ TN + FP+ FN = 150+180+25+15 = 370 = 0.89189

TP 150 150
 Precision = TP+ FP = 150+25 = 175 = 0.857

TP 150 150
 Recall = TP+ FN = 150+15 = 165 = 0.90909

Precision∗Recall 0.857∗0.90909 0.7791

 F1-score = 2 * Precision+ Recall = 2 * 0.857+0.90909 = 2 * 1.76609 = 2 * 0.44114 =

0.8823

Scenario 7: Customer Churn Prediction

Predicted

Churn No churn

Churn 75 50
Actual
No churn 25 850

TP+TN 75+850 925

 Accuracy = TP+ TN + FP+ FN = 75+850+25+50 = 1000 = 0.925

TP 75 75
 Precision = TP+ FP = 75+25 = 100 = 0.75

TP 75 75
 Recall = TP+ FN = 75+50 = 125 = 0.6

Precision∗Recall 0.75∗0.6 0.45

 F1-score = 2 * Precision+ Recall = 2 * 0.75+0.6 = 2 * 1.35 = 2 * 0.3333 = 0.6667

120 50 TP 120
30 800 TN 800
FP 30
FN 50

Accuracy (TP + TN) / (TP + TN + FP + FN) 920

Precision TP / (TP + FP) 120
Recall TP / (TP + FN) 120
F1-Score 2 * (Precision * Recall) / (Precision + Recall) 0.56

Scenario 8: Loan Approval

 Predicted
Approve Reject

Approve 120 50
Actual
Reject 30 800

TP+TN 120+800 920

 Accuracy = TP+ TN + FP+ FN = 120+800+30+50 = 1000 = 0.92

TP 120 120
 Precision = TP+ FP = 120+30 = 150 = 0.8

TP 120 120
 Recall = TP+ FN = 120+50 = 170 = 0.71

Precision∗Recall 0.8∗0.71 0.568

 F1-score = 2 * Precision+ Recall = 2 * 0.8+0.71 = 2 * 1.51 = 2 * 0.376 = 0.752

Scenario 9: Intrusion Detection System

Predicted
Attack Normal

Attack 95 5
Actual
Normal 50 850

TP+TN 95+850 945

 Accuracy = TP+ TN + FP+ FN = 95+850+50+ 5 = 1000 = 0.945

TP 95 95
 Precision = TP+ FP = 95+50 = 145 = 0.66

TP 95 95
 Recall = TP+ FN = 95+5 = 100 = 0.95

Precision∗Recall 0.655∗0.95 0.62225

 F1-score = 2 * Precision+ Recall = 2 * 0.655+0.95 = 2 * 1.605 = 2 * 0. 0.388 = 0.78

70 10 TP 70
30 90 TN 90
FP 30
FN 10

Accuracy (TP + TN) / (TP + TN + FP + FN) 160

Precision TP / (TP + FP) 70
Recall TP / (TP + FN) 70
F1-Score 2 * (Precision * Recall) / (Precision + Recall) 0.61
Scenario 10: Fake News Detection

Predicted
Fake Real

Fake 70 10
Actual
Real 30 90

TP+TN 70+90 160

 Accuracy = TP+ TN + FP+ FN = 70+90+30+10 = 200 = 0.8

TP 70 70
 Precision = TP+ FP = 70+30 = 100 = 0.7

TP 70 70
 Recall = TP+ FN = 70+10 = 80 = 0.88

Precision∗Recall 0.7∗0.88 0.616

 F1-score = 2 * Precision+ Recall = 2 * 0.7+0.88 = 2 * 1.58 = 2 * 0.39 = 0.78