Scribd
Upload
16 views13 pages

ATML Unit2

Uploaded by

aaryasawant2575
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
16 views13 pages

ATML Unit2

Uploaded by

aaryasawant2575
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 13

Advanced Topics in

Machine Learning
Unit 2 : Outlier Detection and Dimensionality
Reduction
Text Books:
1. Charu C. Aggarwal, Outlier Analysis, 2nd edition, Springer, 2017

Web references:
https://www.analyticsvidhya.com/blog/2024/03/one-class-svm-for-anomaly-detection/
Outliers

 An outlier is a data point that significantly deviates from the rest of the data.
 It can be either much higher or much lower than the other data points, and its
presence can have a significant impact on the results of machine learning
algorithms.
 They can be caused by measurement or execution errors.

There are two main types of outliers:


• Global outliers: Global outlier are isolated data points that are far away from
the main body of the data. They are often easy to identify and remove.
• Contextual outliers: Contextual outliers are data points that are unusual in a
specific context but may not be outliers in a different context. They are often
more difficult to identify and may require additional information or domain
knowledge to determine their significance.
Outliers

Algorithm
1. Calculate the mean of each cluster
2. Initialize the Threshold value
3. Calculate the distance of the test data from each cluster mean
4. Find the nearest cluster to the test data
5. If (Distance > Threshold) then, Outlier
Outliers

Importance of outlier detection in machine learning


Outlier detection is important in machine learning for several reasons:
1. Biased models: Outliers can bias a machine learning model towards the outlier
values, leading to poor performance on the rest of the data. This can be particularly
problematic for algorithms that are sensitive to outliers, such as linear regression.
2. Reduced accuracy: Outliers can introduce noise into the data, making it difficult for a
machine learning model to learn the true underlying patterns. This can lead to reduced
accuracy and performance.
3. Increased variance: Outliers can increase the variance of a machine learning
model, making it more sensitive to small changes in the data. This can make it difficult to
train a stable and reliable model.
4. Reduced interpretability: Outliers can make it difficult to understand what a machine
learning model has learned from the data. This can make it difficult to trust the model’s
predictions and can hamper efforts to improve its performance.
One Class SVM (OC-SVM)

Anomalies
 Anomalies are observations or instances that deviate significantly from a
dataset’s normal behavior.
 These deviations can manifest in various forms, such as outliers, noise, errors,
or unexpected patterns.
 Outlier and novelty detection identify anomalies and abnormal or uncommon
observations.

The One-Class Support Vector Machine (SVM) is a variant of the traditional SVM. It is
specifically tailored to detect anomalies. Its primary aim is to locate instances that
notably deviate from the standard.
SVM revisit

Soft margin Hard margin

A new regularization parameter C controls the trade-off between maximizing the margin and
minimizing the loss.
As you can see, the difference between the primal problem and the one for the hard margin is
the addition of slack variables. The new slack variables ( in the figure) add flexibility for
misclassifications of the model:
One Class SVM vs Traditional SVM

 One-class SVMs represent a variant of the traditional SVM algorithm primarily


employed for outlier and novelty detection tasks. Unlike traditional SVMs, which
handle binary classification tasks, One-Class SVM exclusively trains on data
points from a single class, known as the target class.
 Traditional SVMs aim to find a decision boundary that maximizes the margin
between different classes, allowing for optimal classification of new data points.
On the other hand, One-Class SVM seeks to find a boundary that encapsulates
the target class while minimizing the risk of including outliers or novel instances
outside this boundary.
 Traditional SVMs require labeled data with instances from multiple classes,
making them suitable for supervised classification tasks. In contrast, a One-
Class SVM allows application in scenarios where only data from the target class
is available, making it well-suited for unsupervised anomaly detection and
novelty detection tasks.
One Class SVM

 One-class SVM aims to discover a hyperplane with maximum margin within the
feature space by separating the mapped data from the origin. On a dataset Dn
= {x1, . . . , xn} with xi ∈ X (xi is a feature) and n dimensions:

This equation represents the primal problem formulation for OC-SVM,


where w is the separating hyperplane,
ρ is the offset from the origin, and
ξi are slack variables.
They allow for a soft margin but penalize violations ξi.
A hyperparameter ν ∈ (0, 1] controls the effect of the slack variable and should be
adjusted according to need. The objective is to minimize the norm of w while
penalizing deviations from the margin. Further, this allows a fraction of the data to
fall within the margin or on the wrong side of the hyperplane.
One Class SVM

gamma (ρ) : is a crucial parameter that


influences the shape of the decision boundary. A
smaller gamma value results in a broader
decision boundary which makes the model less
sensitive to individual data points. Conversely, a
larger gamma value leads to a more complex
decision boundary, potentially capturing intricate

w.x + b =0 is the decision boundary, and patterns in the data. Fine-tuning gamma is
the slack variables penalize deviations. essential for achieving optimal model
performance.

nu (ν): This is a crucial hyperparameter in One-Class SVM, which controls the


proportion of outliers allowed. It sets an upper bound on the fraction of training
errors and a lower bound on the fraction of support vectors. It typically ranges
between 0 and 1, where lower values imply a stricter margin and may capture fewer
outliers, while higher values are more permissive. The default value is 0.5.
One Class SVM

Kernel Functions in One-Class SVM

 Kernel functions play a crucial role in One-Class SVM by allowing the algorithm to
operate in higher-dimensional feature spaces without explicitly computing the
transformations.
 These kernels map the original input space into a higher-dimensional space, where
data points become linearly separable or exhibit more distinct patterns, facilitating
learning.
One Class SVM

Margin and Support Vectors

 In One-Class SVM, the margin represents the region where most of the data points
belonging to the target class lie.
 Maximizing the margin is crucial for One-Class SVM as it helps generalize new data
points well and improves the model’s robustness.
 In One-Class SVM, support vectors are the data points from the target class closest
to the decision boundary.
 These support vectors play a significant role in determining the shape and
orientation of the decision boundary and, thus, in the overall performance of the
One-Class SVM model.
One Class SVM

The plots allow us to visually inspect the performance of the One-Class SVM models in detecting
outliers in the Wine dataset. By comparing the results of hard margin and soft margin One-Class
SVM models, we can observe how the choice of margin setting (nu parameter) affects outlier
detection.
 The hard margin model with a very small nu value (0.01) likely results in a more
conservative decision boundary. It tightly wraps around the majority of the data points and
potentially classifies fewer points as outliers.
 Conversely, the soft margin model with a larger nu value (0.35) likely results in a more
flexible decision boundary. Thus allowing for a wider margin and potentially capturing more

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy