Statistical Machine Learning

Statistical + Machine Learning

TEXTBOOKS/LEARNING RESOURCES:
a) Masashi Sugiyama, Introduction to Statistical Machine Learning (1 st ed.), Morgan Kaufmann, 2017. ISBN 978-0128021217.
b) T. M. Mitchell, Machine Learning (1st ed.), McGraw Hill, 2017. ISBN 978-1259096952.

REFERENCE BOOKS/LEARNING RESOURCES:

a) Richard Golden, Statistical Machine Learning A Unified Framework (1 st ed.), unknown, 2020.
Dr. Tej Bahadur Chandra October 30, 2024 1
 Lecture - 1

 Background (Statistical Theory)

 Machine Learning

 Types of Machine Learning

 Supervised Learning

 Unsupervised Learning

 Reinforcement Learning

 Domains/Application Areas

Dr. Tej Bahadur Chandra October 30, 2024 2

Statistical Theory

10 Correct Answers

15 Incorrect Answers

3 Don’t know what's going on

Dr. Tej Bahadur Chandra October 30, 2024 3

Statistical Theory

Mathematical theories behind statistics

Statics rely heavily on differential and integral
calculus, linear algebra, and probability theory.

What the Statisticians do?

It is particularly concerned with determining how to
draw reliable conclusions from the given set of data.

Dr. Tej Bahadur Chandra October 30, 2024 4

Statistical Theory

Areas of Statistics

Descriptive Inferential
statistics statistics

• It describes the overall shape of the data by using • Tools that statisticians use to draw conclusions about
properties such as mean, variance, skewness, and the characteristics of a population.
kurtosis.
• Inferential statistics include linear regression analysis,
• The above statistics describe the overall structure analysis of variance (ANOVA), and null hypothesis
of the data. It forms the basis of making testing, etc.
predictions using inferential statistics.

Dr. Tej Bahadur Chandra October 30, 2024 5

Statistical Theory

Examples of Statistics

Investing average trading volume, week low, week high, etc.

Economics GDP, unemployment, inflation, etc.

human resources employee turnover, employee satisfaction, etc.

information technology bandwidth, network capabilities, hardware logistics, etc.

Dr. Tej Bahadur Chandra October 30, 2024 6

Machine Learning (ML)

• ML is a subset of artificial intelligence (AI) that uses

mathematical models of data to help a computer learn without
direct instruction.

• ML uses algorithms to identify patterns within data, and those

patterns are then used to create a data model that can make
predictions.

• The adaptability of ML makes it a great choice in dynamic

scenarios (where the data is always changing).

Dr. Tej Bahadur Chandra October 30, 2024 7

Machine Learning (ML): Why Machine Learning ?

Human can learn from experience ------- Can machine also learn from experience
??
• Humans have limitations (but one can scale-up the machines as
needed)

• Work 24x7

• Can perform complex computational task that human can not

• In emergency situations like pandemic, wars, etc.

Dr. Tej Bahadur Chandra October 30, 2024 8

Machine Learning (ML): Why Machine Learning ?

Problems Tra
in
Problem Solution

Encode the problem Solution

New Problem New Problem

Dr. Tej Bahadur Chandra October 30, 2024 9

Machine Learning (ML): Why Machine Learning ?

Dr. Tej Bahadur Chandra October 30, 2024 10

Machine Learning (ML): Why Machine Learning ?

Training Data

Class Labels

Dr. Tej Bahadur Chandra October 30, 2024 11

Machine Learning (ML):

The term machine learning was first introduced by Arthur

Samuel in 1959 . We can define it in a summarized way as:

Machine learning enables a machine to automatically learn from

data, improve performance from experiences, and predict things
without being explicitly programmed.

With the help of sample historical data, which is known as

training data , machine learning algorithms build a mathematical
model that helps in making predictions or decisions without
being explicitly programmed.

Machine learning brings computer science and statistics together

for creating predictive models.

Machine learning constructs or uses the algorithms that

learn from historical data.

Dr. Tej Bahadur Chandra October 30, 2024 12

Machine Learning (ML): Process

Dr. Tej Bahadur Chandra October 30, 2024 13

Machine Learning (ML): Types of Machine Learning

Unsupervised Learning
No labeled data is present, machine draw
inferences from datasets and assign them
class labels.

Supervised Learning Reinforcement Learning

The machine learns from training data The machine learns on its own, receiving
and labels and make predictions. rewards and punishments and determining
from these what it should do.

Dr. Tej Bahadur Chandra October 30, 2024 14

Machine Learning (ML): Supervised Learning

Supervised
Learning

Examples: Support vector machines (SVM), Neural networks, Naïve Bayes classifier, Decision trees, K-
Nearest neighbors (kNN)
Dr. Tej Bahadur Chandra October 30, 2024 15
Machine Learning (ML): Supervised Learning Real-Life Example

Train Model
Trained Model
Normal
Normal X-ray Images

Abnormal
- Preprocess
- Extract Features
- Select Optimal Features
Test Cases
- Perform Classification
X-ray Images of Pneumonia - Evaluate Model Performance

Dr. Tej Bahadur Chandra October 30, 2024 16

Machine Learning (ML): Supervised Learning

 Supervised learning is a type of machine learning method in which we provide sample labelled data to the
machine learning system in order to train it, and on that basis, it predicts the output.

 The system creates a model using labelled data to understand the datasets and learn about each data, once the
training and processing are done then we test the model by providing a sample data to check whether it is
predicting the exact output or not.

 The goal of supervised learning is to map input data with the output data.

 The supervised learning is based on supervision, and it is the same as when a student learns things in the
supervision of the teacher. The example of supervised learning is spam filtering.

 Supervised learning can be grouped further in two categories of algorithms: Classification and Regression

Dr. Tej Bahadur Chandra October 30, 2024 17

Machine Learning (ML): Unsupervised Learning

Unsupervised
Learning

Examples:
‒ K-means clustering,
‒ Fuzzy c-means clustering
‒ Hierarchical clustering
‒ Density based clustering

Dr. Tej Bahadur Chandra October 30, 2024 18

Machine Learning (ML): Unsupervised Learning

Dr. Tej Bahadur Chandra October 30, 2024 19

Machine Learning (ML): Unsupervised Learning

Input CXR image

Abnormal Patches

Performance Evaluation
Normal Patches
Segmented Extracting Image Patches
Lung Fields Image Patch Clustering

Patch Mapping on CXR images

Dr. Tej Bahadur Chandra October 30, 2024 20

 Machine Learning (ML): Reinforcement Learning

Reinforcement
Learning

Examples
‒ Markov Decision Process
‒ Q Learning

Dr. Tej Bahadur Chandra October 30, 2024 21

Machine Learning (ML): Reinforcement Learning

Dr. Tej Bahadur Chandra October 30, 2024 22

 Machine Learning Domains

Text Domain Signal Domain Image Domain

Data Science

Machine is
learning
trainedintoimage domain
analyze has several
the signals applications
from various like:
sources like:
Machine learning in text domain has several applications like:

FacialBrain
Recognition/
(EEG) Robotic
HeartNavigation
(ECG) Disease
AudioDetection
signals
Text Mining Natural Language Processing
Object Detection

Dr. Tej Bahadur Chandra October 30, 2024 23

 Machine Learning Process in image

Image Acquisition

Enhancement
(a) (b)

Segmentation

Feature Extraction

Feature Selection/
Reduction

Figure 7. Classifying the features by training model [22]

Classification
Figure
Figure 8. Trained
2. Image classification
Acquisition model
from various sources [22]
Figure 3. (a)
Figure 4.FigureImage with
6. Feature
(a) Complete low contrast
selection
chest [22] (b) Enhancement
and reduction
X-Ray Image, Image
(b) Segmented chest X-Ray Image

Figure 5. Extracting features from whole training image set [22]

Use the Model

Dr. Tej Bahadur Chandra October 30, 2024 24

Dr. Tej Bahadur Chandra October 30, 2024 25
 Machine Learning Applications

Virtual Personal Assistants Analyze huge data Health care sector Others

Google Assistant Weather Prediction Automated disease diagnosis

Space exploration program

Cortana on Windows 10
Share market price
Personalized medicine Google car

Jarvis Earthquake/ Tsunami

Prediction Smart Wearable's Internet of Things
Dr. Tej Bahadur Chandra October 30, 2024 26
Machine Learning Applications

Dr. Tej Bahadur Chandra October 30, 2024 27

 Machine Learning Applications

Future
of
Machine
Learning

Dr. Tej Bahadur Chandra October 30, 2024 28

Further Readings:
• Statistics, https://www.investopedia.com/terms/s/statistics.asp

• Machine Learning, https://www.ibm.com/cloud/learn/machine-learning

• What Is Machine Learning, https://www.expert.ai/blog/machine-learning-definition/

• Getting started with Machine Learning, https://www.geeksforgeeks.org/getting-started-machine-learning/

• Machine Learning- Sudeshna Sarkar,

https://www.youtube.com/watch?v=T3PsRW6wZSY&list=PLJ5C_6qdAvBGaabKHmVbtryZW9KpICiHC

Dr. Tej Bahadur Chandra October 30, 2024 29