DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

III B. TECH II SEMESTER (R20)

MACHINE LEARNING (R2032051)

UNIT – I

SYLLABUS

INTRODUCTION

• ARTIFICAL INTELLIGENCE

• MACHINE LEARNING

• DEEP LEARNNG

• TPES OF MACHINE LEARNING SYSTEMS

• MAIN CHALLENGES OF MACHINE LEARNING

STATISTCAL LEARNING

• SUPER VISED AND UNSUPERVISED MACHINE LEARNING

• TRANING AND TEST LOSS

• ESTIMATING RISK STATISTICS

• SAMPLING DISTRIBUTION OF AN ESTIMATOR

• EMPIRICAL RISK MINIMIZATION

UNIT – I (MACHINE LEARNING) 1

 UNIT – I

INTRODUCTION TO MACHINE LEARNING

➔ Introduction to AI: -

The AI is branch of computer science by which we can create intelligent machine,

which can be have like a human & think like human & able to make decisions.

Artificial + Intelligence

Man made Thinking Power

• AI can create a machine with programmed algorithm. Which can work with own
intelligence.
• AI cannot need to use preprogram machine to do some work.
(or)

Definition of AI: -

AI is a branch of computer science which deals with helping machines finding

solutions to complex problems like a human.

Human beings
Convert
Algorithms
Implement
Machine
AI is not related to computer science. It is also related with maths, biology, Psychology, etc.,

Example: - 1. Automated Driving Systems (Self Driving Cars)

2. To calcite 100 students marks.

Human takes more time but computer can perform very fast.

Needs of AI: -

• AI can create software / devices which can solve real world problems vary easily&
accuracy.

Example: Health Issues, Supermarket, Traffic issues, etc.,

• AI can create your personal virtual assistant such as “siri”, “cortana”, “Google
Assistant”, “Gemini” etc.,

• AI can create robots. Its work just like a human.

Example: Sofhia, Alexa.

UNIT – I (MACHINE LEARNING) 2

Goals of AI:

1. Replicate human intel

2. Solve Knowledge

3. An intelligence connects of perception tasks.

4. Building a machine

Application of AI: -

Every branch of Science, Engineering and Technology shares the tools and techniques
available in the domains of AI.

1. Game Playing: -

• Game playing is one of the leading domains where AI has been applied with
great successful.

2. Expert Systems: -

• An expert system is a software that manipulated encoded knowledge to

problems in a specialized domain that normally requires human experts.

• An expert system is in AI program in which the system’s knowledge is obtain

from an expert source such that intelligence advice or intelligence decision in
solving a problem.

3. Natural Language Processing: - (NLP)

• NLP is a technique that builds ability in machines to ready & understand the
languages that humans speak.

4. Image Understanding: -

• Many of AI programs are engineered to solve some problems without humans.

• 2D array contains grey levels can be used to received digital images that
recognized by video camera.

5. Robotics: -

• AI is applied in robotics in order to see, hear & react to other sensor

simulations.

Example: Erica and Sophia they can behaves like humans.

6. Finance: -

• The banks we use AI to perform different operations.

✓ Organize operations (credit, debit)

✓ Invest in stocks

UNIT – I (MACHINE LEARNING) 3

 ✓ Finance institutions have used AI to detect changes or claims etc.,

7. Music: -

• Composition of songs, performance sounds and research in music.

8. Transportation: - Fuzzy logic controllers have been developed for automatic gear
box in auto mobiles.

Example: Audi TT (Tourist Trophy), Volkswagen, etc.,

9. Hospitals: -

• A medical clinic use AI system to organize bed schedule, make a staff rotation,
heart sound analysis identify tumors.

10. Computer Vision: -

• It is able to extract information from its vision computer vision plays its role
here to recognize the object as an image & identify the task.

• The image data can be in the form of picture, videos, multidimensional data
from a medical scanner or multiple cameras.

Advantages of AI: -

• High accuracy with less error.

• High speed

• High reliability

• Useful for risk area

• Digital Assistant

• Useful has a public utility

Disadvantages of AI: -

• High Cost

• No feelings / emotions

• No original creations

• Can’t think out of book

UNIT – I (MACHINE LEARNING) 4

➔ MACHINE LEARNING: -

• Machine learning is the branch of artificial

Intelligence concerned with the design and
development of algorithms.

• It allows the computer to behave in a way based on

the empirical data, such as from electronic sensors
or database.

• Machine learning is a field of studying that gives

computers a capability to learn without being
explicitly programmed.

Example: Online shopping

(Or)

• Machine learning is an sub field of AI that provides system that ability tool
automatically improve from experience without being explicitly programmed.

Features of Machine Learning:

• ML is used to detect various patterns in a given data set.

• It can learn from past data and improve performance perfectly.

• It is a data driven technology. ML is similar to data mining and it can work large
amount of data.

Types of ML:

There are three (3) types of Machine Learning:

1. Supervised Learning

2. Unsupervised Learning

3. Reinforcement learning

1) Supervised Learning: -

• In supervised learning we teach the machine using label data to predict

outcome for new unseen data.

• An external supervision is there.

• The labeled data means some input data is already tagged with current output.

• If any wrong data is given then send feed back given to machine.

• There are two (2) types of supervised learning algorithm

i. Regression ii Classification

UNIT – I (MACHINE LEARNING) 5

 i. Regression: It is used for prediction of continuous variables.

Example: Price, age, salary, wheather report, temperature, etc.,

ii. Classification: It is used for prediction of output variables.

Example: True or False, Male or Female, Positive or Negative, Yes or No,

etc.,

2) Unsupervised Learning:
Unsupervised Learning is a learning method with any supervision.

• The training is provided to the machine with the set of data that not been
classified and the algorithms needs to act on that data without any supervision.

• The input data is unlabeled data.

• No supervision and No Feedback.

• There are two (2) types of Unsupervised learning algorithms:

i. Clustering

ii. Association
i. Clustering: Grouping the objects into clusters.

ii. Association: To find out the relationship between two objects.

3) Reinforcement Learning:

• It is a feedback based learning approach. Here an agent learn the environment

and perform the action to get result of actions.

• The reinforcement learning is an example of semi - supervised learning,

because it works based on supervised data and unsupervised learning data.

• There are two (2) types of reinforcement learning:

i. Positive reinforcement learning

ii. Negative reinforcement learning

UNIT – I (MACHINE LEARNING) 6

Applications of Machine learning:-

 Machine learning is a buzzword for today's technology, and it is growing very rapidly
day by day.
 We are using machine learning in our daily life even without knowing it such as Google
Maps, Google assistant, Alexa, etc.
 Below are some most trending real-world applications of Machine Learning:

1. Image Recognition:

 Image recognition is one of the most common applications of machine learning.

 It is used to identify objects, persons, places, digital images, etc.

2. Speech Recognition:

 While using Google, we get an option of "Search by voice," it comes under speech
recognition, and it's a popular application of machine learning.
 Example: Google Assistant, Siri, Cortana, and Alexa

3. Traffic prediction:
 If we want to visit a new place, we take help of Google Maps, which shows us the
correct path with the shortest route and predicts the traffic conditions.
 It predicts the traffic conditions such as whether traffic is cleared, slow-moving, or
heavily congested with the help of two ways:
 Real Time location of the vehicle form Google Map app and sensors
 Average time has taken on past days at the same time.

4. Product recommendations:

 Machine learning is widely used by various e-commerce and entertainment

companies such as Amazon, Netflix, etc., for product recommendation to the
user.
5. Self-driving cars:
 One of the most exciting applications of machine learning is self-driving cars.
 Machine learning plays a significant role in self-driving cars.
 Tesla, the most popular car manufacturing company is working on self-driving
car.
 6. Virtual Personal Assistant:
 We have various virtual personal assistants such as Google
assistant, Alexa, Cortana, Siri.

7. Medical Diagnosis:

 In medical science, machine learning is used for diseases diagnoses.

 With this, medical technology is growing very fast and able to build 3D models that
can predict the exact position of lesions in the brain.

Advantages of Machine Learning:

 Easily identifies trends & patterns.

 Continuous improvements.
 No human interaction needed.
 Handling multi-dimensional data.
 Wide range of applicability.
Disadvantages of Machine Learning:
 Changing nature of jobs.
 Highly Expensive
 High error chances
 Results interpretations
 Time & Resources
 Data Acquisition
 DEEP LEARNING

 Deep learning is a synonym for deep structured hierarchy learning.

 It is sub field of machine learning where Supervised, Unsupervised & semi supervised
learning methods are adopted to learn from data representation.
 It contains set of algorithms which have been inspired from the structure & function of
human brain.
 Deep learning is used to feature extraction.

 Deep learning will do imitate the human brain.

 Deep learning is a sub field of ML for learning feature hierarchies that are actually on
Artifical neural networks.
 Deep learning is implemented by the help of deep networks. Which is nothing but neural
networks with multiple hidden layers.
Neural networks:- Neural network is also known as Artifical neural networks (ANN) /
Simulated Neural network.

 Most popular techniques used for implementing deep architecture are:

1. Artifical Neural Networks (or) Multilayer Perception (or) Feed forward N/W.
2. Convolutional Neural Network (or) deep forward N/W.
3. Recurrent Neural Network
1) Artifical Neural Networks:-
 Artifical Neural Network (ANN) is inspired by the working of a human brain.
 The human brain has neurons interconnected to one another, ANN also have neurons
that are interconnected to one another in various layers of the networks.
 These neurons are known as nodes (or) units.
 Neural network are a set of algorithm that tries to recognize the patterns, relationships
& information from the data.
 Artifical Neural Network is also called as Multilayer perception or Feed forward
network.
 Components of ANN: -
A simple neural network consists of three (3) components:
i. Input layer
ii. Hidden layer (Middle layer)
iii. Output layer

Input Layer: -
Input nodes receives inputs/ information from
the outside world.

Hidden Layer: -
 Hidden layer is set of neurons where all the computations are performed on the
input data.
 Any number of hidden layers are used in neural network. But the simple network
consists only one hidden layer.

Output Layer: -
 The output layer is the output / Conclusion derived from all the computations
performed.
 There can be single or multiple nodes in the output layer.
 If we give binary classification problem the output node is 1.
 If we are give input as multi class classification, the output nodes are more than
one.
2) Convolutional Neural Network: -
 Convolutional Neural Network (CNN) is similar to a multilayer perceptron
network.
 The difference is network learns structures & purpose they are mostly used.
 CNN can be applied in domain of computer vision problems / image processing
etc.,
 If the given image is color then use three (3) as size or gray image use 1.
 The CNN used to reduce the number of parameters & speed up the training of the
model.
3) Recurrent Neural Network:- (RNN)
 The RNN is used to solve the problem occurs in CNN.
 Problems with CNN. We have input & It produce one output. In CNN it doesn’t
maintain internal memory.
  Every input is independent from other input.
 CNN cannot solve problems like sentences, stock price & Time Series.
 In RNN, each neuron or unit of RNN uses its internal memory to maintain
information about the previous output.
 When it is required to predict the next word of sentence. The previous word are
required.
 The RNN is used to member the previous output.
 The most & main important features of RNN is Hidden layers.
 The Hidden layers have a memory which remembers all the information what has
be calculated.
 It can also reduce the complexity of parameters.
Applications of Deep Learning:-
 Automatic colouring of black & white images.
 Automatically adding sounds to silent movies.
 Automatic machine translation.
 Object classification & detection in photographs.
 Automatic text generations.
 Automatic handwriting generation.
 Recommendation engine.
 Chat bots & Speech recognization.
 Image recognization, IOT, Computer Vision.
 MAIN CHALLENGES OF MACHINE LEARNING
There are a lot of challenges that machine learning professionals face to inculcate ML skills
and create an application from scratch.

1. Poor Quality of Data:-

 Data plays a significant role in the machine learning process.
 One of the significant issues that machine learning professionals face is the
absence of good quality data. Unclean and noisy data can make the whole
process extremely exhausting.
 We don’t want our algorithm to make inaccurate or faulty predictions. Hence
the quality of data is essential to enhance the output.
2. Underfitting of Training Data:-
 This process occurs when data is unable to establish an accurate relationship
between input and output variables.
 To overcome this issue:
i. Maximize the training time
ii. Enhance the complexity of the model
iii. Add more features to the data
iv. Reduce regular parameters
v. Increasing the training time of model
3. Overfitting of Training data:-
 An overfitting is occurs when the low bias & high variance. That means the
model training is very well but testing is very low.
 Training is properly done & but Testing is not properly done.
How to avoid the overfitting:-
a) Cross Validation
b) Training with more data
c) Removing features
d) Early stopping of training
 Both overfitting & underfitting cause the degraded performance of the machine
learning model.

4. Lack of Training Data:-

 The most important task you need to do in the machine learning process is to
train the data to achieve an accurate output.
  Less amount training data will produce inaccurate result.
5. Slow Implementation:-
 This is one of the common issues faced by machine learning professionals.
 The machine learning models are highly efficient in providing accurate results.
 But it takes more time to provide accurate results.
Difference Between Artificial Intelligence vs Machine Learning vs Deep
Learning:-

Artificial Intelligence Machine Learning Deep Learning

AI stands for Artificial ML stands for Machine DL stands for Deep Learning, and
Intelligence, and is Learning, and is the study is the study that makes use of
basically the that uses statistical methods Neural Networks (similar to
study/process which enabling machines to neurons present in human brain)
enables machines to improve with experience. to imitate functionality just like a
mimic human behaviour human brain.
through particular
algorithm.
AI is the broader family
consisting of ML and DL ML is the subset of AI. DL is the subset of ML.
as it’s components.

AI is a computer DL is a ML algorithm that uses

ML is an AI algorithm
algorithm which exhibits deep(more than one layer) neural
which allows system to learn
intelligence through networks to analyze data and
from data.
decision making. provide output accordingly.

If you have a clear idea

If you are clear about the math
about the logic(math)
involved in it but don’t have idea
involved in behind and you
Search Trees and much about the features, so you break
can visualize the complex
complex math is involved the complex functionalities into
functionalities like K-Mean,
in AI. linear/lower dimension features
Support Vector Machines,
by adding more layers, then it
etc., then it defines the ML
defines the DL aspect.
aspect.

The aim is to basically The aim is to increase It attains the highest rank in terms
increase chances of accuracy not caring much of accuracy when it is trained
success and not accuracy. about the success ratio. with large amount of data.

Three broad Three broad categories/types DL can be considered as neural

categories/types Of AI Of ML are: Supervised networks with a large number of
are: Artificial Narrow Learning, Unsupervised parameters layers lying in one of
Intelligence (ANI), Learning and Reinforcement the four fundamental network
Artificial General Learning architectures: Unsupervised Pre-
Intelligence (AGI) and trained Networks, Convolutional
Artificial Super Neural Networks, Recurrent
Intelligence (ASI) Neural Networks and Recursive
Neural Networks

The efficiency Of AI is Less efficient than DL as it

basically the efficiency can’t work for longer More powerful than ML as it can
provided by ML and DL dimensions or higher amount easily work for larger sets of data.
respectively. of data.

Examples of AI:
Examples of ML: Virtual
Google’s AI-Powered Examples of DL: Sentiment
Personal Assistants: Siri,
Predictions, Ridesharing based news aggregation, Image
Alexa, Google, etc., Email
Apps Like Uber and Lyft, analysis and caption generation,
Spam and Malware
Commercial Flights Use etc.
Filtering.
an AI Autopilot, etc.

DL networks consist of multiple

In reinforcement learning,
layers of interconnected neurons
AI systems can be rule- the algorithm learns by trial
that process data in a hierarchical
based, knowledge-based, and error, receiving
manner, allowing them to learn
or data-driven. feedback in the form of
increasingly complex
rewards or punishments.
representations of the data.
 STATISTICAL LEARNING
INTRODUCTION:

• Structuring & Visualizing data are important aspect of data science.

• Statistical Learning: - In ML, to interpret the model & quantify the uncertainty in
the data to this analysis is usually refered to as statistical learning (Mathematical
calculations).
• There are two (2) major goals for modelling data: -
i. To accurately predict some future quantify of interest, given some observed
data.
ii. To discover or interesting patterns in the data.

To achieve these goals one must rely on knowledge from three (3) important pillars of the
mathematical science:

1. Function Approximation
2. Optimization
3. Probability & Statistics
1) Functional Approximation: -
• A mathematical function is used to represent the relationship between the
variables.
Example: f (x) = x → y
y determines x
• As a data scientist, you need to understand how data to be represented using least
mount of computer processing time & memory.
Understanding data

This process is Mathematical Function

called as Function
Approximation. Computer Program

It takes less time & memory

2) Optimization: -
• The optimization is used to find the best possible model in given set of
mathematical models.
• It requires knowledge of optimization algorithms & efficient computer coding or
programming.
Example: Search procedures
Loss functions These are Optimization
Algorithms.
Convex Programming
 3) Probability & Statistics: -
• The probability & Statistics knowledge needed to fit or train algorithm &
generate a model.
➔ TRAINING AND TESTING LOSS: -
In machine learning training & testing loss are used to evaluated to
performance of a model.
Training Loss: -
Training Loss is the error rate on the training data during the training process.
The goal of the training process is to minimize the loss, that means the model is
learning to make accurate predictions on the training data.
Testing Loss: -
• Testing Loss is the error rate on the testing data, which is a separate dataset
that is not used during the training process.
• This is used to evaluate the generalization performance of the model, meaning
how well it can make accurate predictions on new unseen data.
The goal of ML is to minimize both the training loss and testing loss, while also
preventing overfitting.
This is achieved by using techniques such as regularization, early stopping and cross
validation.

➔ ESTIMATING RISK STATISTICS: -

Estimation: - Estimation means find a values that is nearest to the original values but not
exact value.

• Risk management plays a crucial role in machine learning of building effective and
reliable.

• At a high level the statistical risk measures the quality of the learning algorithm.

• In ML risks occurs when training data is more & testing data is less otherwise
training data is less & testing data is more.

Techniques of Risk Statistics:

1. Empirical Risk Minimization [ERM]

2. Regularization Risk Minimization [RRM]

3. Cross Validation

4. Loss Functions
1) Empirical Risk Minimization: [ERM]

• As we don’t know parameter “p” we can’t compute the true risk but we can
compute the empirical risk based on given input data (or) training data (xi, yi.
where i = 1, 2, 3, ……….., n).
• ERM compute only Temporarily risk not True risk.
• It is a statistical learning algorithm used to find the optional solution out of a
set of possible solution based on sample data.

• It is flexible and easy to implement.

• Risk occurs when there is a over fitting and underfitting in data models.
Overfitting: It occurs when there is a more training data.
Underfitting: It occurs when there is a more testing data than training data.
2) Regularization Risk Minimization: [RRM]
• It is a machine learning technique that uses regularization to reduce overfitting
in data models.
• By using set of methods (maths, statistics, computer science, et.,) that trade a
small decrease in training accuracy for better generalizability.
• There are three (3) primary regularization methods:
a) L1 Regularization
b) L2 Regularization
c) Elastic net
A. L1 Regularization: The process of adding sum of absolute values to the loss
function. It is also called “LASSO”.
B. L2 Regularization: The process of adding sum of squared values to the loss
function. It is also called “RIDGE”.
C. Elastic Net: The combination of L1 Regularization (LASSO) and L2
Regularization (RIDGE) is called Elastic Net.
3) Cross Validation:
In machine Language it is not suitable to use same data for both training and
testing.

Training data Testing data

75% 25%
Cross validation is the mechanism to identify how much data is used for
training & how much data is used for testing to get an accurate results.
 Block is used for Testing
1 2 3 4

3 Block are used for Training

4) LOSS Function:
• Loss function quantifies the difference between the predefined values and
actual values.
• It is also known as Error function (or) Lost function.
• There are two (2) types of LOSS functions
i. Classification Model
ii. Regression Model

➔ SAMPLING DISTRIBUTION OF AN ESTIMATOR: -

Population: Population is the totality of statistical data forming a subject of
investigation.
Size: The number of observations in the populations is defined to be the size of
population. It may be finite or infinite.
⸪ Size of the population denoted by “N”.
Size of the sample denote by “n”.
Sample: A sample is a part of whole selected object from the whole population.
Sampling: The process of selecting the sample is called sampling.
Sampling Distribution:
• A sampling distribution is a probability of a statistic obtained from a larger
number of samples drawn from a specific population.
• Sample statistic only estimate population parameters means, standard deviation.
• The sample mean will be different to the population mean.
• A researcher will never known the exact amount of sampling. But using sampling
distribution they can estimate the sampling error.
i. Number of samples with replacement = Nn

ii. Number of samples without replacement = NCn

• Samples are classified in two (2) ways:

1. Large Sample: n > 30
2. Small Sample: n < = 30
Parameters: Population related mean, standard deviation, variance, median etc are
called parameters (σ, µ, σ2).
Statistics: Sample related means, standard deviation, variance, median etc are called
statistics.
Empirical Risk Minimization (ERM)

Empirical Risk Minimization is a fundamental principle in statistical learning theory used to

find a predictive model by minimizing the error or loss on a given training dataset. The
objective is to choose a hypothesis from a hypothesis set that minimizes the empirical risk,
which is the average loss on the training data.

Key Concepts

Hypothesis Set: A set of possible functions (models) from which we aim to choose the best
one.

Loss Function: Measures the error between the predicted and actual values. Common
examples include mean squared error for regression and cross-entropy for classification.

Empirical Risk: The average loss over the training sample. If the training set consists of

𝑁 samples and the loss function is 𝐿, the empirical risk is given by:

𝑅𝑒𝑚𝑝(ℎ)=(1/𝑁) ∑ 𝐿(ℎ(𝑥𝑖),𝑦𝑖)

where ℎ(𝑥𝑖) is the prediction of the model ℎ for the input 𝑥𝑖, and 𝑦𝑖 is the actual output.

Example
Let's consider a simple example with linear regression.

1.Data: Suppose we have a training set with three data points:

{(1,2),(2,2.5),(3,3.5)}

2.Hypothesis Set: We consider linear functions of the form ℎ(𝑥)=𝑤𝑥+𝑏.

3.Loss Function: Mean Squared Error (MSE):

𝐿(ℎ(𝑥𝑖),𝑦𝑖)=(ℎ(𝑥𝑖)−𝑦𝑖)2
4.Empirical Risk Calculation:

Suppose our model is ℎ(𝑥)=1.2𝑥+0.5.

Why ERM?
Practical: Provides a concrete criterion to select the best model based on observed data.
Theoretical Foundation: Forms the basis for many machine learning algorithms, ensuring
they generalize well to unseen data.
Flexibility: Can be applied to various types of loss functions and hypothesis sets.
ERM is a core concept in machine learning and helps in building models that perform well on
training data, hoping they will also perform well on new, unseen data.
Advantages of Empirical Risk Minimization

Foundation of Supervised Learning: ERM is a core principle in supervised learning,

providing a systematic approach to model training and evaluation.

Flexibility: ERM can be applied to various types of models and loss functions, making it
versatile across different problem domains.

Optimization Framework: Provides a clear objective for optimization: minimizing the

empirical risk (average loss on training data).

Generalization: When combined with techniques like cross-validation and regularization,

ERM helps models generalize well to unseen data.

Adaptability: Can be adapted to different learning algorithms, from linear regression to deep
neural networks, by choosing appropriate loss functions and hypothesis sets.

Disadvantages of Empirical Risk Minimization

Overfitting: Focusing solely on minimizing empirical risk may lead to overfitting, where the
model performs well on training data but poorly on new data.

Dependence on Training Data Quality: ERM relies heavily on the quality and
representativeness of the training data. If the data is biased or noisy, the model's performance
will suffer.

Computational Complexity: For large datasets or complex models, minimizing empirical

risk can be computationally intensive, requiring significant resources.

Choice of Loss Function: The effectiveness of ERM depends on the choice of loss function,
which can be challenging to select and may not always capture the true cost of prediction
errors.

Local Minima: In non-convex optimization problems, ERM can get stuck in local minima,
leading to suboptimal models.