TABLE OF CONTENTS

Unit No Contents Page No

1 INTRODUCTION TO MACNINE LEARNING 1
1.1 Learning problem 1
1.1.1 Well-Defined Learning Problem 2
1.2 Concept Learning in ML 2
1.3 Perspectives in Machine Learning 5
1.4 Issues in Machine Learning 6
1.5 Version Spaces 12
1.5.1 The List-Then-Elimination Algorithm 13
1.5.2 Theorem: Version Space representation 14
1.6 Candidate Elimination Learning Algorithm 15
1.7 Inductive Bias in Machine Learning 17
1.7.1 An Unbiased Learner 18
1.8 Decision Tree Classification Algorithm 22
1.8.1 Decision Tree algorithm – Working 24
1.8.2 Classification of Machine Learning 27
1.9 Heuristic techniques 35
1.9.1 Heuristic search techniques 37
1.9.2 Types of heuristics 41

2 NEURAL NETWORKS AND GENETIC 44

ALGORITHMS 44
2.1 Neural Networks ( ANN)
2.1.1 Advantages of Artificial Neural Network 48
(ANN) 49
2.1.2 Artificial Neural Networks – Working 51
2.1.3 Types of Artificial Neural Network 52
2.2 Problems in Machine Learning 58
2.3 Perceptron in Machine Learning
2.3.1 Basic Components of Perceptron 59
2.3.2 Perceptron – Working 60
2.4 Multilayer Networks and Back Propagation 65
Algorithms
2.4.1 Backpropagation 66
2.4.2 Backpropagation Algorithm – Working 66
2.4.3 Types of Backpropagation Networks 68
2.5 Neural Network - Advances 69
2.6 Genetic Algorithm in Machine Learning 71
2.6.1 Genetic Algorithm- Working 72
2.6.2 Workflow of a Simple Genetic 76
Algorithm
2.7 Hypothesis Search – ML 78
2.8 Genetic programming 82
 2.9 Machine Learning Model Evaluation 85
2.10 Evaluating Multiclass Classifier Predictions 88
3 BAYESIAN AND COMPUTATIONAL 91
LEARNING
3.1 Bayes Theorem 91
3.1.1 Introduction to Bayes Theorem 91
3.1.2 Prerequisites for Bayes Theorem 93
3.2 Concept Learning 98
3.3 Maximum Likelihood Estimation 101
3.3.1 Working of Maximum Likelihood 102
Estimation
3.4 Minimum Description Length Principle 107
3.5 Bayes Optimal Classifier 109
3.6 Gibbs Algorithm 112
3.7 Naïve Bayes Classifier 116
3.7.1 Types of Naïve Bayes Model 121
3.8 Bayesian Belief Network 121
3.9 Expectation-Maximization Algorithm 124
3.10 Probability Theory Basics 127
3.11 Sample Complexity 131
3.11.1 Sample Complexity – Finite Hypothesis 132
3.11.2 Sample Complexity – Infinite Hypothesis 133
3.12 Mistake Bound Model 134

4 INSTANT BASED LEARNING 138

4.1 Introduction 138
4.2 K-Nearest Neighbor Learning 139
4.2.1 Classification of Algorithm 139
4.2.2 Distance-Weighted Algorithm 141
4.3 Locally Weighted Regression 142
4.3.1 Regression Analysis in ML 144
4.3.2 Types of Regression 147
4.4 Radial Basis Functions 154
4.5 Case-Based Reasoning 156
4.5.1 Reinforcement Learning 158
4.6 Q Learning 163
4.7 Evaluating Hypothesis 168
4.8 Basics of Sampling Theory 171

5 ADVANCED LEARNING 175

5.1 Learning Set of Rules 175
5.2 Sequential Covering Algorithms 177
5.3 Learning Rule Sets : Summary 183
5.4 Learning First Order Rules 187
5.5 Induction on Inverted Deduction 190
5.6 Inverting Resolution 192
5.7 Analytical Learning 195
5.8 Learning Perfect Domain:PROLOG-EB 201
5.9 Training Example - Explanation 203
5.9.1 Analyze the Explanation 205
5.10 Refine the Current Hypothesis 211
5.11 Deductive Learning 215
5.12 FOCL Algorithm 216
5.13 Reinforcement Learning 221
5.14 The Learning Task 224
5.15 Q Learning 231
5.16 Temporal Difference (TD) 236