MACHINE

LEARNING
ALGORITHMS :
Machine learning algorithms are computational models that allow
computers to understand patterns and forecast or make judgments based
on data without the need for explicit programming. These algorithms
form the foundation of modern artificial intelligence and are used in a
wide range of applications, including image and speech recognition,
natural language processing, recommendation systems, fraud detection,
autonomous cars etc.

TYPES OF MACHINE LEARNING ALGORITHMS:

• Supervised Learning

• Unsupervised Learning
 • Reinforcement Learning

SUPERVISED MACHINE LEARNING:

 1. Linear Model:

Regression

• Ordinary Least Square Regression

• Simple Linear Regression

• Multiple Linear Regression

• Polynomial Regression

• Orthogonal Matching Pursuit (OMP)

• Bayesian Regression

• Quantile Regression

• Isotonic regression

• Stepwise regression

• Least-angle regression (LARS)

Classification:

• Logistic Regression

• Sigmoid & Softmax functions

Regularization:

• Lasso (L1 Regularization)

 • Ridge (L2 Regularization)

• Ridge regression

• Ridge Classifier

• Elastic Net

• LARS Lasso

2. K-Nearest Neighbors (KNN):

• Brute Force Algorithms

• Ball Tree and KD Tree Algorithms

• K-Nearest Neighbors (KNN) Classifier

• K-Nearest Neighbors (KNN) Regressor

3. Support Vector Machines:

• Support Vector Machines Classifier

• Support Vector Machines Regressor

• Different Kernel functions in SVM

4. Stochastic Gradient Descent

• Stochastic Gradient Descent Classifier

• Stochastic Gradient Descent Regressor

 • Different Loss functions in SGD

5. Decision Tree:

• Decision Tree Algorithms

• Iterative Dichotomiser 3 (ID3) Algorithms

• C5. Algorithms

• Classification and Regression Trees Algorithms

• Decision Tree Classifier

• Decision Tree Regressor

6. Ensemble Learning:

Bagging (Bootstrap Aggregating)

• Random Forest

• Extra Trees

Boosting

• AdaBoost

• XGBoost

• CatBoost

• Gradient Boosting Machines (GBM)

 • LightGBM

Stacking

7. Generative Model

Naive Bayes

• Gaussian Naive Bayes

• Multinomial Naive Bayes

• Bernoulli Naive Bayes

Gaussian Processes

• Gaussian Process Regression (GPR)

• Gaussian Process Classification (GPC)

Gaussian Discriminant Analysis

• Linear Discriminant Analysis (LDA)

• Quadratic Discriminant Analysis (QDA)

Bayesian Belief Networks

Hidden Markov Models (HMMs)

8. Time Series Forecasting:

Time Series Visualization and Analysis:

• Time Series Components: Trend, Seasonality, and Noise

• Time Series Decomposition Techniques

• Seasonal Adjustment and Differencing

• Autocorrelation and Partial Autocorrelation Functions

• Stationarity

• Augmented Dickey-Fuller Test

• Seasonal Decomposition of Time Series (STL Decomposition)

• Box-Jenkins Methodology for ARIMA Models

Time Series Forecasting Algorithms:

• Moving Average (MA) and Weighted Moving Average

• Exponential Smoothing Methods (Simple, Double, and Triple)

• Autoregressive (AR) Models

• Moving Average (MA) Models

• Autoregressive Integrated Moving Average (ARIMA) Models

• Seasonal Decomposition of Time Series by Loess (STL)

• Seasonal Autoregressive Integrated Moving Average (SARIMA)

Models
 • ARIMAX and SARIMAX Models

9. Supervised Dimensionality Reduction Technique:

• Linear Discriminant Analysis (LDA)

Some of the Supervised Machine Learning Algorithms

can be used used for both Classification & Regression with a little bit of
modification

Multiclass and multioutput algorithms:

• Multiclass classification

• OneVsRest Classifier

• Multilabel classification

• Multioutput Regression

Metrics for Classification & Regression Algorithms:

Regression Metrics:

• Mean squared error (MSE)

• Root mean squared error (RMSE)

• Mean absolute error (MAE)

• R-squared
• Adjusted R-squared

Classification Metrics:

• Confusion matrix

• Accuracy

• Precision

• Recall

• Specificity

• F1 score

• Area under the ROC curve (AUC-ROC)

Probability calibration

• Calibration curves

• Calibrating a classifier

Cross Validation Technique:

• K-fold cross-validation

• Stratified k-Fold Cross-Validation

• Leave-one-out cross-validation

• Shuffle Split Cross-Validation

 • Time Series Cross-Validation

Optimization Technique:

Gradient Descent

• Stochastic Gradient Descent

• Mini-batch Gradient Descent

• Momentum-based Gradient Descent

Newton-Based Optimization Techniques

• Newton’s algorithm

• Quasi-Newton Methods (BFGS, L-BFGS)

• Conjugate Gradient

Local Search Optimization Techniques

• Hill Climbing

• Tabu Search

UNSUPERVISED MACHINE LEARNING:

Clustering

Centroid-based Methods

• K-Means clustering

• K-Means++ clustering

• K-Mode clustering

• Fuzzy C-Means (FCM) Clustering

Distribution-based Methods

• Gaussian mixture models (GMMs)

• Expectation-Maximization Algorithms
 • Dirichlet process mixture models (DPMMs)

Connectivity based methods

• Hierarchical clustering

• Agglomerative Clustering

• Divisive clustering

• Affinity propagation

Density Based methods

• DBSCAN (Density-Based Spatial Clustering of Applications with

Noise)

• OPTICS (Ordering Points To Identify the Clustering Structure)

Association Rule Mining

• Apriori algorithm

• FP-Growth (Frequent Pattern-Growth)

• ECLAT (Equivalence Class Clustering and bottom-up Lattice

Traversal)

Anomaly Detection:

• Z-Score
 • Local Outlier Factor (LOF)

• Isolation Forest

Dimensionality Reduction Technique:

• Principal Component Analysis (PCA)

• t-distributed Stochastic Neighbor Embedding (t-SNE)

• Non-negative Matrix Factorization (NMF)

• Independent Component Analysis (ICA)

• Factor Analysis

• Latent Dirichlet Allocation (LDA)

• Isomap

• Locally Linear Embedding (LLE)

• Latent Semantic Analysis (LSA)

REINFORCEMENT LEARNING:
Model-Based Methods:

• Markov decision processes (MDPs)

• Bellman equation

• Value iteration algorithm

• Monte Carlo Tree Search

Model-Free Methods:

Value-Based Methods:

• Q-Learning

• SARSA

• Monte Carlo Methods

 Policy-based Methods:

• REINFORCE Algorithm

• Actor-Critic Algorithm

Actor-Critic Methods

• Asynchronous Advantage Actor-Critic (A3C)