Scribd
Upload
11 views3 pages

UNIT 4 - EM Alg

Uploaded by

esmritypoudel
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
11 views3 pages

UNIT 4 - EM Alg

Uploaded by

esmritypoudel
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Expectation-Maximization Algorithm

1) Observable variables that can be directly observed or measured.


2) Latent variables are those variables that are not directly observable
and that can only be inferred indirectly through a mathematical model
from the values of the other observed variables.
a) Hidden variables reflect the fact that the variables are meaningful,
but not observable.
b) Hypothetical variables correspond to abstract concepts, like
categories, behavioural or mental states, or data structures.

Ex: Quality of Life is a latent variable in Economics field.

In the real-world applications of machine learning, it is very common that


there are many relevant features available for learning but only a small
subset of them are observable.

So, for the variables which are sometimes observable and sometimes not,
then we can use the instances when that variable is visible is observed for
the purpose of learning and then predict its value in the instances when it
is not observable.

It was explained, proposed and given its name in a paper published in


1977 by Arthur Dempster, Nan Laird, and Donald Rubin.

This algorithm is actually at the base of many unsupervised clustering


algorithms in the field of machine learning.

EM Algorithm is used to

a) find the local maximum likelihood (MLE) parameters or MAP


Parameters of a statistical model in the cases where latent
variables are involved
b) Determine values of missing or incomplete data.
In summary, the expectation maximization algorithm alternates between
the steps of

Step 1) guessing a probability distribution over completions of missing


data given the current model (known as the E-step) and then

Step 2) re-estimating the model parameters using these completions


(known as the M-step).

Update
Variables

Update
Hypothesis

Algorithm:
1. Given a set of incomplete data, consider a set of starting
parameters.
2. Expectation step (E – step): Using the observed available
data of the dataset, estimate (guess) the values of the missing
data.
3. Maximization step (M – step): Complete data generated
after the expectation (E) step is used in order to update the
parameters. It is basically used to update the hypothesis.
4. Repeat step 2 and step 3 until convergence.
Usage of EM algorithm –
• It can be used to fill the missing data in a sample.
• It can be used as the basis of unsupervised learning of clusters.
• It is also used to compute the Gaussian density of a function.
• It can be used for the purpose of estimating the parameters of
Hidden Markov Model (HMM).
• It can be used for discovering the values of latent variables.
• EM algorithm finds plenty of use in natural language processing
(NLP), computer vision, and quantitative genetics.
• Other important applications of the EM algorithm include image
reconstruction in the field of medicine and structural engineering.

Advantages of EM algorithm –
• It is always guaranteed that likelihood will increase with each
iteration.
• The E-step and M-step are often pretty easy for many problems
in terms of implementation.
• Solutions to the M-steps often exist in the closed form.

Disadvantages of EM algorithm –
• It has slow convergence.
• It makes convergence to the local optima only.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy