Scribd
Upload
313 views22 pages

Ensemble Learning

Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem and their predictions are combined. It contains base learners that are usually weak learners. Ensembles reduce bias and variance by combining multiple trained models. Boosting is an ensemble meta-algorithm that focuses new learners on examples others got wrong to convert weak learners into strong ones like Adaboost. Decision lists are a type of logical expression used in some ensemble methods.

Uploaded by

Aleli Pamplona
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
313 views22 pages

Ensemble Learning

Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem and their predictions are combined. It contains base learners that are usually weak learners. Ensembles reduce bias and variance by combining multiple trained models. Boosting is an ensemble meta-algorithm that focuses new learners on examples others got wrong to convert weak learners into strong ones like Adaboost. Decision lists are a type of logical expression used in some ensemble methods.

Uploaded by

Aleli Pamplona
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 22

ENSEMBLE

LEARNING
ENSEMBLE LEARNING

• machine learning paradigm where multiple


learners are trained to solve the same problem.
• the idea is to select a whole collection, or
ensemble of hypotheses from hypothesis space
and combine their predictions.
• contains a number of learners which are usually
called base learners or also referred as weak
learners.
• Ensembles use multiple trained models.
ENSEMBLE LEARNING

• BIAS - measure of how flexible the model is.


- error from erroneous assumptions in the
learning algorithm. High bias can cause an algorithm to
miss the relevant relations between features and target
outputs (under fitting).
• VARIANCE - error from sensitivity to small
fluctuations in the training set. High variance can cause
an algorithm to model the random noise in the training
data, rather than the intended outputs (over fitting).
ENSEMBLE LEARNING
APPLICATION OF
ENSEMBLE LEARNING
Weak
learning
Weak learning

• consists of a learning algorithm that classifies the


input data better than random.
• an error is weak if its error is high, but the error
cannot be higher than 0.5
• ε < 0.5
• if an error is not very love, it’s a weak learner.
Weighted
training
set
Weighted
training set

• In such a training set, each example has


an associated weight wj > 0.
• The higher the weight, the higher is the
importance attached during the learning
of a hypothesis.
Weighted
training set
boosting
boosting

• machine learning ensemble meta-


algorithm for primarily reducing bias, and
also variance in supervised learning, and a
family of machine learning algorithms
that convert weak learners to strong ones.
boosting

• Focus new experts on examples that others get


wrong.
• Train experts sequentially.
• Errors of early experts indicate the “hard”
examples.
• Focus later classifier on getting these examples.
• Combine the whole set in the end.
• Convert many “weak” learners into complex
classifier.
Adaboost
(Adaptive boosting)
Adaboost
• Any learning algorithm that returns
hypotheses that are probably
approximately correct is called a PAC-
learning algorithm.
Decision
list
Decision list
• Logical expression of a restricted form.
• Consists of a series of tests, each of which is a
conjunction of literal.
• If a test succeeds when applied to an example
description, the decision list specifies the value to be
returned.
• If the test fails, processing continues with the next test
in the list.
• Decision lists resemble decision trees, but their
overall structure is simpler. In contrast, the individual
tests are more complex.
Decision list
Decision list
Decision list
references

• https://www.youtube.com/watch?v=nelJ3svz0_o
• https://www.youtube.com/watch?v=UHBmv7qCey4
• https://en.wikipedia.org/wiki/Boosting_(machine_learning)
• Artificial Intelligence A Modern Approach, Second
Edition (Stuart J. Russell and Peter Norvig )

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy