Scribd
Upload
109 views13 pages

Restricted Boltzmann Machines (RBMS)

Restricted Boltzmann Machines (RBMs) are stochastic neural networks used for tasks like dimensionality reduction, classification, and collaborative filtering. An RBM has visible and hidden layers fully connected by weights, but no intra-layer connections. It learns features from input data through updating weights to minimize the difference between real and reconstructed inputs. RBMs have been applied successfully to tasks like handwritten digit recognition and recommender systems.

Uploaded by

groverwalia
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
109 views13 pages

Restricted Boltzmann Machines (RBMS)

Restricted Boltzmann Machines (RBMs) are stochastic neural networks used for tasks like dimensionality reduction, classification, and collaborative filtering. An RBM has visible and hidden layers fully connected by weights, but no intra-layer connections. It learns features from input data through updating weights to minimize the difference between real and reconstructed inputs. RBMs have been applied successfully to tasks like handwritten digit recognition and recommender systems.

Uploaded by

groverwalia
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

Boltzmann Machine

• each nodes connects to every


other node in the network

• useful for factory safety when


an unusual state occurs to
alert the workers (the state
might not have occurred
previously).
Restricted Boltzmann Machine (RBM)
• RBMs
• models probability
• feature extraction
• Dimensionality reduction
• perform unsupervised pretraining of the rst layer based on the input data
• They are feed-forward networks in which data is fed through them in one
direction with two biases rather than one bias as in traditional back
propagation feed-forward networks.
fi
Restricted Boltzmann Machine (RBM)
• The “restricted” part of the name
“Restricted Boltzmann Machines” means
that connections between nodes of the
same layer are prohibited (e.g., there are
no visible-visible or hidden-hidden
connections along which signal passes)

• " A network of symmetrically connected,


neuron-like units that make stochastic
decisions about whether to be on or o ."

• RBM is a Stochastic Neural Network


which means that each neuron will have
random behavior when activated.
ff
Restricted Boltzmann Machine (RBM)
• Parts of RBM
• Visible units
• Hidden units
• Weights
• Visible bias units
• Hidden bias units
Restricted Boltzmann Machine (RBM)
• Visible and hidden layers
• every single node of the input (visible) layer is connected by weights to every
single node of the hidden layer, but no two nodes of the same layer are connected

• The second layer is known as the “hidden” layer. Hidden units are feature
detectors, learning features from the input data

• Each layer has a bias unit with its state always set to on.
• Each node performs computation based on the input to the node and outputs a
result based on a stochastic decision whether or not to transmit data through an
activation.

• The activation computation is based on weights on the connections and the input
values. The initial weights are randomly generated.
Restricted Boltzmann Machine (RBM)
• Connections and weights
• All connections are visible-hidden; none are visible-visible or hidden-
hidden.

• Those circles, or nodes, act like human neurons. They are decision
units.

• They make decisions about whether to be on or o through acts of


computation. “On” means that they pass a signal further through the
net; “o ” means that they don’t.

• ON is valuable and OFF signals may be due to a noisy input.


ff
ff
Restricted Boltzmann Machine (RBM)
• Hidden bias and visible bias
• This is what makes RBMs di erent from autoencoders.
• The hidden bias RBM produce the activation on the forward pass.
• The visible bias helps RBM to reconstruct the input during a backward
pass.

• The reconstructed input is always di erent from the actual input.


• As there are no connections among the visible units and therefore, no
way of transferring information among themselves.
ff
ff
Training RBM
• Forward Pass
• There can be multiple inputs. All inputs are multiplied by the weights and then added to the
bias. Then the result is passed through sigmoid activation function.

• Reconstruction Phase
• It is like the forward pass but in the opposite direction.
• h1: hidden layer vector, v0: input vector, W: weight matrix, a: hidden layer bias vector, S:
sigmoid activation function, b: visible layer bias

• The di erence (v0-v1) can be considered as the reconstruction error that we need to reduce
in subsequent steps of the training process. The weights are adjusted in each iteration to
minimize this error.
ff
Training RBM
Training RBM
• Suppose we have two normal
distributions,

• one from the input data (p(x))


• one from the reconstructed input (q(x)).
• The di erence between these two
distributions is the error and the goal is
to minimize it.

• KL-divergence (Kullback-Leibler
divergence) measures the non-
overlapping areas under the two
graphs.
ff
Case Study on RBM
• MNIST (Mixed National Institute of
Standards and Technology)

• 60000 records and 10000 test


records

• Cross entropy
• Cross means comparison
between two distributions

• entropy means uncertainty


(where the points fall on a normal
distribution curve if the variance is
wide spread).
Case Study on RBM
Applications of RBM
• Recommender System:
• Amazon product suggestions
• Net ix movie recommendations
• Handwritten digit recognition
• Criminal evidence,
• O ce computerization,
• Check (Cheque) veri cation,
• Data entry applications
• Vehicle number plate identi cation
ffi
fl
fi
fi

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy