Scribd
Upload
37 views11 pages

Perceptrons

This document provides information about a video course on machine learning taught by Professor Carl Gustaf Jansson of KTH. It includes an outline of the topics to be covered in Week 6, which focuses on machine learning based on artificial neural networks. Specifically, Lecture 6.2 will cover perceptrons, the simplest type of feedforward neural network. Perceptrons are linear classifiers that use a threshold function to map an input vector to a binary output. The document then provides details on the structure and learning rule of perceptrons, including examples of how they are used for binary classification.

Uploaded by

polinati.vinesh2023
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
37 views11 pages

Perceptrons

This document provides information about a video course on machine learning taught by Professor Carl Gustaf Jansson of KTH. It includes an outline of the topics to be covered in Week 6, which focuses on machine learning based on artificial neural networks. Specifically, Lecture 6.2 will cover perceptrons, the simplest type of feedforward neural network. Perceptrons are linear classifiers that use a threshold function to map an input vector to a binary output. The document then provides details on the structure and learning rule of perceptrons, including examples of how they are used for binary classification.

Uploaded by

polinati.vinesh2023
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 11

NPTEL

Video Course on Machine Learning

Professor Carl Gustaf Jansson, KTH

Week 6 Machine Learning based


on Artificial Neural Networks

Video 6.2 Perceptrons


Structure of Lectures in week 6
L1 Fundamentals of
We are her now Neural Networks

McCulloch and Pitts

Supervised learning L2 Perceptrons Linear L6 Hebbian Learning and


- classification classification Associative Memory
- regression
L3 och L4 Feed forward multiple layer Reinforcement
networks and Backpropagation learning Unsupervised
learning

L5 Recurrent Neural Sequence and L7 Hopfield Networks and


Perception Networks (RNN) temporal data Boltzman Machines

L8 Convolutional Neural
Networks (CNN)
L9 Deep Learning and Development of
recent developments the ANN field
L10 Tutorial on assignments
Perceptron
The perceptron is an algorithm for learning a linear binary
classifier in the form of a threshold function that maps its
input X, a real-valued input vector, to a single output
binary value Y.

In the context of neural networks, a perceptron is a single


artificial neuron using the Heaviside step function as the
activation function.

As a linear classifier, the single-layer perceptron is the


simplest feedforward neural network.

Y j= if W * X = Sum Wi j * Xi j > then 1 otherwise= 0.


i=1 .. N
W is a real valued vector, j = # of iteration

The binary value of Y is used to classify a data-item as either a


positive or a negative instance.
Intuitions for Perceptron Learning
A perceptron is a liner classifier that tries to find a
hyperplane in the space spanned by its input.

The adjustments of weights corresponds to changes in


orientation of the hyper plane.The adjustments of the bias
corresponds to changes in the hyper plan intercepts with
the axes of the space.

For a two dimensional space the perceptron corresponds a


line in the plane, the orientation of which is decided by the
weights and the intercepts of which are decided by the bias.

An optimal perceptron finds the hyperplane (e.g. line) that


best separates the data-items.
Which problems can a perceptron solve?
= Binary Classification for linearly separable
instance spaces

A multi class classification problem can be reduced to


multiple binary classification problems and can therefor
indirectly be solved by parallel multiple perceptrons.

As a perceptron only has binary output, regression cannot


be performed. A trick can be to discretize the regression
problem into number ranges and view it as a multiclass
classification problem (see above).
Then non-linear case

The perceptron learning algorithm does


not terminate if the learning set is
not linearly separable.

The most famous example of the


perceptron's inability to solve problems
with linearly non-separable vectors is the
Boolean XOR problem.
Perceptron Learning Rule
A simpler version of the delta rule for ONE neuron with specific properties.

Wi j+1= Wi j+ a * (Tj-Yj)* Xi j, a = learning rate


, parameter.
delta weight
T
Yj = Xi j Wi j, i = input, j =iteration, T = target

The treshold can also be learned by transforming it to a bias = - threshold and completing the
dataitems with an extra input x0j=1 and a corresponding weight w0j=bias.

Steps in a Learning Procedure


Initialize the weights and the learning rate.
Weights may be initialized to 0 or to a small random value.

For each example j in the dataset, perform the following steps until total training set error ceases to improve:
• calculate the output
• calculate the new weights
Perceptron

x1 w1 Y j =1 if W * X+B = (Sum wi j * xi j) >


i=1 .. n
x2 w2 𝜃 Y otherwise Y= 0.
Wi j+1= Wi j+ a * (Tj-Yj)* Xi j a=0.2
delta weight
xn wn

X11=0.8 x21=0.3 T1= 0 W11=0.4 W21=-0.2


0.8 0.4

0.1 Y1=1

0.3 -0.2
0.4*0.8 -0.2*0.3 =0 .26 > 0.1 -> Y1 = 1

W12=0.4+0.2*(0-1)*0.8 = 0.24 W22= -0.2 +0.2*(0-1) *0.3=-0.26


8
Perceptron with threshold handled as a bias that can be learned

x1 w1 Y j =1 if W * X+B = (Sum wi j * xi j) > 0


i= .. n
x2 w2 Y otherwise Y= 0.
Wi j+1= Wi j+ a * (Tj-Yj)* Xi j a=0.2
delta weight
xn wn

X0=1 w0=
X11=0.8 x21=0.3 T1= 0 W11=0.4 W21=-0.2 W01=-0.1
0.8 0.4

Y1=1

0.3 -0.2
0.4*0.8 + -0.2*0.3 - 0.1 =0 .16 > 0 -> Y1 = 1

1 -0.1 W12=0.4+0.2*(0-1)*0.8 = 0.24 W22= -0.2 +0.2*(0-1) *0.3=-0.26


W02=-0.1 + 0.2*(0-1)*1=-0.3 9
Example
Assume a perceptron:
Training set Y j =1 if W * X = (Sum wi j * xi j) > 0
• with 3 inputs plus 1 for bias i= .. n
• where net = 0 0 1 -> 0 otherwise Y= 0.
x0*w0+x1*w1+x2*w2+x3*w3 1 1 1 -> 1 Wi j+1= Wi j+ a * (Tj-Yj)* Xi j
1 0 1 -> 1
• that outputs 1 if net > 0, else 0 delta weight
• with a learning rate a=1
• initial weights all 0

Instance Target Weight Vector Net Output Delta Weight


001 1 0 0000 0 0 0 0 0 0
111 1 1 0000 0 0 1 1 1 1
101 1 1 1111 3 1 0 0 0 0
NPTEL

Video Course on Machine Learning

Professor Carl Gustaf Jansson, KTH

Thanks for your attention!

The next lecture 6.3 will be on the topic:

Model of a Neuron in an ANN

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy