Scribd
Upload
78 views12 pages

Lecture 03 Perceptron PDF

The document discusses the perceptron, which is a single artificial neuron that can classify input vectors into two categories based on a weighted sum of the inputs and a threshold. It learns to do this classification by adjusting the weights according to a learning rule. The perceptron has limitations in that it can only learn linearly separable functions and cannot model XOR. Beyond perceptrons, neural networks need to learn representations and features from data rather than just weights, use recurrent connections for sequences, and be able to learn without a teacher's supervision.

Uploaded by

Colin Nyamiaka
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
78 views12 pages

Lecture 03 Perceptron PDF

The document discusses the perceptron, which is a single artificial neuron that can classify input vectors into two categories based on a weighted sum of the inputs and a threshold. It learns to do this classification by adjusting the weights according to a learning rule. The perceptron has limitations in that it can only learn linearly separable functions and cannot model XOR. Beyond perceptrons, neural networks need to learn representations and features from data rather than just weights, use recurrent connections for sequences, and be able to learn without a teacher's supervision.

Uploaded by

Colin Nyamiaka
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

COM 424 E: NEURAL

NETWORKS

Lecture 03: The Perceptron


The Perceptron
• A single artificial neuron that computes its
weighted input and uses a threshold
activation function.
• It effectively separates the input space into
two categories by
the hyperplane:
wTx + b = 0

2
Perceptron: Applications


The perceptron is used for classification:
classify correctly a set of examples into
one of the two classes C1, C2:


If the output of the perceptron is +1 then the
input is assigned to class C1

If the output is -1 then the input is
assigned to C2
3
The Perceptron
The Perceptron can automatically learn to categorise or classify
input vectors into types.

It obeyed the following rule:


If the sum of the weighted inputs
exceeds a threshold, output 1, else
Σxi wi output -1.

1 if Σ inputi * weighti > threshold


-1 if Σ inputi * weighti < threshold
weights
inputs

sum output
*

4
Perceptron: Classification

The equation below describes a hyperplane in the input


space. This hyperplane is used to separate the two
classes C1 and C2
decision
region for C1
x2 w1x1 + w2x2 + b > 0

decision
boundary C1
decision x1
C2
region for C2
w1x1 + w2x2 + b <= 0 w1x1 + w2x2 + b = 0
5
Perceptron: Limitations

• The perceptron can only model linearly


separable functions.
• The perceptron can be used to model the
following Boolean functions:
• AND
• OR
• COMPLEMENT
But it cannot model the XOR. Why?

6
Perceptron: Limitations

The XOR is not linear separable


It is impossible to separate the classes C1 and
C2 with only one line

x2 C1 C2
1 1 -1

0 -1 1 C1

0 1 x1 7
Perceptron: Learning Algorithm

• Variables and parameters


x(n) = input vector
= [+1, x1(n), x2(n), …, xm(n)]T
w(n) = weight vector
= [b(n), w1(n), w2(n), …, wm(n)]T
b(n) = bias
y(n) = actual response
d(n) = desired response
 = learning rate parameter
8
The fixed-increment learning algorithm
• Initialization: set w(0) =0
• Activation: activate perceptron by applying input
example (vector x(n) and desired response d(n))
• Compute actual response of perceptron:
y(n) = sgn[wT(n)x(n)]
• Adapt weight vector: if d(n) and y(n) are different
then
w(n + 1) = w(n) +  [d(n)-y(n)]x(n)
+1 if x(n)  C1
Where d(n) =
-1 if x(n)  C2

• Continuation: increment time step n by 1 and go


to Activation step 9
Example

Consider a training set {C1 , C2 } where:


C1 = {(1,1), (1, -1), (0, -1)} elements of class 1
C2 = {(-1,-1), (-1,1), (0,1)} elements of class -1

Use the perceptron learning algorithm to classify these


examples.

w(0) = [1, 0, 0]T

10
Example

x2
1
- - +
Decision boundary:
C2 2x1 - x2 = 0

-1 1/2 1 x1

- + -1 + C1

11
Beyond perceptrons
• Need to learn the features, not just how to weight them to
make a decision.
• Need to make use of recurrent connections, especially for
modeling sequences.
» The network needs a memory (in the
activities) for events that happened some time
ago, and we cannot easily put an upper bound
on this time.
• Need to learn representations without a teacher.
» This makes it much harder to define what the
goal of learning is.

12

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy