Data Mining and Analytics

Lec.4.

© Dr. Magda Madbouly

Types of clustering:
They are different types of clustering methods,
including:
 Partitioning methods k-means & Medoids
 Hierarchical clustering

 Fuzzy clustering
 Density-based clustering
 Model-based clustering
Fuzzy cluster

© Dr. Magda Madbouly

Types of clustering :

 Hard clustering: Clusters do not overlap

element either belongs to cluster or it does not.
Each document belongs to exactly one cluster

 Soft clustering (Fuzzy) : Clusters may overlap.

A document can belong to more than one
cluster.
Fuzzy Logic and Fuzzy
Inference System
 Fuzzy Logic
 Fuzzy logic is an approach to computing based on "degrees of
truth" rather than the usual "true or false" (1 or 0) Boolean
logic on which the modern computer is based.
 First introduced by Dr. Lotfi Zadeh of UC Berkeley in the
1965's as a mean to model the uncertainty of natural language.
 Uses the mathematical theory of fuzzy sets
 Simulates the process of normal human reasoning
 Allows the computer to behave less precisely
 Decision making involves gray areas.
 fuzzy logic variables may have different degrees, called
grades of membership, that ranges in degree between 0
and 1.
Traditional Representation of Logic
Fuzzy Logic Representation
Foundation of fuzzy logic
 Fuzzy set
 Membership Function
 Logical operations
 If – then Rules

10
Crisp sets v.s. Fuzzy sets:
◦ The crisp set: is defined in such a way as to the individuals in
some given universe of discourse into two groups: members
and nonmembers.
 However, many classification concepts do not exhibit this
characteristic.
 For example, the set of tall people, expensive cars, or sunny
days.

◦ A fuzzy set: can be defined mathematically by assigning to

each possible individual in the universe of discourse a value
representing its grade of membership in the fuzzy set.
 For example: a fuzzy set representing our concept of sunny
might assign a degree of membership of 1 to a cloud cover
of 0%, 0.8 to a cloud cover of 20%, 0.4 to a cloud cover of
30%, and 0 to a cloud cover of 75%.
 Fuzzy Logic Example: Tallness

◼ The classical example in fuzzy sets is tall men. The

elements of the fuzzy set “tall men” are all men, but
their degrees of membership depend on their height.

You must be taller

than this line to be
considered “tall”

Crisp Set
1.0
Degree of membership

0.8
0.6
Short Average Tall
0.4
0.2
Crisp and fuzzy sets of “tall men”

Degree of
Crisp Sets
Membership
1.0
0.8

0.6

0.4

0.2

0.0
150 160 170 180 190 200 210
Height, cm
Degree of
Membership Fuzzy Sets
1.0

0.8

0.6
0.4

0.2

0.0
150 160 170 180 190 200 210
0.8

Crisp and fuzzy sets of “tall men”

0.6

0.4

0.2

0.0
150 160 170 180 190 200 210
Height, cm
Degree of
Membership Fuzzy Sets
1.0

0.8

0.6
0.4

0.2

0.0
150 160 170 180 190 200 210
Height, cm
Crisp and fuzzy sets of short, average and tall men
Degreeof Crisp Sets
Membership
1.0
0.8 Short Average Tall

0.6
0.4
0.2

0.0
150 160 170 180 190 200 210
Height, cm
Degreeof Fuzzy Sets
Membership
1.0

0.8
0.6 Short Average Tall

0.4
0.2

0.0
150 160 170 180 190 200 210
Fuzzy Logic Example: Tallness
Fuzzy Logic Example: Tallness
 Membership Functions

 A membership Function (MF) is a curve that

defines how each point in the input space is
mapped to a membership value (or degree of
membership) between 0 and 1.
Membership Functions
 There are different forms of membership functions
such as triangular, trapezoidal, or Gaussian. The
most common types of membership functions are
triangular and trapezoidal shapes. The type of the
membership function can be context dependent and
it is generally chosen arbitrarily according to the user
experience.
Membership Functions (cont.)
Membership Function for Fuzzy Set “Speed”
Confidence

1000

0
Speed 0 20 40 60 80

Slow
Medium
Fast

22
Membership Function for Fuzzy Set “Speed”
Confidence

1000

500

0
Speed 0 20 30 40 60 80

Slow
Medium
Fast

23
Advantages of Fuzzy Logic
 More natural to construct
 Easy to understand - Frees the imagination
 Provides flexibility
 More forgiving
 Shortens system development time
 Increases the system's maintainability
 Uses less expensive hardware
 Handles control or decision-making problems not easily
defined by mathematical models
 …more…
Fuzzy Inference System (FIS) = Expert
System + Fuzzy Logic
●A Fuzzy Inference System (FIS) is a way of
mapping an input space to an output space
using fuzzy logic

●FIS uses a collection of fuzzy membership

functions and rules, instead of Boolean
logic, to reason about data.
Fuzzy Inference System (FIS)…con.
● The rules in FIS (sometimes may be called as fuzzy
expert system) are fuzzy production rules of the
form:
− if p then q, where p and q are fuzzy statements.

● For example, in a fuzzy rule

−if x is low and y is high then z is medium.
−Here x is low; y is high; z is medium are fuzzy
statements; x and y are input variables; z is an
output variable, low, high, and medium are fuzzy
sets.
Cont…
●Most tools for working with fuzzy expert systems
allow more than one conclusion per rule.

●The functional operations in fuzzy expert system

proceed in the following steps.
−Fuzzification
−Fuzzy Inference (apply implication method)
Rule Evaluation
−Aggregation of all outputs
−Defuzzification
Structure of a Fuzzy Expert System
 IF service is poor or food is bad
Rule 1 THEN tip is low

Defuzzyfication
Fuzzyfication
Input 1
Service (0-10)
IF service is good Output
Rule 2 THEN tip is average Summation
Tip (5 - 25%)

Input 2
Food (0-10)
IF service is excellent or food is delicious
Rule 3 THEN tip is generous

Fuzzy Inferencing Process

Crisp Crisp
Fuzzification Inferencing Composition Defuzzification
Inputs Outputs

Membership Fuzzy Composition Defuzzification

functions rules heuristics heuristics
 Fuzzification
• The first step in fuzzy logic processing involves a domain
transformation called fuzzification,

• Crisp inputs are transformed into fuzzy inputs

• To transform crisp inputs into fuzzy inputs, member

functions are needed

30
 Fuzzification
● In the process of fuzzification, membership functions
defined on input variables are applied to their actual
values so that the degree of truth for each rule premise
can be determined.

● Fuzzy statements in the antecedent are resolved to a

degree of membership between 0 and 1.
−If there is only one part to the antecedent, then this is
the degree of support for the rule.
−If there are multiple parts to the antecedent, apply
fuzzy logic operators and resolve the antecedent to a
single number between 0 and 1.
Fuzzy Inferencing or Rule valuation
• In step two of fuzzy logic processing, the fuzzy
processor uses linguistic rules to determine what
control action should occur in response to a given set
of input values,

• Rule Evaluation, also referred to as Fuzzy Inference,

applies and evaluates the rules (fuzzy rules) to the
fuzzy inputs that were generated in the fuzzification
process.
32
 Fuzzy Inferencing….con
●In the process of inference
−Truth value for the premise of each rule is
computed and applied to the conclusion part of
each rule.
−This results in one fuzzy set to be assigned to
each output variable for each rule.

●The use of degree of support for the entire rule is

to shape the output fuzzy set.
 Aggregation of all outputs

●It is the process where the outputs of each

rule are combined into a single fuzzy set.
●The input of the aggregation process is the
list of truncated output functions returned
by the implication process for each rule.
●The output of the aggregation process is
one fuzzy set for each output variable.
 Aggregation of all outputs…con.
❑Here, all fuzzy sets assigned to each output variable
are combined together to form a single fuzzy set for
each output variable using a fuzzy aggregation
operator.
❑Some of the most commonly used aggregation
operators are
−the maximum : point-wise maximum over all of the
fuzzy sets
−the sum : point-wise sum over all of the fuzzy
−the probabilistic sum.
 Defuzzification
● In Defuzzificztion, the fuzzy output set is converted to a
crisp number.
● Some commonly used techniques are the centroid and
maximum methods.
−In the centroid method, the crisp value of the output
variable is computed by finding the variable value of
the centre of gravity of the membership function for
the fuzzy value.
−In the maximum method, one of the variable values at
which the fuzzy set has its maximum truth value is
chosen as the crisp value for the output variable.
 Fuzzy Rules
• Fuzzy rules are usually if-then statements that
describe the action to be taken in response to
various fuzzy inputs, see next slide for examples,

• Rules follow common sense behavior of the

system and are written in terms of the
membership function linguistic labels.
Fuzzy Rules (cont.) antecedent

IF soil is wet and temperature is hot

THEN watering duration is short.
consequent

IF the car is moving fast and

pavement is dry
THEN apply brakes hard.

IF shower water is too hot

THEN gently increase cold water flow.
Logical operations

39
 Example:
●We will explain these steps using an example
of Tipping Problem
●Two inputs : Quality of food and Service at a
restaurant rated at scale from 0-10
●One output: Amount of tip to be given
●Tip should reflect the quality of the food and
service.
●The tip might be in the range 5-15% of total
bill paid.
Rules for Tipping
● Let us consider the following three rules
− If service is poor or food is bad, then tip is cheap
− If service is good, then tip is average
− If service is excellent or food is delicious, then
tip is generous
● Input variables
− Service : represented by poor, good, excellent
− Food : represented by bad, delicious
● Output Variable:
− Tip : represented by cheap, average, generous
 The Reasoning Process for FIS
(the tipping example)
Example: What % tip to leave at a restaurant?

“Given the
quality of Rule 1
IF service is poor or food is bad
THEN tip is low
service and

Defuzzyfication
Fuzzyfication
Input 1
the food, Service (0-10)

how much Rule 2

IF service is good
THEN tip is average Summation Output
Tip (5 - 25%)
should I
Input 2
tip?” Food (0-10)
IF service is excellent or food is delicious
Rule 3 THEN tip is generous

Fuzzy Inferencing Process

Crisp Crisp
Fuzzification Inferencing Composition Defuzzification
Inputs Outputs

Membership Fuzzy Composition Defuzzification

functions rules heuristics heuristics
Antecedent for each rule
Rule's Conclusion
Aggregate Conclusions
Fuzzy Applications
 In Manufacturing and Management
◦ Space shuttle vehicle orbiting
◦ Regulation of water temperature in shower
heads
◦ Selection of stocks to purchase
◦ Inspection of beverage cans for printing
defects
◦ Matching of golf clubs to customers'
swings
◦ Risk assessment, project selection
Fuzzy Applications….con.

◦ Consumer products (air

conditioners, cameras,
dishwashers), …
◦ In Business
◦ Strategic planning
◦ Real estate appraisals and valuation
◦ Bond evaluation and portfolio
design, …