Q1.

A) Differentiate between Supervised and Unsupervised Learning.

B) Describe any two real-world applications of clustering.

A) Difference between Supervised and Unsupervised Learning:

Feature Supervised Learning Unsupervised Learning

A type of learning where the model is trained A type of learning where the model
Definition on a labeled dataset (input and correct finds patterns from data without labeled
output given). outputs.

Labeled Data Required Not required

Goal Predict output from input Discover hidden patterns or groups

Examples Classification, Regression Clustering, Association

Use Case Customer Segmentation based on

Email Spam Detection (Spam/Not Spam)
Example buying behavior

B) Two Real-World Applications of Clustering:

1. Customer Segmentation in Marketing:

o Companies use clustering to group customers based on behavior, preferences, or

spending habits.

o This helps in sending personalized marketing messages.

2. Image Segmentation in Computer Vision:

o Clustering is used to divide an image into meaningful parts.

o For example, separating sky, trees, and buildings in a satellite image.

Q2. Explain the DBSCAN clustering algorithm with an illustration.

✅ Definition of DBSCAN:

DBSCAN stands for Density-Based Spatial Clustering of Applications with Noise.

It is a popular unsupervised learning algorithm used for clustering data based on density.

✅ Key Concepts:

 Epsilon (ε): The maximum distance between two points to be considered neighbors.
  MinPts: Minimum number of points required to form a dense region (cluster).

 Core Point: A point with at least MinPts within its ε-radius.

 Border Point: A point that is within the ε-radius of a core point but has fewer than MinPts
neighbors.

 Noise Point (Outlier): A point that is neither a core point nor a border point.

✅ Working Steps of DBSCAN:

1. Choose ε and MinPts values.

2. For each point:

o If it has at least MinPts points within ε → mark as core point and start a new cluster.

3. Expand the cluster:

o Add all directly density-reachable points (within ε of a core point).

o Then add all density-connected points (reachable through other core points).

4. Repeat until all points are classified as:

o Core, Border, or Noise.

✅ Illustration:

Assume we have points scattered in 2D space:

 Points in dense regions (crowded areas) will be grouped as clusters.

 Points lying far from any group will be labeled as noise.

Example:

Let’s say ε = 2 units, MinPts = 3.

 Point A has 4 neighbors within 2 units → Core point.

 Point B has 2 neighbors, but one is a core → Border point.

 Point C has 1 neighbor only → Noise.

✅ Advantages of DBSCAN:

 Can find arbitrarily shaped clusters.

 Can detect noise and outliers.

 Doesn’t require specifying number of clusters in advance.

✅ Limitations:

 Not good when clusters have varying densities.

 Choosing a proper ε and MinPts is tricky.

Q3A. What is Association Rule Mining? Explain in detail.

✅ Definition:

Association Rule Mining is a data mining technique used to find interesting relationships (patterns)
among items in large datasets.
It is mainly used in market basket analysis to understand what items are frequently bought together.

✅ Basic Terms:

 Itemset: A collection of one or more items.

 Transaction: A record of items bought together (e.g., one customer's shopping cart).

 Rule: A → B, meaning if a customer buys A, they are likely to buy B.

✅ Purpose:

To generate rules like:

{Bread, Butter} → {Jam}

Which means: if a customer buys Bread and Butter, they are likely to buy Jam too.

✅ Important Metrics:

1. Support:

o How often the itemset appears in the dataset.

o Formula:

Support(A→B) = Transactions containing A and B / Total Transactions

2. Confidence:

o How often B appears in transactions that contain A.

o Formula:

Confidence(A→B )= Transactions containing A and B / Transactions containing A

3. Lift (optional for exams):

 o Measures the strength of a rule over random chance.

o Lift > 1 means a strong rule.

✅ Steps in Association Rule Mining:

1. Find all frequent itemsets that meet the minimum support.

2. Generate strong rules from these itemsets using minimum confidence.

✅ Example:

Let’s say we have 5 transactions:

TID Items Bought

1 Milk, Bread

2 Milk, Diaper, Beer

3 Milk, Bread, Diaper

4 Bread, Butter

5 Milk, Bread, Diaper

Rule: Milk → Diaper

 Support = 3/5 = 0.6 (3 transactions have both Milk and Diaper)

 Confidence = 3/4 = 0.75 (4 transactions have Milk, and in 3 of them Diaper is also present)

Q3B. Define Support and Confidence with examples.

✅ 1. Support:

Definition:
Support tells us how frequently an item or itemset appears in the total transactions.
It helps identify frequent patterns in the data.

Formula:

Support(A→B) = Transactions containing A and B / Total Transactions

Example:
Suppose there are 5 transactions:
TID Items Bought

1 Milk, Bread

2 Milk, Diaper, Beer

3 Milk, Bread, Diaper

4 Bread, Butter

5 Milk, Bread, Diaper

Rule: Milk → Diaper

Transactions containing both Milk and Diaper = 3
Total transactions = 5

Support=3/5=0.6 or 60%

So, the rule Milk → Diaper appears in 60% of all transactions.

✅ 2. Confidence:

Definition:
Confidence tells how often item B appears in transactions that contain item A.
It measures the strength of the rule.

Formula:

Confidence(A→B) = Transactions containing A and B / Transactions containing A

Example (continued):
Transactions containing Milk = 4
Transactions containing both Milk and Diaper = 3

Confidence=3/4=0.75 or 75%

So, in 75% of the cases where Milk is bought, Diaper is also bought.

Q4. Describe the Apriori algorithm with an example dataset.

✅ Definition:

The Apriori algorithm is a classic algorithm used for Association Rule Mining.
It is used to find frequent itemsets and then generate association rules from those itemsets.

✅ Key Idea of Apriori:

 An itemset is frequent only if all of its subsets are also frequent.

  This is called the Apriori Principle.

✅ Steps in the Apriori Algorithm:

1. Set Minimum Support (min_sup).

2. Generate 1-itemsets (single items) with support ≥ min_sup.

3. Generate 2-itemsets, 3-itemsets, and so on, by:

o Joining frequent itemsets.

o Pruning itemsets whose subsets are not frequent.

4. Continue until no more frequent itemsets can be generated.

5. From the frequent itemsets, generate strong association rules using minimum confidence.

✅ Example Dataset:

TID Items Bought

1 Milk, Bread

2 Milk, Diaper, Beer

3 Milk, Bread, Diaper

4 Bread, Butter

5 Milk, Bread, Diaper

Let’s take min_sup = 2 (i.e., itemset must appear in at least 2 transactions).

✅ Step-by-Step Apriori:

1-itemsets:

 Milk → 4 times ✅

 Bread → 4 times ✅

 Diaper → 3 times ✅

 Beer → 1 time ❌

 Butter → 1 time ❌

Frequent 1-itemsets: Milk, Bread, Diaper

2-itemsets:

 Milk, Bread → 3 times ✅

  Milk, Diaper → 3 times ✅

 Bread, Diaper → 2 times ✅

Frequent 2-itemsets:
{Milk, Bread}, {Milk, Diaper}, {Bread, Diaper}

3-itemset:

 Milk, Bread, Diaper → 2 times ✅

Frequent 3-itemset: {Milk, Bread, Diaper}

✅ Final Output:

Frequent Itemsets:

 1-itemsets: Milk, Bread, Diaper

 2-itemsets: {Milk, Bread}, {Milk, Diaper}, {Bread, Diaper}

 3-itemset: {Milk, Bread, Diaper}

From these, rules like:

 Milk → Bread

 Milk & Diaper → Bread

...can be generated based on confidence values.

Q5A. Define clustering and mention different types of clustering techniques.

✅ Definition of Clustering:

Clustering is an unsupervised learning technique in which data points are grouped into clusters
based on similarity.
The goal is to group similar data points together and separate dissimilar ones.

Each group is called a cluster, and data points in the same cluster have similar properties.

✅ Example:

If we have data about customer purchases, clustering can help identify customer groups such as:

 Customers who buy sports goods,

 Customers who prefer electronics, etc.

✅ Types of Clustering Techniques:

1. Partitioning Methods:

o Divide the dataset into k groups or clusters.

o Each data point belongs to exactly one cluster.

o Example: K-Means, K-Medoids

2. Hierarchical Clustering:

o Builds a tree-like structure of clusters (dendrogram).

o Two types:

 Agglomerative (Bottom-Up): Start with individual points and merge.

 Divisive (Top-Down): Start with one big cluster and divide.

o No need to specify number of clusters.

3. Density-Based Methods:

o Form clusters based on dense areas of data points.

o Can find arbitrary-shaped clusters.

o Good at handling noise.

o Example: DBSCAN

4. Grid-Based Methods:

o The data space is divided into a grid of cells.

o Clusters are formed from dense cells.

o Example: STING (Statistical Information Grid)

5. Model-Based Clustering:

o Assume a model (like Gaussian Mixture) for each cluster.

o Use probability/statistics to assign data points.

o Example: EM (Expectation-Maximization)

✅ Summary Table:

Type Description Example

Partitioning Fixed number of clusters K-Means

Hierarchical Tree structure of clusters Agglomerative

Type Description Example

Density-Based Based on data density DBSCAN

Grid-Based Data space divided into grids STING

Model-Based Uses statistical models EM Algorithm

Q5B. Describe Partitioning methods with example.

✅ Definition of Partitioning Methods:

Partitioning methods are clustering techniques that:

 Divide a dataset into a fixed number of non-overlapping clusters.

 Each data point belongs to exactly one cluster.

 Aim to maximize intra-cluster similarity and minimize inter-cluster similarity.

✅ How it Works:

1. Choose the number of clusters k.

2. Assign each data point to one of the k clusters.

3. Use an objective function (like distance) to improve cluster assignments.

4. Repeat the steps until clusters become stable (converge).

✅ Popular Partitioning Algorithms:

1. K-Means

2. K-Medoids

✅ Example: K-Means Clustering (Partitioning Method)

Let’s say we have this dataset of students with their Math and Science scores:

Student Math Science

A 85 90

B 82 88

C 30 40
Student Math Science

D 25 38

E 70 65

We want to divide them into k = 2 clusters.

Steps in K-Means:

1. Initialize two centroids randomly.

2. Assign each student to the nearest centroid (cluster).

3. Recalculate new centroids.

4. Repeat steps 2 and 3 until no more changes.

✅ Output:

 Cluster 1: Students A, B (high scorers)

 Cluster 2: Students C, D (low scorers)

 Student E may fall into the cluster where it's closest based on average.

✅ Advantages of Partitioning Methods:

 Simple and efficient for large datasets.

 Easy to implement.

✅ Disadvantages:

 Need to specify k in advance.

 Sensitive to initial cluster positions.

 Not suitable for non-spherical clusters or noisy data.

Q6. Explain the K-Medoids algorithm and how it differs from K-Means.

✅ What is K-Medoids?

K-Medoids is a partitioning-based clustering algorithm similar to K-Means, but instead of using the
mean (centroid) of data points, it uses medoids (actual data points) as the center of a cluster.
✅ Definition of Medoid:

A medoid is the most centrally located point in a cluster (i.e., the point with the minimum total
distance to all other points in the same cluster).

✅ Steps in K-Medoids Algorithm:

1. Choose k initial medoids randomly from the dataset.

2. Assign each data point to the nearest medoid using a distance metric (e.g., Manhattan or
Euclidean).

3. For each cluster, try replacing the medoid with a non-medoid point and check if the overall
cost (total distance) reduces.

4. If yes, update the medoid.

5. Repeat the process until medoids do not change.

✅ Example:

Let’s assume we have these 1D data points:

[1, 2, 3, 10, 11, 12]
Let k = 2

 Initial Medoids: 2 and 11

 Cluster 1: [1, 2, 3] → Medoid = 2

 Cluster 2: [10, 11, 12] → Medoid = 11

Resulting clusters are grouped around actual data points (2 and 11).

✅ Difference between K-Means and K-Medoids:

Feature K-Means K-Medoids

Center Type Centroid (mean of points) Medoid (real data point)

Sensitive to Outliers Yes (mean is affected) No (medoid is more robust)

Cost Function Sum of squared distances Sum of pairwise distances

Output Clusters May include non-data centers Always data points as centers

✅ Advantages of K-Medoids:

 More robust to noise and outliers.

 Works well with categorical data and non-Euclidean distances.

✅ Disadvantages:

 Slower than K-Means for large datasets.

 More complex due to the pairwise distance calculations.

Q7A. Discuss the concept of finding patterns using Association Rules.

✅ What are Association Rules?

Association Rules are a data mining technique used to find interesting patterns or relationships
between items in large datasets, especially in market basket analysis.

These rules help identify if-then relationships:

 If item A is bought, then item B is also likely to be bought.

✅ Purpose:

To uncover hidden patterns in transaction data to support:

 Business decisions

 Product recommendations

 Cross-selling strategies

✅ Structure of an Association Rule:

If (Antecedent) → Then (Consequent)

Example:

 If a customer buys bread and butter → Then they also buy milk.

This can be written as:

{Bread, Butter} → {Milk}

✅ Important Metrics to Find Patterns:

1. Support:
How often the itemset appears in the dataset.

o Example: If 2 out of 10 transactions include {Bread, Milk}, then

Support = 2/10 = 0.2 (20%)
 2. Confidence:
How often the rule has been found to be true.

o Example: Out of 5 times bread was bought, if 4 times milk was also bought,
Confidence = 4/5 = 0.8 (80%)

3. Lift:
How much more likely item B is purchased when item A is purchased.

o If Lift > 1 → Strong association.

✅ Steps to Find Patterns:

1. Collect transaction data.

2. Identify frequent itemsets (groups of items often bought together).

3. Generate association rules from frequent itemsets.

4. Calculate support, confidence, and lift.

5. Filter rules based on thresholds.

✅ Applications:

 Market Basket Analysis (Retail)

 Recommender Systems (Amazon, Netflix)

 Web Usage Mining (Pages clicked together)

 Fraud Detection (Unusual spending patterns)

✅ Summary Example:

Transaction ID Items Bought

1 Milk, Bread

2 Milk, Bread, Butter

3 Bread, Butter

4 Milk, Bread, Butter

From the above:

 Frequent itemset: {Milk, Bread}

 Rule: {Milk} → {Bread} has high support and confidence.

Q7B. Explain the Apriori principle used in association rule mining.

✅ What is the Apriori Principle?

The Apriori principle is a key concept in association rule mining. It helps in finding frequent itemsets
(combinations of items that appear together frequently) efficiently.

The principle is based on the idea that:

 If an itemset is frequent, then all of its subsets must also be frequent.

How it Works:

 The Apriori algorithm works by first identifying individual items (1-itemsets) that appear
frequently in the dataset.

 Then, it uses these frequent 1-itemsets to generate larger itemsets (2-itemsets, 3-itemsets,
etc.), filtering out itemsets that do not meet the minimum support threshold.

 This process continues until no further frequent itemsets can be found.

✅ Steps in the Apriori Algorithm:

1. Find frequent 1-itemsets:

Identify items that appear frequently in the dataset.

2. Generate candidate itemsets:

Use the frequent 1-itemsets to generate candidate 2-itemsets.

3. Prune non-frequent itemsets:

Remove itemsets that do not meet the minimum support.

4. Repeat:
Continue the process for larger itemsets (3-itemsets, 4-itemsets) using the frequent itemsets
found in the previous step.

5. Generate association rules:

From the frequent itemsets, generate association rules with high confidence and lift.

✅ Example of the Apriori Algorithm:

Consider the following transactions:

Transaction ID Items Bought

T1 Milk, Bread, Butter

T2 Milk, Bread

T3 Milk, Butter

T4 Bread, Butter

T5 Milk, Bread, Butter

Transaction ID Items Bought

Let the minimum support be 60% (i.e., at least 3 out of 5 transactions).

1. Frequent 1-itemsets:

o {Milk}, {Bread}, {Butter} (Support of each is 60%)

2. Generate candidate 2-itemsets:

o {Milk, Bread}, {Milk, Butter}, {Bread, Butter}

3. Prune non-frequent 2-itemsets:

o All 2-itemsets are frequent, as their support is above 60%.

4. Generate association rules from {Milk, Bread}:

o {Milk} → {Bread} (Confidence: 3/4 = 75%)

o {Bread} → {Milk} (Confidence: 3/4 = 75%)

✅ Advantages of the Apriori Algorithm:

 Efficiency: It reduces the search space by eliminating itemsets that are not frequent.

 Simplicity: Easy to understand and implement.

✅ Disadvantages of the Apriori Algorithm:

 Combinatorial Explosion: As the size of the itemset increases, the number of candidate
itemsets grows exponentially.

 Slow for Large Datasets: It can be computationally expensive for large datasets.

✅ Applications of Apriori:

 Market Basket Analysis: Identifying products that are often bought together (e.g., bread and
butter).

 Recommendation Systems: Recommending products based on previous customer

purchases.

 Fraud Detection: Identifying unusual patterns of behavior or transactions.

Q8. Demonstrate the process of building association rules from a transaction dataset.

✅ Steps to Build Association Rules from a Transaction Dataset:

To build association rules, we follow a systematic process that includes identifying frequent itemsets
and then generating association rules. Here's how to do it step-by-step:

✅ Step 1: Collect Transaction Data

Let's start with a small dataset of transactions:

Transaction ID Items Bought

T1 Milk, Bread, Butter

T2 Milk, Bread

T3 Milk, Butter

T4 Bread, Butter

T5 Milk, Bread, Butter

✅ Step 2: Set the Minimum Support and Confidence

 Minimum Support: Defines how frequently an itemset should appear in the dataset to be
considered frequent. Let's assume Support ≥ 60% (at least 3 transactions).

 Minimum Confidence: Defines how often an association rule holds true. Let's assume
Confidence ≥ 75%.

✅ Step 3: Find Frequent Itemsets

 Frequent 1-itemsets:
Identify individual items that appear frequently in the dataset.

Item Support (%)

Milk 4/5 = 80%

Bread 4/5 = 80%

Butter 4/5 = 80%

 Frequent 2-itemsets:
Now, identify pairs of items that appear frequently together.

Itemset Support (%)

{Milk, Bread} 3/5 = 60%

{Milk, Butter} 3/5 = 60%

{Bread, Butter} 4/5 = 80%

✅ Step 4: Generate Association Rules

Now, using the frequent itemsets, we generate association rules that satisfy the minimum
confidence.

 For {Milk, Bread}:

o Rule 1: {Milk} → {Bread}

 Confidence = 3/4 = 75%

 This rule is valid because it meets the minimum confidence (75%).

o Rule 2: {Bread} → {Milk}

 Confidence = 3/4 = 75%

 This rule is valid because it meets the minimum confidence (75%).

 For {Milk, Butter}:

o Rule 3: {Milk} → {Butter}

 Confidence = 3/4 = 75%

 This rule is valid because it meets the minimum confidence (75%).

o Rule 4: {Butter} → {Milk}

 Confidence = 3/4 = 75%

 This rule is valid because it meets the minimum confidence (75%).

 For {Bread, Butter}:

o Rule 5: {Bread} → {Butter}

 Confidence = 4/4 = 100%

 This rule is valid because it has high confidence.

o Rule 6: {Butter} → {Bread}

 Confidence = 4/4 = 100%

 This rule is valid because it has high confidence.

✅ Step 5: Calculate Lift (Optional)

The Lift of a rule measures how much more likely the consequent is to occur, given the antecedent.

 Lift = (Support of {Milk, Bread}) / (Support of {Milk} * Support of {Bread})

For Rule 1: {Milk} → {Bread}

 Support of {Milk, Bread} = 60%

 Support of {Milk} = 80%

  Support of {Bread} = 80%

Lift = 60% / (80% * 80%) = 0.9375

✅ Step 6: Filter the Best Rules

After calculating support, confidence, and lift, filter the association rules based on your thresholds.

 Rules to keep:

o {Milk} → {Bread} with confidence of 75% and lift of 0.9375.

o {Bread} → {Milk} with confidence of 75% and lift of 0.9375.

o {Bread} → {Butter} with confidence of 100% and lift of 1.25.

✅ Final Association Rules:

After filtering based on confidence and lift, we are left with:

 {Milk} → {Bread}

 {Bread} → {Milk}

 {Bread} → {Butter}

 {Butter} → {Bread}

These are the association rules that describe the relationships between items in the dataset.

✅ Applications of Association Rules:

 Market Basket Analysis: Finding items frequently bought together.

 Recommender Systems: Suggesting products based on what other customers bought.

 Web Usage Mining: Analyzing patterns of web page clicks.