INTERNATIONAL ISLAMIC UNIVERSITY

ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES

Department of _Technology and Project Management____

Date of Exam -1-2025 SPRING FALL
Semester
Class BSBA Male Female 2024
Exam Midterm Final
Course Code TMB-351 Term
✘

Time Hours. Minutes

Course Title Analytics with Python 2 30
Teacher’s Ms. Muthara-Tul-Ain Total Marks 60
Name

Teacher’s HoD’s Signature

Signature

INSTRUCTIONS:
 Attempt all questions.
 Read the questions carefully.
 Write in a concise manner.
 Make your assumptions where required but state them clearly in the
answer

Question:1 [20 Marks]

1. What are the key differences between simple linear regression and multiple linear
regressions? How can linear regression be applied to solve real-world problems in
fields such as economics and Business?

Key Differences Between Simple Linear Regression and Multiple Linear Regression
1. Number of Independent Variables:
o Simple Linear Regression: Involves one independent variable and one
dependent variable. The relationship is modeled as Y=β0+β1X+ϵY = \beta_0 + \
beta_1 X + \epsilonY=β0+β1X+ϵ, where XXX is the independent variable.
o Multiple Linear Regression: Involves two or more independent variables and
one dependent variable. The model is represented as Y=β0+β1X1+β2X2+…
+βnXn+ϵY = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \ldots + \beta_n X_n + \
epsilonY=β0+β1X1+β2X2+…+βnXn+ϵ.
2. Complexity:
o Simple Linear Regression: Straightforward and easier to interpret due to the
involvement of only one predictor variable.
o Multiple Linear Regression: More complex as it accounts for multiple factors
that may influence the dependent variable.
3. Interpretation:
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
o Simple Linear Regression: The coefficient β1\beta_1β1 represents the change
in YYY for a one-unit change in XXX.
o Multiple Linear Regression: Each coefficient βi\beta_iβi represents the change
in YYY for a one-unit change in XiX_iXi, holding other variables constant.
4. Applications:
o Simple Linear Regression: Often used when the relationship between the
dependent and independent variable is straightforward.
o Multiple Linear Regression: Suitable when the outcome is influenced by several
factors simultaneously.

Application of Linear Regression in Real-World Problems

Linear regression is a powerful tool for predicting outcomes, analyzing relationships, and
making data-driven decisions. Below are examples of its application in economics and
business:

1. Economics:
o Demand and Supply Analysis: Predicting demand for a product based on price,
income levels, and market trends.
o Economic Forecasting: Estimating GDP growth or unemployment rates using
multiple economic indicators like inflation, interest rates, and consumer
spending.
o Income Prediction: Modeling the impact of education level, experience, and
industry on wages.
2. Business:
o Sales Prediction: Estimating future sales based on advertising spend, pricing,
and seasonal factors.
o Customer Behavior Analysis: Understanding factors influencing customer
satisfaction, such as service quality, pricing, and product features.
o Operational Efficiency: Predicting production output or delivery times using
variables like labor hours, machine usage, and raw material availability.
o Risk Management: Assessing the likelihood of loan defaults based on credit
score, income, and other demographic factors.

By using these models, organizations can make informed decisions, optimize resources, and
plan strategically for future growth.

2. How do underfitting and overfitting affect the performance of linear regression

models, and what techniques can be used to identify and address these issues
effectively?
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
Underfitting and Overfitting in Linear Regression Models
1. Underfitting:
o Definition: Occurs when a model is too simple to capture the underlying
patterns in the data. It fails to fit the training data well and performs poorly on
both training and test data.
o Causes:
 Model complexity is too low (e.g., using a simple linear model when the
relationship is non-linear).
 Missing important features or predictors.
 Insufficient training or incorrect assumptions about the data.
o Symptoms:
 High training error.
 High test error.
 Residuals show systematic patterns instead of randomness.
o Remedies:
 Increase model complexity (e.g., adding polynomial terms or interaction
terms).
 Include more relevant features.
 Verify and preprocess data appropriately.
2. Overfitting:
o Definition: Occurs when a model is too complex and captures not only the true
patterns but also the noise in the data. It performs well on the training data
but poorly on unseen data.
o Causes:
 Excessive number of features relative to the number of observations.
 Using too complex a model (e.g., higher-order polynomials).
 Over-reliance on small fluctuations in the data.
o Symptoms:
 Low training error but high test error.
 Model behaves erratically for new or unseen data.
o Remedies:
 Simplify the model by removing irrelevant or less significant features.
 Regularization techniques (e.g., Ridge Regression or Lasso Regression).
 Increase the size of the training dataset.
 Use cross-validation to tune hyperparameters and assess model
performance.

Techniques to Identify and Address Underfitting and Overfitting

1. Cross-Validation

 Split the data into training and validation sets (e.g., k-fold cross-validation).
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
 Compare training and validation performance. Large discrepancies often indicate
overfitting, while poor performance on both indicates underfitting.

2. Learning Curves

 Plot training and validation errors against the size of the training data.
 If both errors are high and close, the model is underfitting.
 If the training error is low but validation error is high, the model is overfitting.

3. Regularization

 Introduce penalties for large coefficients to prevent the model from overfitting.
o Ridge Regression: Adds an L2L_2L2 penalty.
o Lasso Regression: Adds an L1L_1L1 penalty, which can also perform feature
selection.

4. Feature Selection

 Eliminate irrelevant or redundant features to reduce complexity and mitigate

overfitting.

5. Increase Training Data

 For overfitting, provide the model with more data to generalize better.

6. Use Polynomial or Interaction Terms:

 For underfitting, increase the model's capacity to capture complex relationships by

adding polynomial or interaction terms.

7. Early Stopping

 Monitor performance on a validation set during training and stop once performance
starts to degrade (common in iterative optimization).

8. Adjusting Hyperparameters

 Use techniques like grid search or random search to optimize parameters such as
learning rate, regularization strength, and polynomial degree.

By balancing model complexity and training data, linear regression models can achieve better
generalization and predictive performance.
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
Question:2 [20 Marks]
Apply a decision tree on the following dataset to predict whether a person will buy a product:

Age Income Buys Product?

Young Low No
Young High Yes
Middle Low Yes
Old High No

1. What is the first split the tree will make based on this dataset? How would you build
the decision tree step by step for this data?

To build a decision tree and determine the first split for the given dataset, we use a measure
like information gain (based on entropy) or Gini index to evaluate which feature provides the
best split. Here, we'll use the entropy and information gain approach.

Step 1: Calculate the Overall Entropy

The dataset has the following labels:

 Buys Product? = Yes: 2 occurrences

 Buys Product? = No: 2 occurrences

The entropy formula is:

H=−p1log⁡2(p1)−p2log⁡2(p2)H = -p_1 \log_2(p_1) - p_2 \log_2(p_2)H=−p1log2(p1)−p2log2(p2)

where p1p_1p1 and p2p_2p2 are the probabilities of each class.

H=−24log⁡2(24)−24log⁡2(24)=−0.5×log⁡2(0.5)−0.5×log⁡2(0.5)=1H = -\frac{2}{4} \log_2\left(\frac{2}

{4}\right) - \frac{2}{4} \log_2\left(\frac{2}{4}\right) = -0.5 \times \log_2(0.5) - 0.5 \times \
log_2(0.5) = 1H=−42log2(42)−42log2(42)=−0.5×log2(0.5)−0.5×log2(0.5)=1

The overall entropy is 1.

 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
Step 2: Calculate Entropy for Each Feature

2.1 Split by Age

 For Young:
o 2 instances: 1 Yes, 1 No
o Entropy = −0.5log⁡2(0.5)−0.5log⁡2(0.5)=1-0.5 \log_2(0.5) - 0.5 \log_2(0.5) =
1−0.5log2(0.5)−0.5log2(0.5)=1
 For Middle:
o 1 instance: 1 Yes, 0 No
o Entropy = 0 (pure subset).
 For Old:
o 1 instance: 0 Yes, 1 No
o Entropy = 0 (pure subset).

Weighted Entropy for Age:

Entropy(Age)=24(1)+14(0)+14(0)=0.5\text{Entropy(Age)} = \frac{2}{4}(1) + \frac{1}{4}(0) + \

frac{1}{4}(0) = 0.5Entropy(Age)=42(1)+41(0)+41(0)=0.5

Information Gain for Age:

Gain(Age)=1−0.5=0.5\text{Gain(Age)} = 1 - 0.5 = 0.5Gain(Age)=1−0.5=0.5

2.2 Split by Income

 For Low:
o 2 instances: 1 Yes, 1 No
o Entropy = 111
 For High:
o 2 instances: 1 Yes, 1 No
o Entropy = 111

Weighted Entropy for Income:

Entropy(Income)=24(1)+24(1)=1\text{Entropy(Income)} = \frac{2}{4}(1) + \frac{2}{4}(1) =

1Entropy(Income)=42(1)+42(1)=1

Information Gain for Income:

Gain(Income)=1−1=0\text{Gain(Income)} = 1 - 1 = 0Gain(Income)=1−1=0
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
Step 3: Determine the First Split

 Gain(Age) = 0.5
 Gain(Income) = 0

The first split is made on the feature Age, as it provides the highest information gain.

Step 4: Build the Decision Tree

1. Split on Age:
o Young: Subset is: Young, Low, NoYoung, High, Yes\begin{aligned} \text{Young,
Low, No}\\ \text{Young, High, Yes} \
end{aligned}Young, Low, NoYoung, High, Yes Entropy = 111 (further split
required).
o Middle: Subset is: Middle, Low, Yes\text{Middle, Low, Yes}Middle, Low, Yes
Pure subset (no further split needed).
o Old: Subset is: Old, High, No\text{Old, High, No}Old, High, No Pure subset (no
further split needed).
2. Split the Young Subset on Income:
o Low: Result is No.
o High: Result is Yes.

Final Decision Tree

yaml
Copy code
Age?
├── Young
│ ├── Low → No
│ └── High → Yes
├── Middle → Yes
└── Old → No

This tree predicts the target variable "Buys Product?" based on the features Age and Income.

2. What Python libraries can be used to build decision trees for business analytics?

Several Python libraries are well-suited for building decision trees for business analytics. Here
are the most commonly used ones:
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
1. Scikit-learn

 Overview: One of the most popular libraries for machine learning in Python, providing
tools for creating, visualizing, and evaluating decision trees.
 Features:
o Easy-to-use implementation of decision trees (DecisionTreeClassifier and
DecisionTreeRegressor).
o Offers Gini Impurity and Entropy-based splitting.
o Tools for hyperparameter tuning like max_depth, min_samples_split, and
min_samples_leaf.
o Provides visualization tools (plot_tree).
 Example:

python
Copy code
from sklearn.tree import DecisionTreeClassifier, plot_tree
import matplotlib.pyplot as plt

# Sample data
X = [[0, 0], [1, 0], [0, 1], [1, 1]]
y = [0, 0, 1, 1]

# Create and train the model

model = DecisionTreeClassifier(criterion="entropy", max_depth=3)
model.fit(X, y)

# Visualize the decision tree

plt.figure(figsize=(10, 6))
plot_tree(model, filled=True)
plt.show()

2. XGBoost

 Overview: A high-performance library that specializes in gradient-boosted decision

trees, ideal for large datasets and complex models.
 Features:
o Highly efficient and optimized for speed.
o Supports advanced tree-based models like Gradient Boosting.
o Handles missing data automatically.
 Example:

python
Copy code
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Sample data
X = [[0, 0], [1, 0], [0, 1], [1, 1]]
y = [0, 0, 1, 1]

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

# Train an XGBoost classifier

model = xgb.XGBClassifier(use_label_encoder=False, eval_metric='logloss')
model.fit(X_train, y_train)

# Predict and evaluate

y_pred = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))

3. LightGBM

 Overview: A gradient-boosting framework that focuses on efficiency and scalability.

 Features:
o Optimized for large datasets with categorical features.
o Excellent speed and low memory usage.
 Use Case: Suitable for high-dimensional data in business applications.

4. PyCaret

 Overview: A low-code library for automating machine learning workflows, including

decision tree models.
 Features:
o Simplifies model training, tuning, and comparison.
o Built-in support for decision trees and ensembles.
 Example:

python
Copy code
from pycaret.classification import *

# Load dataset
data = pd.read_csv('data.csv')
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES

# Set up PyCaret for classification

setup(data=data, target='target_column')

# Train and compare models

best_model = compare_models()

5. Statsmodels

 Overview: Focuses on statistical models and analysis. Though not specifically

optimized for decision trees, it complements tree-based models with statistical tests
and insights.

6. H2O.ai

 Overview: A scalable and distributed platform for machine learning, including decision
trees.
 Features:
o Distributed and cloud-ready.
o Provides AutoML for decision tree-based models.

7. TensorFlow Decision Forests (TF-DF)

 Overview: A library by TensorFlow for training decision tree models.

 Features:
o Integrates with TensorFlow for advanced workflows.
o Can train Random Forests and Gradient Boosted Trees.
 Example:

python
Copy code
import tensorflow_decision_forests as tfdf

# Load dataset
dataset = pd.read_csv('data.csv')
train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(dataset, label='target')

# Train model
model = tfdf.keras.GradientBoostedTreesModel()
model.fit(train_ds)
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES

These libraries cover a range of use cases, from simple decision trees to advanced ensemble
methods, making them versatile tools for business analytics. For most scenarios, Scikit-learn
is a good starting point due to its simplicity and integration with other tools.

Question:3 [20 Marks]

1. What are the different components of time series analysis, and how can they be
identified and modeled to improve the accuracy of forecasts?

Components of Time Series Analysis

Time series analysis involves identifying and modeling the underlying patterns in data that
vary over time. The key components are:

1. Trend

 Definition: The long-term movement or direction in the data over time. It represents
the overall increase, decrease, or stability in the series.
 Examples:
o An upward trend in sales revenue over several years.
o A downward trend in unemployment rates over time.
 Identification:
o Visualization: Plot the time series to observe overall direction.
o Statistical Methods: Use techniques like moving averages or regression analysis
to extract the trend.
 Modeling:
o Linear or polynomial regression models.
o Smoothing techniques such as moving averages.

2. Seasonality

 Definition: Regular, repeating patterns or cycles in the data due to seasonal or periodic
factors.
 Examples:
o Higher ice cream sales in summer.
o Increased online shopping during holiday seasons.
 Identification:
o Visualization: Plot the data and look for periodic patterns.
o Decomposition: Use time series decomposition to separate the seasonal
component.
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
o Autocorrelation: Identify repeating cycles using autocorrelation functions.
 Modeling:
o Add seasonal components explicitly (e.g., sine/cosine terms for periodicity).
o Use seasonal decomposition (e.g., STL decomposition).
o Apply seasonal ARIMA or SARIMAX models.

3. Cyclic Patterns

 Definition: Fluctuations in the data that occur over periods longer than a season, often
tied to economic or business cycles.
 Examples:
o Economic boom-and-bust cycles.
o Industry demand cycles that span multiple years.
 Identification:
o Visual inspection of long-term data.
o Spectral analysis to detect dominant cycles.
 Modeling:
o Long-term regression models.
o Business cycle indicators or econometric models.

4. Irregular or Random Component (Noise)

 Definition: Unpredictable variations in the data caused by random or unforeseen

factors.
 Examples:
o Sudden spikes in sales due to one-off promotions.
o External disruptions like natural disasters or pandemics.
 Identification:
o Residual analysis after removing trend, seasonality, and cyclic components.
 Modeling:
o Consider as white noise or use advanced models to account for any structure
within the noise (e.g., ARCH or GARCH models for volatility).

Approaches to Identify and Model Components

1. Decomposition

 Break the series into trend, seasonality, and residuals.

 Methods:
o Additive Model: Yt=Tt+St+RtY_t = T_t + S_t + R_tYt=Tt+St+Rt
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
o Multiplicative Model: Yt=Tt×St×RtY_t = T_t \times S_t \times R_tYt=Tt×St×Rt
 Libraries: statsmodels.tsa.seasonal_decompose, STL in Python.

2. Autocorrelation and Partial Autocorrelation

 Use autocorrelation (ACF) and partial autocorrelation (PACF) plots to identify lags and
seasonal patterns.

3. Smoothing

 Techniques like moving averages, exponential smoothing, or LOESS help to smooth

data for better trend and seasonality detection.

4. Modeling Techniques

 ARIMA (Auto-Regressive Integrated Moving Average):

o Captures trend and autocorrelation.
 SARIMA (Seasonal ARIMA):
o Handles seasonality explicitly.
 Holt-Winters Exponential Smoothing:
o Ideal for series with trend and seasonality.
 Machine Learning Models:
o Gradient boosting, neural networks, or LSTM for capturing non-linear
relationships.

5. Performance Metrics

 Use metrics like Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), or
Mean Absolute Percentage Error (MAPE) to evaluate forecasting accuracy.

By identifying and accurately modeling these components, forecasters can improve

predictions, leading to better decision-making and planning.

2. How can the ARIMA model be applied in business forecasting, and what are the key
steps in using ARIMA to predict future trends in sales, inventory, or demand?

Applying ARIMA in Business Forecasting

The ARIMA (Auto-Regressive Integrated Moving Average) model is a widely used method for
time series forecasting, especially in business contexts such as sales, inventory, and demand
forecasting. ARIMA is effective for data that exhibits patterns like trends or seasonality (when
paired with seasonal extensions like SARIMA).
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES

Key Steps in Using ARIMA for Business Forecasting

1. Understand the Problem and Prepare the Data

 Clearly define the objective, such as forecasting future sales, inventory requirements,
or customer demand.
 Collect and organize historical time series data at an appropriate granularity (e.g.,
daily, monthly).
 Ensure the data is consistent and free of anomalies, such as missing or erroneous
values.

2. Visualize the Time Series

 Plot the data to identify:

o Trends (long-term direction).
o Seasonality (regular patterns).
o Stationarity (constant mean and variance over time).

3. Check for Stationarity

 ARIMA requires a stationary time series to perform well.

 Perform a stationarity test, such as the Augmented Dickey-Fuller (ADF) test:
o Null Hypothesis (H0H_0H0): The series is non-stationary.
o Reject H0H_0H0 if the p-value is below a threshold (e.g., 0.05).

4. Transform the Data if Necessary

 If the series is not stationary:

o Remove trends using differencing.
o Address seasonality using seasonal differencing or transformations like
logarithms.

5. Determine ARIMA Parameters (p,d,qp, d, qp,d,q)

 AR (Auto-Regressive): Number of lagged observations included (ppp).

 I (Integrated): Degree of differencing applied (ddd).
 MA (Moving Average): Number of lagged forecast errors in the model (qqq).
 Use tools like:
o ACF (Auto-Correlation Function): Helps identify qqq.
o PACF (Partial Auto-Correlation Function): Helps identify ppp.
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
6. Fit the ARIMA Model

 Use libraries like statsmodels in Python to fit the ARIMA model:

python
Copy code
from statsmodels.tsa.arima.model import ARIMA

# Fit ARIMA model

model = ARIMA(data, order=(p, d, q))
model_fit = model.fit()

# Summary of the model

print(model_fit.summary())

7. Evaluate the Model

 Split the data into training and testing sets.

 Assess performance using metrics like:
o Mean Absolute Error (MAE)
o Mean Absolute Percentage Error (MAPE)
o Root Mean Squared Error (RMSE)
 Check residuals to ensure they are random (white noise).

8. Forecast Future Trends

 Generate forecasts for the required horizon using the fitted model:

python
Copy code
forecast = model_fit.forecast(steps=forecast_horizon)
print(forecast)

9. Incorporate Seasonality (if needed)

 Use SARIMA (Seasonal ARIMA) if the data exhibits strong seasonal patterns.
 SARIMA extends ARIMA with seasonal parameters (P,D,Q,mP, D, Q, mP,D,Q,m):
o PPP: Seasonal Auto-Regressive order.
o DDD: Seasonal Differencing.
o QQQ: Seasonal Moving Average order.
o mmm: Number of periods in a season.

Example for SARIMA:

python
 INTERNATIONAL ISLAMIC UNIVERSITY
ISLAMABAD
FACULTY OF MANAGEMENT SCIENCES
Copy code
from statsmodels.tsa.statespace.sarimax import SARIMAX

model = SARIMAX(data, order=(p, d, q), seasonal_order=(P, D, Q, m))

model_fit = model.fit()

10. Refine and Iterate

 Adjust the model based on:

o Residual diagnostics.
o Performance on validation data.
 Experiment with alternative parameter combinations for better forecasts.

Applications of ARIMA in Business Forecasting

1. Sales Forecasting:
o Predict future revenue or unit sales based on historical sales data.
o Use for budget planning, marketing strategies, and financial analysis.
2. Inventory Management:
o Anticipate inventory needs to avoid stockouts or overstock situations.
o Optimize supply chain operations.
3. Demand Forecasting:
o Predict customer demand for better production planning.
o Adjust staffing levels during peak and off-peak periods.
4. Revenue Projections:
o Estimate future earnings for financial planning and investor reporting.
5. Risk Management:
o Forecast financial market trends or customer defaults for proactive risk
mitigation.

By carefully following these steps and iteratively refining the model, ARIMA can serve as a
powerful tool for accurate and actionable business forecasts.

GOOD LUCK!   

--------------------------- End ----------------------