Machine Learning for Chemical Engineers

CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-1
02-08-2024
BITS Pilani
Pilani Campus
 Introduction
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Outline

What is Machine Learning

Applications
Why in Chemical Engineering
Hand-out

15 August 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

What is Machine Learning

Certain tasks are difficult to program by hand
How are you?

All learning problems can be formulated as mappings (complex)

between inputs and outputs
Mathematically, we search for a good function F: I → O
This function can be derived from available data
Data
Both I and O
Only I
No O but we can get measure of O quality produced by model

15 August 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

What is Machine Learning

Use of past experience in decision making

Use of historical data to discover patterns and improve
decision making
When there is excess data, learning must be automatic
ML technology  automatic learning from voluminous data

Output Prediction
Classification
Data ML Algorithm Pattern recognition
--
--

15 August 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

What is Machine Learning

Data
In the form of table (matrix)
Rows –
instances/samples/measurements/observations/records/
patterns/objects/events
Columns –
attributes/variables/features

15 August 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Types of Machine Learning

Algorithms
Machine
learning
algorithms

Supervised Unsupervised Semisupervised Reinforced

Feature
Classification
extraction

Regression Clustering

15 August 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

AI Vs ML

Machine learning is a subset of the broader category of AI

AI refers to the general ability of computers to emulate
human thought and intelligence
ML refers to the technologies and algorithms that enable
systems to identify patterns and make decisions by
learning from data

15 August 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Applications
Spam filter
Biometric/Handwriting/Speech/Text recognition
Image processing
Stock market forecasting/demand forecasting
Medical diagnostics
Drug discovery and optimization
Bioinformatics
Robotics and automation
--
--

15 August 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Why in Chemical Engineering

• Chemical engineering is usually a data-rich field that

offers opportunities to exploit data science and machine
learning techniques for improved automation,
profitability, product quality maintenance, reduced
emissions and downtimes for several industrial
processes.
• There is a need to evaluate the potential of this new
wave of machine learning to influence the process
systems field
• Fourth industrial revolution call for a combination of
expertise in chemical engineering and information
technology(AI/ML/Data science)

15 August 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Why in Chemical Engineering

• Third industrial revolution (analog  digital)

• Advanced instrumentation, data storage
• No utilization of available huge process data except some routine technical
checking
• Modern process industries are data rich and information poor

• Fourth industrial revolution (Industry 4.0)

Stringent emission norms, minimum tolerance on product quality, advanced process
control
• Intelligent manufacturing
• Cyber physical system
• Internet of things
• Smart factories

15 August 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Why in Chemical Engineering

• Modern processes are quite complex (difficult to develop

first principle models)
• Solution??  why not utilize the huge amount of data
available in the historical database
• Data mining and analytics is a tool for transformation
from machine automation  information automation
• Analyze patterns of process data and relationships
among variables to extract useful information  develop
models from data [Statistical/AI (soft computing)]

15 August 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Why in Chemical Engineering

Data driven
process
modeling

Process
Soft sensing of Parameter
monitoring/fault
key variables optimization
detection

15 August 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Applications

Prediction of molecular properties from structures

Prediction and optimization of reaction outcome and
reactor performance based on the reaction conditions
(e.g., reagents, catalysts, solvents, reactants,
products,flow rate, temperature, and pressure)
Understanding flow patterns
Understanding catalytic mechanisms
Prediction of process response to control actions

15 August 2024 17
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

Ferreira, J., Pedemonte, M., & Torres, A. I. (2022). Development of a machine learning-
based soft sensor for an oil refinery’s distillation column. Computers & Chemical
Engineering, 161, 107756.
15 August 2024 18
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

Zhu, Q. X., Wang, X. W., Li, K., Xu, Y., & He, Y. L. (2022). Enhanced multicorrelation block
process monitoring and abnormity root cause analysis for distributed industrial process:
A visual data-driven approach. Journal of Process Control, 118, 1-15.

15 August 2024 19
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Popular journals in this field

IEEE Transactions on industrial informatics
IEEE Transactions on Industry Applications
IEEE Transactions on Automation Science and Engineering
Applied Soft Computing
ISA Transactions
Computers and Industrial Engineering
Computers in Industry
Computers & Chemical Engineering
Engineering Applications of Artificial Intelligence
Journal of Industrial Information Integration
Digital Chemical Engineering
Journal of Advanced Manufacturing and Processing

15 August 2024 20
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Handout

15 August 2024 21
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024
22 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-2
05-08-2024
BITS Pilani
Pilani Campus
 Data Preprocessing
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Outline

Industrial data characteristics

Missing values
Outlier

15 August 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial data characteristics

and treatment
• Wide use of distributed control system, increasing use of
online sensors with low sampling time, improved data
transmission and storage facility have resulted in
availability of huge amount of past process data
• Data-driven process modeling, monitoring, prediction
and control have received much attention in recent
years.
• By analyzing the patterns of process data and
relationships among variables, useful information can be
extracted, based on which statistical models can then be
developed for various applications, such as process
monitoring, fault diagnosis, mode clustering, soft sensing
of key variables/quality variables, etc.

15 August 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial data characteristics

and treatment
• Big Data in process industries is characterized by
volume, variety, and velocity, or simply V3
• volume refers to size of ever-growing data sets which
range from terabytes (1012 bytes) to zettabytes (1021
bytes)
• “variety” describes various types of data: process
measurements, text, audio, and images
• “velocity” refers to the speed of big data generation

15 August 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial data characteristics

and treatment
• Data preparation is an initial step for machine learning model
development
• The main tasks of this step are to extract the dataset from the
historical database, examine the structure of the dataset, and
make data selections through sample and variable directions,
etc.
• In order to extract an effective dataset from the historical
database, the operating regions of the process need to be
analyzed, and any changes of operating condition also need
to be identified.
• To ensure the efficiency for the information extraction step,
the natures or characteristics of the process data should be
analyzed, such as non-Gaussianity, linear/nonlinear
relationships among different variables, time-series
correlations, etc.

15 August 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial data characteristics

and treatment

Thebelt, A., Wiebe, J., Kronqvist, J., Tsay, C., & Misener, R. (2022). Maximizing information from chemical
engineering data sets: Applications to machine learning. Chemical Engineering Science, 252, 117469.

15 August 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial data characteristics

and treatment
Common descriptors that characterize data together with examples

Thebelt, A., Wiebe, J., Kronqvist, J., Tsay, C., & Misener, R. (2022). Maximizing information from chemical
engineering data sets: Applications to machine learning. Chemical Engineering Science, 252, 117469.

15 August 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial data characteristics

and treatment
Normal Distribution
• The normal distribution is also known as the Gaussian
distribution.

Probabilities associated with the normal distribution

15 August 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

Data pre-processing is carried out in order to improve the

quality of the data
outliers and gross errors should be removed from the
modeling dataset, which will otherwise greatly
deteriorate the performance of the machine learning
model
missing values need to be addressed, e.g. deletion of the
sample, missing value estimation, Bayesian inference,
etc
the scale difference among process variables needs to be
considered

15 August 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

The raw data of different formats stored in databases are

not useful until they are cleaned and transformed
Data cleaning consists of four steps:
• missing data imputation
• Outlier detection and noise removal
• time alignment
• Delay estimation

15 August 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing
Missing data imputation
Missing values in process industries refer to entries in the
data set that have no connection with the real state of
the process and take values such as ±∞, 0, nan (not a
number)
There are generally three missing patterns:
Missing completely at random (MCAR)
Missing at random (MAR)
Missing not at random (MNAR)

15 August 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing
Missing data imputation
A and C – missing values for
single/multiple variables 
due to sensor failure
B – values of some variables
missing at same time
instances  fault
D – single variable showing
regular missing values 
multirate sampling

Common missing patterns in the process industries

Xu, S., Lu, B., Baldea, M., Edgar, T. F., Wojsznis, W., Blevins, T., & Nixon, M. (2015). Data cleaning in the process
industries. Reviews in Chemical Engineering, 31(5), 453-490.
15 August 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

Missing data imputation

Deletion
eliminate any time point that contains missing values
Works well for large datasets
Will sacrifice a large amount of data, reduce the statisti
cal power, and lead to biased parameter estimation with
more uncertainty
Replacement
Mean replacement
Interpolation replacement

15 August 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing
Outlier detection and removal

• Observations or subsets of
observations that do not show a
consistent behavior with the rest
of the data set from a statistical
perspective
• Causes: malfunction of sensors
Pani, A. K., & Mohanta, H. K. (2016). Online monitoring of cement
and inappropriate treatment of clinker quality using multivariate statistics and Takagi-Sugeno fuzzy-
missing data inference technique. Control Engineering Practice, 57, 1-17.

• Two types of outliers: univariate

and multivariate

15 August 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024
17 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-3
07-08-2024
BITS Pilani
Pilani Campus
 Data Preprocessing
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap

What is machine learning

Scope of ML in Chemical Engineering
Data Preprocessing
missing value
outlier detection
univariate methods

15 August 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

EDA

Use of descriptive statistics and data visualization to

understand nature of data

Descriptive statistics
Does dataset summarization
Data visualization
– Bar chart
– Pie chart
– histogram

15 August 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Descriptive statistics
(Univariate)
Central tendency
Mean, median
Dispersion
Range, variance, standard deviation, quartiles and
interquartile range
Distribution
Skewness, kurtosis

15 August 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing
Quartile-based identifier and boxplots:
Uses the interquartile distance Q as the scale parameter
Q = Q3 – Q1
where Q1 is the lower quartile, x0.25 and Q3 is the upper
quartile, x0.75
Med = (Q1+ Q3)/2
For a symmetric data distribution, the following condition to
detect outliers:
|xk -med| >2Q
A boxplot is used as a graphical demonstration
of the quartile-based detector
In the plot, any point that lies outside the
upper or lower fences, is considered as an
outlier.

15 August 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Statistics
(Bivariate/Multivariate)
Scatter plot
Covariance
Correlation
Heatmap

15 August 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

Univariate outlier detection

3σ rule: The 3σ rule is widely used for detecting outliers
from an i.i.d. data set {xk} subject to a normal distribution
N(μ, σ2 ). If the following condition holds:
xk is an outlier if
Hampel identifier: Instead of using mean and standard
deviation, the Hampel identifier uses the median med
and MAD:
|xk-med| > 3×1.483MAD

15 August 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing
Quartile-based identifier and boxplots:
Uses the interquartile distance Q as the scale parameter
Q = Q3 – Q1
where Q1 is the lower quartile, x0.25 and Q3 is the upper quartile,
x0.75
13
med = (Q1+ Q3)/2
For a symmetric data distribution, the following condition to detect
outliers:
|xk -med| >2Q
A boxplot is used as a graphical demonstration
of the quartile-based detector
In the plot, any point that lies outside the
upper or lower fences, is considered as an
outlier.

15 August 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing
Multivariate outlier detection

Mahalanobis distance
Minimum covariance
determinant (MCD)
estimator
Minimum volume ellipsoid
(MVE) estimator
Smallest half volume

Pani, A. K., & Mohanta, H. K. (2016). Online monitoring of cement

clinker quality using multivariate statistics and Takagi-Sugeno fuzzy-
inference technique. Control Engineering Practice, 57, 1-17.
15 August 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Useful References
https://www.machinelearningplus.com/statistics/mahalanobi
s-distance/
Chiang, L. H., Pell, R. J., & Seasholtz, M. B. (2003).
Exploring process data with the use of robust outlier
detection algorithms. Journal of Process Control, 13(5),
437-449.

15 August 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024
13 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-4
09-08-2024
BITS Pilani
Pilani Campus
 Data Preprocessing
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap

Missing value
Descriptive statistics
– Central tendency
– Dispersion
– Distribution

15 August 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Outlier detection (Univariate)

Three sigma edit rule

MAD
Box and whisker plot

15 August 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Outlier detection (Univariate)

Quartile-based identifier and boxplots:
Uses the interquartile distance Q as the scale parameter
Q = Q3 – Q1
where Q1 is the lower quartile, x0.25 and Q3 is the upper
quartile, x0.75
Med = (Q1+ Q3)/2
For a symmetric data distribution, the following condition to
detect outliers:
|xk -med| >2Q
A boxplot is used as a graphical demonstration
of the quartile-based detector
In the plot, any point that lies outside the
upper or lower fences, is considered as an
outlier.

15 August 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Outlier detection (Multivariate)

Pani, A. K., & Mohanta, H. K. (2016). Online monitoring of cement clinker quality using multivariate statistics and
Takagi-Sugeno fuzzy-inference technique. Control Engineering Practice, 57, 1-17.
15 August 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Outlier detection (Multivariate)

Mahalanobis distance

If the underlying distribution is a multivariate normal

distribution, it is common to use the 0.975 quantile of a
chi-square distribution with p degrees of freedom
2p;0.975 as a cut off value

Multivariate trimming (MVT)

Minimum covariance determinant (MCD) estimator
Minimum volume ellipsoid (MVE) estimator
Smallest half volume

15 August 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Pani, A. K., & Mohanta, H. K. (2016). Online monitoring of cement clinker quality using multivariate statistics and
Takagi-Sugeno fuzzy-inference technique. Control Engineering Practice, 57, 1-17.

15 August 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Useful References
https://www.machinelearningplus.com/statistics/mahalanobi
s-distance/
Hodge, V., & Austin, J. (2004). A survey of outlier detection
methodologies. Artificial intelligence review, 22, 85-126.
Chiang, L. H., Pell, R. J., & Seasholtz, M. B. (2003).
Exploring process data with the use of robust outlier
detection algorithms. Journal of Process Control, 13(5),
437-449.

15 August 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data transformation (scaling)

Min-max
z-score

15 August 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

ACROSS
2. Mean and mode are examples of
______________ of univariate data.
4. Noisy data is
(normal/abnormal) data.
1
7. The branch of statistics that is used for
2
summarizing data is called ______
statistics.
10. Kurtosis characterized
the__________ of data.
3 12. The assumption of testing of data is
called a ___________ .
4 5
13. Raw facts are called _______.
6 14. Data wrangling refers to making data
suitable for processing. (Yes/ No)
7 8 9 15. Pairplot is used to visualize univariate
data. (Yes/No)
10

DOWN
1. The averaged square distance from
its mean is called ____________.
3. The characteristics of Big Data are
11
volume, velocity and
12
__________________.
5. The Dataset of two variables is
13 called __________________ data.
6. Visualiztion helps in presentation of
14 data. (Yes/ No)
8. Normalized covariance is called
________________.
9. Processed data is
________________.
15 11. Incorrect rejection of true
hypothesis is called _____________
error.
15 August 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024
13 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-5
12-08-2024
BITS Pilani
Pilani Campus
 Data Preprocessing
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap

Multivariate data
Euclidean and Mahalanobis distance
Multivariate outlier detection
Data transformation

15 August 2024 4
BITS Pilani, Pilani Campus
 CHE F315 Machine Learning for Chemical Engineers

Dimensionality reduction

Rallo, R., Ferre-Gine, J., Arenas, A.,

& Giralt, F. (2002). Neural virtual
sensor for the inferential
prediction of product quality from
process variables. Computers &
Chemical Engineering, 26(12),
1735-1754.

15 August 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Dimensionality reduction
As the number of dimensions increases time/computation
complexity increases
• Variable (feature) selection
Reduces dataset size by removing irrelevant variables
• Variable (feature) extraction (transformation)

15 August 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Feature selection
Measure of relevant feature
– Mutual information
– Correlation based similarity
– Distance-based similarity
A typical feature selection process consists of four steps:
– Generation of possible subsets
– Subset evaluation
– Stop searching based on some stopping criterion
– Validation of the result

15 August 2024 7
BITS Pilani, Pilani Campus
 ET ZC362 Environmental Pollution Control

Feature selection

Supervised learning Unsupervised learning

A matrix of unlabeled data
Predictor and response Similarity of data samples
variable are evaluated based on
Each predictor variable is values of the variables
expected to contribute
to the value of If a variable does not
response variable contribute in deciding
When this contribution is the similarity or
very little, the variable dissimilarity of samples,
is weakly relevant then that variable is
(irrelevant) weakly relevant

15 August 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Dimensionality reduction

Variable (feature) selection

Filter based
– Correlation
– Distance based similarity
– Mutual information

Wrapper based
– Stepwise forward selection
– Stepwise backward elimination

15 August 2024 9
BITS Pilani, Pilani Campus
 Probability
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Why Probability in ML
Designing machines that learn from observed data
Uncertainty in learning from data
Observed data can be consistent with many models and
therefore which model is appropriate, given the data, is
uncertain
Predictions about future data and the future consequences
of actions are uncertain
Many aspects of learning and intelligence crucially depend
on the careful probabilistic representation of uncertainty.
Probabilistic framework describes how to represent and
manipulate uncertainty about models and predictions
Bayesian interpretation  use of probability to quantify
uncertainty
15 August 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Review of basics

p(A) – probability that event A is true

p(A) – 1  A will definitely happen
p(A) – 0  A will definitely not happen
p(Ā) = 1-p(A)
Joint probability p(A,B)
Conditional probability p(AΙB)

15 August 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

An industry produces few defective products. It is observed

that in a lot of 1000 products, 25 are defective. If two
random samples are selected for testing without
replacement, calculate the probability that both products
are defective

15 August 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Probability distribution

A mathematical model that relates the value of the variable

with the probability of occurrence of that value in a
population
Continuous distribution  Variable being measured is
expressed on a continuous scale
Discrete distribution  when the measured parameter can
only take certain values (such as integers)

15 August 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Probability distribution

15 August 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

The metal layer thickness on silicon wafers in a CVD

process is normally distributed with mean 0.2508 and std
dev 0.0005. The specification is 0.2500 0.0015. What
fractions of the wafers produced conform to the
specification?

15 August 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Probability distribution

Central limit theorem

If x1, x2, …, xn are independent random variables with
mean  and variance 2, then
𝑦−σ𝑛
𝑖=1 𝜇𝑖
The distribution of approaches the N(0,1)
σ𝑛 𝜎
𝑖=1 𝑖
2

distribution as n approaches infinity

15 August 2024 17
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

References

Montgomery, D. C. (2019). Introduction to statistical quality

control. John wiley & sons.
Ghahramani, Z. (2015). Probabilistic machine learning and
artificial intelligence. Nature, 521(7553), 452-459.

15 August 2024 18
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024
19 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-6
14-08-2024
BITS Pilani
Pilani Campus
 Data Preprocessing
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap
Dimensionality reduction
– Variable selection
– Variable extraction
Variable selection
Filter based
– Mutual information
– Correlation based similarity
– Distance-based similarity

Wrapper based
– Forward selection
– Backward elimination
Embedded methods
– LASSO
– Elastic net
– Ridge regression

15 August 2024 4
BITS Pilani, Pilani Campus
 Probability
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Why Probability in ML
Designing machines that learn from observed data
Uncertainty in learning from data
Observed data can be consistent with many models and
therefore which model is appropriate, given the data, is
uncertain
Predictions about future data and the future consequences
of actions are uncertain
Many aspects of learning and intelligence crucially depend
on the careful probabilistic representation of uncertainty.
Probabilistic framework describes how to represent and
manipulate uncertainty about models and predictions
Bayesian interpretation  use of probability to quantify
uncertainty
15 August 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Review of basics

p(A) – probability that event A is true

p(A) – 1  A will definitely happen
p(A) – 0  A will definitely not happen
p(Ā) = 1-p(A)
Joint probability p(A,B)
Conditional probability p(AΙB)

15 August 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

An industry produces few defective products. It is observed

that in a lot of 1000 products, 25 are defective. If two
random samples are selected for testing without
replacement, calculate the probability that both products
are defective

15 August 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Probability distribution

A mathematical model that relates the value of the variable

with the probability of occurrence of that value in a
population
Continuous distribution  Variable being measured is
expressed on a continuous scale
Discrete distribution  when the measured parameter can
only take certain values (such as integers)

15 August 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Probability distribution

15 August 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

A sample of 100 dry battery cells tested to find the length of

life produced the following results
Mean – 12 hours, std dev – 3 hrs
Assuming the data to be normally distributed, what
percentage of battery cells are expected to have life
(a) More than 15 hrs
(b) Less than 6 hrs
(c) Between 10 and 14 hrs

15 August 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

The metal layer thickness on silicon wafers in a CVD

process is normally distributed with mean 0.2508 and std
dev 0.0005. The specification is 0.2500 0.0015. What
fractions of the wafers produced conform to the
specification?

15 August 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Probability distribution

Central limit theorem

If x1, x2, …, xn are independent random variables with
mean  and variance 2, then
𝑦−σ𝑛
𝑖=1 𝜇𝑖
The distribution of approaches the N(0,1)
σ𝑛 𝜎
𝑖=1 𝑖
2

distribution as n approaches infinity

15 August 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

References

Montgomery, D. C. (2019). Introduction to statistical quality

control. John wiley & sons.
Ghahramani, Z. (2015). Probabilistic machine learning and
artificial intelligence. Nature, 521(7553), 452-459.

15 August 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

15 August 2024
15 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-12
02-09-2024
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap

• Introduction
• Normal and faulty operation
• Traditional Monitoring Techniques
• Quality Control Charts
• Shewhart control charts (for subgroup data)
• Mean control chart
• Variability control chart (std dev and R)

16 September 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Control charts for individual

observations
There are many situations in which the sample size used for
process monitoring is n = 1 the sample consists of an
individual unit
Some examples of these situations
Automated inspection and measurement technology is used, and
every unit manufactured is analyzed so there is no basis for
rational subgrouping
Repeat measurements on the process differ only because of
laboratory or analysis error, as in many chemical processes
In process plants, measurements on some parameter differ very
little

In such situations, the control chart for individual units is

useful

16 September 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The Shewhart Control Chart

for Individual Measurements
use the moving range two successive observations as the
basis of estimating the process variability

If a moving range of n = 2 observations is used, then d2 =

1.128

16 September 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Day to day yield of plate glass panels in a glass sheet

manufacturing process is given below:
Sample 1 2 3 4 5 6 7 8 9
no
Yield 79 81 72 57 73 68 75 67 70

Using Shewhart control chart principles, determine the target,

upper control limit and lower control limit of yield and from this
information, determine if any sample is produced in an out of
control process.
Assuming that data samples are in time series form, compute
C+(k) and C-(k) of each sample for a CUSUM chart. Also
compute the control limit(s) and determine if any product is
formed in an out of control process. Use a slack parameter
value of 0.5 and threshold value of four times standard
deviation.
16 September 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Theoretical basis of Shewhart

charts
Traiditional SPC is based on assumption that natural
variability for in control conditions are characterized by
random variations around a constant average value
x(k) = x*+e(k)
Additive errors
Have zero mean and normal distribution
Are IID (statistically independent and uncorrelated, e(k)
f(e(j))

16 September 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Theoretical basis of Shewhart

charts
ARL – average number of points that must be plotted
before a point indicates an out-of-control condition
A Shewhart chart with 3 control limits will an average of
one control chart violation every 370 samples even when
the process is in a state of control
ARL0 = 370

16 September 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Limitations of Shewhart chart

16 September 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

True random points should evenly

distribute
19 of 25 points below center line
After 4th, run-up for 5 points
After 18th run-down for 6 points
A run of length 8 or more has a
very low probability in a random
distribution
Cyclic pattern

16 September 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Pattern tests and western

electric rules
Pattern tests can be used to augment Shewhart charts.
The following excerpts from the Western Electric Rules
indicate that the process is out of control if one or more
of the following conditions occur:
1. One data point is outside the 3 σ control limits.
2. Two out of three consecutive data points are beyond a 2σ
limit.
3. Four out of five consecutive data points are beyond a 1σ
limit and on one side of the center line.
4. Eight consecutive points are on one side of the center
line.

16 September 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• Although Shewhart charts with 3 σ limits can quickly

detect large process changes, they are ineffective for
small, sustained process changes (for example, changes
smaller than 1.5 σ )
• Two alternative control charts have been developed to
detect small changes: the CUSUM and EWMA control
charts.
• They also can detect large process changes (for
example, 3 σ shifts), but detection is usually somewhat
slower than for Shewhart charts.

16 September 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

CUSUM Control Chart

• The cumulative sum (CUSUM) is defined to be a running

summation of the deviations of the plotted variable from
its target.
• If the sample mean is plotted, the cumulative sum, C(k),
is

• where T is the target for x

16 September 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• During normal process operation, C(k) fluctuates around zero.

• But if a process change causes a small shift in x , C(k) will drift
either upward or downward.
• The CUSUM control chart was originally developed using a
graphical approach based on V-masks.
• However, for computer calculations, it is more convenient to
use an equivalent algebraic version that consists of two
recursive equations,

where C+ and C- denote the sums for the high and low directions and K is a
constant, the slack parameter.
16 September 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• The CUSUM calculations are initialized by setting C+(0) = C-

(0) = 0.
• A deviation from the target that is larger than K increases either C+ or
C -.
• A control limit violation occurs when either C+ or C- exceeds a
specified control limit (or threshold), H.
• After a limit violation occurs, that sum is reset to zero or to a specified
value.
• The selection of the threshold H can be based on considerations of
average run length.
• Suppose that we want to detect whether the sample mean x has
shifted from the target by a small amount,δ .
• The slack parameter K is usually specified as K = 0.5δ .

16 September 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024
17 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-13
04-09-2024
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap

• Introduction
• Normal and faulty operation
• Traditional Monitoring Techniques
• Quality Control Charts
• Shewhart control charts (for subgroup data)
• Shewhart control charts (for individual data)
• Limitations of Shewhart chart
• Western electric rules
• CUSUM chart

16 September 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Day to day yield of plate glass panels in a glass sheet

manufacturing process is given below:
Sample 1 2 3 4 5 6 7 8 9
no
Yield 79 81 72 57 73 68 75 67 70

Using Shewhart control chart principles, determine the target,

upper control limit and lower control limit of yield and from this
information, determine if any sample is produced in an out of
control process.
Assuming that data samples are in time series form, compute
C+(k) and C-(k) of each sample for a CUSUM chart. Also
compute the control limit(s) and determine if any product is
formed in an out of control process. Use a slack parameter
value of 0.5 and threshold value of four times standard
deviation.
16 September 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• Although Shewhart charts with 3 σ limits can quickly

detect large process changes, they are ineffective for
small, sustained process changes (for example, changes
smaller than 1.5 σ )  Phase I process monitoring
• Two alternative control charts have been developed to
detect small changes: the CUSUM and EWMA control
charts.
• They also can detect large process changes (for
example, 3 σ shifts), but detection is usually somewhat
slower than for Shewhart charts.

16 September 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

CUSUM Control Chart

• The cumulative sum (CUSUM) is defined to be a running

summation of the deviations of the plotted variable from
its target.
• If the sample mean is plotted, the cumulative sum, C(k),
is

• where T is the target for x

16 September 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• During normal process operation, C(k) fluctuates around zero.

• But if a process change causes a small shift in x , C(k) will drift
either upward or downward.
• The CUSUM control chart was originally developed using a
graphical approach based on V-masks.
• However, for computer calculations, it is more convenient to
use an equivalent algebraic version that consists of two
recursive equations,

where C+ and C- denote the sums for the high and low directions and K is a
constant, the slack parameter.
16 September 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• The CUSUM calculations are initialized by setting C+(0) = C-

(0) = 0.
• A deviation from the target that is larger than K increases either C+ or
C -.
• A control limit violation occurs when either C+ or C- exceeds a
specified control limit (or threshold), H.
• After a limit violation occurs, that sum is reset to zero or to a specified
value.
• The selection of the threshold H can be based on considerations of
average run length.
• Suppose that we want to detect whether the sample mean x has
shifted from the target by a small amount,δ .
• The slack parameter K is usually specified as K = 0.5δ .

16 September 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• For the ideal situation where the normally distributed and IID assumptions are valid,
ARL values have been tabulated for specified values ofδ , K, and H

16 September 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example
You are producing a chemical that
contains a small amount of
component Y that is important in the
use of the chemical. In addition,
you know that the optimum
concentration for that component is
0.16 wt. %. You want to control the
manufacturing process as close to
that as possible. You take a sample
every batch. The results for the last
25 batches are shown below.

https://www.spcforexcel.com/knowledge/variable-control-charts/keeping-process-target-cusum-charts
16 September 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Remember that our action limits are +/- 0.1116. This value
is exceeded by SH(23), our 23rd sample. This tells us that
the process has moved significantly off target and needs to
be adjusted to return the process to the target value.

16 September 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024
16 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-14
06-09-2024
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap

• Introduction
• Normal and faulty operation
• Traditional Monitoring Techniques
• Quality Control Charts
• Shewhart control charts (for subgroup data)
• Shewhart control charts (for individual data)
• Limitations of Shewhart chart
• Western electric rules
• CUSUM chart

16 September 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

EWMA Control Chart

• Information about past measurements can also be included in

the control chart calculations by exponentially weighting the
data.
• This strategy provides the basis for the exponentially-weighted
moving-average (EWMA) control chart.
• EWMA control chart is also a good alternative to the Shewhart
control chart in detecting small shifts
• Performance of EWMA chart is approximately equivalent to
CUSUM chart, easier to set up and operate

16 September 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• Let 𝑥ҧ denote the sample mean of the measured variable and z

denote the EWMA of 𝑥ҧ . A recursive equation is used to calculate
zi ,

• where  is a constant, 0 ≤ 𝜆 ≤ 1.
• The EWMA control chart consists of a plot of zi vs. i, as well as a
target and upper and lower control limits.
• Note that the EWMA control chart reduces to a Shewhart chart for
 = 1.
• The EWMA calculations are initialized by setting z(0) = T.
• If the measurements satisfy the IID condition, the EWMA control
limits can be derived.
16 September 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The term [1-(1-)2i] approaches unity as i gets larger

 After several time periods, control limits will approach
steady state values

16 September 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

• The theoretical 3 limits are given by

• where x is determined from a set of test data taken when the

process is in a state of control.
• The target T is selected to be either the desired value of 𝑥ҧ or
the grand mean for the test data, 𝑥Ӗ .
• The EWMA performance can be adjusted by specifying 
• EWMA control charts can also be constructed for measures of
variability such as the range and standard deviation.

16 September 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

Set up CUSUM and

EWMA control chart
( = 0.1 and L = 2.7)

16 September 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

16 September 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate process
monitoring

Kourti, T., & MacGregor, J. F. (1995). Process analysis, monitoring and diagnosis, using multivariate projection methods. Chemometrics and intelligent laboratory systems, 28(1),
16 September 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024
15 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-15
09-09-2024
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Revision

Shewhart control charts (Subgroup data) - mean

16 September 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Revision

Shewhart control charts (Subgroup data) - variability

16 September 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Revision

Shewhart control charts (individual observation) – mean

CUSUM (Tabular)

a reasonable value for H is five times the process standard

deviation 
K = (/2)

16 September 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Revision

EWMA

16 September 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Control chart using MATLAB

Function controlchart
'charttype' The name of a chart type chosen from among the
following:
'xbar' X-bar or mean
's' Standard deviation
'r' Range
'ewma' Exponentially weighted moving average
'i' Individual observation
'mr' Moving range of individual observations
'ma' Moving average of individual observations
'p' Proportion defective
'np' Number of defectives
'u' Defects per unit
'c' Count of defects
Alternatively this parameter can be a cell array listing
multiple compatible chart types. There are four sets
of compatible types: XBAR, S, R, and EWMA;
16 September 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example 17.1 (Seborg et al, 4th

Edition)
A manufacturing plant produces
10,000 plastic bottles per day.
Because the product is inexpensive
and the plant operation is normally
satisfactory, it is not economically
feasible to inspect every bottle.
Instead, a sample of 3 bottles is
randomly selected and inspected
each day. The inspection includes
measuring the toughness of x of
each bottle in the subgroup. Values
for a 10 days period are shown.
Plot Shewhart charts (𝑥ҧ and R) for
bottle quality monitoring and
determine if any subgroup fails to
meet the quality criterion

16 September 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example 17.1 (Seborg et al, 4th

Edition)

16 September 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example 17.1 (Seborg et al, 4th

Edition)
Solve using MATLAB
Store all values in a 10×3 matrix (x) ------ >> controlchart(x)

16 September 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example 17.1 (Seborg et al, 4th

Edition)
>> controlchart(x,'charttype',{'r'});

16 September 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example 17.2

In semiconductor processing, the photolithography process

is used to transfer the circuit design to silicon wafers. In
the first step of the process, a specified amount of a
polymer solution, photoresist, is applied to a wafer as it
spins at high speed on a turntable. The resulting
photoresist thickness x is a key process variable.
Thickness data for 25 subgroups are shown in Table.
Each subgroup consists of three randomly selected
wafers. Construct x and s control charts for these test
data and critically evaluate the results.

16 September 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Thickness Data (in Å) for

Example 17.2 209.6
183.5
207.6
193.1
211.1
202.4
190.1 206.8 201.6
206.9 189.3 204.1
260 209 212.2
193.9 178.8 214.5
206.9 202.8 189.7
200.2 192.7 202.1
210.6 192.3 205.9
186.6 201.5 197.4
204.8 196.6 225
183.7 209.7 208.6
185.6 198.9 191.5
202.9 210.1 208.1
198.6 195.2 150
188.7 200.7 207.6
197.1 204 182.9
194.2 211.2 215.4
191 206.2 183.9
202.5 197.1 211.1
185.1 186.3 188.9
203.1 193.1 203.9
179.7 203.3 209.7
205.3 190 208.2
203.4 202.9 200.4
16 September 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

MATLAB Use
controlchart(x,'charttype',{'s'});

16 September 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

controlchart(x,'charttype',{'xbar'});

16 September 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Shewhart Chart
controlchart(runout,'rules','we2');

16 September 2024 17
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

The data in Table are temperature readings

from a chemical process in C, taken every
two minutes.
(Read the observations down, from left.)
The target value for the mean is 0 = 950.
(a) Estimate the process standard
deviation.
(b) Develop xbar chart, R chart, s chart,
CUSUM chart, EWMA chart
standardized values for CUSUM h = 5 and
k = 1/2.

16 September 2024 18
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

CUSUM

16 September 2024 19
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

EWMA
controlchart(x,'charttype',{'ewma'});

16 September 2024 20
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024
21 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-16
11-09-2024
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The Multivariate Problem

Kourti, T., & MacGregor, J. F. (1995). Process analysis, monitoring and diagnosis, using multivariate projection methods. Chemometrics and intelligent laboratory systems, 28(1),
16 September 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The Multivariate Problem

In most industries, traditional univariate control charts

(Shewhart, CUSUM and EWMA) are used to separately
monitor key measurements on the final product which in
some way define the quality of that product.
The difficulty with this approach is that these quality
variables are not independent of one another, nor does
any one of them adequately define product quality by
itself.
In practice, however, many if not most process monitoring
and control scenarios involve several related variables

16 September 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate Statistical
Techniques
• For example, ten or more quality variables are typically
measured for synthetic fibers.
Although applying univariate control charts to each
individual variable is a possible solution, we will see that
this is inefficient and can lead to erroneous conclusions
For these situations, multivariable SPC techniques can
offer significant advantages over the single-variable
methods discussed in.
• In the statistics literature, these techniques are referred to
as multivariate methods, while the standard Shewhart
and CUSUM control charts are examples of univariate
methods.
16 September 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Seborg, D. E., Edgar, T. F., Mellichamp, D. A., & Doyle III, F. J. (2016). Process dynamics and control. John Wiley & Sons.
16 September 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Univariate and bivariate confidence regions for two random variables,

x1 and x2
16 September 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

AI, Data Science, and Statistics  Statistics and Machine Learning Toolbox 
Descriptive Statistics and Visualization  Statistical Visualization

gscatter
biplot

16 September 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Control charts for inner (x1) and outer (x2) bearing diameters
Montgomery, D. C. (2019). Introduction to statistical quality control. John wiley & sons.
16 September 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Examples

Simultaneous monitoring of ten or more quality variables

for synthetic fibers
Both ID and OD determine the usefulness of a bearing
Actuators (sticking, leakage etc.)
Drifting of reaction kinetics (catalyst decay)
Reduction of heat transfer rate (fouling)
Steam pressure upset
Upsets in inputs (flow, temperature, composition)

16 September 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The Tennessee Eastman

Challenge Problem

16 September 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The Tennessee Eastman

Challenge Problem
The TE challenge process was published by the Tennessee
Eastman Company as a process simulation for academic
research
It contains 41 measured variables and 12 manipulated
variables.
The TE process produces two products (labeled G and H)
from four reactants (labeled A, C, D and E).
The problem statement defines process constraints, 20
types of process disturbances, and six operating modes
corresponding to different production rates and G/H
mass ratios in the product stream.

16 September 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The process consists of three units: a reactor, product

separator, and stripper.
A recycle stream returns unused reactants in the product to
the reaction section.
The 41 measurements are a mixture of continuous and
sampled measurements.
The process is nonlinear, open-loop unstable, and contains
a mixture of fast and slow dynamics.

http://depts.washington.edu/control/LARRY/TE/download.html#updated_TE_code

16 September 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Descriptions of
process faults in
TE process

16 September 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024
16 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-17
13-09-2024
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Contents

Hotelling’s T2 chart
Multivariate EWMA
PCA
ICA
Clustering

16 September 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Hotelling’s T2 control chart

Multivariate extension of Shewhart 𝑥ҧ chart

Used for monitoring the mean vector of the process
Let two variables x1 and x2
chi-square distribution with 2 degrees of freedom

If the process means remain at the values 1 and  2, then

values of 𝜒02 should be less than the upper control limit
UCL = 𝜒 2𝛼,2 where 𝜒 2𝛼,2 is the upper 𝛼 percentage point
of the chi-square distribution with 2 degrees of freedom.
16 September 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Bivariate case

A control ellipse for two independent variables A control ellipse for two dependent variables

16 September 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Hotelling’s T2 control chart

Let p related quality characteristics are controlled jointly

The procedure requires computing the sample mean for
each of the p quality characteristics from a sample of
size n. This set of quality characteristic means is
represented by the p × 1 vector

16 September 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

The tensile strength and diameter of a textile fiber are two

important quality characteristics that are to be jointly
controlled. The quality engineer has decided to use n =
10 fiber specimens in each sample. He has taken 20
preliminary samples. Set up the T2 control chart.

16 September 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

16 September 2024
11 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-18
18-09-2024
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Contents

Hotelling’s T2 chart
Multivariate EWMA
PCA
ICA
Clustering

2 October 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Hotelling’s T2 control chart

UCL = 𝜒 2𝛼,2 where 𝜒 2𝛼,2 is the upper 𝛼 percentage point of the chi-square
distribution with 2 degrees of freedom

2 October 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Hotelling’s T2 control chart

Individual observations
subgroup size is naturally n = 1
This situation occurs frequently in the chemical and
process industries
Suppose that m samples, each of size n = 1, are available
and that p is the number of quality characteristics
observed in each sample.
Let 𝑥ҧ and S be the sample mean vector and covariance
matrix, respectively, of these observations.
The Hotelling T2 statistic is

2 October 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Hotelling’s T2 control chart

2 October 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate EWMA

The chi-square and T2 charts are Shewhart-type control

charts.
The UCL is obtained from the fact that the plotted statistic
has a chi-squared distribution with p degrees of freedom
when the process is on target
They use information only from the current sample
Consequently, they are relatively insensitive to small and
moderate shifts in the mean vector.
Cumulative sum (cusum) and EWMA control charts were
developed to provide more sensitivity to small shifts in
the univariate case, and they can be extended to
multivariate quality control problems.

2 October 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate EWMA

Recap of univariate EWMA

0≤𝜆≤1
calculations are initialized by setting
z(0) = 0.
In general 0.05 ≤ 𝜆 ≤ 0.25 works
well
Use smaller values of to detect
small shifts.
L = 3 works reasonably well
If  ≤ 0.1, L is preferred between
2.6 to 2.8

2 October 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate EWMA
The MEWMA is a logical extension of the univariate EWMA
and is defined as
where 0 ≤  ≤ 1 and Z0 = 0
The quantity plotted on the control chart is
the covariance matrix

2 October 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate EWMA

Process is out of control

when T2H
Choose H from the table so as
to have ARL0  200

Prabhu, S. S., & Runger, G. C. (1997).

Designing a multivariate EWMA control
chart. Journal of Quality Technology,
29(1), 8-15.

2 October 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example
Bivariate normal distribution data (Lowry et al 1992)

-0.12 0.06
-0.10 0.14 0.49
-0.25 0.17 0.82
-0.20 0.20 2.55
2.26
-0.09 0.10
0.51
0.00 0.19 0.76
-0.03 0.40 3.54
0.04 0.53 5.62
0.19 0.64 7.66
14.48
0.32 0.88

Lowry, C. A., Woodall, W. H., Champ, C. W., & Rigdon, S. E. (1992). A multivariate exponentially
weighted moving average control chart. Technometrics, 34(1), 46-53.
2 October 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

2 October 2024
14 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-19
20-09-2024
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Contents

Hotelling’s T2 chart
Multivariate EWMA
PCA
ICA
Clustering

2 October 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Hotelling’s T2 control chart

(Subgroup data)

UCL = 𝜒 2𝛼,2 where 𝜒 2𝛼,2 is the upper 𝛼 percentage point of the chi-square
distribution with 2 degrees of freedom

2 October 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Hotelling’s T2 control chart

(Individual data)

2 October 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate EWMA
The MEWMA is a logical extension of the univariate EWMA
and is defined as
where 0 ≤  ≤ 1 and Z0 = 0
The quantity plotted on the control chart is
the covariance matrix

2 October 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multivariate EWMA

Process is out of control

when T2H
Choose H from the table so as
to have ARL0  200

Prabhu, S. S., & Runger, G. C. (1997).

Designing a multivariate EWMA control
chart. Journal of Quality Technology,
29(1), 8-15.

2 October 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example
Control chart on pH and Viscosity data

2 October 2024 10
BITS Pilani, Pilani Campus
ET ZC362 Environmental Pollution Control

https://www.spcforexcel.com/knowledge/variable-control-charts/hotelling-
t2-control-chart/

11 BITS Pilani, Pilani Campus

CHE F315 Machine Learning for Chemical Engineers

2 October 2024
12 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-20
23-09-2024
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Contents

Hotelling’s T2 chart
Multivariate EWMA
PCA
ICA
Clustering

2 October 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

PCA for process monitoring

Teaching on board

2 October 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

2 October 2024
7 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-21
25-09-2024
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

PCA for process monitoring

(Demonstration)

Changing heat transfer fouling in a

𝑑𝐶 𝑄 multiplicative manner (𝑏 = 𝑏0 𝑒 −0.001𝑡 )
= 𝐶 − 𝐶 − 𝑎𝑘𝐶 + 𝑣1
𝑑𝑡 𝑉 𝑖
𝑑𝑇 𝑄 Δ𝐻𝑟 kC 𝑈𝐴
= 𝑇𝑖 − 𝑇 − 𝑎 −𝑏 𝑇 − 𝑇𝑐 + 𝑣2
𝑑𝑡 𝑉 𝜌𝐶𝑝 𝜌𝐶𝑝 𝑉
𝑑𝑇𝑐 𝑄𝐶 𝑈𝐴
= 𝑇𝑐𝑖 − 𝑇𝑐 + 𝑏 𝑇 − 𝑇𝑐 + 𝑣3
𝑑𝑡 𝑉𝑐 𝜌𝑐 𝐶𝑝𝑐 𝑉𝑐
2 October 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Variables Description
Ci Inlet concentration
Ti Input reactor temperature
C Output concentration
T Output reactor temperature
Qc Coolant flow rate
Tci Coolant inlet temperature
Tc Coolant outlet temperature

Introduction of heat transfer area fouling

Changing heat transfer fouling in a multiplicative manner (𝑏 = 𝑏0 𝑒 −0.001𝑡 )

2 October 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

2 October 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

2 October 2024
8 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-22
27-09-2024
BITS Pilani
Pilani Campus
 Mid Semester Revision
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Basic statistics

Central tendency
Dispersion
Variance, St dev, Covariance, Correlation
Distribution
Probability distribution
Continuous
Normal

Discrete

2 October 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

Missing values
– Deletion
– Replacement

Outliers
Univariate
|xk-µ| > 3
|xk-med| > 3×1.483MAD

2 October 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

Outliers
Multivariate
– MD
– MVT
– MCD
– MVE

2 October 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

Dimensionality reduction
Variable selection
Correlation
Similarity
Forward selection
Backward selection
Variable transformation
PCA, SVD, LDA, ICA

Data scaling

2 October 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Data preprocessing

Raw Missing Outlier

data value

Dimensionality
reduction
• Variable selection Normalization
• Variable
transormation

2 October 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

2 October 2024
9 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-23
30-09-2024
BITS Pilani
Pilani Campus
 Mid Semester Revision
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Univariate process monitoring

Need of process monitoring

• Safety
• Environment
• Product quality
• Production efficiency
• Profitability

Normal/abnormal operation
Shewhart control chart
(for process mean)

2 October 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Univariate process monitoring

Shewhart control chart (for Shewhart control chart (for
process variability) individual measurements)

2 October 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Univariate process monitoring

CUSUM (Tabular)

EWMA

2 October 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Unsupervised techniques
(Multivariate process monitoring)
Hotelling’s T2 chart

UCL = 𝜒 2𝛼,𝑝

Multivariate EWMA

PCA

2 October 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

2 October 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

2 October 2024
9 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-24
11-10-2024
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 Statistical Process
Monitoring/Control
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

PCA (Assumptions)

Latent variables are obtained by performing a linear

transformation on the dataset
high signal-to-noise ratio
The dataset follows a normal distribution
The dataset is from a steady state process

5 BITS Pilani, Pilani Campus

CHE F315 Machine Learning for Chemical Engineers

PCA (Modifications)

Batch process - Multi-way PCA

4 November 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

PCA (Modifications)

Time varying process

– Dynamic PCA
– Recursive PCA
– MW-PCA
– EWMA-PCA

Non-linear process
– Associative network
– Neural network using principal curve
– IT-net
– Kernel PCA

4 November 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Kernel PCA

4 November 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Kernel PCA

Müller, K. R., Mika, S., Tsuda, K., & Schölkopf, K. (2018). An introduction to kernel-based learning algorithms.
IEEE Transactions on Neural Networks, 12, 2001, 181-201.
4 November 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Kernel PCA

Instead of using the points xi as is, we go to some different feature space

φ(xi) ∈ R
Using polar coordinates instead of cartesian coordinates would help us
deal with the circle
We avoid explicitly going to feature space by working with kernels:
K(xi, xk) = φ(xi)T φ(xk)
In the higher dimensional space, we can then do PCA
The result will be non-linear in the original data space
4 November 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Kernel PCA

Kernel PCA with Gaussian Kernel (σ = 20)

https://www.cs.mcgill.ca/~dprecup/courses/ML/Lectures/ml-
lecture13.pdf

4 November 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Kernel PCA
𝑑
Polynomial kernel: 𝒌 𝒙𝒊 𝒙𝒋 = 𝒙𝒊 . 𝒙𝒋 + 𝟏
Sigmoid kernel: 𝒌 𝒙𝒊 𝒙𝒋 = 𝑡𝑎𝑛𝒽 𝛽0 𝒙𝒊, 𝒙𝒋 + 𝛽1
1
Inverse multiquadratic kernel: 𝒌 𝒙𝒊 𝒙𝒋 =
2
𝒙𝒊 −𝒙𝒋 +𝑑 2

Gaussian or radial basis function kernel: 𝒌 𝒙𝒊 𝒙𝒋 =

2
− 𝒌(𝒙𝒊 − 𝒙𝒋 )
𝑒𝑥𝑝
2𝜎 2
𝑑, 𝛽0 , 𝛽1 𝑎𝑛𝑑 𝜎 are kernel parameters.

4 November 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Pani, A. K. (2022). Non-linear process monitoring

using kernel principal component analysis: A review
of the basic and modified techniques with industrial
applications. Brazilian Journal of Chemical
Engineering, 39(2), 327-344.

4 November 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

ICA

Applicable to non-Gaussian process

In PCA, the principal components are linear combinations
of actual variables.
In ICA, the process variables are assumed to be linear
combinations of independent components.
Like PCA, number of independent components is equal to
the number of variables.
The purpose of any ICA algorithm is to determine the
independent components and the coefficients of the
linear combination from a given process dataset.

4 November 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

𝑋 = 𝐴𝑆 + 𝑌 𝑋𝜖 𝑅𝑛𝑥𝑟

A mixing matrix 𝐴 𝜖 𝑅𝑛𝑥𝑞 .

𝑆 independent component matrix 𝑆 𝜖 𝑅𝑞𝑥𝑟 .
𝑌 𝜖 𝑅𝑛𝑥𝑟 residual matrix with ‘𝑟’ number of samples.
Evaluation of 𝐴 (mixing matrix) and 𝑆 (independent
components) is done by the independent component
analysis algorithm. However, ICA can determine the
demixing matrix 𝑊 from the data matrix, which helps us
መ It is given by
to estimate the reconstruction matrix 𝑆.
𝑆መ = 𝑊𝑋
𝑊 = 𝐴−1

4 November 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Palla, G. L. P., & Pani, A. K. (2023). Independent component analysis application for fault detection in process industries: Literature
review and an application case study for fault detection in multiphase flow systems. Measurement, 209, 112504.

4 November 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Applications
ICA application on multiphase flow system

Palla, G. L. P., & Pani, A. K. (2023). Independent component analysis application for fault detection in process industries: Literature review
and an application case study for fault detection in multiphase flow systems. Measurement, 209, 112504.

4 November 2024 17
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Applications
KPCA application on Biological WWTP

(a) PCA monitoring charts, and (b) KPCA monitoring for the case of a linear decrease
in the nitrification rate (benchmark example).
Lee, J. M., Yoo, C., Choi, S. W., Vanrolleghem, P. A., & Lee, I. B. (2004). Nonlinear process monitoring using kernel
principal component analysis. Chemical engineering science, 59(1), 223-234.
4 November 2024 18
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Ge, Z., Song, Z., Ding, S. X., & Huang, B.

(2017). Data mining and analytics in the
process industry: The role of machine
learning. Ieee Access, 5, 20590-20616.

4 November 2024 19
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Ge, Z., Song, Z., Ding, S. X., & Huang, B. (2017). Data mining and analytics in the process industry: The role of machine learning. Ieee
Access, 5, 20590-20616.
4 November 2024 20
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
21 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-25
14-10-2024
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 Clustering
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Introduction
An unsupervised machine learning task that automatically
divides the data into clusters (groups of similar items)
Techniques for finding subgroups, or clusters, in a data set
on the basis of the characteristics of the objects
Clustering enables a large set of diverse and varied data to
be represented in a smaller number of groups (reduces
complexity)
As a stand-alone tool to get insight into data distribution
As a preprocessing step for other algorithms
Done using a trial and error approach

4 November 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

 A good clustering method will produce high quality clusters

with
 high intra-class similarity
 low inter-class similarity
Objects within a cluster are similar to each other (Homogeneity) -
but different from objects of other clusters (heterogeneity)
Level of similarity or dissimilarity is evaluated on the basis of
variable characteristics
Algorithms need a measure to find similarity or dissimilarity
among objects to group them  proximity measures
Distance measures – more distance  less similarity

4 November 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Proximity measures

Distance
Dij is the distance between objects i and j
Properties of distance measure:
Data types  quantitative and categorical
Quantitative data
Minkowski distance
Euclidean, city block (Manhattan), Chebyshev (maximum
value)

4 November 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Proximity measures

Example
Two samples have values of (0,3) and (5,8). Compute the
distance between the two samples

4 November 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Proximity measures

Cosine similarity
Used to measure similarity between objects
Measures the cosine of the angle between two vectors
projected in a multi-dimensional space
Two samples have values of (1,1,0) and (0,1,1). Compute
the cosine similarity between the two samples

4 November 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Techniques

• Partitioning methods
• Hierarchical methods
Creates hierarchical structure through decomposition
• Density-based methods
Identification of dense regions are the basis of cluster
formation
Useful in case of arbitrarily shaped clusters
• Grid-based methods
• Probabilistic model based methods

4 November 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Partitioning methods

Uses mean or medoid to represent cluster centre

Effective from small to medium datasets
Example –k means and k-mediod

4 November 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

k-means

Clusters do not overlap

A sample can belong to only one cluster
Assigns each of the ‘n’ data points to one of the K clusters.
Centroids is representative points for each cluster

4 November 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Step 1:Select K points in the data space and mark them as initial
centroids
loop
Step 2: Assign each point in the data space to the nearest
centroid to form K clusters
Step 3: Measure the distance of each point in the cluster from
the centroid
Step 4: Calculate the Sum of Squared Error (SSE) to measure
the quality of the clusters
Step 5: Identify the new centroid of each cluster on the basis of
distance between points
Step 6: Repeat Steps 2 to 5 to refine until centroids do not
change
end loop

4 November 2024 14
BITS Pilani, Pilani Campus
 ( t - 1)
- N(kkF315
x(1i )CHE t - 1)
x(i ) -  l(Learning
<Machine t - 1)
for Chemical Engineers
å x (i )
, k = 1, , K

Summary of K-means
N k(t - 1) i =1

clustering algorithm
Step 1: Initialize X, K, 1(0), …, K(0). Set t = 1.
Step 2: Classify N samples according to nearest k:
x(i) Î k cluster if

x(i ) - (kt - 1) < x(i ) - l(t - 1) for each l ¹ k

Identify Nk(t–1); k = 1, …, K.
Step 3: Recompute k:

N k( t - 1)
k(t) = 1
N k(t - 1)
å x(i ) , k = 1, ,K
i =1

Step 4: If stopping criterion is satisfied, stop; otherwise go to step 2. Set t = t + 1.

Step 5: Return

4 November 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Choosing optimum number of

clusters
The elbow method
The Silhouette Method
Gap statistics Technique
Calinski-Harabasz index
Davies-Bouldin index
Density Canopy Algorithm

4 November 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

Perform k-means clustering with the initial value of objects

2 and 5 with the coordinates (4,6) and (12,4) as initial
seeds
Objects X-coordinate Y-coordinate

1 2 4
2 4 6
3 6 8
4 10 4
5 12 4

4 November 2024 17
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Disadvantage

Skewed clusters when the means of the data points are

used as centroids.
example - 1, 2, 3, 5, 9, 10, 11, and 25.
Where Point 25 is the outlier

4 November 2024 18
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

k-medoids

Takes k point from the data as the center

Partitioning Around Medoids (PAM) algorithm

4 November 2024 19
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

PAM Algorithm
Step 1: Randomly choose k points in the data set as the initial
representative points
loop
Step 2: Assign each of the remaining points to the cluster which
has the nearest representative point
Step 3: Randomly select a non-representative point o in each
cluster
Step 4: Swap the representative point o with o and compute the
new SSE after swapping
Step 5: If SSEnew < SSEold, then swap o with o to form the new
set of k representative objects;
Step 6: Refine the k clusters on the basis of the nearest
representative point. Logic continues until there is no change
end loop

4 November 2024 20
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
21 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-26
16-10-2024
BITS Pilani
Pilani Campus
 Unsupervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 Clustering
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Techniques
Partitioning methods
– K-means/K-medoid
– Fuzzy K-means
– Mixture of Gaussian
– Spectral
Hierarchical methods
– Bottom up – Agglomerative
– Top down - divisive
Density-based methods
– DBSCAN
– OPTICS
Grid-based methods
Probabilistic model based methods
4 November 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Centroid: the “middle” of a cluster

Radius: square root of squared average distance from any

point of the cluster to its centroid

Diameter: square root of average mean squared distance

between all pairs of points in the cluster

4 November 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The temperature and pressure

values at a location in an industry
are recorded as given below

800 1.8
700 1.4
500 1.5

Determine the distance of the three

samples from the centroid.

4 November 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

k means clustering

Ways to assess convergence

Change in the positions of the centroids between iterations

Change in the within-cluster sum of squares (WCSS)

A lower WCSS value indicates that points are on average

closer to their cluster centroids

4 November 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Assess quality of clusters

Evaluate the goodness of a clustering by considering how

well the clusters are separated, and how compact the
clusters are
high intra-class similarity
low inter-class similarity

4 November 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Choosing optimum number of

clusters
The elbow method
The Silhouette Method
Gap statistics Technique
Calinski-Harabasz index
Davies-Bouldin index
Density Canopy Algorithm

4 November 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Running time per iteration:

1. Assign data points to closest cluster center O(KN) time
2. Change the cluster center to the average of its assigned
points O(N)
Need to specify k, the number of clusters, in advance
Sensitive to noisy data and outliers
Not suitable to discover clusters with non-convex shapes

4 November 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The elbow method

Run K-Means several times,
each time with a different
number of clusters
Plot the WCSS against the
number of clusters.
With increase in k, WCSS
decreases because the points
are closer to centroids.
There is a point where adding
more clusters doesn’t result in
substantial decrease in
WCSS.
This point, where the rate of
decrease sharply changes,
looks like an elbow on the
graph.

4 November 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

k-medoids

Takes k point from the data as the center

Partitioning Around Medoids (PAM) algorithm

4 November 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

PAM Algorithm
Step 1: Randomly choose k points in the data set as the initial
representative points
loop
Step 2: Assign each of the remaining points to the cluster which
has the nearest representative point
Step 3: Randomly select a non-representative point o in each
cluster
Step 4: Swap the representative point o with o and compute the
new SSE after swapping
Step 5: If SSEnew < SSEold, then swap o with o to form the new
set of k representative objects;
Step 6: Refine the k clusters on the basis of the nearest
representative point. Logic continues until there is no change
end loop

4 November 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial applications

Modern chemical plants maintain large historical databases

recording past sensor measurements.
Advanced process monitoring techniques analyze to help
plant operators and engineers interpret the meaning of
live trends in databases.
However, many of the best process monitoring methods
require data organized into groups before training is
possible.
Data mining and knowledge discovery techniques drawn
from computer science literature can help engineers find
fault states in historical databases and group them
together with little detailed knowledge of the process.

4 November 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Industrial applications

Thomas, M. C., Zhu, W., & Romagnoli, J. A. (2018). Data

mining and clustering in chemical process databases for
monitoring and knowledge discovery. Journal of Process
Control, 67, 160-175.

4 November 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Apply classical k-means clustering algorithm to classify the

following points into three clusters
Take points 2, 7 and 15 as initial cluster centers and
perform two iterations

Point 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Coordinat 2,10 2,6 11, 6,9 6,4 1,2 5,1 4,9 10, 7,5 9,1 4,6 3,1 3,8 6,11
e 11 0 12 1 0

4 November 2024 17
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
18 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-27
18-10-2024
BITS Pilani
Pilani Campus
 Supervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 ET ZC362 Environmental Pollution Control

Introduction

Applies to labelled data

Two types  classification, regression

Classification Regression
• K-NN • Linear
• Decision tree • Non-linear
• Random forest • ANN
• SVM • SVM

4 November 2024 4
BITS Pilani, Pilani Campus
 Linear regression
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Regression

Observe i=1,2,...,n pairs of variables

Yi = ith response variable
Xi = ith explanatory variable
Types of regression
Simple linear regression
Multiple linear regression
Polynomial regression
MARS
Logistics regression

4 November 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Simple Linear Regression

Explanatory and Response Variables are Numeric

Relationship between the mean of the response variable
and the level of the explanatory variable assumed to be
approximately linear (straight line)
Model:

• b1 > 0  Positive Association

• b1 < 0  Negative Association
• b1 = 0  No Association
•  is random error term (i and j are uncorrelated)
4 November 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Least square estimation of regression coefficients

n å xy - å x å y
b1 =
nå x - (å x )
2
2

b0 =
å y - b åx1

4 November 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Regression through origin

Chemical Chemical A in
A in input Y = 98.248+0.1098X
product(Y) (X)
245 1400 b0 is the estimated mean value of Y when the value
312 1600 of X is zero (if X = 0 is in the range of observed X
values)
279 1700
308 1875
199 1100
219 1550
Y = 0.1641X
405 2350
324 2450
319 1425
255 1700

4 November 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Assumptions of regression

Linearity
– The relationship between X and Y is linear
Independence of Errors
– Error values are statistically independent
– Particularly important when data are collected over a
period of time
Normality of Error
– Error values are normally distributed for any given value
of X
Equal Variance (also called homoscedasticity)
– The probability distribution of the errors has constant
variance

4 November 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Residual analysis

4 November 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Residual Analysis for

Homoscedasticity

4 November 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Residual Analysis for

Independence

4 November 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multiple linear regression

Numeric Response variable (Y)

p Numeric predictor variables
Model:
Y = b0 + b1x1 +  + bpxp + 

Partial Regression Coefficients: bi  effect (on the

mean response) of increasing the ith predictor
variable by 1 unit, holding all other predictors
constant

4 November 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multiple linear regression

Matrix Approach to Multiple Linear Regression

least squares estimate

4 November 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
17 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-28
21-10-2024
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Mid Sem Paper Distribution

4 November 2024 3
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
4 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-29
23-10-2024
BITS Pilani
Pilani Campus
 Supervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 ET ZC362 Environmental Pollution Control

Introduction

Applies to labelled data

Two types  classification, regression

Classification Regression
• K-NN • Linear
• Decision tree • Non-linear
• Random forest • ANN
• SVM • SVM

4 November 2024 4
BITS Pilani, Pilani Campus
 Linear regression
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Recap

Simple Linear Regression

n å xy - å x å y
b1 = å y - b åx
( ) b0 =
1
nå x 2 - åx
2
n

𝑌 = 𝛽𝑥 + 𝜀

4 November 2024 6
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Assumptions of regression

Linearity
– The relationship between X and Y is linear
Independence of Errors
– Error values are statistically independent
– Particularly important when data are collected over a
period of time
Normality of Error
– Error values are normally distributed for any given value
of X
Equal Variance (also called homoscedasticity)
– The probability distribution of the errors has constant
variance

4 November 2024 7
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multiple linear regression

Numeric Response variable (Y)

p Numeric predictor variables
Model:
Y = b0 + b1x1 +  + bpxp + e

Partial Regression Coefficients: bi  effect (on the

mean response) of increasing the ith predictor
variable by 1 unit, holding all other predictors
constant

4 November 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Multiple linear regression

Matrix Approach to Multiple Linear Regression

least squares estimate

When to apply:
• Errors should be independent, normal and randomly distributed
• Variance of errors should be constant

4 November 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Polynomial regression

4 November 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Model accuracy

How to decide whether the predicted values

are close enough to the actual values and if
our assumption of a particular functional
form for y and x is actually correct
4 November 2024
11 BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Model Evaluation Techniques

Graphical Statistical
• Standard Regression
• Dimensionless
• Error index

4 November 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Standard Regression

Coefficient of Determination (R2)

Proportion of variance in measured data explained by the
regression model
Lies between 0 to 1
Higher values  less error variance

4 November 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Standard Regression

Pearson Correlation Coefficient (R)

Describes the degree of collinearity
Lies between -1 and 1 

   y  y  ˆ
y  ˆ
y  


R = 0  No relationship
   y  y 2   yˆ  yˆ  
 2 
 
1 or -1 perfect positive or perfect negative linear relation

4 November 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Graphical
Slope and y-Intercept

 1 and  0

4 November 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Error Index
N
Mean of squared errors (MSE)   y  ˆ
y 2

i 1

Root Mean of squared errors (RMSE)   y  yˆ 

i 1
2

Mean Absolute Error (MAE) y

i 1
i  yˆ i

N
N

Mean Absolute Percentage Error (MAPE)  y i  yˆ i

i 1
 100
N

4 November 2024 16
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Error Index

• RMSE and MAE values less than half of the standard deviation of the measured
data are acceptable
• RMSE and MAE both have the units of difference between actual and predicted
value, for any model, the RMSE value is higher than MAE value because large
errors get amplified because of squaring phenomenon.
• MAE has been stated to be a better performance measuring criterion than the
RMSE

4 November 2024 17
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
18 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-30
28-10-2024
BITS Pilani
Pilani Campus
 Supervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 ET ZC362 Environmental Pollution Control

Introduction

Applies to labelled data

Two types  classification, regression

Classification Regression
• K-NN • Linear
• Decision tree • Non-linear
• Random forest • ANN
• SVM • SVM

4 November 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Model validation techniques

Splitting data into training and testing sets

Cross-validation using k-folds
Leave-one-out cross-validation method

4 November 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Overfitting and underfitting

Two problems to be avoided during model building
• Underfitting (high bias) - when model neither fits the training data
nor generalizes on the new data
• Overfitting (high variance) - model fits training data well but performs
poorly on the test data

https://in.mathworks.com/discovery/overfitting.html
4 November 2024 6
BITS Pilani, Pilani Campus
 Linear regression
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Linear Regression

Simple Linear Regression

n å xy - å x å y
b1 = å y - b åx
( ) b0 =
1
nå x 2 - åx
2
n

𝑌 = 𝛽𝑥 + 𝜀

Multiple linear regression

4 November 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Linear Regression

Given x and y data in columns, solve for the

coefficients of the best fit line for y=0+1x+2x2
Z = [ones(size(x)) x x.^2]
 = inv((Z’*Z))*(Z’*y)
– Note also that MATLAB’s left-divide will automatically
include the [Z]T terms if the matrix is not square, so
 = Z\y
would work as well

4 November 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Linear Regression
Built-in function polyfit
MATLAB has a built-in function polyfit that fits a least-
squares nth order polynomial to data:
– p = polyfit(x, y, n)
• x: independent data
• y: dependent data
• n: order of polynomial to fit
• p: coefficients of polynomial
f(x)=p1xn+p2xn-1+…+pnx+pn+1
MATLAB’s polyval command can be used to compute a
value using the coefficients.

4 November 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Example

X = [1 2 4 5 7 9 11 13 14 16]
Y = [101 105 109 112 117 116 122 123 129 130]
Use the built-in function Use MATLAB’s left divide operator

Use the Matrix Formula

Also try fitlm

4 November 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

The ordinary least squares (OLS) estimates are obtained

by minimizing the residual squared error
Two problems of OLS estimate
Accuracy
The OLS estimates often have low bias but large variance;
Prediction accuracy can sometimes be improved or setting
to 0 some coefficients.
By doing so we sacrifice a little variance of the predicted
values and hence may improve the accuracy.
Interpretation
With a large number of predictors, we often would like to
determine a smaller subset that exhibit the strongest
effects
4 November 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

when the independent variables are highly correlated or

multicollinearity, the matrix A is ill conditioned
Hoerl and Kennard (1970) proposed the ridge regression
to estimate β.
Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for
nonorthogonal problems. Technometrics, 12(1), 55-67.
4 November 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Techniques for improving OLS estimates

Subset selection – forward selection/backward elimination
Provides interpretable model but can be extremely variable
Ridge regression - a continuous process that shrinks
coefficients and hence is more stable
However, it does not set any coefficients to 0 and hence
does not give an easily interpretable model
LASSO shrinks some coefficients and sets others to 0,
retain the good features of both subset selection and
ridge regression

4 November 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
15 BITS Pilani, Pilani Campus
 Machine Learning for Chemical Engineers
CHE F315

Ajaya Kumar Pani

BITS Pilani Department of Chemical Engineering
B.I.T.S-Pilani, Pilani Campus
Pilani Campus
 Lecture-31
04-11-2024
BITS Pilani
Pilani Campus
 Supervised Machine
Learning Techniques
BITS Pilani
Pilani Campus
 ET ZC362 Environmental Pollution Control

Introduction

Applies to labelled data

Two types  classification, regression

Classification Regression
• K-NN • Linear
• Decision tree • Non-linear
• Random forest • ANN
• SVM • SVM

4 November 2024 4
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Model validation techniques

Splitting data into training and testing sets

Cross-validation using k-folds
Leave-one-out cross-validation method

4 November 2024 5
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Overfitting and underfitting

Two problems to be avoided during model building
• Underfitting (high bias) - when model neither fits the training data
nor generalizes on the new data
• Overfitting (high variance) - model fits training data well but performs
poorly on the test data

https://in.mathworks.com/discovery/overfitting.html
4 November 2024 6
BITS Pilani, Pilani Campus
 Linear regression
BITS Pilani
Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Linear Regression

Simple Linear Regression

n å xy - å x å y
b1 = å y - b åx
( ) b0 =
1
nå x 2 - åx
2
n

𝑌 = 𝛽𝑥 + 𝜀

Multiple linear regression

4 November 2024 8
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Linear Regression

Bias and variance in regression model

4 November 2024 9
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Techniques for improving OLS estimates

Subset selection – forward selection/backward elimination
Provides interpretable model but can be extremely variable
Ridge regression - a continuous process that shrinks
coefficients and hence is more stable
However, it does not set any coefficients to 0 and hence
does not give an easily interpretable model
LASSO shrinks some coefficients and sets others to 0,
retain the good features of both subset selection and
ridge regression

4 November 2024 10
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Acetylene data (MATLAB)

x1: Reactor temperature (degrees centigrade)

x2: Rate of H2 to n-heptane (mole ratio)
x3: Contact time (seconds)
y: Conversion of n-heptane to acetylene (%)

4 November 2024 11
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

>> plot(x1,x2,'.')
xlabel('x1')
ylabel('x2')

4 November 2024 12
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

>> X = [x1 x2 x3];

>> D = x2fx(X,'interaction');
D(:,1) = []; % No constant term
>> k = 0:1e-5:5e-3;
>> betahat = ridge(y,D,k);
>> figure
plot(k,betahat,'LineWidth',2)
ylim([-100 100])
grid on
xlabel('Ridge Parameter')
ylabel('Standardized Coefficient')
title('{\bf Ridge Trace}')
legend('x1','x2','x3','x1x2','x1x3','x2x3')

4 November 2024 13
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024 14
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

Compare

>> X = [x1 x2 x3]; >> X1 = [ones(size(x1)) x1 x2 x3];

>> beta = X\y; >> beta1 = X1\y;

Details of battery data: https://data.mendeley.com/datasets/cp3473x7xv/3

4 November 2024 15
BITS Pilani, Pilani Campus
CHE F315 Machine Learning for Chemical Engineers

4 November 2024
16 BITS Pilani, Pilani Campus