Seminar on

Transfer Learning-Based Smart Features Engineering for

Osteoarthritis Diagnosis From Knee X-Ray Images

by

Ananya T S
USN: 4SU20IS003
VIII Semester

Under the guidance of

Dr. G. P. Hegde
Associate Professor & Head

Department of Information Science and Engineering

SDM INSTITUTE OF TECHNOLOGY
UJIRE – 574240
2023-24
 SDM Institute of Technology, Ujire – 574240
Department of Information Science and Engineering

CERTIFICATE
This is to certify that Ms. Ananya T S (USN:4SU20IS003) has satisfactorily
completed seminar on “Transfer Learning-Based Smart Features
Engineering for Osteoarthritis Diagnosis From Knee X-Ray Images” in
partial fulfilment for the award of degree of Bachelor of Engineering in
Information Science and Engineering from Visvesvaraya Technological
University, Belagavi during the year 2023-24.

Date of Seminar presentation:

Signature of the student:

Dr. G. P. Hegde Dr. Usha G. R

Faculty Advisor and Head Seminar Coordinator
 Abstract

Osteoarthritis is a deteriorating joint disease that affects millions worldwide. It is characterized by

deterioration of the joint function and is associated with a high morbidity and mortality. In this study,
a novel transfer learning-based feature engineering technique CRK(CNN Random forest K-
neighbors) is proposed to detect osteoarthritis in knee X-ray images using an efficient mixture of deep
learning and multivariate information. The proposed model achieves 99% cross-validation accuracy
for predicting the progression of the disease. The performance comparison of the proposed study with
other state-of-the-art studies in detecting knee x-rays images shows that the proposed model
outperforms with 99% accuracy score for detecting the disease progression.

i
 Table of Contents

Sl. No Title Page No.

Abstract i
Table of Contents ii
List of Figures iii
1 Introduction 1
2 Literature Survey 2
3 System Design and Working 4
3.1 System Design 4
3.2 Working 5
4 Applications 8
5 Results and Discussions 10
5.1 Results 10
5.2 Discussions 11
6 Conclusion 13
References 14

ii
 List of Figures

Page No.
Figure 3.1 System Design of CRK-based Knee X-ray Data Engineering 4
Figure 3.2 Transfer learning-based feature engineering working flow
architecture 6
Figure 5.1 The confusion matrix analysis of machine learning techniques with
spatial features 10
Figure 5.2 The confusion matrix analysis of machine learning techniques with
probabilistic features. 11

iii
 1. Introduction

Osteoarthritis, a prevalent joint syndrome predominantly impacting the knee, manifests as the gradual
deterioration of cartilage, resulting in pain, swelling, and compromised mobility. Age, obesity, prior
joint injury, and genetic predisposition serve as notable risk factors for this condition. Treatment
modalities encompass lifestyle adjustments, physical therapy, medications, and in severe cases,
surgical intervention. Given its substantial healthcare burden and adverse effects on individuals'
quality of life, the timely prediction and management of osteoarthritis are imperative.

The emergence of neural networks and deep learning has revolutionized medical image recognition,
notably in interpreting X-rays, offering enhanced accuracy and efficiency. Transfer learning-based
feature engineering holds promise in refining X-ray recognition accuracy, allowing for customized
applications in specific medical contexts. Introducing CRK, a novel approach employing spatial
feature extraction from X-ray images, signifies a notable advancement in osteoarthritis prediction.

It embarks on a comprehensive exploration, comparing advanced machine learning and deep learning
methodologies, meticulously optimizing hyperparameters, and employing k-fold cross-validation for
robust accuracy validation. Detailed methodology, experimental evaluations, insightful findings, and
conclusive remarks are meticulously delineated in subsequent sections, offering valuable insights into
the potential impact and future trajectories.

1
 2. Literature Survey

A literature review comprehensively examines books, academic articles, and other relevant sources
pertaining to a specific topic, research area, or theory. It offers a descriptive, summarized, and
critically evaluated analysis of these materials in relation to the research problem at hand. Literature
reviews serve to present an overview of the sources explored during the research process and to
illustrate how the current study contributes to the broader field of inquiry. In the social sciences,
literature reviews typically adopt an organized structure, integrating both summarization and
synthesis, often categorized according to key conceptual themes.

In 2022, Insha Majeed Wani and Sakshi Arora [1] introduced an innovative method to address
osteoporosis diagnosis challenges, crucial for the elderly and postmenopausal women prone to
fractures. Their focus on cost-effective early detection led to leveraging X-ray imaging despite its
interpretational difficulties. Employing deep learning, particularly convolutional neural networks
(CNNs) like AlexNet, VggNet-16, ResNet, and VggNet-19, they classified knee joint X-rays into
normal, osteopenia, and osteoporosis categories, using a dataset of 381 knee X-rays validated by T-
scores.

The conventional diagnosis of osteoarthritis relies on manually examining medical images, presenting
challenges in accessibility and efficiency. Recent research has explored image-based deep learning
techniques for automated osteoarthritis detection, necessitating hospital visits for image collection.
In contrast, leveraging statistical data like medical utilization and health behaviour information offers
a more accessible avenue. Yifan Wang et al. (2021) proposed a novel approach utilizing deep neural
networks with scaled principal component analysis (PCA) to forecast osteoarthritis occurrence. Their
method demonstrates promising results in pre-screening patients for potential osteoarthritis, thereby
reducing healthcare expenses and hospital visits [2].

In 2019, Jihye Lim et al. [3] addressed the challenge of predicting osteoarthritis occurrence using
patient statistical data instead of medical images, aiming to streamline diagnosis and preventive care.
With a focus on knee, hip, and spine osteoarthritis, their study emphasized the importance of accurate
prediction for effective prevention. Utilizing a deep neural network, they analyzed statistical data
from 5749 subjects, employing principal component analysis with quantile transformer scaling to
2
extract features from patients' medical utilization and health behaviour records. Their method
achieved a promising 76.8% area under the curve (AUC), indicating its potential as a pre-screening
tool to reduce healthcare costs and patient hospital visits.

In 2021, Mario Herrera-Pérez et al.[4] conducted a review focusing on ankle osteoarthritis, a

condition impacting around 1% of the population. Unlike other forms of arthritis, ankle osteoarthritis
often arises as a consequence of prior trauma, with over 75% of cases linked to previous injuries.
Notably, this condition predominantly affects a younger, more active demographic, leading to socio-
occupational implications. Herrera-Pérez's study emphasized the role of mechanical factors like
incongruity, instability, malalignment, and impacts in exacerbating stress on specific areas of ankle
cartilage, contributing to osteoarthritis development. Moreover, the paper highlighted the significance
of pro-inflammatory mediators present post-fracture, initiating a cascade leading to chondrocyte cell
death, a pivotal factor in ankle osteoarthritis progression.

Bochen Guan et al. (2022) conducted a study aiming to develop and assess deep learning (DL) risk
assessment models for predicting pain progression in individuals with or at risk for knee osteoarthritis
(OA). Utilizing data from the Osteoarthritis Initiative, the authors performed a retrospective analysis
involving 9348 knees in 4674 subjects. They randomly stratified subsets of knees into training and
hold-out testing datasets and developed DL models using baseline knee radiographs. Additionally,
they created traditional risk assessment models incorporating demographic, clinical, and radiographic
factors. The study found that DL models outperformed traditional models in predicting pain
progression, with DL models achieving higher diagnostic performance. The combined model,
integrating demographic, clinical, and radiographic factors with DL analysis, demonstrated the
highest performance, indicating its potential for enhancing pain progression prediction in knee OA
patients [5].

3
 3. System Design and Working

3.1 System Design

System design is the process of creating a high-level architecture or blueprint of a software or
hardware system. It involves identifying the requirements of the system, decomposing those
requirements into smaller components, defining the interactions between those components, and
creating a plan for implementing and deploying the system.

Figure 3.1: System Design of CRK-based Knee X-ray Data Engineering

4
3.2 Working
The process of knee-based X-ray image data engineering is as follows:
1. Data Acquisition: Acquisition step involves collecting knee X-ray images. These images may
come from various sources, including hospitals, clinics, and imaging centers.
2. Data Splitting: After the data is collected, it’s then divided into two main sets: training and
testing. The training set (typically 80% of the data) is used to train a machine learning model
to identify patterns and relationships within the data. The testing set (typically 20% of the data)
is used to evaluate the performance of the trained model on unseen data.
3. CRK Feature Engineering: CRK likely refers to Cartilage Radiographic Knee (CRK) score,
a semi-quantitative scoring system used to assess the severity of knee osteoarthritis based on
specific radiological features observed on knee X-rays. In this stage, features or characteristics
relevant to diagnosing OA are extracted from the knee X-ray images. Feature engineering can
involve a combination of manual techniques and machine learning algorithms.
4. Model Training: The extracted features and labels (Osteoarthritis or Normal) from the
training set are fed into a machine learning model. The model learns to identify patterns and
relationships between the features and the labels.
5. Model Prediction: Once the model is trained, it can then be used to make predictions on the
testing set. The model predicts whether a new knee X-ray image is indicative of osteoarthritis
or normal.
6. Evaluation: The performance of the model is evaluated on the testing set. This evaluation
helps assess how accurate the model is in classifying new knee X-ray images.
7. Outperformed AI Model: The text “Outperformed AI Model” suggests that the proposed
CRK feature engineering technique resulted in a model that performed better than a previous
AI model on the same task.

Overall, knee-based X-ray image data engineering is a crucial step in developing machine learning
models for diagnosing knee OA. By carefully engineering the data, it’s possible to improve the
accuracy and performance of these models.

5
The novel proposed transfer learning-based feature engineering working flow architecture

Figure 3.2: Transfer learning-based feature engineering working flow architecture

The above image is a diagram of a study comparing two different artificial intelligence (AI) techniques
for analyzing knee X-ray images for osteoarthritis diagnosis.

The text in the image refers to the two different AI techniques used in the study:
Convolutional Neural Network (CNN): This is a type of AI that is commonly used for image
recognition. It works by breaking down an image into smaller parts and then learning to identify
patterns in those parts. In the context of knee X-ray images, a CNN could be used to learn to identify
patterns that are associated with osteoarthritis.

Random Forest (RF): This is a type of AI that is used for classification. It works by creating a series
of decision trees, which are like flowcharts that ask a series of questions about the data. In the context
of knee X-ray images, a random forest could be used to learn to classify images as either normal or
osteoarthritic.

The diagram shows that the CNN approach achieved higher accuracy than the random forest approach
in this particular study. This is likely because CNNs are better at capturing the complex patterns that
exist in medical images.

6
The text Input Size: refers to the size of the input images that the AI models were trained on. In this
case, the input images were 244 pixels wide, 244 pixels tall, and have 3 color channels (red, green,
and blue).
Output Size: This refers to the size of the output that the AI models produce. In this case, the output
is a single number that represents the probability that the knee X-ray image shows signs of
osteoarthritis.

Algorithm
CRK Algorithm
Input: X-ray images data of the knee.
Output: Transfer learning-based features.
initiate;
1- Fcnn ← CNNprediction(XiS) // XiS ϵ X −rayimages set, here XiS is original image data and Fcnn is
extracted spatial features set.
2- Frf ← RFprobabilities prediction(Fcnn) // here Frf is the extracted probabilistic based features set.
3- Fkn ←KNprobabilities prediction(Fcnn) // here Fkn is the extracted probabilistic based features set.
4- FProb ← Frf + Fkn // here FProb is final probabilistic based features set used for osteoarthritis
detection.
end;

7
 4. Applications

The applications of Transfer Learning and Deep Learning Techniques includes the following:
1. Osteoarthritis Diagnosis: Using advanced transfer learning and machine learning techniques
for accurate detection and prediction of osteoarthritis based on knee X-ray images.
2. Automated Disease Detection: Potential for automated detection and classification of diseases
in medical imaging, similar to the proposed method for osteoarthritis prediction.
3. Healthcare AI: Integration of advanced machine learning techniques in healthcare for
improved disease diagnosis and prediction using medical imaging data.
4. Biomedical Imaging: Enhancing feature extraction and analysis in biomedical imaging for
accurate disease detection and monitoring
5. Computer-Aided Diagnosis: Supporting healthcare professionals with AI-based tools for faster
and more accurate diagnosis and treatment planning.
6. Document Information Security: Leveraging research expertise in document information
security and management for practical applications in data protection.
7. Bioinformatics: Applying knowledge in bioinformatics for disease detection and classification
using AI models in various healthcare scenarios.
8. Data Mining: Utilizing data mining techniques for extracting valuable insights from large
datasets, potentially applicable in various industries beyond healthcare.
9. Feature Engineering: Implementing novel CRK approach for spatial feature extraction to
enhance performance in osteoarthritis prediction.
10. Machine Learning Models: Demonstrating improved performance with probabilistic features,
potentially revolutionizing osteoarthritis prediction methods.
11. Image Classification: Applying CNN for spatial feature extraction from X-ray images of the
knee to train machine learning models for high-performance osteoarthritis prediction.

In the realm of healthcare and biomedical research, the applications of advanced transfer learning and
machine learning techniques are vast and impactful. These technologies play a crucial role in
enhancing disease detection and classification in medical imaging, leading to more accurate diagnosis
and treatment planning. By implementing healthcare AI solutions, such as automated disease detection
and prediction models, these techniques have the potential to revolutionize healthcare practices.
Furthermore, the integration of AI models in biomedical research facilitates advancements in disease

8
diagnosis and monitoring, particularly in the field of bioinformatics. Beyond healthcare, the expertise
in document information security can be applied to ensure data protection and management in various
industries. Additionally, the utilization of data mining techniques enables the extraction of valuable
insights from large datasets, supporting decision-making processes across diverse sectors.

9
 5. Results and Discussions

5.1 Results
The confusion matrix analysis of machine learning techniques with spatial features

Figure 5.1: The confusion matrix analysis of machine learning techniques with spatial features.

The confusion matrix analysis of machine learning techniques with spatial features is illustrated in
Figure 5.1. The confusion matrix is a table that summarizes the predictions made by the model against
the ground truth labels of a dataset. By analyzing the metrics in the confusion matrix, we can gain
insights into the strengths and weaknesses of the model and identify areas for improvement. This
confusion matrix analysis concludes that machine learning models achieved low-performance scores
with spatial features. Applied methods achieved a high error rate for X-ray image classification.

10
The confusion matrix analysis of machine learning techniques with probabilistic features

Figure 5.2: The confusion matrix analysis of machine learning techniques with probabilistic
features.

The confusion matrix analysis of machine learning techniques with probabilistic features is illustrated
in Figure 10. The machine learning methods achieved high-performance scores with probabilistic
features by analyzing the confusion matrix. The analysis concludes that applying the probabilistic
features methods achieved a low error rate for x-ray image classification.

5.2 Discussions
The X-ray images of the knee are used to detect osteoarthritis with advanced neural network-based
feature engineering techniques. Extensive results experiments are conducted to validate the
performance of each applied model. The spatial and probabilistic features extracted from the knee’s
X-ray images are mainly utilized to compare the performance of applied machine-learning techniques.
The result comparison shows that the applied machine learning techniques achieved a low-

11
performance score using the spatial features extracted from original x-rays images. However, using
the probabilistic features, the applied method performed the highest score. In addition, they have
applied the feature space analysis to show that the proposed CRK approach significantly enhances the
performance of applied machine learning techniques. The study results demonstrate that the proposed
SVM model outperformed the state-of-the-art studies. Furthermore, they have also analyzed the
computational complexity of applied machine-learning techniques to test the latency for predicting
osteoarthritis.

12
 6. Conclusion

The use of advanced transfer learning and machine learning techniques can improve the diagnosis of
osteoarthritis by analyzing knee X-ray images. In this study, a new method called CRK was
introduced, which uses transfer learning based feature engineering to achieve better results than
traditional CNN and other machine learning methods. By combining a 2D-CNN for spatial feature
extraction with probabilistic feature sets derived from random forest and k-neighbors techniques, the
proposed model achieves an accuracy rate of 99%, as well as high recall, f1 score, and precision
metrics. The proposed approach was validated through a rigorous process of hyperparameter
optimization and k-fold cross-validation, ensuring its reliability and robustness. This study highlights
the potential of transfer learning and machine learning techniques to revolutionize the diagnosis of
osteoarthritis, providing clinicians with a highly accurate tool for early detection and effective
treatment planning.

13
 References

[1] Wani, Insha Majeed, and Sakshi Arora. “Osteoporosis diagnosis in knee X-rays by transfer
learning based on convolution neural network.” Multimedia tools and applications vol. 82,9
(2023): 14193-14217. doi:10.1007/s11042-022-13911-y
[2] Wang, Yifan et al. “An Automatic Knee Osteoarthritis Diagnosis Method Based on Deep
Learning: Data from the Osteoarthritis Initiative.” Journal of healthcare engineering vol.
2021 5586529. 27 Sep. 2021, doi:10.1155/2021/5586529
[3] Lim, Jihye et al. “A Deep Neural Network-Based Method for Early Detection of
Osteoarthritis Using Statistical Data.” International journal of environmental research and
public health vol. 16,7 1281. 10 Apr. 2019, doi:10.3390/ijerph16071281
[4] Herrera-Pérez, Mario et al. “Ankle Osteoarthritis Aetiology.” Journal of clinical medicine
vol. 10,19 4489. 29 Sep. 2021, doi:10.3390/jcm10194489
[5] Guan, Bochen et al. “Deep learning approach to predict pain progression in knee
osteoarthritis.” Skeletal radiology vol. 51,2 (2022): 363-373. doi:10.1007/s00256-021-
03773-0.

14