Combining CNNs for the Detection ofDiabetic

Retinopathy
Nabia Khalid Mohamed Deriche
AIRC, College ofEngineering and Information Technology AIRC, College ofEngineering and Information Technology
Ajman University Ajman University
Ajman, UAE Ajman, UAE
202010117@ajmanuni.ac.ae m.deriche@ajman.ac.ae

Abstract- Diabetes is a major public health issue that affects presents a number of issues for the MoH globally, including:
approximately forty million individuals in the United States Screening for diabetes-related blindness are insufficient,
alone. A common side effect of diabetes is vision loss and Insufficient DR screening resources and Inadequate patient
blindness caused by diabetic retinopathy (DR). The goal of this knowledge or awareness.
research is to introduce a robust deep learning approach for the
2023 24th International Arab Conference on Information Technology (ACIT) | 979-8-3503-8430-7/23/$31.00 ©2023 IEEE | DOI: 10.1109/ACIT58888.2023.10453830

early detection of DR from retinal images into five categories The goal of this work is to develop a computerized robust
namely No DR, Mild DR, Moderate DR, Severe DR, Non- DR detection technique that can help with better management
Proliferative DR. Here, we propose a fusion of CNNs with an of diabetic retinopathy and, ultimately, creates a reliable
optimal weighting scheme to improve classification accuracy. system for DR screening. The proposed system not only detect
The dataset used is Kaggle APTOS 2019 and for cross dataset but also classify DR severity using retinal fundus. This system
validation we used IDRiD dataset. The proposed weighted twin will be used to monitor DR generally, to categorize images
CNN algorithm is implemented using a pair of pre-trained deep into five categories namely No DR, Mild DR, Moderate DR,
networks namely the DenseNet-169 and the InceptionV3. Such Severe DR, Non-Proliferative DR. Creating an effective
a hybrid combination provided a robust and an optimized model for DR detection will offer benefits like early diagnosis,
architecture. A total of 98.43% sensitivity and 88.78% improved accuracy, efficient screening, scalability, resource
specificity are recorded with a Kappa score and accuracy of optimization, and cost savings, ultimately leading to enhanced
95.8% and 94.3%. Our research has achieved a significant patient outcomes and better public health insights.
11.90% improvement as compared to state of the art,
showcasing remarkable performance in this field.
2 LITERATURE REVIEW
Keywords-Diabetic Retinopathy, convolutional neural Diabetes patients should be screened on a frequent basis
network, DenseNet, Inception V3, Multi-CNN, Recursive Region since early diagnosis of exudates can help avert blindness.
Growing Segmentation RRGS.
Nevertheless, manual inspection by ophthalmologists requires
time, and there are not enough specialists to fulfill the demand
1 INTRODUCTION for detection. The possibility of automatic detection of retinal
Diabetes is a significant public health concern, affecting exudates for use in diagnosis and follow-up of a patient's
around forty million individuals in the United States alone, treatment regimen is fascinating given the limits of human
with an increasing prevalence in both type 1 and type 2 testing. It is worth noting that major research efforts have been
diabetes among children in recent years. One of the critical put in recent times in developing advanced ML techniques.
complications associated with diabetes is diabetic retinopathy Reydon et al. [5] used ML with the EyeArt software on
(DR), a leading cause of visual impairment in young 30,405 images to detect DR, achieving a 54% specificity
individuals. Early detection through routine eye checkups is compared to human grading at 68%. Wolf et al. [4] employed
crucial, as DR can result in irreversible vision loss if not AI with fundus images, resulting in a 85.7% specificity and
identified promptly. DR has two main forms: non- 79.3% sensitivity. Abramoff et al. [1] combined AlexNet and
proliferative diabetic retinopathy (NPDR), characterized by VGGNet on 1,748 images, showing a 30% increase in
minimal or no apparent symptoms but potential damage to specificity for deep learning (96.8% sensitivity, 87%
retinal blood vessels and fluid leakage, and proliferative specificity). Zeng et al. [13] introduced a Siamese-like CNN
diabetic retinopathy (PDR), the most advanced and severe model with an AUC of0.95, outperforming Inception V3 for
form, marked by inadequate circulation leading to the growth DR detection.
of weaker blood vessels that can leak into the vitreous, causing
blurred vision. Understanding these aspects of DR is vital for This study introduces a novel deep learning approach for
its prevention and management[2]. diabetic retinopathy (DR) detection, using two
complementary networks to extract deep features and achieve
The pace of deterioration can be reduced or avoided if it is robust classification across five DR classes. It addresses
predicted ahead of time. Adequate glycemic management can challenges like limited data, imbalanced datasets, varying
help to reduce the risk of retinopathy developing and expert annotations, and interpretability, while considering
progressing. Only 17 percent of pediatric patients attain the ethical concerns, computational resources, and generalization
prescribed HbAlc target of7.5 percent for optimum glycemic across diverse populations, offering an innovative solution to
management, according to the American Diabetes improve DR detection.
Association. [3]
Every year, there are more and more diabetics who get DR.
Therefore, addressing instances of diabetic retinopathy

1
Authorized licensed use limited to: Jaypee Insituite of Information Technology-Noida Sec 128 (L3). Downloaded on November 19,2024 at 07:10:05 UTC from IEEE Xplore. Restrictions apply.
 3 PROPOSED MODEL The following eq (1) represents the mathematical
The methodology used in this research is based on three representation of our model.
stages i.e. dataset loading, preprocessing and a data splitting (1)
module followed by Dual CNN model, where the InceptionV3
and the DenseNet-169 are used, and fmally a prediction where x denotes the image, y is the class of x and C(l,x) is the
module which uses a weighted sum function. Our proposed deep learning model that predicts the class y of the WTC
work formulates a weighted twin CNN deep learning model (weighted twin model). l is the function used to update the
whose aim is to predict the five classes of DR with reduced learning rate ifthere are no changes to the validation loss after
computational power and time used as compared to previous 3 epochs. The fmal output layer of the WTC model uses
approaches. The models used are the DenseNet-169 and the softmax function to determine the probability of x belonging
InceptionV3 based on extensive experiments performed as per to classy.
table III to investigate the best ensemble model to predict DR.
the overall proposed workflow is shown in Fig. 1. 4 EXPERIMENTAL RESULTS
The output of the two models is the weighted sum function
as explained later. As the DenseNet-169 gave the highest 4.1 Datasets
overall accuracy, thus the weight assigned to the DenseNet- In this study, two deep learning models were tested using
169 model is smaller than the weight assigned to the the APTOS 2019 blindness detection dataset [6]. The IDRiD
InceptionV3 model since enough learning has been achieved dataset was used for multi-dataset validation and cross dataset
by the DenseNet-169. validation. In the second quarter of 2019, the Asia Pacific
Tele-Ophthalmology society published this dataset. The
The two models were trained for a total of 50 epochs with
dataset contains 3662 images as shown in table I. These
callbacks of early stopping if the validation loss metric
images are of size 350x350 and consists of 5 classes i.e., Class
converges after 5 epochs. The learning rate was also updated
0 is the absence of retinopathy, Class 1 is Mild Non-
if the validation loss stopped improving for at least 3 epochs.
Proliferative Retinopathy with microaneurysms and fluid
The initial learning rate was set to 0.001 with minimum
infiltration, Class 2 is Moderate Non-Proliferative
learning rate drop to 0.000001.
Retinopathy with vessel abnormalities, Class 3 is Severe Non-
The proposed algorithm uses the DenseNet-169 and the Proliferative Retinopathy with blocked blood vessels and new
InceptionV3 as the pretrained models for transfer learning. vessel growth, and Class 4 is Proliferative Diabetic
The model is trained on APTOS 2019 dataset {X,Y} where X Retinopathy with fragile, bleeding vessels and retinal
is the image of size 256x256 with 3 channels and Y denotes detachment risk[6]. These stages help diagnose and manage
the DR class (No DR, Mild, Moderate, Severe, Proliferative diabetic retinopathy in diabetes care. This dataset is a massive
DR). collection of retinal photos obtained with a fundus camera
under various imaging situations. The dataset was used in a
Input Dataset 60, 20, 20 ratio for training, validation and testing respectively
for benchmarking purposes.
Table I
Dataset Loading
THE APTOS 2019 DATASET
& Preproce ing
Classes Number of Images
Module
0 (No DR) 1805
Dataset Splitting
1 (Mild DR) 370
2 (Moderate DR) 999
3 (Severe DR) 193
4 (Proliferative DR) 295

Dual C NModel
For cross dataset validation used here is the IDRiD dataset
Output
InceptionVJ
with 455 photos. A suitable blend of disease stratification
indicative of DR and diabetic macular edema (DME) is
observed, experts confirmed, and all photos are of appropriate
quality, medically significant, and unique.

4.2 Data Preprocessing

Algorithms proposed in this work were implemented using
Prediction Python together with a number of image processing functions
Module for the OpenCV library. As the images for the APTOS 2019
dataset include a large amount of noise and sometimes blurred
Fmal Prediction features, thus image preprocessing is of utmost importance.
Gaussian blur was first used to highlight unique elements in
Figure 1: Flow chart of the proposed methodology the fundus images. The picture is convolved using a Gaussian
filter, a low-pass filter that eliminates the strong components,
in the Gaussian Blur process. Fig. 2 shows an output after

2
Authorized licensed use limited to: Jaypee Insituite of Information Technology-Noida Sec 128 (L3). Downloaded on November 19,2024 at 07:10:05 UTC from IEEE Xplore. Restrictions apply.
 applying Gaussian blur to a class 3 sample image. Ambient accomplish is known as expected accuracy, and it is directly
noise was then removed from the fundus pictures, and a correlated with the number of examples that belong to each
circular crop was performed in order to make the whole class as well as the number of instances when the predicted
dataset of standard size. value matched the proper label.
To enhance the precision and reduce computation time, a
circular region was extracted from fundus images, centered at 4.5 Pre-trained Model Selection
the image's center and extending to the minimum of width or
height. The area outside this circle was filled with pixel values The twin CNN model approach used in this research was
of 0, eliminating redundant data that doesn't contribute to based on the findings of how various pretrained models
diabetic retinopathy identification. This focused circular respond to the preprocessed APTOS 2019 images. Table II
portion with highlighted features in Figure 5 aids the deep shows the accuracy of various pretrained models which were
learning model in extracting relevant information efficiently. tested. It is notable that the models that performed were
DenseNet-169, MobileNet and lnceptionV3.
Class 0- No OR Class 3- Severe DR

Based on the findings of table II, the ensemble model

testing was formulated based on the best performing model in
combination of either of the top three models, i.e. MobileNet,
lnceptionV3, and ResNet50. The best ensemble model was
found to be the DenseNet-169 and the lnceptionV3 i.e. 91.8%.
Thus the ensemble model used in this research was the
InceptionV3 for Modell and the DenseNet-169 for Model2.

Table II
ACCURACY OF DIFFERENT CNN MODELS
Model Accuracy
VGG 16 71.3

ResNet 50 78.2

InceptionV3 79.7
o 3> .«:1 eo eo 100 t20 MO
~~CltO.Ml'Cn,pandGM,--,Blur
MobileNet 83 .6

DenseNet-169 85.2

According to a research conducted by Wan et. al [7], the

AlexNet performance varies greatly when it is used in
ensembled model as such it was not included in this research.
Whereas, the best model when used in the ensemble
framework was InceptionV3 model. Table III shows the
results of the InceptionV3 when combined with the DenseNet-
169 model.
Figure 2: Data preprocessing steps for samples .from class 0 and 3
4.6 DR Classification Experiments
4.3 Computational Environment For DR classification, the proposed model was examined
using the APTOS 2019 and IDRiD datasets. The model was
The experiments were conducted on a Nvidia Telsa PlO0 trained to categorize the DR score using a multi-class
GPU with2 CPU cores and 13Gigabytes of RAM. The code is classification method. To compare the suggested technique
written in Python along with TensorFlow and Keras.
with existing research, Kappa score, accuracy, recall,
precision, specificity, and Fl metrics are used.
4.4 Performance Metrics
A number of performance metrics have been proposed for
evaluating the performance of classifiers in practical setups
0.00 0.00 0.00 0.00
such as accuracy, precision, recall, and Fl score. O • NoDR

0.8
Additionally, we also compared observed accuracies with l · Mild 0.00 0.01
predicted accuracies using a different metric known as the 0.6

Kappa statistic. It is calculated as: 2 • Moderate 0.01 0.03

-
0.4

OA-EA 3 • Severe 0.00 0.03 0.05

Kappa Score = i-EA (3) -0.2
4 - Prohferat1ve DR 0.00 0.00 0.13 0.00
-o.o
Where OA is the observed accuracy and EA is the 0-NoOR 1 - Mlld 2 - Moderate 3 - Severe 4 - Proliferative DR

expected accuracy. The total number of samples that are Figure 3: Average Confusion Matrix on the APTOS 2019 Dataset
properly categorized is known as observed accuracy. The
accuracy that somehow classification would be expected to

3
Authorized licensed use limited to: Jaypee Insituite of Information Technology-Noida Sec 128 (L3). Downloaded on November 19,2024 at 07:10:05 UTC from IEEE Xplore. Restrictions apply.
 The performance analysis was carried over four phases. Table IV
The initial phase of the evaluation involved the selection of PERFORMANCE ANALYSIS ON THE APTOS 2019 DATASET
the two best models for transfer learning. The results Metric Class 0 Class 1 Class 2 Class 3 Class 4
presented in Table II guided the choice for the optimal models Fl-Score 0.99 0.87 0.92 0.89 0.85
of the ensemble CNN. The first CNN model chosen was the
DenseNet-169, primarily due to its strong overall accuracy, Recall 1.00 0.81 0.94 0.82 0.87
after which it was compared to the remaining models. Precision 0.99 0.93 0.90 0.97 0.84
In the subsequent stage, the DenseNet-169 was pitted Sensitivity 0.98 0.99 0.95 0.99 0.98
against the top three models with the highest accuracy. The
outcomes of this comparison were detailed in Table III, Specificity 0.99 0.81 0.94 0.83 0.87
demonstrating the performance of the DenseNet-169 in Accuracy 0.99 0.97 0.96 0.98 0.98
conjunction with MobileNet, lnceptionV3, and ResNet5
which shows better accuracy when the DenseNet-169 is
combined with the lnceptionV3 with an accuracy of91.8%.
The suggested WTCM was tested using cross dataset
Table III validation by using APTOS 2019 dataset and IDRiD dataset
ACCURACY OF ENSEMBLE MODELS ON APTOS in order to further analyze its efficiency. The proposed model
Model I Model2 Accuracy divided the labeled data into training, validation, and test sets
DenseNet-169 MobileNet 90.3 using splits of 60, 20 and 20 respectively. In our research, we
chose a 60-20-20 split instead of an 80-10-10 split due to the
DenseNet-169 InceptionV3 91.8 APTOS dataset's moderate size. This allocation provided a
DenseNet-169 ResNet50 88.9 larger training set, which was valuable for handling
complexity. The 20% validation set supported effective
hyperparameter tuning, and the 20% test set ensured robust
The third phase entailed testing on the APTOS 2019 model performance assessment. This split balanced training
dataset, with the results summarized in Table IV, showcasing data volume, validation robustness, and test result
a noteworthy Kappa score of 96.2% in comparison to prior significance, aligning with our research objectives and dataset
research[8][9][10]. Table V demonstrates that the Kappa score characteristics.
achieved with the APTOS dataset fell below 79%, whereas The findings of the comparison between the suggested
our proposed method reached a significant improvement of approach and the earlier work are shown in Table V which
21.52% in Kappa Scores. makes it obvious that the suggested WTCM performed better
Lastly, the final stage encompassed cross-dataset than all earlier studies. The model construction makes it clear
validation using the IDRiD dataset Cross-dataset validation how well the weighted dual CNN models detect retinopathy.
was conducted by utilizing a pre-trained model from the Fig. 3 for the APTOS 2019 dataset offers the normalized
APTOS dataset and assessing its performance on the IDRiD confusion matrix to further analyze the WTCM's
dataset. This validation was essential to evaluate how well the discriminative capacity for differentiating across diabetes
model generalized across different datasets, highlighting the groups. The confusion matrix shows that the model's
importance of assessing its adaptability to variations and classification error is nearly same for each class. This
differences between the APTOS 2019 dataset and the IDRiD information was obtained by integrating the findings of two
dataset. These differences could include variations in image CNNs. The performance of the weighted sum approach
quality, patient demographics, and disease prevalence, and substantially enhanced classification accuracy.
ensuring the model's effectiveness in this cross-dataset The suggested WTCM produced more reliable
scenario was crucial for broader applicability in real-world classification results for diagnosing diabetic retinopathy than
settings. any other approach. The performance measures for APTOS
2019 dataset in table IV shows how well the WTCM model
performed based on each class. Fig. 4 also show loss and
accuracy graphs to further illustrate the performance of the
proposed model.
TableV
COMPARISON OF PROPOSED MODEL WITH EXISTING
RESEARCH ON APTOS 2019 DATASET
Model Kappa Accuracy Sensitivity Specificity
Score
ResNet50 [8] 0.78 0.74 0.56 0.85

InceptionV3 [9] 0.79 0.78 0.63 0.85

MobileNet [10] 0.77 0.84 0.91 0.90

Proposed 0.96 0.94 0.90 0.99

Method
Figure 4: Loss & Accuracy for APTOS 2019 dataset

The WTCM model was also tested using cross dataset

validation with IDRiD dataset. Our model surpassed the
performance of the previous research conducted by Li X et

4
Authorized licensed use limited to: Jaypee Insituite of Information Technology-Noida Sec 128 (L3). Downloaded on November 19,2024 at 07:10:05 UTC from IEEE Xplore. Restrictions apply.
 al.[12], achieving an accuracy of 89%, which was 4 6 REFERENCES
percentage points higher than their 85%, indicating a
significant 4% improvement in accuracy. The execution time [l] Abra 'moffMD, Lou Y, Erginay A, et al. Improved automated detection
per epoch was recorded to be 2.93 minutes which is lower than of diabetic retinopathy on a publicly available dataset through
the time required per epoch compared to the research integration of deep learning. Invest Ophthalmol Vis Sci 2016; 57:5200
conducted by Al Antary et. al[l 1] as shown in table VI. - 5206.
[2] Osareh, Alireza & Mirmehdi, Majid & Thomas, Barry. (2002).
Classification and Localisation of Diabetic-Related Eye Disease.
Table VI Proceedings of 7th European conference on computer vision-Part IV,
COMPARISON OF COMPUTATION TIME Vol. 2. 2353. 10.1007/3-540-47979-1_34.
Previous research Computation time per epoch
[3] J. M. Shivaram, R. Patil and Aravind H. S, "Automated Detection and
Al Antary et. al[25] 4 minutes Quantification of Haemorrhages in Diabetic Retinopathy Images Using
Image Arithmetic and Mathematical Morphology Methods",
Proposed Method 2.93 minutes International Journal of Recent Trends in Engineering (IJRTE),
pp.174- 176, Volume 2, 2009.
[4] Wolf, Risa & Liu, T.Y. & Thomas, Chrystal & Prichett, Laura &
Galler, Ingrid & Smith, Kerry & Abramoff, Michael & Channa,
Roomasa. (2021). The SEE Study: Safety, Efficacy, and Equity of
Implementing Autonomous Artificial Intelligence for Diagnosing
5 CONCLUSION Diabetic Retinopathy in Youth. Diabetes Care.
Diabetes stands out as one of the fastest-growing health [5] Heydon, Peter & Egan, Catherine & Bolter, Louis & Chambers, Ryan
concerns worldwide, and diabetic retinopathy (DR) is no & Anderson, John & Aldington, Steve & Stratton, Irene & Scanlon,
exception. This condition unfolds through various phases, Peter & Webster, Laura & Mann, Samantha & Chemin, Alan & Owen,
Christopher & Tufail, Adnan & Rudnicka, Alicja. (2020). Prospective
ranging from mild to severe, culminating in Proliferative evaluation of an artificial intelligence-enabled algorithm for automated
Diabetic Retinopathy (PDR). The advanced stages of this diabetic retinopathy screening of 30 000 patients. British Journal of
ailment can lead to symptoms like impaired vision, eventually Ophthalmology. 105. bjophthalmol-2020. 10.1136/bjophthalmol-
resulting in vision loss. Manual diagnosis of DR is time­ 2020- 316594.
consuming and challenging. Consequently, the scientific [6] Asia Pacific Tele-Ophthalmology Society. APTOS 2019 Blindness
literature has explored computer vision-based methods for Detection dataset. 2019.
automated disease identification. [7] Wan, S.; Liang, Y.; Zhang, Y.: Deep convolutional neural networks for
diabetic retinopathy detection by image classification. Comput. Electr.
In our present study, we've achieved the categorization of Eng. 72, 274-282 (2018)
distinct DR stages by employing an ensembled weighted [8] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image
approach. This approach integrates models such as VGG16, recognition," in Proc. IEEE Conj Comput. Vis. Pattern Recognit.
(CVPR), Jun. 2016, pp. 770-778.
MobileNet, ResNet50, lnceptionV3, and DenseNet-169. To
augment the quality offundus images, we've applied various [9] C.Szegedy,W.Liu,Y.Jia,P.Serrnanet,S.Reed,D.Anguelov,D.Erhan, V.
Vanhoucke, and A. Rabinovich, "Going deeper with convolutions,"
data preprocessing techniques before feeding them into deep in Proc. IEEE Conj Comput. Vis. Pattern Recognit. (CVPR), Jun.
neural networks. Our substantial experimental findings, based 2015, pp. 1-9.
on assessments with the APTOS 2019 dataset, underscore the [10] A.G.Howard,M.Zhu,B.Chen,D.Kalenichenko,W.Wang,T.Weyand, M.
impressive performance of our recommended weighted twin Andreetto, and H. Adam, "MobileNets: Efficient convolutional neu­
CNN model when compared to state-of-the-art DR ral networks for mobile vision applications," 2017, arXiv: 1704. 04861.
techniques. [Online]. Available: http://arxiv.org/abs/1704.04861
[11] M. T. Al-Antary and Y. Arafa, "Multi-Scale Attention Network for
Our proposed model has yielded exceptional results, Diabetic Retinopathy Classification," in IEEE Access, vol. 9, pp.
boasting a Kappa score of96% and an accuracy rate of94%, 54190-54200, 2021, doi: 10.1109/ACCESS.2021.3070685.
surpassing the benchmarks set by previous models. [12] Li X, Hu X, Yu L, Zhu L, Fu CW, Heng PA. CANet: Cross-Disease
Furthermore, we've recorded an overall sensitivity of90% and Attention Network for Joint Diabetic Retinopathy and Diabetic
Macular Edema Grading. IEEE Trans Med Imaging. 2020
an impressive specificity of 99.5%. These experimental May;39(5):1483-1493. doi: 10.1109/TMI.2019.2951844. Epub 2019
outcomes affirm the effectiveness and efficiency of our Nov 6. PMID: 31714219.
proposed paradigm in the precise identification and [13] X. Zeng, H. Chen, Y. Luo and W. Ye, "Automated Diabetic
categorization of DR. Consequently, our approach holds Retinopathy Detection Based on Binocular Siamese-Like
significant promise for future therapeutic applications in this Convolutional Neural Network," in IEEE Access, vol. 7, pp. 30744-
domain. Our research has demonstrated a substantial 30753, 2019, doi: 10.l109/ACCESS.2019.2903171.
improvement of11.90% when compared to the current state­
of-the-art, underscoring the remarkable advancements made
in this field ofarchitecture.

5
Authorized licensed use limited to: Jaypee Insituite of Information Technology-Noida Sec 128 (L3). Downloaded on November 19,2024 at 07:10:05 UTC from IEEE Xplore. Restrictions apply.