Skip to main content

Advertisement

Springer Nature Link
Log in

Conditional adversarial segmentation and deep learning approach for skin lesion sub-typing from dermoscopic images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Automatic skin lesion subtyping is a crucial step for diagnosing and treating skin cancer and acts as a first level diagnostic aid for medical experts. Although, in general, deep learning is very effective in image processing tasks, there are notable areas of the processing pipeline in the dermoscopic image regime that can benefit from refinement. Our work identifies two such areas for improvement. First, most benchmark dermoscopic datasets for skin cancers and lesions are highly imbalanced due to the relative rarity and commonality in the occurrence of specific lesion types. Deep learning methods tend to exhibit biased performance in favor of the majority classes with such datasets, leading to poor generalization. Second, dermoscopic images can be associated with irrelevant information in the form of skin color, hair, veins, etc.; hence, limiting the information available to a neural network by retaining only relevant portions of an input image has been successful in prompting the network towards learning task-relevant features and thereby improving its performance. Hence, this research work augments the skin lesion characterization pipeline in the following ways. First, it balances the dataset to overcome sample size biases. Two balancing methods, synthetic minority oversampling TEchnique (SMOTE) and Reweighting, are applied, compared, and analyzed. Second, a lesion segmentation stage is introduced before classification, in addition to a preprocessing stage, to retain only the region of interest. A baseline segmentation approach based on Bi-Directional ConvLSTM U-Net is improved using conditional adversarial training for enhanced segmentation performance. Finally, the classification stage is implemented using EfficientNets, where the B2 variant is used to benchmark and choose between the balancing and segmentation techniques, and the architecture is then scaled through to B7 to analyze the performance boost in lesion classification. From these experiments, we find that the pipeline that balances using SMOTE and uses the adversarially trained segmentation network achieves the best baseline performance of 91% classification accuracy with EfficientNet B2. Based on the scaling experiments, we find that optimal performance is reached with the B6 architecture that classifies with a 97% accuracy. Furthermore, the proposed pipeline for lesion characterization outperforms the state of the art performance on the ISIC dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Data supporting the findings of this study are available from the ISIC challenge websites (2018 and 2019) at https://challenge.isic-archive.com/data/#2018 and https://challenge.isic-archive.com/data/#2019. The generated and validated skin lesion masks will be made available by the corresponding author upon reasonable request.

Code availability

The implementation of this research work will be made available at https://github.com/karthik-d/lesion-characterization-using-cgan.

References

  1. Abbasi NR, Shaw HM, Rigel DS et al (2004) Early diagnosis of cutaneous melanoma: revisiting the ABCD criteria. JAMA 292(22):2771–2776

    Article  Google Scholar 

  2. Al-Masni MA, Kim DH, Kim TS (2020) Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification. Comput Methods Programs Biomed 190:105351

    Article  Google Scholar 

  3. Azad R, Asadi-Aghbolaghi M, Fathy M, et al (2019) Bi-directional convlstm u-net with Densley connected convolutions. In: Proceedings of the IEEE/CVF international conference on computer vision workshops, pp 0–0

  4. Barata C, Celebi ME, Marques JS (2019) A survey of feature extraction in dermoscopy image analysis of skin cancer. IEEE J Biomed Health Inform 23(3):1096–1109. https://doi.org/10.1109/JBHI.2018.2845939

    Article  Google Scholar 

  5. Batista GE, Bazzan AL, Monard MC et al (2003) Balancing training data for automated annotation of keywords: a case study. Wob 3:10–8

    Google Scholar 

  6. Bi L, Feng DD, Fulham M et al (2020) Multi-label classification of multi-modality skin lesion via hyper-connected convolutional neural network. Pattern Recogn 107:107502

    Article  Google Scholar 

  7. Celebi ME, Aslandogan YA, Stoecker WV et al (2007) Unsupervised border detection in dermoscopy images. Skin Res Technol 13:454–462

    Article  Google Scholar 

  8. Celebi ME, Wen Q, Hwang S et al (2013) Lesion border detection in dermoscopy images using ensembles of thresholding methods. Skin Res Technol 19:e252–e258

    Google Scholar 

  9. Chawla NV, Bowyer KW, Hall LO et al (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    Article  Google Scholar 

  10. Delgado-Gómez D, Butakoff C, Ersbøll BK et al (2008) Independent histogram pursuit for segmentation of skin lesions. IEEE Tran Biomed Eng 55:157–161

    Article  Google Scholar 

  11. Desingu K, Palaniappan M, Kumar J (2021) Snake species classification using transfer learning. In: Conference and labs of the evaluation forum, vol 2936, pp 1571–1578

  12. Desingu K, Bhaskar A, Palaniappan M, et al (2022) Classification of fungi species: a deep learning based image feature extraction and gradient boosting ensemble approach. In: Conference and labs of the evaluation forum. https://ceur-ws.org/Vol-3180/paper-162.pdf

  13. Desingu K, Chandrabose A (2022) Few-shot classification of skin lesions from dermoscopic images by meta-learning representative embeddings. https://doi.org/10.48550/ARXIV.2210.16954

  14. Erkol B, Moss RH, Stanley RJ et al (2005) Automatic lesion boundary detection in dermoscopy images using gradient vector flow snakes. Skin Re Technol 11:17–26

    Article  Google Scholar 

  15. Gessert N, Nielsen M, Shaikh M et al (2020) Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data. MethodsX 7:100864. https://doi.org/10.1016/j.mex.2020.100864 (https://www.sciencedirect.com/science/article/pii/S2215016120300832)

    Article  Google Scholar 

  16. Gessert N, Nielsen M, Shaikh M et al (2020) Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data. MethodsX 7:100864

    Article  Google Scholar 

  17. Gonzalez-Diaz I (2018) Dermaknet: incorporating the knowledge of dermatologists to convolutional neural networks for skin lesion diagnosis. IEEE J Biomed Health Inform 23(2):547–559

    Article  Google Scholar 

  18. Goyal M, Oakley A, Bansal P et al (2020) Skin lesion segmentation in dermoscopic images with ensemble deep learning methods. IEEE Access 8:4171–4181. https://doi.org/10.1109/ACCESS.2019.2960504

    Article  Google Scholar 

  19. Haenssle H, Fink C, Schneiderbauer R et al (2018) Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol: Off J Eur Soc Med Oncol 29:1836–1842

    Article  Google Scholar 

  20. Hasan SN, Gezer M, Azeez RA, et al (2019) Skin lesion segmentation by using deep learning techniques. In: 2019 Medical technologies congress (TIPTEKNO). IEEE, pp 1–4

  21. Innani S, Dutande P, Baid U et al (2023) Generative adversarial networks based skin lesion segmentation. Sci Rep 13(1):13467

    Article  Google Scholar 

  22. Izadi S, Mirikharaji Z, Kawahara J, et al (2018) Generative adversarial networks to segment skin lesions. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 881–884. https://doi.org/10.1109/ISBI.2018.8363712

  23. Jafari MH, Karimi N, Nasr-Esfahani E, et al (2016) Skin lesion segmentation in clinical images using deep learning. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, pp 337–342

  24. Jaisakthi S, Desingu K, Mirunalini P et al (2023) A deep learning approach for nucleus segmentation and tumor classification from lung histopathological images. Netw Model Anal Health Inform Bioinform 12(1):22

    Article  Google Scholar 

  25. Jaisakthi SM, Mirunalini P, Aravindan C (2018) Automated skin lesion segmentation of dermoscopic images using grabcut and k-means algorithms. IET Comput Vision 12(8):1088–1095

    Article  Google Scholar 

  26. Jensen JD, Elewski B (2015) The ABCDEF rule: combining the “ABCDE rule’’ and the “ugly duckling sign’’ in an effort to improve patient self-screening examinations. J Cin Aesth Dermatol 8(2):15

    Google Scholar 

  27. Kassem MA, Hosny KM, Fouad MM (2020) Skin lesions classification into eight classes for ISIC 2019 using deep convolutional neural network and transfer learning. IEEE Access 8:114822–114832

    Article  Google Scholar 

  28. Kouwenhoven E, Giezen M, Struikmans H (2009) Measuring the similarity of target volume delineations independent of the number of observers. Phys Medi Biol 54(9):2863. https://doi.org/10.1088/0031-9155/54/9/018 (https://dx.doi.org/10.1088/0031-9155/54/9/018)

    Article  Google Scholar 

  29. Krishna SR, Gudur A, Jain S, et al (2024) Deep learning for automatic diagnosis of skin cancer using dermoscopic images. J Artif Intell Technol

  30. Lazo C (2021) Segmentation of skin lesions and their attributes using generative adversarial networks. arXiv:2102.00169

  31. Li Y, Shen L (2018) Skin lesion analysis towards melanoma detection using deep learning network. Sensors 18(2):556

    Article  Google Scholar 

  32. Liu L, Tsui YY, Mandal M (2021) Skin lesion segmentation using deep learning with auxiliary task. J Imaging 7(4):67

    Article  Google Scholar 

  33. Liu S, Chen R, Gu Y et al (2023) AcneTyper: an automatic diagnosis method of dermoscopic acne image via self-ensemble and stacking. Technol Health Care 31(4):1171–1187

    Article  Google Scholar 

  34. Mahbod A, Schaefer G, Wang C et al (2020) Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification. Comput Methods Programs Biomed 193:105475

    Article  Google Scholar 

  35. Mahbod A, Tschandl P, Langs G et al (2020) The effects of skin lesion segmentation on the performance of dermatoscopic image classification. Comput Methods Programs Biomed 197:105725

    Article  Google Scholar 

  36. Mirikharaji Z, Abhishek K, Bissoto A et al (2023) A survey on deep learning for skin lesion segmentation. Med Image Anal 88:102863. https://doi.org/10.1016/j.media.2023.102863

    Article  Google Scholar 

  37. Mohamed EH, El-Behaidy WH (2019) Enhanced skin lesions classification using deep convolutional networks. In: 2019 Ninth international conference on intelligent computing and information systems (ICICIS), pp 180–188. https://doi.org/10.1109/ICICIS46948.2019.9014823

  38. Muthulakshmi V, Hemapriya N (2023) CNN-based dermoscopic analysis of vascular skin lesions in the prognosis of skin lesion sarcoma based on ensemble learning. J Intell Fuzzy Syst 1–20

  39. Nachbar F, Stolz W, Merkle T et al (1994) The ABCD rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions. J Am Acad Dermatol 30(4):551–559

    Article  Google Scholar 

  40. Palaniappan M, Desingu K, Bharathi H, et al (2022) Deep learning and gradient boosting ensembles for classification of snake species. In: Conference and labs of the evaluation forum. https://ceur-ws.org/Vol-3180/paper-173.pdf

  41. Pizer SM, Amburn EP, Austin JD et al (1987) Adaptive histogram equalization and its variations. Comput Vision Graph Image Process 39(3):355–368. https://doi.org/10.1016/S0734-189X(87)80186-X (https://www.sciencedirect.com/science/article/pii/S0734189X8780186X)

    Article  Google Scholar 

  42. Pollastri F, Bolelli F, Palacios RP, et al (2018) Improving skin lesion segmentation with generative adversarial networks. In: 2018 IEEE 31St international symposium on computer-based medical systems (CBMS). IEEE, pp 442–443

  43. Pollastri F, Parreño M, Maroñas J et al (2021) A deep analysis on high-resolution dermoscopic image classification. IET Comput Vision 15(7):514–526

    Article  Google Scholar 

  44. Shah V, Autee P, Sonawane P (2020) Detection of melanoma from skin lesion images using deep learning techniques. In: 2020 International conference on data science and engineering (ICDSE). IEEE, pp 1–8

  45. Sivanesan U, Braga LH, Sonnadara RR, et al (2019) Unsupervised medical image segmentation with adversarial networks: from edge diagrams to segmentation maps. arXiv:1911.05140

  46. Song Y, Cai W, Huang H et al (2015) Large margin local estimate with applications to medical image classification. IEEE Trans Med Imaging 34(6):1362–1377. https://doi.org/10.1109/TMI.2015.2393954

    Article  Google Scholar 

  47. Sonia R, Joseph J, Kalaiyarasi D et al (2024) Segmenting and classifying skin lesions using a fruit fly optimization algorithm with a machine learning framework. Automatika 65(1):217–231

    Article  Google Scholar 

  48. Suganyadevi S, Seethalakshmi V, Balasamy K (2022) A review on deep learning in medical image analysis. Int J Multim Inf Retr 11(1):19–38

    Article  Google Scholar 

  49. Tahir M, Naeem A, Malik H et al (2023) Dscc_net: multi-classification deep learning models for diagnosing of skin cancer using dermoscopic images. Cancers 15(7):2179

    Article  Google Scholar 

  50. Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: International conference on machine learning, PMLR, pp 6105–6114

  51. Trang K, Nguyen HA, TonThat L, et al (2022) An ensemble voting method of pre-trained deep learning models for skin disease identification. In: 2022 IEEE international conference on cybernetics and computational intelligence (CyberneticsCom), pp 445–450. https://doi.org/10.1109/CyberneticsCom55287.2022.9865634

  52. Venkataramana L, Prasad DVV, Saraswathi S et al (2022) Classification of Covid-19 from tuberculosis and pneumonia using deep learning techniques. Med Biol Eng Comput 60(9):2681–2691

    Article  Google Scholar 

  53. Vestergaard M, Macaskill P, Holt P et al (2008) Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical setting. Br J Dermatol 159(3):669–676

    Google Scholar 

  54. Yu L, Chen H, Dou Q et al (2016) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004

    Article  Google Scholar 

  55. Zhang J, Xie Y, Xia Y et al (2019) Attention residual learning for skin lesion classification. IEEE Trans Med Imaging 38(9):2092–2103

    Article  Google Scholar 

  56. Zhou S, Zhuang Y, Meng R (2019) Multi-category skin lesion diagnosis using dermoscopy images and deep cnn ensembles. línea], ISIC Chellange

  57. Zuiderveld K (1994) Contrast Limited Adaptive Histogram Equalization. Academic Press Professional Inc, Williston, pp 474–485

    Google Scholar 

Download references

Funding

No funding was received for this research work, or for the preparation of this manuscript.

Author information

Author notes

  1. P. Mirunalini, Karthik Desingu and S M Jaisakthi have contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Science Engineering, Sri Sivasubramaniya Nadar College of Engineering, Chennai, 603110, India

    P. Mirunalini, Karthik Desingu, S. Aswatha, R. Deepika & V. Deepika

  2. School of Computer Science and Engineering, Vellore Institute of Technology, Chennai Campus, Chennai, 600127, India

    S. M. Jaisakthi

Authors
  1. P. Mirunalini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karthik Desingu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Aswatha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Deepika
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. V. Deepika
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. S. M. Jaisakthi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Study design and conceptualization were done by KD, MP, JSM; Experiments and implementation were done by KD, AS, DR, and DV; Validation were done by MP, JSM, KD; Manuscript preparation, review, and editing were done by JSM, MP, KD. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to S. M. Jaisakthi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Compute resources used

The implementation of these experiments was carried out using an Intel Xeon 2.20GHz CPU, CUDA-enabled Tesla T4 and Tesla P100 GPUs with RAM sizes of 25 GB each, and a NVIDIA-SMI 460.32.03 TPU with a RAM size of 35 GB.

Ethical approval

The data used in this study is publicly available at: https://challenge.isic-archive.com/data/#2018 and https://challenge.isic-archive.com/data/#2019.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mirunalini, P., Desingu, K., Aswatha, S. et al. Conditional adversarial segmentation and deep learning approach for skin lesion sub-typing from dermoscopic images. Neural Comput & Applic 36, 16445–16463 (2024). https://doi.org/10.1007/s00521-024-09964-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09964-9

Keywords

Search

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy