Abstract
With the recent advancement in the medical diagnostic tools, multi-modality medical images are extensively utilized as a lifesaving tool. An efficient fusion of medical images can improve the performance of various medical diagnostic tools. But, gathering of all modalities for a given patient is defined as an ill-posed problem as medical images suffer from poor visibility and frequent patient dropout. Therefore, in this paper, an efficient multi-modality image fusion model is proposed to fuse multi-modality medical images. To tune the hyper-parameters of the proposed model, a multi-objective differential evolution is used. The fusion factor and edge strength metrics are utilized to form a multi-objective fitness function. Performance of the proposed model is compared with nine competitive models over fifteen benchmark images. Performance analyses reveal that the proposed model outperforms the competitive fusion models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
References
Chen J, Zhang L, Lu L, Li Q, Hu M, Yang X (2020) A novel medical image fusion method based on rolling guidance filtering. Internet Things 14:100172
Corbat L, Nauval M, Henriet J, Lapayre J-C (2020) A fusion method based on deep learning and case-based reasoning which improves the resulting medical image segmentations. Expert Syst Appl 147:113200
Daniel E, Anitha J, Kamaleshwaran K, Rani I (2017) Optimum spectrum mask based medical image fusion using gray wolf optimization. Biomed Signal Process Control 34:36–43
De Luca G (2022) A survey of nisq era hybrid quantum-classical machine learning research. J Artif Intell Technol 2(1):9–15
Du J, Li W, Lu K, Xiao B (2016) An overview of multi-modal medical image fusion. Neurocomputing 215:3–20
Du J, Li W, Tan H (2020) Three-layer medical image fusion with tensor-based features. Inf Sci 525:93–108
El-Hoseny HM, El-Rahman WA, El-Rabaie E-SM, El-Samie FEA, Faragallah OS (2018) An efficient dt-cwt medical image fusion system based on modified central force optimization and histogram matching. Infrared Phys Technol 94:223–231
Fan Q, Yan X (2018) Multi-objective modified differential evolution algorithm with archive-base mutation for solving multi-objective xylene oxidation rocess. J Intell Manuf 29(1):35–49
Hu Q, Hu S, Zhang F (2020) Multi-modality medical image fusion based on separable dictionary learning and gabor filtering. Signal Process: Image Commun 83:115758
James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fus 19:4–19
Jiang W, Yang X, Wu W, Liu K, Ahmad A, Sangaiah AK, Jeon G (2018) Medical images fusion by using weighted least squares filter and sparse representation. Comput Electr Eng 67:252–266
Jin X, Chen G, Hou J, Jiang Q, Zhou D, Yao S (2018) Multimodal sensor medical image fusion based on nonsubsampled shearlet transform and s-pcnns in hsv space. Signal Process 153:379–395
Karim AM, Güzel MS, Tolun MR, Kaya H, Çelebi FV (2019) A new framework using deep auto-encoder and energy spectral density for medical waveform data classification and processing. Biocybernet Biomed Eng 39(1):148–159
Karthik P, Sekhar K (2021) Resource scheduling approach in cloud testing as a service using deep reinforcement learning algorithms. CAAI Trans Intell Technol 6(2):147–154
Kaur M, Singh D (2021) Multi-modality medical image fusion technique using multi-objective differential evolution based deep neural networks. J Ambient Intell Humaniz Comput 12(2):2483–2493
Kaur M, Singh D, Kumar V, Gupta B, Abd El-Latif AA (2021) Secure and energy efficient-based e-health care framework for green internet of things. IEEE Trans Green Commun Netw 5(3):1223–1231
Kaushik H, Singh D, Kaur M, Alshazly H, Zaguia A, Hamam H (2021) Diabetic retinopathy diagnosis from fundus images using stacked generalization of deep models. IEEE Access 9:108276–108292
Kaushik R, Jain S, Jain S, Dash T (2021) Performance evaluation of deep neural networks for forecasting time-series with multiple structural breaks and high volatility. CAAI Trans Intell Technol 6(3):265–280. https://doi.org/10.1049/cit2.12002
Li Z-K, Tan J-R, Feng Y-X, Fang H (2008) Multi-objective particle swarm optimization algorithm based on crowding distance sorting and its application. Comput Integr Manuf Syst 7:1329–1336
Li H, He X, Tao D, Tang Y, Wang R (2018) Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning. Pattern Recogn 79:130–146
Liu J, Liu Z, Sun C, Zhuang J (2022) A data transmission approach based on ant colony optimization and threshold proxy re-encryption in wsns. J Artif Intell Technol 2(1):23–31
Manchanda M, Sharma R (2018) An improved multimodal medical image fusion algorithm based on fuzzy transform. J Vis Commun Image Represent 51:76–94
Maqsood S, Javed U (2020) Multi-modal medical image fusion based on two-scale image decomposition and sparse representation. Biomed Signal Process Control 57:101810
Padmavathi K, Asha C, Maya VK (2020) A novel medical image fusion by combining tv-l1 decomposed textures based on adaptive weighting scheme. Eng Sci Technol, Int J 23(1):225–239
Polinati S, Dhuli R (2020) Multimodal medical image fusion using empirical wavelet decomposition and local energy maxima. Optik 205:163947
Prakash O, Park CM, Khare A, Jeon M, Gwak J (2019) Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform. Optik 182:995–1014
Prakash O, Park CM, Khare A, Jeon M, Gwak J (2019) Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform. Optik 182:995–1014
Rajalingam B, Priya R, Bhavani R (2019) Hybrid multimodal medical image fusion using combination of transform techniques for disease analysis. Procedia Comput Sci 152:150–157 (international Conference on Pervasive Computing Advances and Applications- PerCAA 2019)
Ravi P, Krishnan J (2018) Image enhancement with medical image fusion using multiresolution discrete cosine transform. Materials Today: Proceedings 5(1, Part 1), 1936 – 1942. international Conference on Processing of Materials, Minerals and Energy (July 29th - 30th) 2016, Ongole, Andhra Pradesh, India
Rezaeipanah A, Mojarad M (2021) Modeling the scheduling problem in cellular manufacturing systems using genetic algorithm as an efficient meta-heuristic approach. J Artif Intell Technol 1(4):228–234
Sheng Guan J, Kang S bo, Sun Y (2019) Medical image fusion algorithm based on multi-resolution analysis coupling approximate spare representation. Futur Gener Comput Syst 98:201–207
Singh S, Anand R (2018) Ripplet domain fusion approach for ct and mr medical image information. Biomed Signal Process Control 46:281–292
Singh D, Kumar V, Kaur M, Jabarulla MY, Lee H-N (2021) Screening of covid-19 suspected subjects using multi-crossover genetic algorithm based dense convolutional neural network. IEEE Access 9:142566–142580
Singh D, Kaur M, Jabarulla MY, Kumar V, Lee H-N (2022) Evolving fusion-based visibility restoration model for hazy remote sensing images using dynamic differential evolution. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2022.3155765
Ullah H, Ullah B, Wu L, Abdalla FY, Ren G, Zhao Y (2020) Multi-modality medical images fusion based on local-features fuzzy sets and novel sum-modified-laplacian in non-subsampled shearlet transform domain. Biomed Signal Process Control 57:101724
Wang X, Zheng Z, He Y, Yan F, Zeng Z, Yang Y (2020) Progressive local filter pruning for image retrieval acceleration, arXiv preprint arXiv:2001.08878
Wani A, Khaliq R (2021) Sdn-based intrusion detection system for iot using deep learning classifier (idsiot-sdl). CAAI Trans Intell Technol 6(3):281–290
Xia K-J, Yin H-S, Wang J-Q (2019) A novel improved deep convolutional neural network model for medical image fusion. Clust Comput 22(1):1515–1527
Xu L, Si Y, Jiang S, Sun Y, Ebrahimian H (2020) Medical image fusion using a modified shark smell optimization algorithm and hybrid wavelet-homomorphic filter. Biomed Signal Process Control 59:101885
Xydeas C, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309
Zhou T, Fu H, Chen G, Shen J, Shao L (2020) Hi-net: hybrid-fusion network for multi-modal mr image synthesis. IEEE Trans Med Imaging 39(9):2772–27781
Zhu Z, Chai Y, Yin H, Li Y, Liu Z (2016) A novel dictionary learning approach for multi-modality medical image fusion. Neurocomputing 214:471–482
Zong J. jing, Qiu T. shuang (2017) Biomedical signal processing and control. Med Image Fus Based Sparse Represent Classif Image Patches 34:195-205
Funding
The authors have not disclosed any funding.
Author information
Authors and Affiliations
Contributions
MG contributed to conceptualization, methodology and software. NK performed data curation and writing—original draft preparation. NG contributed to visualization, investigation, supervision, software and validation. AZ performed writing—reviewing and editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest regarding the publication of this paper.
Additional information
Communicated by Irfan Uddin.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gupta, M., Kumar, N., Gupta, N. et al. Fusion of multi-modality biomedical images using deep neural networks. Soft Comput 26, 8025–8036 (2022). https://doi.org/10.1007/s00500-022-07047-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-022-07047-2