Abstract
Insufficiency of training data is a persistent issue in medical image analysis, especially for task-based functional magnetic resonance images (fMRI) with spatio-temporal imaging data acquired using specific cognitive tasks. In this paper, we propose an approach for generating synthetic fMRI sequences that can then be used to create augmented training datasets in downstream learning tasks. To synthesize high-resolution task-specific fMRI, we adapt the \(\alpha \)-GAN structure, leveraging advantages of both GAN and variational autoencoder models, and propose different alternatives in aggregating temporal information. The synthetic images are evaluated from multiple perspectives including visualizations and an autism spectrum disorder (ASD) classification task. The results show that the synthetic task-based fMRI can provide effective data augmentation in learning the ASD classification task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abbasi-Sureshjani, S., Amirrajab, S., Lorenz, C., Weese, J., Pluim, J., Breeuwer, M.: 4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model. In: Arbel, T., Ben Ayed, I., de Bruijne, M., Descoteaux, M., Lombaert, H., Pal, C. (eds.) Proceedings of the Third Conference on Medical Imaging with Deep Learning, 06–08 Jul 2020, vol. 121, pp. 6–18. PMLR. Proceedings of Machine Learning Research (2020). http://proceedings.mlr.press/v121/abbasi-sureshjani20a.html
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018). arXiv arxiv.org/abs/1810.04805
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. CoRR abs/2010.11929 (2020). arXiv arxiv.org/abs/2010.11929
Dvornek, N., Ventola, P., Pelphrey, K., Duncan, J.: Identifying autism from resting-state fMRI using long short-term memory networks. In: Machine Learning in Medical Imaging, MLMI (Workshop), vol. 10541, pp. 362–370, September 2017. https://doi.org/10.1007/978-3-319-67389-9_42
Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Synthetic data augmentation using GAN for improved liver lesion classification. In: 2018 IEEE 15th International Symposium on Biomedical Imaging, ISBI 2018, pp. 289–293 (2018). https://doi.org/10.1109/ISBI.2018.8363576
Goodfellow, I.J., et al.: Generative adversarial networks (2014). arXiv arxiv.org/abs/1406.2661
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Kaiser, M.D., et al.: Neural signatures of autism. Proc. Natl. Acad. Sci. 107(49), 21223–21228 (2010)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes (2013). arXiv arxiv.org/abs/1312.6114
Kwon, G., Han, C., Kim, D.: Generation of 3D brain MRI using auto-encoding generative adversarial networks. In: Dinggang, S., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 118–126. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_14
Li, X., et al.: BrainGNN: interpretable brain graph neural network for fMRI analysis. Med. Image Anal. 74, 102233 (2021)
Liu, Z., et al.: Video Swin transformer. arXiv arxiv.org/abs/2106.13230 (2021)
van der Maaten, L., Hinton, G.E.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Madan, Y., Veetil, I.K., Sowmya, V., Gopalakrishnan E.A., Soman, K.P.: Synthetic data augmentation of MRI using generative variational autoencoder for Parkinson’s disease detection. In: Bhateja, V., Tang, J., Satapathy, S.C., Peer, P., Das, R. (eds.) Evolution in Computational Intelligence. Smart Innovation, Systems and Technologies, vol. 267. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-6616-2_16
Mirza, M., Osindero, S.: Conditional generative adversarial nets. CoRR abs/1411.1784 (2014). arXiv arxiv.org/abs/1411.1784
Paszke, A., et al.: Automatic differentiation in PyTorch (2017)
Qureshi, M.N.I., Oh, J., Lee, B.: 3D-CNN based discrimination of schizophrenia using resting-state fMRI. Artif. Intell. Med. 98, 10–17 (2019)
Rolls, E.T., Huang, C.C., Lin, C.P., Feng, J., Joliot, M.: Automated anatomical labelling atlas 3. Neuroimage 206, 116189 (2020)
Rosca, M., Lakshminarayanan, B., Warde-Farley, D., Mohamed, S.: Variational approaches for auto-encoding generative adversarial networks (2017). arXiv arxiv.org/abs/1706.04987
Tay, Y., Dehghani, M., Gupta, J.P., Bahri, D., Aribandi, V., Qin, Z., Metzler, D.: Are pre-trained convolutions better than pre-trained transformers? CoRR abs/2105.03322 (2021). arXiv arxiv.org/abs/2105.03322
Vaswani, A., et al.: Attention is all you need. arXiv arxiv.org/abs/1706.03762 (2017)
Waheed, A., Goyal, M., Gupta, D., Khanna, A., Al-Turjman, F., Pinheiro, P.R.: CovidGAN: data augmentation using auxiliary classifier GAN for improved Covid-19 detection. IEEE Access 8, 91916–91923 (2020). https://doi.org/10.1109/ACCESS.2020.2994762
Yang, D., et al.: Brain responses to biological motion predict treatment outcome in young children with autism. Transl. Psychiatry 6(11), e948 (2016). https://doi.org/10.1038/tp.2016.213
Zhuang, P., Schwing, A.G., Koyejo, O.: fMRI data augmentation via synthesis. In: 2019 IEEE 16th International Symposium on Biomedical Imaging, ISBI 2019, pp. 1783–1787 (2019). https://doi.org/10.1109/ISBI.2019.8759585
Acknowledgement
The data collection and study included in this paper are supported under NIH grant R01NS035193.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, J., Dvornek, N.C., Staib, L.H., Duncan, J.S. (2023). Learning Sequential Information in Task-Based fMRI for Synthetic Data Augmentation. In: Abdulkadir, A., et al. Machine Learning in Clinical Neuroimaging. MLCN 2023. Lecture Notes in Computer Science, vol 14312. Springer, Cham. https://doi.org/10.1007/978-3-031-44858-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-44858-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44857-7
Online ISBN: 978-3-031-44858-4
eBook Packages: Computer ScienceComputer Science (R0)
