Abstract
X-ray Bragg coherent diffraction imaging is a powerful technique for 3D materials characterization. However, obtaining X-ray diffraction data is difficult and computationally intensive, motivating the need for automated processing of coherent diffraction images, with the goal of minimizing the number of X-ray datasets needed. We automate a machine learning approach to identify crystalline line defects in samples from the raw coherent diffraction data, in a workflow coupling coherent diffraction data generation with training and inference of deep neural network defect classifiers. In particular, we adopt a continual learning approach, where we generate training data as needed based on the accuracy of the defect classifier instead of generating all training data a priori. Moreover, we develop a novel data generation mechanism to improve the efficiency of defect identification beyond the previously published continual learning approach. We call the improved method smart continual learning. The results show that our approach improves the accuracy of defect classifiers and reduces training data requirements by up to 98% compared with prior approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availibility
Data will be made available on reasonable request.
References
Yao Y, Chan H, Sankaranarayanan S, Balaprakash P, Harder RJ, Cherukara MJ (2022) Autophasenn: unsupervised physics-aware deep learning of 3d nanoscale bragg coherent diffraction imaging. npj Comput Mater 8(1):1–8
Cherukara MJ, Zhou T, Nashed Y, Enfedaque P, Hexemer A, Harder RJ, Holt MV (2020) Ai-enabled high-resolution scanning coherent diffraction imaging. Appl Phys Lett 117(4):044103
Benmore C, Bicer T, Chan MK, Di Z, Gürsoy DA, Hwang I, Kuklev N, Lin D, Liu Z, Lobach I et al (2022) Advancing ai/ml at the advanced photon source. Synchrotron Radiat News 35(4):28–35
Ulvestad A, Menickelly M, Wild S (2018) Accurate, rapid identification of dislocation lines in coherent diffractive imaging via a min-max optimization formulation. AIP Adv 8(1):015114
Ulvestad A, Nashed Y, Beutier G, Verdier M, Hruszkewycz S, Dupraz M (2017) Identifying defects with guided algorithms in bragg coherent diffractive imaging. Sci Rep 7(1):1–9
Lim B, Bellec E, Dupraz M, Leake S, Resta A, Coati A, Sprung M, Almog E, Rabkin E, Schulli T et al (2021) A convolutional neural network for defect classification in bragg coherent x-ray diffraction. npj Comput Mater 7(1):1–8
Judge W, Chan H, Sankaranarayanan S, Harder RJ, Cabana J, Cherukara MJ (2023) Defect identification in simulated bragg coherent diffraction imaging by automated ai. MRS Bull 48(2):124–133
Babu AV, Zhou T, Kandel S, Bicer T, Liu Z, Judge W, Ching DJ, Jiang Y, Veseli S, Henke S, et al. (2022) Deep learning at the edge enables real-time streaming ptychographic imaging. arXiv preprint arXiv:2209.09408
Babu AV, Bicer T, Kandel S, Zhou T, Ching DJ, Henke S, Veseli S, Chard R, Miceli A, Cherukara MJ (2023) Ai-assisted automated workflow for real-time x-ray ptychography data analysis via federated resources. arXiv preprint arXiv:2304.04297
Marchesini S (2007) Invited article: a unified evaluation of iterative projection algorithms for phase retrieval. Rev Sci Instrum 78(1):011301
Cherukara MJ, Nashed YS, Harder RJ (2018) Real-time coherent diffraction inversion using deep generative networks. Sci Rep 8(1):1–8
Scheinker A, Pokharel R (2020) Adaptive 3d convolutional neural network-based reconstruction method for 3d coherent diffraction imaging. J Appl Phys 128(18):184901
Wu L, Juhas P, Yoo S, Robinson I (2021) Complex imaging of phase domains by deep neural networks. IUCrJ 8(1):12–21
Harder R (2021) Deep neural networks in real-time coherent diffraction imaging. IUCrJ 8(Pt 1):1
Chan H, Nashed YS, Kandel S, Hruszkewycz SO, Sankaranarayanan SK, Harder RJ, Cherukara MJ (2021) Rapid 3d nanoscale coherent imaging via physics-aware deep learning. Appl Phys Rev 8(2):021407
Deelman E, Gannon D, Shields M, Taylor I (2009) Workflows and e-science: an overview of workflow system features and capabilities. Futur Gener Comput Syst 25(5):528–540
Altintas I, Purawat S, Crawl D, Singh A, Marcus K (2019) Toward a methodology and framework for workflow-driven team science. Comput Sci Eng 21(4):37–48
Ayachit U, Bauer A, Geveci B, O’Leary P, Moreland K, Fabian N, Mauldin J (2015) In Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ACM), pp. 25–29
Kuhlen T, Pajarola R, Zhou K (2011) Proceedings of the 11th Eurographics Conference on Parallel Graphics and Visualization (EGPGV)
Boyuka DA, Lakshminarasimham S, Zou X, Gong Z, Jenkins J, Schendel ER, Podhorszki N, Liu Q, Klasky S, Samatova NF (2014) In 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (IEEE), pp. 256–266
Ayachit U, Whitlock B, Wolf M, Loring B, Geveci B, Lonie D, Bethel E (2016) Proceedings of the 2nd Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization (IEEE Press), pp. 40–44
Dorier M, Antoniu G, Cappello F, Snir M, Sisneros R, Yildiz O, Ibrahim S, Peterka T, Orf L (2016) Damaris: addressing performance variability in data management for post-petascale simulations. ACM Trans Parallel Comput (TOPC) 3(3):15
Yildiz O, Dreher M, Peterka T (2022) Situ Visualization for Computational Science. Springer, Cham, pp 137–158
Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, Milan K, Quan J, Ramalho T, Grabska-Barwinska A et al (2017) Overcoming catastrophic forgetting in neural networks. Proc Natl Acad Sci 114(13):3521–3526
Van de Ven GM, Tolias AS (2019) Three scenarios for continual learning. arXiv preprint arXiv:1904.07734
Raghavan K, Balaprakash P (2021) Formalizing the generalization-forgetting trade-off in continual learning. Adv Neural Inf Process Syst 34:17284–17297
Krishnan R, Balaprakash P (2020) Meta continual learning via dynamic programming. arXiv preprint arXiv:2008.02219
Hirel P (2015) Atomsk: a tool for manipulating and converting atomic data files. Comput Phys Commun 197:212–219. https://doi.org/10.1016/j.cpc.2015.07.012
Plimpton S, Crozier P, Thompson A (2007) Lammps-large-scale atomic/molecular massively parallel simulator. Sandia Nation Lab 18:43
Favre-Nicolin V, Girard G, Leake S, Carnis J, Chushkin Y, Kieffer J, Paleo P, Richard MI (2020) Pynx: high-performance computing toolkit for coherent x-ray imaging based on operators. J Appl Crystallogr 53(5):1404–1413
Favre-Nicolin V, Coraux J, Richard MI, Renevier H (2011) Fast computation of scattering maps of nanostructures using graphical processing units. J Appl Crystallogr 44(3):635–640
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8024–8035
Zhao L, Liu T, Peng X, Metaxas D (2020) Maximum-entropy adversarial data augmentation for improved generalization and robustness. Adv Neural Inf Process Syst 33:14435–14447
Fornek TE (2017) Advanced photon source upgrade project preliminary design report (No. APSU-2.01-RPT-002). Tech. rep., Argonne National Laboratory (ANL)(United States). Funding organisation: USDOE Office of Science-Office of Basic Energy Sciences, United States
Acknowledgements
This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, under contract number DE-AC02-06CH11357. We gratefully acknowledge the computing resources provided on Swing, a high-performance computing cluster operated by the Laboratory Computing Resource Center at Argonne National Laboratory. Work performed at the Center for Nanoscale Materials and Advanced Photon Source, both U.S. Department of Energy Office of Science User Facilities, was supported by the U.S. DOE, Office of Basic Energy Sciences, under Contract No. DE-AC02-06CH11357.
Author information
Authors and Affiliations
Contributions
Orcun Yildiz: Conceptualization, Methodology, Software, Validation, Investigation, Writing - Original Draft. Krishnan Raghavan: Conceptualization, Software, Writing - Original Draft. Henry Chan: Software, Writing - Review & Editing. Mathew J. Cherukara: Provision, Writing - Review & Editing. Prasanna Balaprakash: Conceptualization. Subramanian Sankaranarayanan: Project Administration, Funding acquisition. Tom Peterka: Conceptualization, Provision, Writing - Review & Editing, Funding acquisition.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no Conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yildiz, O., Raghavan, K., Chan, H. et al. Automated defect identification in coherent diffraction imaging with smart continual learning. Neural Comput & Applic 36, 22335–22346 (2024). https://doi.org/10.1007/s00521-024-10415-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-024-10415-8