Abstract
The Shack-Hartmann wavefront sensor (SHWS) is an essential tool for wavefront sensing in adaptive optical microscopes. However, the distorted spots induced by the complex wavefront challenge its detection performance. Here, we propose a deep learning based wavefront detection method which combines point spread function image based Zernike coefficient estimation and wavefront stitching. Rather than using the centroid displacements of each micro-lens, this method first estimates the Zernike coefficients of local wavefront distribution over each micro-lens and then stitches the local wavefronts for reconstruction. The proposed method can offer low root mean square wavefront errors and high accuracy for complex wavefront detection, and has potential to be applied in adaptive optical microscopes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Booth MJ, 2014. Adaptive optical microscopy: the ongoing quest for a perfect image. Light Sci Appl, 3(4):e165. https://doi.org/10.1038/lsa.2014.46
Booth MJ, Neil MAA, Juškaitis R, et al., 2002. Adaptive aberration correction in a confocal microscope. Proc Nat Acad Sci, 99(9):5788–5792. https://doi.org/10.1073/pnas.082544799
Cheng SF, Li HH, Luo YQ, et al., 2019. Artificial intelligence-assisted light control and computational imaging through scattering media. J Innov Opt Health Sci, 12(4):193006. https://doi.org/10.1142/s1793545819300064
Cornea A, Conn PM, 2014. Fluorescence Microscopy: Super Resolution and Other Novel Techniques. Elsevier, London, UK, p.249.
Cui M, 2011. Parallel wavefront optimization method for focusing light through random scattering media. Opt Lett, 36(6):870–872. https://doi.org/10.1364/ol.36.000870
Cumming BP, Gu M, 2020. Direct determination of aberration functions in microscopy by an artificial neural network. Opt Expr, 28(10):14511–14521. https://doi.org/10.1364/OE.390856
Dai GM, 2008. Wavefront Optics for Vision Correction. SPIE Press, Bellingham, USA. https://doi.org/10.1117/3.769212
Drozdzal M, Vorontsov E, Chartrand G, et al., 2016. The importance of skip connections in biomedical image segmentation. Int Workshop on Deep Learning in Medical Image Analysis and Int Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, p.179–187. https://doi.org/10.1007/978-3-319-46976-8_19
Dubose TB, Gardner DF, Watnik AT, 2020. Intensity-enhanced deep network wavefront reconstruction in Shack-Hartmann sensors. Opt Lett, 45(7):1699–1702. https://doi.org/10.1364/OL.389895
Gómez SLS, González-Gutiérrez C, Alonso ED, et al., 2018. Improving adaptive optics reconstructions with a deep learning approach. Int Conf on Hybrid Artificial Intelligence Systems, p.74–83. https://doi.org/10.1007/978-3-319-92639-1_7
Hu LJ, Hu SW, Gong W, et al., 2019. Learning-based Shack-Hartmann wavefront sensor for high-order aberration detection. Opt Expr, 27(23):33504–33517. https://doi.org/10.1364/OE.27.033504
Hu LJ, Hu SW, Li YN, et al., 2020. Reliability of wavefront shaping based on coherent optical adaptive technique in deep tissue focusing. J Biophoton, 13(1):e201900245. https://doi.org/10.1002/jbio.201900245
Hu SW, Hu LJ, Zhang BW, et al., 2020. Simplifying the detection of optical distortions by machine learning. J Innov Opt Health Sci, 13(3):2040001. https://doi.org/10.1142/s1793545820400015
Ji N, 2017. Adaptive optical fluorescence microscopy. Nat Methods, 14(4):374–280. https://doi.org/10.1038/nmeth.4218
Jin YC, Zhang YY, Hu LJ, et al., 2018. Machine learning guided rapid focusing with sensor-less aberration corrections. Opt Expr, 26(23):30162–30171. https://doi.org/10.1364/OE.26.030162
Li ZH, Yu ZP, Hui H, et al., 2020. Edge enhancement through scattering media enabled by optical wavefront shaping. Photon Res, 8(6):954–962. https://doi.org/10.1364/PRJ.388062
Liu R, Li ZY, Marvin JS, et al., 2019. Direct wavefront sensing enables functional imaging of infragranular axons and spines. Nat Methods, 16(7):615–618. https://doi.org/10.1038/s41592-019-0434-7
Liu TL, Upadhyayula S, Milkie DE, et al., 2018. Observing the cell in its native state: imaging subcellular dynamics in multicellular organisms. Science, 360(6386):eaaq1392. https://doi.org/10.1126/science.aaq1392
Mahajan VN, Dai GM, 2007. Orthonormal polynomials in wavefront analysis: analytical solution. J Opt Soc Am A, 24(9):2994–3016. https://doi.org/10.1364/JOSAA.24.002994
Nishizaki Y, Valdivia M, Horisaki R, et al., 2019. Deep learning wavefront sensing. Opt Expr, 27(1):240–251. https://doi.org/10.1364/OE.27.000240
Paine SW, Fienup JR, 2018. Machine learning for improved image-based wavefront sensing. Opt Lett, 43(6):1235–1238. https://doi.org/10.1364/OL.43.001235
Park JH, Kong LJ, Zhou YF, et al., 2017. Large-field-of-view imaging by multi-pupil adaptive optics. Nat Methods, 14(6):581–583. https://doi.org/10.1038/nmeth.4290
Rodríguez C, Ji N, 2018. Adaptive optical microscopy for neurobiology. Curr Opin Neurobiol, 50:83–91. https://doi.org/10.1016/j.conb.2018.01.011
Schott S, Bertolotti J, Léger JF, et al., 2015. Characterization of the angular memory effect of scattered light in biological tissues. Opt Expr, 23(10):13505–13516. https://doi.org/10.1364/OE.23.013505
Swanson R, Lamb M, Correia C, et al., 2018. Wavefront reconstruction and prediction with convolutional neural networks. Adaptive Optics Systems VI, Article 10703F. https://doi.org/10.1117/12.2312590
Tang JY, Germain RN, Cui M, 2012. Superpenetration optical microscopy by iterative multiphoton adaptive compensation technique. Proc Nat Acad Sci, 109(22): 8434–8439. https://doi.org/10.1073/pnas.1119590109
Vanberg PO, de Xivry GO, Absil O, et al., 2019. Machine learning for image-based wavefront sensing. 33rd Conf on Neural Information Processing Systems, p.1–6.
Wang BK, Barbiero M, Zhang QM, et al., 2019. Superresolution optical microscope: principle, instrumentation, and application. Front Inform Technol Electron Eng, 20(5):608–630. https://doi.org/10.1631/FITEE.1800449
Wang K, Milkie DE, Saxena A, et al., 2014. Rapid adaptive optical recovery of optimal resolution over large volumes. Nat Methods, 11(6):625–628. https://doi.org/10.1038/nmeth.2925
Wang K, Sun WZ, Richie CT, et al., 2015. Direct wavefront sensing for high-resolution in vivo imaging in scattering tissue. Nat Commun, 6:7276. https://doi.org/10.1038/ncomms8276
Yoon J, Lee M, Lee K, et al., 2015. Optogenetic control of cell signaling pathway through scattering skull using wavefront shaping. Sci Rep, 5:13289. https://doi.org/10.1038/srep13289
Yu ZP, Xia MY, Li HH, et al., 2019. Implementation of digital optical phase conjugation with embedded calibration and phase rectification. Sci Rep, 9(1):1537. https://doi.org/10.1038/s41598-018-38326-4
Zeng ZP, Xie H, Chen L, et al., 2017. Computational methods in super-resolution microscopy. Front Inform Technol Electron Eng, 18(9):1222–1235. https://doi.org/10.1631/FITEE.1601628
Author information
Authors and Affiliations
Corresponding authors
Additional information
Project supported by the National Natural Science Foundation of China (Nos. 61735016, 81771877, and 61975178), the Zhejiang Provincial Natural Science Foundation of China (No. LR20F050002), the Key R&D Program of Zhejiang Province, China (No. 2021C03001), the CAMS Innovation Fund for Medical Sciences, China (No. 2019-I2M-5-057), and the Fundamental Research Funds for the Central Universities, China
Contributors
Wei GONG and Ke SI designed the research. Shuwen HU, Lejia HU, Wei GONG, and Ke SI performed the research. Shuwen HU, Lejia HU, and Zhenghan LI processed the data. Shuwen HU and Lejia HU drafted the manuscript. Wei GONG and Ke SI revised and finalized the paper.
Compliance with ethics guidelines
Shuwen HU, Lejia HU, Wei GONG, Zhenghan LI, and Ke SI declare that they have no conflict of interest.
Shuwen HU, first author of this invited paper, received her BS degree from Tianjin University, China. She is currently a master degree candidate at the College of Optical Science and Engineering, Zhejiang University, China. Her research interests include adaptive optics, machine learning, and deep tissue imaging.
Wei GONG, corresponding author of this invited paper, is a PI at Zhejiang University School of Medicine, China. She received her BS and MS degrees from Zhejiang University, China, and her PhD degree from the National University of Singapore, Singapore. She is a special expert of the Ministry of Education (MOE) and the Outstanding Youth of Zhejiang Province. Her research interests include biomedical imaging, optical clearing, and artificial intelligence in biomedicine.
Ke SI, corresponding author of this invited paper, is a professor at the College of Optical Science and Engineering, Zhejiang University, China, and a joint professor at Zhejiang University School of Medicine. He is the Vice Director of MOE Frontier Science Center for Brain Science and Brain-Machine Integration, and the Vice Dean of the School of Brain Science and Brain Medicine. He is now a corresponding expert of Front Inform Technol Electron Eng. His research focuses on biophotonics, deep tissue imaging, adaptive optics, and optogenetics.
Rights and permissions
About this article
Cite this article
Hu, S., Hu, L., Gong, W. et al. Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes. Front Inform Technol Electron Eng 22, 1277–1288 (2021). https://doi.org/10.1631/FITEE.2000422
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2000422