Abstract
Diagnosing medical conditions from histopathology data requires a thorough analysis across the various resolutions of Whole Slide Images (WSI). However, existing generative methods fail to consistently represent the hierarchical structure of WSIs due to a focus on high-fidelity patches. To tackle this, we propose Ultra-Resolution Cascaded Diffusion Models (URCDMs) which are capable of synthesising entire histopathology images at high resolutions whilst authentically capturing the details of both the underlying anatomy and pathology at all magnification levels. We evaluate our method on three separate datasets, consisting of brain, breast and kidney tissue, and surpass existing state-of-the-art multi-resolution models. Furthermore, an expert evaluation study was conducted, demonstrating that URCDMs consistently generate outputs across various resolutions that trained evaluators cannot distinguish from real images. All code and additional examples can be found on GitHub.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cechnicka, S., Ball, J., Reynaud, H., Arthurs, C., Roufosse, C., Kainz, B.: Realistic data enrichment for robust image segmentation in histopathology. In: Koch, L., Cardoso, M.J., Ferrante, E., Kamnitsas, K., Islam, M., Jiang, M., Rieke, N., Tsaftaris, S.A., Yang, D. (eds.) Domain Adaptation and Representation Transfer. pp. 63–72. Springer Nature Switzerland, Cham (2024)
Chai, L., Gharbi, M., Shechtman, E., Isola, P., Zhang, R.: Any-resolution training for high-resolution image synthesis. In: European Conference on Computer Vision (2022)
Cheng, Y.C., Lin, C.H., Lee, H.Y., Ren, J., Tulyakov, S., Yang, M.H.: Inout: Diverse image outpainting via gan inversion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11431–11440 (2022)
Ciga, O., Xu, T., Nofech-Mozes, S., Noy, S., Lu, F.I., Martel, A.L.: Overcoming the limitations of patch-based learning to detect cancer in whole slide images. Scientific Reports | 11, 8894 (123). https://doi.org/10.1038/s41598-021-88494-z
Etten, A.V.: You only look twice: Rapid multi-scale object detection in satellite imagery (2018)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Communications of the ACM 63(11), 139–144 (2020)
Gupta, L., Klinkhammer, B.M., Boor, P., Merhof, D., Gadermayr, M.: Gan-based image enrichment in digital pathology boosts segmentation accuracy. In: MICCAI 2019, Part I 22. pp. 631–639. Springer (2019)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium (2017). https://doi.org/10.48550/ARXIV.1706.08500, https://arxiv.org/abs/1706.08500
Jose, L., Liu, S., Russo, C., Nadort, A., Di Ieva, A.: Generative adversarial networks in digital pathology and histopathological image processing: A review. Journal of Pathology Informatics 12(1), 43 (2021). https://doi.org/10.4103/jpi.jpi_103_20, https://www.sciencedirect.com/science/article/pii/S2153353922001651
Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., Aila, T.: Alias-free generative adversarial networks. In: Proc. NeurIPS (2021)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., Aila, T.: Improved precision and recall metric for assessing generative models. CoRR abs/1904.06991 (2019)
Laak, J., Litjens, G., Ciompi, F.: Deep learning in histopathology: the path to the clinic. Nature Medicine (2021). https://doi.org/10.1038/s41591-021-01343-4, https://doi.org/10.1038/s41591-021-01343-4
Lin, Y., Wang, Z., Cheng, K.T., Chen, H.: Insmix: Towards realistic generative data augmentation for nuclei instance segmentation. In: MICCAI 2022, Part II. pp. 140–149. Springer (2022)
Macenko, M., Niethammer, M., Marron, J.S., Borland, D., Woosley, J.T., Guan, X., Schmitt, C., Thomas, N.E.: A method for normalizing histology slides for quantitative analysis. In: Proceedings of the 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA, June 28 - July 1, 2009. pp. 1107–1110. IEEE (2009). https://doi.org/10.1109/ISBI.2009.5193250, https://drive.google.com/file/d/1eZGi1wUdyxVOYADXUbxZiVtajlztSnGL
Meng, C., Rombach, R., Gao, R., Kingma, D.P., Ermon, S., Ho, J., Salimans, T.: On distillation of guided diffusion models (2023)
Moghadam, P.A., Dalen, S.V., Martin, K.C., Lennerz, J., Yip, S., Farahani, H., Bashashati, A.: A morphology focused diffusion probabilistic model for synthesis of histopathology images. In: 2023 IEEE conference on computer vision and pattern recognition (2023)
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, Alec, e.a.: Zero-Shot Text-to-Image Generation (February 2021), arXiv:2102.12092
Reynaud, H., Qiao, M., Dombrowski, M., Day, T., Razavi, R., Gomez, A., Leeson, P., Kainz, B.: Feature-conditioned cascaded video diffusion models for precise echocardiogram synthesis. In: Greenspan, H., Madabhushi, A., Mousavi, P., Salcudean, S., Duncan, J., Syeda-Mahmood, T., Taylor, R. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. pp. 142–152. Springer Nature Switzerland, Cham (2023)
RL, G., AP, H., V, F., HE, V., DR, L., WA, K., LM., S.: Toward a shared vision for cancer genomic data. N Engl J Med. (2016). https://doi.org/10.1056/NEJMp1607591
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-Resolution Image Synthesis with Latent Diffusion Models (April 2022), arXiv:2112.10752
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S.K.S., Ayan, B.K., Mahdavi, S.S., Lopes, R.G., Salimans, T., Ho, J., Fleet, D.J., Norouzi, M.: Photorealistic text-to-image diffusion models with deep language understanding (2022). https://doi.org/10.48550/ARXIV.2205.11487, https://arxiv.org/abs/2205.11487
Shrivastava, A., Fletcher, P.T.: Nasdm: Nuclei-aware semantic histopathology image generation using diffusion models. In: Greenspan, H., Madabhushi, A., Mousavi, P., Salcudean, S., Duncan, J., Syeda-Mahmood, T., Taylor, R. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. pp. 786–796. Springer Nature Switzerland, Cham (2023)
stanford Song, Y.S., Sohl-Dickstein, J., Brain, G., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score-based generative modeling through stochastic differential equations. ICLR (2021)
Wagner, N., Fuchs, M., Tolkach, Y., Mukhopadhyay, A.: Federated stain normalization for computational pathology. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. pp. 14–23. Springer Nature Switzerland, Cham (2022)
Wang, J., Perez, L., et al.: The effectiveness of data augmentation in image classification using deep learning. Convolutional Neural Networks Vis. Recognit 11(2017), 1–8 (2017)
Wang, P.: lucidrains/imagen-pytorch: Implementation of Imagen, Google’s Text-to-Image Neural Network, in Pytorch — github.com. https://github.com/lucidrains/imagen-pytorch (2022), [Accessed 12-Nov-2022]
Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R.M., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M.: The cancer genome atlas pan-cancer analysis project. Nature Publishing Group (2013). https://doi.org/10.1038/ng.2764, http://www.cancergenome.nih.gov/.
Wu, B., Moeckel, G.: Application of digital pathology and machine learning in the liver, kidney and lung diseases. Journal of Pathology Informatics 14, 100184 (2023). https://doi.org/10.1016/j.jpi.2022.100184
Acknowledgments
S. Cechnicka is supported by the UKRI Centre for Doctoral Training AI4Health (EP/S023283/1). Support was also received from the ERC project MIA-NORMAL 101083647, the State of Bavaria (HTA) and DFG 512819079. HPC resources were provided by NHR@FAU of FAU Erlangen-Nürnberg under the NHR project b180dc. NHR@FAU hardware is partially funded by the DFG - 440719683. Dr. Roufosse is supported by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Imperial College Healthcare NHS Trust and Imperial College London (ICL). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. Dr Roufosse’s research activity is made possible with generous support from Sidharth and Indira Burman. Human samples used in this research project were obtained from the Imperial College Healthcare Tissue & Biobank (ICHTB). ICHTB is supported by NIHR Biomedical Research Centre based at Imperial College Healthcare NHS Trust and ICL. ICHTB is approved by Wales REC3 to release human material for research (22/WA/2836)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cechnicka, S. et al. (2024). URCDM: Ultra-Resolution Image Synthesis in Histopathology. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15004. Springer, Cham. https://doi.org/10.1007/978-3-031-72083-3_50
Download citation
DOI: https://doi.org/10.1007/978-3-031-72083-3_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72082-6
Online ISBN: 978-3-031-72083-3
eBook Packages: Computer ScienceComputer Science (R0)
