Abstract
Purpose
Laparoscopic liver resection is a challenging procedure because of the difficulty to localise inner structures such as tumours and vessels. Augmented reality overcomes this problem by overlaying preoperative 3D models on the laparoscopic views. It requires deformable registration of the preoperative 3D models to the laparoscopic views, which is a challenging task due to the liver flexibility and partial visibility.
Methods
We propose several multi-view registration methods exploiting information from multiple views simultaneously in order to improve registration accuracy. They are designed to work on two scenarios: on rigidly related views and on non-rigidly related views. These methods exploit the liver’s anatomical landmarks and texture information available in all the views to constrain registration.
Results
We evaluated the registration accuracy of our methods quantitatively on synthetic and phantom data, and qualitatively on patient data. We measured 3D target registration errors in mm for the whole liver for the quantitative case, and 2D reprojection errors in pixels for the qualitative case.
Conclusion
The proposed rigidly related multi-view methods improve registration accuracy compared to the baseline single-view method. They comply with the 1 cm oncologic resection margin advised for hepatocellular carcinoma interventions, depending on the available registration constraints. The non-rigidly related multi-view method does not provide a noticeable improvement. This means that using multiple views with the rigidity assumption achieves the best overall registration error.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Koo B, Ozgur E, Le Roy B, Buc E, Bartoli A (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. In: MICCAI
Adagolodjo Y, Trivisonne R, Haouchine N, Cotin S, Courtecuisse H (2017) Silhouette-based pose estimation for deformable organs application to surgical augmented reality. In: IROS
Modrzejewski R, Collins T, Seeliger B, Bartoli A, Hostettler A, Marescaux J (2019) An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions. In: IPCAI
Chen L, Tang W, John NW, Wuan TR, Zhang JJ (2018) SLAM-based dense surface reconstruction in monocular minimally invasive surgery and its application to augmented reality. Comput Methods Programs Biomed 158:135–146
Haouchine N, Roy F, Untereiner L, Cotin S (2016) Using contours as boundary conditions for elastic registration during minimally invasive hepatic surgery. In: IROS
Robu MR, Ramalhinho J, Thompson S, Gurusamy K, Davidson B, Hawkes D, Stoyanov D, Clarkson M (2018) Global rigid registration of CT to video in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 13:947–956
Thompson S, Totz J, Song Y, Johnsen S, Stoyanov D, Ourselin S, Gurusamy K, Schneider C, Davidson B, Hawkes D, Clarkson M (2015) Accuracy validation of an image guided laparoscopy system for liver resection. Proceedings of SPIE - The International Society for Optical Engineering 9415(09):1–12
Plantefeve R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44:139–153
Clements L, Collins J, Weis J, Simpson A, Kingham T, Jarnagin W, Miga M (2017) Deformation correction for image guided liver surgery: an intraoperative fidelity assessment. Surgery 162(3):537–547
Bernhardt S, Nicolau S, Bartoli A, Agnus V, Soler L, Doignon C (2015) Using shading to register an intraoperative CT scan to a laparoscopic image. In: Computer-assisted and robotic endoscopy. CARE
Plantefeve R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44:139–153
Thompson S, Hu M, Johnsen S, Gurusamy K, Davidson B, Hawkes D (2011) Towards image guided laparoscopic liver surgery, defining the system requirement. In: LIVIM
Bender J, Koschier D, Charrier P, Weber D (2014) Position-based simulation of continuous materials. Comput Graph 44:1–10
Zhong FP, Zhang YJ, Liu Y, Zou SB (2017) Prognostic impact of surgical margin in patients with hepatocellular carcinoma: a meta-analysis. Medicine 96(37):e8043
Totz J, Thompson S, Stoyanov D, Gurusamy K, Davidson B, Hawkes D, Clarkson M (2014) Fast semi-dense surface reconstruction from stereoscopic video in laparoscopic surgery. In: IPCAI
Pfeiffer M, Riediger C, Leger S, Kühn J, Seppelt D, Hoffmann R, Weitz J, Speidel S (2020) Non-rigid volume to surface registration using a data-driven biomechanical model. In: MICCAI
Chui C, Kobayashi E, Chen X, Hisada T, Sakuma I (2004) Combined compression and elongation experiments and non-linear modelling of liver tissue for surgical simulation. Med Biol Eng Comput 44:787–798
Shi H, Farag A, Fahmi R, Chen D (2008) Validation of finite element models of liver tissue using micro-CT. IEEE Trans Bio-med Eng 55:978–984
Pizarro D, Bartoli A (2010) Feature-based deformable surface detection with self-occlusion reasoning. Int J Comput Vis 97:54–70
Bartoli A, Perriollat M, Chambon S (2010) Generalized thin-plate spline warps. Int J Comput Vis 88:85–110
3DS Abaqus. http://edu.3ds.com/en/software/abaqus-student-edition. Accessed 2 Mar 2021
Ali S, Espinel Y, Jin Y, Maier-Hein L, Bartoli A (2022) Preoperative to intraoperative laparoscopy fusion (P2ILF). In: MICCAI, 2022. https://p2ilf.grand-challenge.org/
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Yamid Espinel declares to have no potential conflicts of interest. Lilian Calvet declares to have no potential conflicts of interest. Karim Botros declares to have no potential conflicts of interest. Emmanuel Buc declares to have no potential conflicts of interest. Christophe Tilmant declares to have no potential conflicts of interest. Adrien Bartoli declares to have no potential conflicts of interest. All procedures involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study is also supported by an ethical approval with ID IRB00008526-2019-CE58 issued by CPP Sud-Est VI in Clermont-Ferrand, France. Informed consent was obtained from the patients included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Espinel, Y., Calvet, L., Botros, K. et al. Using multiple images and contours for deformable 3D–2D registration of a preoperative CT in laparoscopic liver surgery. Int J CARS 17, 2211–2219 (2022). https://doi.org/10.1007/s11548-022-02774-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-022-02774-1