Abstract
Several computer-based medical systems have been proposed for automatic detection of abnormalities in a variety of medical imaging domains. The majority of these systems are based on binary supervised classification algorithms capable of discriminating abnormal from normal image patterns. However, this approach usually does not take into account that the normal content of images is diverse, including various kinds of tissues and artifacts. In the context of gastrointestinal video-endoscopy, which is addressed in this study, the semantics of the normal content include mucosal tissues, the hole of the lumen, bubbles, and debris. In this paper we investigate such a semantic interpretation of the endoscopy video content as an approach to improve lesion detection in a weakly supervised framework. This framework is based on a novel salient point detection algorithm, the bag-of-words image representation technique and multi-label classification. Advantages of the proposed method include: (a) It does not require detailed, pixel-level annotation of training images, instead image-level annotations are sufficient; (b) It enables a richer description of image content, which is beneficial for the discrimination of lesions. The annotation of the multi-labeled training images was performed using a novel annotation tool called RATStream. The results of the experiments performed in a wireless capsule endoscopy dataset with inflammatory lesions promises an improved performance for future generation diagnostic systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Ratstream is available upon request to the authors.
- 2.
- 3.
References
Smeulders, A., Worring, M., Santini, S., et al.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1349–1380 (2000). doi:10.1109/34.895972
Li, H., Liu, L., Sun, F., et al.: Multi-level feature representations for video semantic concept detection. Neurocomputing 172, 64–70 (2016). doi:10.1016/j.neucom.2014.09.096
Iakovidis, D., Koulaouzidis, A.: Software for enhanced video capsule endoscopy: challenges for essential progress. Nat. Rev. Gastroenterol. Hepatol. 12, 172–186 (2015). doi:10.1038/nrgastro.2015.13
Georgakopoulos, S., Iakovidis, D., Vasilakakis, M., et al.: Weakly-supervised convolutional learning for detection of inflammatory gastrointestinal lesions. In: IEEE International Conference on Imaging Systems and Techniques (IST), pp. 510–514. IEEE (2016)
Vasilakakis, M., Iakovidis, D., Spyrou, V., Koulaouzidis, A.: Weakly-supervised lesion detection in video capsule endoscopy based on a bag-of-colour features model. In: International Workshop on Computer-Assisted Robotic Endoscopy (CARE) at International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) (2016)
Koulaouzidis, A.: Wireless endoscopy in 2020: will it still be a capsule? World J. Gastroenterol. 21, 5119 (2015). doi:10.3748/wjg.v21.i17.5119
Yung, D., Fernandez-Urien, I., Douglas, S., Plevris, J., Sidhu, R., McAlindon, M., Panter, S., Koulaouzidis, A.: Systematic review and meta-analysis of the performance of nurses in small bowel capsule endoscopy reading. United Eur. Gastroenterol. J., 205064061668723. (2017) doi:10.1177/2050640616687232
Zheng, Y., Hawkins, L., Wolff, J., Goloubeva, O., Goldberg, E.: Detection of lesions during capsule endoscopy: physician performance is disappointing. Am. J. Gastroenterol. 107, 554–560 (2012). doi:10.1038/ajg.2011.461
Iakovidis, D., Sarmiento, R., Silva, J., Histace, A., Romain, O., Koulaouzidis, A., Dehollain, C., Pinna, A., Granado, B., Dray, X.: Towards intelligent capsules for robust wireless endoscopic imaging of the gut. In: IEEE International Conference on Imaging Systems and Techniques, pp. 95–100. IEEE (2014)
Koulaouzidis, A.: Small-bowel capsule endoscopy: a ten-point contemporary review. World J. Gastroenterol. 19, 3726 (2013). doi:10.3748/wjg.v19.i24.3726
Riphaus, A., Richter, S., Vonderach, M., Wehrmann, T.: Capsule Endoscopy Interpretation by an Endoscopy Nurse – a Comparative Trial. Zeitschrift für Gastroenterologie 47, 273–276 (2009). doi:10.1055/s-2008-1027822
Koulaouzidis, A.: KID: Koulaouzidis-iakovidis database for capsule endoscopy (2015). http://is-innovation.eu/kid
Iakovidis, D., Koulaouzidis, A.: Automatic lesion detection in capsule endoscopy based on color saliency: closer to an essential adjunct for reviewing software. Gastrointest. Endosc. 80, 877–883 (2014). doi:10.1016/j.gie.2014.06.026
Hoai, M., Torresani, L., De la Torre, F., Rother, C.: Learning discriminative localization from weakly labeled data. Pattern Recogn. 47, 1523–1534 (2014). doi:10.1016/j.patcog.2013.09.028
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004). doi:10.1023/b:visi.0000029664.99615.94
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110, 346–359 (2008). doi:10.1016/j.cviu.2007.09.014
Tuytelaars, T.: Dense interest points. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2281–2288 (2010)
Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Elsevier/Academic Press, Amsterdam (2008)
Tsoumakas, G., Katakis, I.: Multi-label classification. Int. J. Data Warehouse. Min. 3, 1–13 (2007). doi:10.4018/jdwm.2007070101
Zhang, M., Zhou, Z.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26, 1819–1837 (2014). doi:10.1109/tkde.2013.39
Zhang, M., Zhou, Z.: ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40, 2038–2048 (2007). doi:10.1016/j.patcog.2006.12.019
Elisseeff, A., Weston, J.: A kernel method for multi-labeled classification. In: NIPS, pp. 681–687 (2001)
Fürnkranz, J., Hüllermeier, E., Loza Mencía, E., Brinker, K.: Multilabel classification via calibrated label ranking. Mach. Learn. 73, 133–153 (2008). doi:10.1007/s10994-008-5064-8
Mencia, E., Furnkranz, J.: Pairwise learning of multilabel classifications with perceptrons. In: 2008 IEEE International Joint Conference on Neural Networks, IJCNN 2008, (IEEE World Congress on Computational Intelligence), pp. 2899–2906. IEEE (2008)
Read, J., Pfahringer, B., Holmes, G.: Multi-label classification using ensembles of pruned sets. In: 2008 Eighth IEEE International Conference Data Mining, ICDM 2008 (2008)
Iakovidis, D., Goudas, T., Smailis, C., Maglogiannis, I.: Ratsnake: a versatile image annotation tool with application to computer-aided diagnosis. Sci. World J. 2014, 1–12 (2014). doi:10.1155/2014/286856
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012). doi:10.1109/tpami.2012.120
Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27, 861–874 (2006). doi:10.1016/j.patrec.2005.10.010
Provost, F., Fawcett, T.: Analysis and visualization of classifier performance: comparison under imprecise class and cost distributions. In: KDD, pp. 43–48 (1997)
Witten, I., Frank, E., Hall, M., Pal, C.: Data Mining, 1st edn. Morgan Kaufmann, Amsterdam (2017)
Read, J., Reutemann, P., Pfahringer, B., Holmes, G.: MEKA: a multi-label/multi-target extension to WEKA. J. Mach. Learn. Res. 17, 1–5 (2017)
Yuan, Y., Li, B., Meng, M.: Improved bag of feature for automatic polyp detection in wireless capsule endoscopy images. IEEE Trans. Autom. Sci. Eng. 13, 529–535 (2016). doi:10.1109/tase.2015.2395429
Acknowledgements
The research presented in this paper was financially supported by the project “Klearchos Koulaouzidis” Grant No. 5151 and the Special Account of Research Grants of the University of Thessaly, Greece.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Vasilakakis, M.D., Iakovidis, D.K., Spyrou, E., Chatzis, D., Koulaouzidis, A. (2017). Beyond Lesion Detection: Towards Semantic Interpretation of Endoscopy Videos. In: Boracchi, G., Iliadis, L., Jayne, C., Likas, A. (eds) Engineering Applications of Neural Networks. EANN 2017. Communications in Computer and Information Science, vol 744. Springer, Cham. https://doi.org/10.1007/978-3-319-65172-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-65172-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65171-2
Online ISBN: 978-3-319-65172-9
eBook Packages: Computer ScienceComputer Science (R0)