Abstract
An innovative background modeling technique that is able to accurately segment foreground regions in RGB-D imagery (RGB plus depth) has been presented in this paper. The technique is based on a Bayesian framework that efficiently fuses different sources of information to segment the foreground. In particular, the final segmentation is obtained by considering a prediction of the foreground regions, carried out by a novel Bayesian Network with a depth-based dynamic model, and, by considering two independent depth and color-based mixture of Gaussians background models. The efficient Bayesian combination of all these data reduces the noise and uncertainties introduced by the color and depth features and the corresponding models. As a result, more compact segmentations, and refined foreground object silhouettes are obtained. Experimental results with different databases suggest that the proposed technique outperforms existing state-of-the-art algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albiol, A., Albiol, A., Mossi, J., Oliver, J.: Who is who at different cameras: people re-identification using depth cameras. IET Comput. Vision 6(5), 378–387 (2012)
Arulampalam, M., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)
Barbosa, I., Cristani, M., Bue, A., Bazzani, L., Murino, V.: Re-identification with RGB-D sensors. In: Computer Vision ECCV 2012. Workshops and Demonstrations. Lecture Notes in Computer Science 7583, 433–442 (2012)
Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)
del Blanco, C.R., Jaureguizar, F., García, N.: Robust tracking in aerial imagery based on an ego-motion Bayesian model. EURASIP J. Adv. Signal Process. 2010, 1–19 (2010)
del Blanco, C.R., Jaureguizar, F., García, N.: An advanced Bayesian model for the visual tracking of multiple interacting objects. EURASIP J. Adv. Signal Process. 1, 130 (2011)
Bouwmans, T.: Recent advanced statistical background modeling for foreground detection - a systematic survey. Recent Patents Comput. Sci. 4(3), 147–176 (2011)
Bouwmans, T., Baf, F.E.: Background modeling using mixture of gaussians for foreground detection-a survey. Recent Patents Comput. Sci. 3, 219–237 (2008)
Camplani, M., Salgado, L.: Background foreground segmentation with RGB-D Kinect data: an efficient combination of classifiers. J. Vis. Commun. Image Represent. (2013) (in press)
Camplani, M., Mantecon, T., Salgado, L.: Accurate depth-color scene modeling for 3D contents generation with low cost depth cameras. In: 2012 19th IEEE International Conference on Image Processing (ICIP), pp 1741–1744 (2012)
Camplani, M., Mantecon, T., Salgado, L.: Depth-Color Fusion Strategy for 3D scene modeling with Kinect. IEEE Transactions on Cybernetics (accepted paper) (2013)
Cristani, M., Farenzena, M., Bloisi, D., Murino, V.: Background subtraction for automated multisensor surveillance: a comprehensive review. EURASIP J. Adv. Signal Process. 2010, 1–24 (2010)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification, 2nd edn. Wiley-Interscience, Newyork (2001)
Frick, A., Kellner, F., Bartczak, B., Koch, R.: Generation of 3D-TV LDV-content with time-of-flight camera. In: IEEE 3DTV Conference, pp 1–4 (2009)
Gordon, G., Darrell, T., Harville, M., Woodfill, J.: Background estimation and removal based on range and color. In: IEEE Computer Society Conference on Computer Vision and. Pattern Recognition 2, 464 (1999)
Goyette, N., Jodoin, P.M., Porikli, F., Konrad, J., Ishwar, P.: Changedetection.net: a new change detection benchmark dataset. In: IEEE computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, pp 1–8 (2012)
Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with microsoft Kinect Sensor: a review. IEEE transactions on Cybernetics (accepted paper) (2013).
Hofmann, M.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 38–43 (2012)
KaewTraKulPong, P., Bowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: European Workshop on Advanced Video Based Surveillance Systems, pp 149–158 (2001)
Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)
Klare, B., Sarkar, S.: Background subtraction in varying illuminations using an ensemble based on an enlarged feature set. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 66–73 (2009)
Kuncheva, L.: Combining pattern classifiers: methods and algorithms. Wiley-Interscience, Newyork (2004)
Leens, J., Barnich, O., Piérard, S., Droogenbroeck, M., Wagner, J.M.: Combining color, depth, and motion for video segmentation. In: Computer Vision Systems. Lecture Notes in Computer Science 5815, 104–113 (2009)
Li, L., Huang, W., Gu, I.Y.H., Tian, Q.: Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans. Image Process. 13(11), 1459–1472 (2004)
Maddalena, L., Petrosino, A.: A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans. Image Process. 17(7), 1168–1177 (2008)
Mastorakis, G., Makris, D.: Fall detection system using kinect’s infrared sensor. J. Real-Time Image Process. pp 1–12 (2012). doi:10.1007/s11554-012-0246-9
Molina, J., Escudero-Viñolo, M., Signoriello, A., Pardàs, M., Ferrán, C., Bescós, J., Marqués, F., Martínez, J.M.: Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models. Mach. Vis. Appl. 24(1), 187–204 (2011)
Spinello, L., Arras, K.: People detection in rgb-d data. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 3838–3843 (2011)
Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and, Pattern Recognition, pp 246–252 (1999)
Stone, E., Skubic, M.: Evaluation of an inexpensive depth camera for in-home gait assessment. J. Ambient Intell. Smart Environ. 3(4), 349–361 (2011)
Stormer, A., Hofmann, M., Rigoll, G.: Depth gradient based segmentation of overlapping foreground objects in range images. In: IEEE Conference on, Information Fusion, pp 1–4 (2010)
Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Proceedings of the Seventh IEEE International Conference on Computer Vision pp 255–261 (1999)
Acknowledgments
This work has been partially supported by the Ministerio de Economía y Competitividad of the Spanish Government under the project TEC2010-20412 (Enhanced 3DTV). M. Camplani would like to acknowledge the European Union and the Universidad Politécnica de Madrid (UPM) for supporting his activities through the Marie Curie-Cofund research grant.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Camplani, M., del Blanco, C.R., Salgado, L. et al. Advanced background modeling with RGB-D sensors through classifiers combination and inter-frame foreground prediction. Machine Vision and Applications 25, 1197–1210 (2014). https://doi.org/10.1007/s00138-013-0557-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-013-0557-2