Abstract
Real-time automatic detection and tracking of moving vehicles in videos acquired by airborne cameras is a challenging problem due to vehicle occlusion, camera movement, and high computational cost. This paper presents an efficient and robust real-time approach for automatic vehicle detection and tracking in aerial videos that employ both detections and tracking features to enhance the final decision. The use of Top-hat and Bottom-hat transformation aided by the morphological operation in the detection phase has been adopted. After detection, background regions are eliminated by motion feature points’ analysis of the obtained object regions using a combined technique between KLT tracker and K-means clustering. Obtained object features are clustered into separate objects based on their motion characteristic. Finally, an efficient connecting algorithm is introduced to assign the vehicle labels with their corresponding cluster trajectories. The proposed method was tested on videos taken in different scenarios. The experimental results showed that the recall, precision, and tracking accuracy of the proposed method were about 95.1 %, 97.5%, and 95.2%, respectively. The method also achieves a fast processing speed. Thus, the proposed approach has superior overall performance compared to newly published approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdelwahab M A, Abdelwahab M M (2015) A novel algorithm for vehicle detection and tracking in airborne videos. In: Multimedia (ISM), 2015 IEEE International Symposium on, Miami, FL, USA, pp 65–68
Al-Kaff A, de la Escalera A, Armingol J M (2015) Sift and surf performance evaluation and the effect of freak descriptor in the context of visual odometry for unmanned aerial vehicles. In: International Conference on Computer Aided Systems Theory, Las Palmas de Gran Canaria, Spain, pp 739–747
Alkanat T, Tunali E, Öz S (2015) Fully-automatic target detection and tracking for real-time, airborne imaging applications. In: International Joint Conference on Computer Vision, Imaging and Computer Graphics, Berlin, Germany, pp 240–255
Bar-Shalom Y, Fortmann T, Scheffe M, et al. (1980) Joint probabilistic data association for multiple targets in clutter. In: Prof. conf. on information sciences and systems, Albuquerque, pp 404–409
Barth A, Franke U (2009) Estimating the driving state of oncoming vehicles from a moving platform using stereo vision. IEEE Trans Intell Transp Syst 10 (4):560–571
Bouguet J-Y (2001) Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corporation 5(1-10):4
Cao X, Jiang X, Li X, Yan P (2018) Correlation-based tracking of multiple targets with hierarchical layered structure. IEEE transactions on cybernetics 48(1):90–102
Cao X, Shi Z, Yan P, Li X (2013) Tracking vehicles as groups in airborne videos. Neurocomputing 99:38–45
Collins R, Zhou X, Teh S K (2005) An open source tracking testbed and evaluation web site. In: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2005), January 2005, Breckenridge, Colorado
Gao Z, Gao L-S, Zhang H, Cheng Z, Hong R (2019) Deep spatial pyramid features collaborative reconstruction for partial person reid. In: Proceedings of the 27th ACM International Conference on Multimedia, pp 1879–1887
Gao T, Li G, Lian S, Zhang J (2012) Tracking video objects with feature points based particle filtering. Multimedia Tools and Applications 58(1):1–21
Gao Z, Xu C, Zhang H, Li S, de Albuquerque V H C (2020) Trustful internet of surveillance things based on deeply-represented visual co-saliency detection. IEEE Internet Things J 7(5):4092–4100
Gomaa A, Abdelwahab M M, Abo-Zahhad M (2018) Real-time algorithm for simultaneous vehicle detection and tracking in aerial view videos. In: 2018 IEEE 61st International Midwest Symposium on Circuits and Systems (MWSCAS), IEEE, pp 222–225
Gomaa A, Abdelwahab M M, Abo-Zahhad M, Minematsu T, Taniguchi R- (2019) Robust vehicle detection and counting algorithm employing a convolution neural network and optical flow. Sensors 19(20):4588
Hess R, Fern A (2009) Discriminatively trained particle filters for complex multi-object tracking. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEE, Miami, FL, USA, pp 240–247
Jiang X, Cao X (2016) Surveillance from above: A detection-and-prediction based multiple target tracking method on aerial videos. In: Integrated Communications Navigation and Surveillance (ICNS), 2016, IEEE, pp 4D2–1
Kalantar B, Mansor S B, Halin A A, Shafri H Z M, Zand M (2017) Multiple moving object detection from uav videos using trajectories of matched regional adjacency graphs. IEEE Trans Geosci Remote Sens 55(9):5198–5213
Kanistras K, Martins G, Rutherford M J, Valavanis K P (2015) Survey of unmanned aerial vehicles (uavs) for traffic monitoring. In: Handbook of unmanned aerial vehicles. Springer, pp 2643–2666
Kasturi R, Goldgof D, Soundararajan P, Manohar V, Garofolo J, Bowers R, Boonstra M, Korzhova V, Zhang J (2008) Framework for performance evaluation of face, text, and vehicle detection and tracking in video: Data, metrics, and protocol. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(2):319–336
Ke R, Li Z, Kim S, Ash J, Cui Z, Wang Y (2017) Real-time bidirectional traffic flow parameter estimation from aerial videos. IEEE Trans Intell Transp Syst 18(4):890–901
Kent P, Maskell S, Payne O, Richardson S, Scarff L (2012) Robust background subtraction for automated detection and tracking of targets in wide area motion imagery. In: Optics and Photonics for Counterterrorism, Crime Fighting, and Defence VIII, vol 8546, International Society for Optics and Photonics, p 85460Q
Nguyen V D, Tran D T, Byun J Y, Jeon J W (2018) Real-time vehicle detection using an effective region proposal-based depth and 3-channel pattern. IEEE Trans Intell Transp Syst 20(10):3634–3646
Niknejad H T, Takeuchi A, Mita S, McAllester D (2012) On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation. IEEE Trans Intell Transp Syst 13(2):748–758
Noh S, Shim D, Jeon M (2016) Adaptive sliding-window strategy for vehicle detection in highway environments. IEEE Trans Intell Transp Syst 17 (2):323–335
Rad R, Jamzad M (2005) Real time classification and tracking of multiple vehicles in highways. Pattern Recogn Lett 26(10):1597–1607
Reid D, et al. (1979) An algorithm for tracking multiple targets. IEEE transactions on Automatic Control 24(6):843–854
Rosenbaum D, Leitloff J, Kurz F, Meynberg O, Reize T (2010) Real-time image processing for road traffic data extraction from aerial images. Proc. of ISPRS TC VII Symposium IAPRS XXXVIII:469–474
Su A, Sun X, Liu H, Zhang X, Yu Q (2015) Online cascaded boosting with histogram of orient gradient features for car detection from unmanned aerial vehicle images. J Appl Remote Sens 9(1):096063
Tang Y, Zhang C, Gu R, Li P, Yang B (2017) Vehicle detection and recognition for intelligent traffic surveillance system. Multimedia tools and applications 76(4):5817–5832
Teutsch M, Kruger W (2015) Robust and fast detection of moving vehicles in aerial videos using sliding windows. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA, pp 26–34
Teutsch M, Krüger W (2012) Spatio-temporal fusion of object segmentation approaches for moving distant targets. In: Information Fusion (FUSION), 2012 15th International Conference on, IEEE, pp 1988–1995
Teutsch M, Krüger W, Beyerer J (2017) Moving object detection in top-view aerial videos improved by image stacking. Opt Eng 56(8):083102
Wan E A, Van Der Merwe R (2001) The unscented Kalman filter. In: Haykin S. (ed) Kalman filtering and neural networks. ch. 7. Wiley, New York
Yang S, Xu J, Chen Y, Wang M (2014) On-road vehicle tracking using keypoint-based representation and online co-training. Multimedia tools and applications 72(2):1561–1583
Yao Y, Li J, Wu T, Zhang L (2017) Retracted article: Moving object surveillance using object proposals and background prior prediction. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-017-4820-9
Yu X, Shi Z (2015) Vehicle detection in remote sensing imagery based on salient information and local shape feature. Optik-International Journal for Light and Electron Optics 126(20):2485–2490
Zhao Z-Q, Zheng P, Xu S-, Wu X (2019) Object detection with deep learning: A review. IEEE transactions on neural networks and learning systems 30 (11):3212–3232
Zheng Z, Zhou G, Wang Y, Liu Y, Li X, Wang X, Jiang L (2013) A novel vehicle detection method with high resolution highway aerial image. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 6(6):2338–2343
Acknowledgments
This work was supported in part by the Egyptian Ministry of Higher Education (MoHE), Cairo, Egypt, and in part by the Egypt Japan University of Science and Technology (E-JUST), Alexandria, Egypt.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gomaa, A., Abdelwahab, M.M. & Abo-Zahhad, M. Efficient vehicle detection and tracking strategy in aerial videos by employing morphological operations and feature points motion analysis. Multimed Tools Appl 79, 26023–26043 (2020). https://doi.org/10.1007/s11042-020-09242-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09242-5