Abstract
Drone swarms consist of multiple drones that can achieve tasks that individual drones can not, such as search and recovery or surveillance over a large area. A swarm’s internal structure typically consists of multiple drones operating autonomously. Reliable detection and tracking of swarms and individual drones allow a greater understanding of the behaviour and movement of a swarm. Increased understanding of drone behaviour allows better coordination, collision avoidance, and performance monitoring of individual drones in the swarm. The research presented in this paper proposes a deep learning-based approach for reliable detection and tracking of individual drones within a swarm using stereo-vision cameras in real time. The motivation behind this research is in the need to gain a deeper understanding of swarm dynamics, enabling improved coordination, collision avoidance, and performance monitoring of individual drones within a swarm. The proposed solution provides a precise tracking system and considers the highly dense and dynamic behaviour of drones. The approach is evaluated in both sparse and dense networks in a variety of configurations. The accuracy and efficiency of the proposed solution have been analysed by implementing a series of comparative experiments that demonstrate reasonable accuracy in detecting and tracking drones within a swarm.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Code or data availability
The code used during the current study is available from the corresponding author on a reasonable request.
References
Navarro, I., Matia, F.: An introduction to gswarm robotics. Int Scholarly Res Notices 2013 (2013)
Khaldi, B., Cherif, F.: An overview of swarm robotics: Swarm intelligence applied to multi-robotics. Int J Comput Appl 126(2), 31–37 (2015)
Saha, H.N., Das, N.K., Pal, S.K., Basu, S., Auddy, S., Dey, R., Nandy, A., Pal, D., Roy, N., Mitra, D., et al.: A cloud based autonomous multipurpose system with self-communicating bots and swarm of drones. In: 2018 IEEE Annual Computing and Communication Workshop and Conference (CCWC), pp. 649–653 (2018)
Wang, X., Green, D., Barca, J.C.: Guidelines for improving the robustness of swarm robotic systems through adjustment of network topology. In: 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 1399–1405 (2017)
Sparrow, R.J.: Killer robots: Ethical issues in the design of unmanned systems for military applications. In: Handbook of Unmanned Aerial Vehicles, pp. 2965–2983 (2015)
Gupta, M., Saxena, D., Kumari, S., Kaur, D.: Issues and applications of swarm robotics. Int. J. Res. Eng., Technol. Sci. 6, 1–5 (2016)
Xiaohong, W., Zhang, Y., Lizhi, W., Dawei, L., Guoqi, Z.: Robustness evaluation method for unmanned aerial vehicle swarms based on complex network theory. Chinese J. Aeronautics 33(1), 352–364 (2020)
Scharre, P.: Counter-swarm: a guide to defeating robotic swarms. https://warontherocks.com/2015/03/counter-swarm-a-guide-to-defeating-robotic-swarms (Accessed 02 April 2023)
Scharre, P.: Army of None: Autonomous Weapons and the Future of War. WW Norton & Company, New York (2018)
Beauchamp, G.: Social Predation: How Group Living Benefits Predators and Prey. Elsevier, UK (2013)
Procaccini, A., Orlandi, A., Cavagna, A., Giardina, I., Zoratto, F., Santucci, D., Chiarotti, F., Hemelrijk, C.K., Alleva, E., Parisi, G., et al.: Propagating waves in starling, sturnus vulgaris, flocks under predation. Animal Behaviour 82(4), 759–765 (2011)
Hettiarachchige, Y., Khan, A., Barca, J.C.: Multi-object tracking of swarms with active target avoidance. In: 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1204–1209 (2018)
Scharre, P.: How swarming will change warfare. Bullet. Atomic Scientists 74(6), 385–389 (2018)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: 2016 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Campion, M., Ranganathan, P., Faruque, S.: A review and future directions of uav swarm communication architectures. In: 2018 IEEE International Conference on Electro/information Technology (EIT), pp. 0903–0908 (2018)
Teague, E., Kewley Jr, R.H.: Swarming unmanned aircraft systems. Operations Research Center of Excellence (ORCEN) Technical Report DSE-TR-0808. West Point, NY: US Military Academy ORCEN (2008)
Burkle, A., Segor, F., Kollmann, M.: Towards autonomous micro uav swarms. J. Intell. & Robotic Syst. 61, 339–353 (2011)
Husseini, T.: Gremlins are coming: DARPA enters Phase III of its UAV programme. https://www.army-technology.com/features/gremlins-darpa-uav-programme/ (Accessed 19 February 2023)
Airforce Technology: Initial Operating Capability for Swarm Drone Technology Could Be Achieved in Less Than Ten Years: Poll. https://www.airforce-technology.com/news/initial-operating-capability-for-swarm-drone-technology-could-be-achieved-in-less-than-ten-years-poll/ (Accessed 02 April 2023)
Bakhshipour, M., Ghadi, M.J., Namdari, F.: Swarm robotics search & rescue: A novel artificial intelligence-inspired optimization approach. Appl. Soft Comput. 57, 708–726 (2017)
Liao, Y.-L., Su, K.-L.: Multi-robot-based intelligent security system. Artif. Life Robotics 16(2), 137 (2011)
Shiomi, M., Kamei, K., Kondo, T., Miyashita, T., Hagita, N.: Robotic service coordination for elderly people and caregivers with ubiquitous network robot platform. In: 2013 IEEE Workshop on Advanced Robotics and Its Social Impacts, pp. 57–62 (2013)
Gerkey, B.P., Thrun, S., Gordon, G.: Parallel stochastic hill-climbing with small teams. In: Multi-Robot Systems. From Swarms to Intelligent Automata vol. III, pp. 65–77 (2005)
Lemaire, T., Alami, R., Lacroix, S.: A distributed tasks allocation scheme in multi-uav context. In: 2004 IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04. 2004, vol. 4, pp. 3622–3627 (2004)
D’Emidio, M., Frigioni, D., Navarra, A.: Exploring and making safe dangerous networks using mobile entities. In: 2013 International Conference on Ad-Hoc Networks and Wireless, pp. 136–147 (2013)
Ducatelle, F., Di Caro, G.A., Forster, A., Gambardella, L.: Mobile stigmergic markers for navigation in a heterogeneous robotic swarm. In: 2010 International Conference on Swarm Intelligence, pp. 456–463 (2010)
Kallenborn, Z.: The Era of the Drone Swarm is Coming, and We Need to be Ready for it. https://mwi.usma.edu/era-drone-swarm-coming-need-ready/ (Accessed 25 March 2023)
Embention: Drone Swarm Performance and Applications. https://www.embention.com/news/drone-swarm-performance-and-applications/ (Accessed 10 April 2023)
Chang, X., Yang, C., Wu, J., Shi, X., Shi, Z.: A surveillance system for drone localization and tracking using acoustic arrays. In: 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), pp. 573–577 (2018)
Chang, X., Yang, C., Shi, X., Li, P., Shi, Z., Chen, J.: Feature extracted doa estimation algorithm using acoustic array for drone surveillance. In: 2018 IEEE 87th Vehicular Technology Conference (VTC Spring), pp. 1–5 (2018)
Yang, C., Wu, Z., Chang, X., Shi, X., Wo, J., Shi, Z.: Doa estimation using amateur drones harmonic acoustic signals. In: 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), pp. 587–591 (2018)
Ganti, S.R., Kim, Y.: Implementation of detection and tracking mechanism for small uas. In: 2016 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1254–1260 (2016)
Hommes, A., Shoykhetbrod, A., Noetel, D., Stanko, S., Laurenzis, M., Hengy, S., Christnacher, F.: Detection of acoustic, electro-optical and radar signatures of small unmanned aerial vehicles. In: 2016 Target and Background Signatures II, vol. 9997, p. 999701 (2016)
Hauzenberger, L., Holmberg Ohlsson, E.: Drone detection using audio analysis (2015)
Abdelkader, M., Guler, S., Jaleel, H., Shamma, J.S.: Aerial swarms: Recent applications and challenges. Current Robotics Reports, 1–12 (2021)
Sevil, H.E., Dogan, A., Subbarao, K., Huff, B.: Evaluation of extant computer vision techniques for detecting intruder suas. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 929–938 (2017)
Hwang, S., Lee, J., Shin, H., Cho, S., Shim, D.H.: Aircraft detection using deep convolutional neural network in small unmanned aircraft systems. In: 2018 AIAA Information Systems-AIAA Infotech@ Aerospace, p. 2137 (2018)
Censi, A., Strubel, J., Brandli, C., Delbruck, T., Scaramuzza, D.: Low-latency localization by active led markers tracking using a dynamic vision sensor. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 891–898 (2013)
Nguyen, P.H., Kim, K.W., Lee, Y.W., Park, K.R.: Remote marker-based tracking for uav landing using visible-light camera sensor. Sensors 17(9), 1987 (2017)
Ruiz, C., Pan, S., Bannis, A., Chen, X., Joe-Wong, C., Noh, H.Y., Zhang, P.: Idrone: Robust drone identification through motion actuation feedback. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2(2), 1–22 (2018)
Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Kim, T.-K.: Multiple object tracking: A literature review. Artif Intell 293, 103448 (2021)
Senanayake, M., Senthooran, I., Barca, J.C., Chung, H., Kamruzzaman, J., Murshed, M.: Search and tracking algorithms for swarms of robots: A survey. Robot. Autonomous Syst. 75, 422–434 (2016)
Cimino, M.G., Massimiliano, L., Monaco, M., Vaglini, G., et al.: Adaptive exploration of a uavs swarm for distributed targets detection and tracking. In: 2019 Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods, vol. 1, pp. 837–844 (2019)
Kumari, N., Lee, K., Ranaweera, C., Barca, J.C.: A comparison of clustering vs yolo for drone swarm centroid detection. In: 2023 8th International Conference on Robotics and Automation Engineering (ICRAE), pp. 106–111 (2023). https://doi.org/10.1109/ICRAE59816.2023.10458512
Kumari, N., Lee, K., Ranaweera, C., Barca, J.C.: Visually detecting drones in drone swarm formations topologies. In: International Conference on Information Technology and Applications, pp. 21–30 (2022). https://doi.org/10.1007/978-981-99-8324-7_3 . Springer
Coops, N.C., Goodbody, T.R., Cao, L.: Four steps to extend drone use in research. Nature 572(7770), 433–435 (2019)
Johnson, J.: Artificial intelligence, drone swarming and escalation risks in future warfare. RUSI J. 165(2), 26–36 (2020)
Yang, B., Huang, C., Nevatia, R.: Learning affinities and dependencies for multi-target tracking using a crf model. In: 2011 CVPR, pp. 1233–1240 (2011)
Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 261–268 (2009)
Case, E.E., Zelnio, A.M., Rigling, B.D.: Low-cost acoustic array for small uav detection and tracking. In: 2008 IEEE National Aerospace and Electronics Conference, pp. 110–113 (2008)
Tang, Q., Yu, F., Ding, L.: A grouping method for multiple targets search using swarm robots. In: 2016 International Conference on Swarm Intelligence, pp. 470–478 (2016)
Wang, X.: Intelligent multi-camera video surveillance: A review. Pattern Recognition Lett. 34(1), 3–19 (2013)
Candamo, J., Shreve, M., Goldgof, D.B., Sapper, D.B., Kasturi, R.: Understanding transit scenes: A survey on human behavior-recognition algorithms. IEEE Trans. Intell. Transportation Syst. 11(1), 206–224 (2009)
Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., Hengel, A.V.D.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. (TIST) 4(4), 1–48 (2013)
Wu, Y., Lim, J., Yang, M.-H.: Online object tracking: A benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2411–2418 (2013)
Leal-Taixe, L., Milan, A., Reid, I., Roth, S., Schindler, K.: MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking (2015)
Scott-Samuel, N.E., Holmes, G., Baddeley, R., Cuthill, I.C.: Moving in groups: how density and unpredictable motion affect predation risk. Behavioral Ecol. Sociobiol. 69(6), 867–872 (2015)
Ioannou, C., Tosh, C., Neville, L., Krause, J.: The confusion effect–from neural networks to reduced predation risk. Behavioral Ecol. 19(1), 126–130 (2008)
Cavagna, A., Giardina, I., Orlandi, A., Parisi, G., Procaccini, A.: The STARFLAG handbook on collective animal behaviour: Part II, three-dimensional analysis (2008)
Singha, S., Aydin, B.: Automated drone detection using yolov4. Drones 5(3), 95 (2021)
Jung, H.-K., Choi, G.-S.: Improved yolov5: Efficient object detection using drone images under various conditions. Appl. Sci. 12(14), 7255 (2022)
Aydin, B., Singha, S.: Drone detection using yolov5. Eng 4(1), 416–433 (2023)
Alsanad, H.R., Sadik, A.Z., Ucan, O.N., Ilyas, M., Bayat, O.: Yolo-v3 based real-time drone detection algorithm. Multimed. Tools Appl. 81(18), 26185–26198 (2022)
Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., Ling, H.: Lasot: A high-quality benchmark for large-scale single object tracking. In: 2019 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5374–5383 (2019)
Le, T.: Real-time object detection and tracking on drones. Undergraduate Res. & Mentoring Program 25 (2018)
Fan, H., Du, D., Wen, L., Zhu, P., Hu, Q., Ling, H., Shah, M., Pan, J., Schumann, A., Dong, B., et al.: Visdrone-mot2020: The vision meets drone multiple object tracking challenge results. In: 2020 European Conference on Computer Vision, pp. 713–727 (2020)
Li, Q., Li, R., Ji, K., Dai, W.: Kalman filter and its application. In: 2015 8th International Conference on Intelligent Networks and Intelligent Systems (ICINIS), pp. 74–77 (2015)
Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)
Cipra, T., Romera, R.: Robust kalman filter and its application in time series analysis. Kybernetika 27(6), 481–494 (1991)
Lo, L.-Y., Yiu, C.H., Tang, Y., Yang, A.-S., Li, B., Wen, C.-Y.: Dynamic object tracking on autonomous uav system for surveillance applications. Sensors 21(23), 7888 (2021)
Kumari, D., Kaur, K.: A survey on stereo matching techniques for 3d vision in image processing. Int. J. Eng. Manuf. 4, 40–49 (2016)
Pereira, R., Carvalho, G., Garrote, L., Nunes, U.J.: Sort and deep-sort based multi-object tracking for mobile robotics: Evaluation with new data association metrics. Appl. Sci. 12(3), 1319 (2022)
Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., Yang, M.-H.: Online multi-object tracking with dual matching attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 366–382 (2018)
Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., Luo, P., Liu, W., Wang, X.: Bytetrack: Multi-object tracking by associating every detection box. In: European Conference on Computer Vision, pp. 1–21 (2022). Springer
Campbell, M.E., Grocott, S.C.: Parametric uncertainty model for control design and analysis. IEEE Trans. Control Syst. Technol. 7(1), 85–96 (1999)
Bhattacharyya, S.: Robust control under parametric uncertainty: An overview and recent results. Annual Rev. Control 44, 45–77 (2017)
Funding
Open Access funding enabled and organized by CAUL and its Member Institutions.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Nisha Kumari. Experiments were conducted by Nisha Kumari. The first draft of the manuscript was written by Nisha Kumari and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflicts of interest to this work.
Ethics Approval
Not applicable.
Consent to Participate
Not applicable.
Consent for Publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kumari, N., Lee, K., Barca, J.C. et al. Towards Reliable Identification and Tracking of Drones Within a Swarm. J Intell Robot Syst 110, 84 (2024). https://doi.org/10.1007/s10846-024-02115-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-024-02115-1