Skip to main content

Advertisement

Springer Nature Link
Log in

GTAN: graph-based tracklet association network for multi-object tracking

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Multi-object tracking (MOT) is a thriving research field in computer vision. The tracklet-based MOT frameworks are frequently employed to generate long and stable trajectories in work scenes that involve long-term occlusion. However, most of these methods train tracklet feature encoders using complex loss functions, lacking an end-to-end paradigm guided by association results, which ultimately leads to limited MOT performance. To address this issue, a graph-based tracklet association framework that seamlessly integrates tracklet feature learning with tracklet association, thereby achieving tracklet association in an end-to-end manner. Specifically, we perform tracklet-based MOT in the graph domain and transform the tracklet association problem into an edge classification task. A message passing network (MPN) is used to update the tracklet features globally, which enhances the robustness of the tracklet features. Additionally, an attention-based feature update function is proposed to ensure the significance of current object. The effectiveness of the proposed framework is demonstrated using MOT17 and MOT20 benchmark datasets, and the experimental results show that the graph-based tracklet association network is a model-independent and plug-and-play component that could combine with different frame-based trackers to boost the MOT performance significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Tong K, Wu Y (2020) Zhou F Recent advances in small object detection based on deep learning: A review. Image Vis Comput 97:103910

    Article  Google Scholar 

  2. Suljagic H, Bayraktar E (2022) Celebi N Similarity based person re-identification for multi-object tracking using deep siamese network. Neural Comput Appl 34(20):18171–18182

    Article  Google Scholar 

  3. Yang B, Nevatia R Multi-target tracking by online learning of non-linear motion patterns and robust appearance models. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) (2012)

  4. Yuan D, Shu X, Liu Q, Zhang X (2023) He Z Robust thermal infrared tracking via an adaptively multi-feature fusion model. Neural Comput Appl 35(4):3423–3434

    Article  PubMed  Google Scholar 

  5. Yang K, Song H, Zhang K (2020) Liu Q Hierarchical attentive siamese network for real-time visual tracking. Neural Comput Appl 32(18):14335–14346

    Article  Google Scholar 

  6. Ma C, Yang C, Yang F, Zhuang Y, Zhang Z, Jia H, Xie X Trajectory factory: Tracklet cleaving and re-connection by deep siamese bi-gru for multiple object tracking. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2018). https://doi.org/10.1109/ICME.2018.8486454

  7. Milan A, Roth S (2014) Schindler K Continuous energy minimization for multitarget tracking. IEEE Trans Pattern Anal Mach Intell 36(1):58–72. https://doi.org/10.1109/TPAMI.2013.103

    Article  PubMed  Google Scholar 

  8. Oh S, Russell S (2009) Sastry S Markov chain monte carlo data association for multi-target tracking. IEEE Trans Autom Control 54(3):481–497. https://doi.org/10.1109/TAC.2009.2012975

    Article  Google Scholar 

  9. Ban Y, Ba S.O, Alameda-Pineda X, Horaud R Tracking multiple persons based on a variational bayesian model. In: Proceedings of the European Conference on Computer Vision(ECCV), vol. 9914, pp. 52–67 (2016)

  10. Choi W Near-online multi-target tracking with aggregated local flow descriptor. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3029–3037 (2015). https://doi.org/10.1109/ICCV.2015.347

  11. Leibe B, Schindler K, Cornelis N (2008) Van Gool L Coupled object detection and tracking from static cameras and moving vehicles. IEEE Trans Pattern Anal Mach Intell 30(10):1683–1698. https://doi.org/10.1109/TPAMI.2008.170

    Article  PubMed  Google Scholar 

  12. Kim C, Li F, Ciptadi A, Rehg J.M Multiple hypothesis tracking revisited. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 4696–4704 (2015). https://doi.org/10.1109/ICCV.2015.533

  13. Welch G, Bishop G An introduction to the kalman filter (2006)

  14. Kuhn H.W The hungarian method for the assignment problem. Naval Research Logistics Quarterly 2(1-2), 83–97 (1955)

  15. Yang B, Nevatia R An online learned crf model for multi-target tracking. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 2034–2041 (2012). https://doi.org/10.1109/CVPR.2012.6247907

  16. Leal-Taixé L, Canton-Ferrer C, Schindler K Learning by tracking: Siamese cnn for robust target association. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR), pp. 418–425 (2016). https://doi.org/10.1109/CVPRW.2016.59

  17. Zhang Y, Wang C, Wang X, Zeng W (2021) Liu W Fairmot: On the fairness of detection and re-identification in multiple object tracking. Int J Comput Vision 129(11):3069–3087. https://doi.org/10.1007/s11263-021-01513-4

    Article  Google Scholar 

  18. Zhou X, Koltun V, Krähenbühl P Tracking objects as points. In: European Conference on Computer Vision, pp. 474–490 (2020)

  19. Wang B, Wang G, Chan KL (2017) Wang L Tracklet association by online target-specific metric learning and coherent dynamics estimation. IEEE Trans Pattern Anal Mach Intell 39(3):589–602. https://doi.org/10.1109/TPAMI.2016.2551245

    Article  Google Scholar 

  20. Peng J, Wang T, Lin W, Wang J, See J, Wen S, Ding E Tpm: Multiple object tracking with tracklet-plane matching. Pattern Recognition 107, 107480 (2020) https://doi.org/10.1016/j.patcog.2020.107480

  21. Li G, Peng M, Nai K, Li Z (2020) Li K Multi-view correlation tracking with adaptive memory-improved update model. Neural Comput Appl 32:9047–9063

    Article  Google Scholar 

  22. Wang G, Wang Y, Gu R, Hu W, Hwang J.-N Split and connect: A universal tracklet booster for multi-object tracking. IEEE Transactions on Multimedia, 1–1 (2022) https://doi.org/10.1109/TMM.2022.3140919

  23. Wang Y, Kitani K, Weng X Joint object detection and multi-object tracking with graph neural networks. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 13708–13715 (2021). https://doi.org/10.1109/ICRA48506.2021.9561110

  24. Brasó G, Leal-Taixé L Learning a neural solver for multiple object tracking. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6246–6256 (2020). https://doi.org/10.1109/CVPR42600.2020.00628

  25. Shen H, Huang L, Huang C, Xu W Tracklet association tracker: An end-to-end learning-based association approach for multi-object tracking. arXiv preprint arXiv:1808.01562 (2018)

  26. Lan L, Wang X, Zhang S, Tao D, Gao W, Huang T.S Interacting tracklets for multi-object tracking. IEEE Transactions on Image Processing 27(9), 4585–4597 (2018) https://doi.org/10.1109/TIP.2018.2843129

  27. Yang K, He Z, Pei W, Zhou Z, Li X, Yuan D (2021) Zhang H Siamcorners: Siamese corner networks for visual tracking. IEEE Trans Multimedia 24:1956–1967

    Article  Google Scholar 

  28. Yuan D, Chang X, Huang P-Y, Liu Q (2020) He Z Self-supervised deep correlation tracking. IEEE Trans Image Process 30:976–985

    Article  ADS  PubMed  Google Scholar 

  29. Yuan D, Chang X, Liu Q, Yang Y, Wang D, Shu M, He Z, Shi G Active learning for deep visual tracking. IEEE Transactions on Neural Networks and Learning Systems (2023)

  30. Kipf T.N, Fetaya E, Wang K.-C, Welling M, Zemel R.S Neural relational inference for interacting systems. In: International Conference on Computational Linguistics(ICML), vol. 80, pp. 2693–2702 (2018)

  31. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  32. Leal-Taixé L, Milan A, Reid I, Roth S Motchallenge 2015: Toward a benchmark for multi-target tracking. arXiv preprint arXiv:1504.01942 (2015)

  33. Milan A, Leal-Taixé L, Reid I.D, Roth S, Schindler K Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)

  34. Dendorfer P, Rezatofighi H, Milan A, Shi J, Cremers D, Reid I, Roth S, Leal-Taixé L Mot20: A benchmark for multi object tracking in crowded scenes (2020)

  35. Luiten J, Osep A, Dendorfer P, Torr P, Geiger A, Leal-Taixé L (2021) Leibe B Hota: A higher order metric for evaluating multi-object tracking. Int J Comput Vision 129(2):548–578

    Article  Google Scholar 

  36. Ge Z, Liu S, Wang F, Li Z, Sun J Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 (2021)

  37. Luo H, Jiang W, Gu Y, Liu F, Liao X, Lai S (2019) Gu J A strong baseline and batch normalization neck for deep person re-identification. IEEE Trans Multimedia 22(10):2597–2609

    Article  Google Scholar 

  38. Ristani E, Solera F, Zou R, Cucchiara R, Tomasi C Performance measures and a data set for multi-target, multi-camera tracking. In: Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II, pp. 17–35 (2016). Springer

  39. Wojke N, Bewley A, Paulus D Simple online and realtime tracking with a deep association metric. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3645–3649 (2017). IEEE

  40. Bergmann P, Meinhardt T, Leal-Taixé L Tracking without bells and whistles. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 941–951 (2019). https://doi.org/10.1109/ICCV.2019.00103

  41. Sun P, Cao J, Jiang Y, Zhang R, Xie E, Yuan Z, Wang C, Luo P Transtrack: Multiple object tracking with transformer. arXiv preprint arXiv:2012.15460 (2020)

  42. Wang S, Sheng H, Zhang Y, Wu Y, Xiong Z A general recurrent tracking framework without real data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13219–13228 (2021)

  43. Du Y, Song Y, Yang B, Zhao Y Strongsort: Make deepsort great again. arXiv preprint arXiv:2202.13514 (2022)

  44. Zhang Y, Sun P, Jiang Y, Yu D, Weng F, Yuan Z, Luo P, Liu W, Wang X Bytetrack: Multi-object tracking by associating every detection box. In: European Conference on Computer Vision, pp. 1–21 (2022). Springer

  45. Yang F, Chang X, Sakti S, Wu Y (2021) Nakamura S Remot: A model-agnostic refinement for multiple object tracking. Image Vis Comput 106:104091

    Article  Google Scholar 

Download references

Funding

Funds was provided by the Key Program of the National Natural Science Foundation of China (Grant No. U23A20346), and by the National Natural Science Foundation of China (Grant No. 62173107).

Author information

Authors and Affiliations

  1. The Key Laboratory of Autonomous Intelligent Unmanned Systems, Harbin Institute of Technology, Harbin, 150001, Heilongjiang, China

    Lv Jianfeng, Liu Yifan & Sun Guanghui

  2. The College of Automation, Chongqing University, Chongqing, 400044, China

    Yu Zhongliang

Authors
  1. Lv Jianfeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Zhongliang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liu Yifan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sun Guanghui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sun Guanghui.

Ethics declarations

Conflict of interest

The authors certify that there is no actual or potential conflict of interest in relation to this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jianfeng, L., Zhongliang, Y., Yifan, L. et al. GTAN: graph-based tracklet association network for multi-object tracking. Neural Comput & Applic 36, 3889–3902 (2024). https://doi.org/10.1007/s00521-023-09287-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09287-1

Keywords

Search

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy