skip to main content
research-article

Diversity-Representativeness Replay and Knowledge Alignment for Lifelong Vehicle Re-identification

Published: 30 December 2024 Publication History

Abstract

Lifelong Vehicle Re-Identification (LVReID) aims to match a target vehicle across multiple cameras, considering non-stationary and continuous data streams, which fits the needs of the practical application better than traditional vehicle re-identification. Nonetheless, this area has received relatively little attention. Recently, methods for Lifelong Person Re-Identification (LPReID) have been emerging, with replay-based methods achieving the best results by storing a small number of instances from previous tasks for retraining, thus effectively reducing catastrophic forgetting. However, these methods cannot be directly applied to LVReID because they fail to simultaneously consider the diversity and representativeness of replayed data, resulting in biases between the subset stored in the memory buffer and the original data. They randomly sample classes, which may not adequately represent the distribution of the original data. Additionally, these methods fail to consider the rich variation in instances of the same vehicle class due to factors such as vehicle orientation and lighting conditions. Therefore, preserving more informative classes and instances for replay helps maintain information from previous tasks and may mitigate the model's forgetting of old knowledge. In view of this, we propose a novel Diversity-Representativeness Dual-Stage Sampling Replay (DDSR) strategy for LVReID that constructs an effective memory buffer through two stages, i.e., Cluster-Centric Class Selection and Diverse Instance Mining. Specifically, we first perform class-level sampling based on density in the clustered class-centered feature space and then further mine the diverse, high-quality instances within the selected classes. In addition, we introduce Maximum Mean Discrepancy loss to align the feature distribution between replay data and the new arrivals and apply L2 regularization in the parameter space to facilitate knowledge transfer, thus enhancing the model's generalization ability to new tasks. Extensive experiments demonstrate effective improvements of our method compared to current state-of-the-art lifelong ReID methods on the VeRi-776, VehicleID, and VERI-Wild datasets.

References

[1]
Xinchen Liu, Wu Liu, Huadong Ma, and Huiyuan Fu. 2016. Large-scale vehicle re-identification in urban surveillance videos. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME ’16). IEEE, 1–6.
[2]
Nan Pu, Wei Chen, Yu Liu, Erwin M. Bakker, and Michael S. Lew. 2021. Lifelong person re-identification via adaptive knowledge accumulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR ’21), 7901–7910.
[3]
Guile Wu and Shaogang Gong. 2021. Generalising without forgetting for lifelong person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI ’21), Vol. 35, 2889–2897.
[4]
Wenhang Ge, Junlong Du, Ancong Wu, Yuqiao Xian, Ke Yan, Feiyue Huang, and Wei-Shi Zheng. 2022. Lifelong person re-identification by pseudo task knowledge preservation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI ’22), Vol. 36, 688–696.
[5]
Chunlin Yu, Ye Shi, Zimo Liu, Shenghua Gao, and Jingya Wang. 2023. Lifelong person re-identification via knowledge refreshing and consolidation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI ’23), Vol. 37, 3295–3303.
[6]
Xinchen Liu, Wu Liu, Tao Mei, and Huadong Ma. 2017. Provid: Progressive and multimodal vehicle reidentification for large-scale urban surveillance. IEEE Transactions on Multimedia 20, 3 (2017), 645–658.
[7]
Yuqi Li, Yanghao Li, Hongfei Yan, and Jiaying Liu. 2017. Deep joint discriminative learning for vehicle re-identification and retrieval. In Proceedings of the IEEE International Conference on Image Processing (ICIP ’17). IEEE, 395–399.
[8]
Yiheng Zhang, Dong Liu, and Zheng-Jun Zha. 2017. Improving triplet-wise training of convolutional neural network for vehicle re-identification. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME ’17). IEEE, 1386–1391.
[9]
Xin Yuan, Xin Xu, Zheng Wang, Kai Zhang, Wei Liu, and Ruimin Hu. 2023. Searching parameterized retrieval & verification loss for re-identification. IEEE Journal of Selected Topics in Signal Processing 17, 3 (2023), 560–574.
[10]
Shuting He, Hao Luo, Weihua Chen, Miao Zhang, Yuqi Zhang, Fan Wang, Hao Li, and Wei Jiang. 2020. Multi-domain learning and identity mining for vehicle re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR ’20) Workshops, 582–583.
[11]
Xinchen Liu, Wu Liu, Tao Mei, and Huadong Ma. 2016. A deep learning-based approach to progressive vehicle re-identification for urban surveillance. In Proceedings of the 14th European Conference on Computer Vision (ECCV ’16). Springer, 869–884.
[12]
Hongye Liu, Yonghong Tian, Yaowei Yang, Lu Pang, and Tiejun Huang. 2016. Deep relative distance learning: Tell the difference between similar vehicles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’16), 2167–2175.
[13]
Yihang Lou, Yan Bai, Jun Liu, Shiqi Wang, and Lingyu Duan. 2019. Veri-wild: A large dataset and a new method for vehicle re-identification in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR ’19), 3235–3243.
[14]
Yue Yao, Liang Zheng, Xiaodong Yang, Milind Naphade, and Tom Gedeon. 2020. Simulating content consistent vehicle datasets with attribute descent. In Proceedings of the 16th European Conference on Computer Vision (ECCV ’20). Springer, 775–791.
[15]
Zhijing Wan, Xin Xu, Zheng Wang, Zhixiang Wang, and Ruimin Hu. 2023. From multi-source virtual to real: Effective virtual data search for vehicle re-identification. IEEE Transactions on Intelligent Transportation Systems 25, 5 (2023), 3433–3444.
[16]
David Abel, Dilip Arumugam, Lucas Lehnert, and Michael Littman. 2018. State abstractions for lifelong reinforcement learning. In Proceedings of the International Conference on Machine Learning (ICML ’18). PMLR, 10–19.
[17]
Bo Xiao, Hak-Keung Lam, Xiaojie Su, Ziwei Wang, Frank P.-W. Lo, Shihong Chen, and Eric Yeatman. 2023. Sampled-data control through model-free reinforcement learning with effective experience replay. Journal of Automation and Intelligence 2, 1 (2023), 20–30.
[18]
Dushyant Rao, Francesco Visin, Andrei Rusu, Razvan Pascanu, Yee Whye Teh, and Raia Hadsell. 2019. Continual unsupervised representation learning. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc., 1–11.
[19]
Kun Wei, Cheng Deng and Xu Yang. 2020. Lifelong zero-shot learning. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI ’20), 551–557.
[20]
Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu. 2018. Memory replay gans: Learning to generate new categories without forgetting. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 31. Curran Associates, Inc., 1–11.
[21]
Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. 2017. Incremental learning of object detectors without catastrophic forgetting. In Proceedings of the IEEE International Conference on Computer Vision (ICCV ’17), 3400–3409.
[22]
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. 2017. ICARL: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’17), 2001–2010.
[23]
Francisco M. Castro, Manuel J. Marín-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. 2018. End-to-end incremental learning. In Proceedings of the 15th European Conference on Computer Vision (ECCV ’18), 241–257.
[24]
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114, 13 (2017), 3521–3526.
[25]
Zhijing Wan, Zhixiang Wang, CheukTing Chung, and Zheng Wang. 2024. A survey of dataset refinement for problems in computer vision datasets. ACM Computing Surveys 56, 7 (2024), 1–34.
[26]
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. 2018. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv:1810.11910. Retrieved from https://doi.org/10.48550/arXiv.1810.11910
[27]
Jeffrey S. Vitter. 1985. Random sampling with a reservoir. ACM Transactions on Mathematical Software 11, 1 (1985), 37–57.
[28]
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K. Dokania, Philip H. S. Torr, and Marc’Aurelio Ranzato. 2019. On tiny episodic memories in continual learning. arXiv:1902.10486. Retrieved from https://doi.org/10.48550/arXiv.1902.10486
[29]
Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. 2022. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning. In Proceedings of the 11th International Conference on Learning Representations, 1–20.
[30]
Zhijing Wan, Zhixiang Wang, Yuran Wang, Zheng Wang, Hongyuan Zhu, and Shin’ichi Satoh. 2024. Contributing dimension structure of deep feature for coreset selection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, 9080–9088.
[31]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11 (1998), 2278–2324.
[32]
Vincenzo Lomonaco and Davide Maltoni. 2017. Core50: A new dataset and benchmark for continuous object recognition. In Proceedings of the Conference on Robot Learning. PMLR, 17–26.
[33]
Alex Krizhevsky and Geoffrey Hinton. 2009. Learning Multiple Layers of Features From Tiny Images. University of Toronto, 1–60. Retrieved from https://api.semanticscholar.org/CorpusID:18268744
[34]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115 (2015), 211–252.
[35]
Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. 2015. Scalable person re-identification: A benchmark. In Proceedings of the IEEE International Conference on Computer Vision (ICCV ’15), 1116–1124.
[36]
Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. 2018. Person transfer gan to bridge domain gap for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’18), 79–88.
[37]
Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. 2014. Deepreid: Deep filter pairing neural network for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’14), 152–159.
[38]
Wei Chen, Yu Liu, Weiping Wang, Tinne Tuytelaars, Erwin M. Bakker, and Michael Lew. 2020. On the exploration of incremental learning for fine-grained image retrieval. arXiv:2010.08020.
[39]
Xin Xu, Wei Liu, Zheng Wang, Ruimin Hu, and Qi Tian. 2022. Towards generalizable person re-identification with a bi-stream generative model. Pattern Recognition 132 (2022), 108954.
[40]
Zhicheng Sun and Yadong Mu. 2022. Patch-based knowledge distillation for lifelong person re-identification. In Proceedings of the 30th ACM International Conference on Multimedia (MM ’22), 696–707.
[41]
David Arthur and Sergei Vassilvitskii. 2007. K-means++: The advantages of careful seeding. In Proceedings of the 18th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), Vol. 7, 1027–1035.
[42]
James Seale Smith, Junjiao Tian, Shaunak Halbe, Yen-Chang Hsu, and Zsolt Kira. 2023. A closer look at rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR ’23), 2409–2419.
[43]
Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 12 (2017), 2935–2947.
[44]
He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’16), Vol. 34, 770–778.
[45]
Frederick Tung and Greg Mori. 2019. Similarity-preserving knowledge distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV ’19), 1365–1374.
[46]
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. 2020. Maintaining discrimination and fairness in class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR ’20), 13208–13217.

Index Terms

  1. Diversity-Representativeness Replay and Knowledge Alignment for Lifelong Vehicle Re-identification

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 21, Issue 2
    February 2025
    651 pages
    EISSN:1551-6865
    DOI:10.1145/3703007
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 December 2024
    Online AM: 05 November 2024
    Accepted: 13 October 2024
    Revised: 09 September 2024
    Received: 13 May 2024
    Published in TOMM Volume 21, Issue 2

    Check for updates

    Author Tags

    1. Lifelong learning
    2. Vehicle re-identification
    3. Replay
    4. Catastrophic forgetting

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 148
      Total Downloads
    • Downloads (Last 12 months)148
    • Downloads (Last 6 weeks)27
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media

    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy