Abstract
Video anomaly detection is always formulated as frame prediction task which only learned on normal data and detects deviations as anomalies. However, previous methods lack sufficient spatiotemporal constraints on moving objects, making it difficult to learn compact normal distributions and anomalies near the boundary will be misclassified as normal. Besides, the inadequate exploration of diverse normal patterns results in mode missing and unlearned normal patterns will be misclassified as anomalies. To address these problems, we propose an object-level Diverse Motion-conditioned Adversarial Predictive Network for video anomaly detection which combines conditional variational generation with adversarial learning to mitigate false detection. We design a motion-guided generator that controls the generation process conditioned on optical flows to accurately memorize spatiotemporal correlations of normal data. We employ the diversity regularization strategy which explicitly preserves the recurrent structure of normal data in continuous latent space to ensure full utilization of diverse patterns. Additionally, we combine an input clip with the object it generates to synthesize an anomaly near the boundary, then employ a video discriminator to perceive subtle differences between normal and abnormal data, making them more distinguishable. Extensive experiments conducted on public datasets illustrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The datasets used in this paper are all publicly available.
References
Astrid M, Zaheer MZ, Lee S-I (2023) Pseudobound: limiting the anomaly reconstruction capability of one-class classifiers using pseudo anomalies. Neurocomputing 534:147–160
Yang M, Tian S, Rao AS, Rajasegarar S, Palaniswami M, Zhou Z (2023) An efficient deep neural model for detecting crowd anomalies in videos. Appl Intell 53(12):15695–15710
Lee S, Kim HG, Ro YM (2018) Stan: spatio-temporal adversarial networks for abnormal event detection. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1323–1327. IEEE
Nguyen T-N, Meunier J (2019) Anomaly detection in video sequence with appearance-motion correspondence. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1273–1283
Liu W, Luo W, Lian D, Gao S (2018) Future frame prediction for anomaly detection–a new baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6536–6545
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Advances in neural information processing systems 27
Li S, Cheng Y, Tian Y, Liu Y (2022) Anomaly detection based on superpixels in videos. Neural Comput Appl 34(15):12617–12631
Larsen ABL, Sønderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. In: International Conference on Machine Learning, pp. 1558–1566. PMLR
Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114
Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–742
Hyun W, Nam W-J, Lee S-W (2023) Dissimilate-and-assimilate strategy for video anomaly detection and localization. Neurocomputing 522:203–213
Park H, Noh J, Ham B (2020) Learning memory-guided normality for anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14372–14381
Cai R, Zhang H, Liu W, Gao S, Hao Z (2021) Appearance-motion memory consistency network for video anomaly detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 938–946
Xu D, Yan Y, Ricci E, Sebe N (2017) Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput Vis Image Underst 156:117–127
Zhao Y, Deng B, Shen C, Liu Y, Lu H, Hua X-S (2017) Spatio-temporal autoencoder for video anomaly detection. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1933–1941
Luo W, Liu W, Gao S (2017) Remembering history with convolutional lstm for anomaly detection. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 439–444. IEEE
Chong YS, Tay YH (2017) Abnormal event detection in videos using spatiotemporal autoencoder. In: Advances in Neural Networks-ISNN 2017: 14th International Symposium, ISNN 2017, Sapporo, Hakodate, and Muroran, Hokkaido, Japan, June 21–26, 2017, Proceedings, Part II 14, pp. 189–196. Springer
Wang Y, Long M, Wang J, Gao Z, Yu PS (2017) Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. Advances in neural information processing systems 30
Doshi K, Yilmaz Y (2020) Continual learning for anomaly detection in surveillance videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 254–255
Morais R, Le V, Tran T, Saha B, Mansour M, Venkatesh S (2019) Learning regularity in skeleton trajectories for anomaly detection in videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11996–12004
Ouyang Y, Sanchez V (2021) Video anomaly detection by estimating likelihood of representations. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 8984–8991. IEEE
Ionescu RT, Khan FS, Georgescu M-I, Shao L (2019) Object-centric auto-encoders and dummy anomalies for abnormal event detection in video. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7842–7851
Georgescu M-I, Barbalau A, Ionescu RT, Khan FS, Popescu M, Shah M (2021) Anomaly detection in video via self-supervised and multi-task learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12742–12752
Flaborea A, Collorone L, Melendugno GMD, D’Arrigo S, Prenkaj B, Galasso F (2023) Multimodal motion conditioned diffusion model for skeleton-based video anomaly detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10318–10329
Li N, Chang F, Liu C (2022) Human-related anomalous event detection via spatial-temporal graph convolutional autoencoder with embedded long short-term memory network. Neurocomputing 490:482–494
Liu Z, Nie Y, Long C, Zhang Q, Li G (2021) A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13588–13597
Fan Y, Wen G, Li D, Qiu S, Levine MD, Xiao F (2020) Video anomaly detection and localization via gaussian mixture fully convolutional variational autoencoder. Comput Vis Image Underst 195:102920
Lu Y, Kumar KM, Nabavi S, Wang Y (2019) Future frame prediction using convolutional vrnn for anomaly detection. In: 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–8. IEEE
Ravanbakhsh M, Nabi M, Sangineto E, Marcenaro L, Regazzoni C, Sebe N (2017) Abnormal event detection in videos using generative adversarial nets. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1577–1581. IEEE
Yu J, Kim J-G, Gwak J, Lee B-G, Jeon M (2022) Abnormal event detection using adversarial predictive coding for motion and appearance. Inf Sci 586:59–73
Ganokratanaa T, Aramvith S, Sebe N (2022) Video anomaly detection using deep residual-spatiotemporal translation network. Pattern Recogn Lett 155:143–150
Hao Y, Li J, Wang N, Wang X, Gao X (2022) Spatiotemporal consistency-enhanced network for video anomaly detection. Pattern Recogn 121:108232
Yu J, Lee Y, Yow KC, Jeon M, Pedrycz W (2021) Abnormal event detection and localization via adversarial event prediction. IEEE Trans Neural Netw Learn Syst 33(8):3572–3586
Singh R, Sethi A, Saini K, Saurav S, Tiwari A, Singh S (2024) Vald-gan: video anomaly detection using latent discriminator augmented gan. SIViP 18(1):821–831
Bao J, Chen D, Wen F, Li H, Hua G (2017) Cvae-gan: fine-grained image generation through asymmetric training. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2745–2754
Kanu-Asiegbu AM, Vasudevan R, Du X (2022) Bipoco: Bi-directional trajectory prediction with pose constraints for pedestrian anomaly detection. arXiv preprint arXiv:2207.02281
Cai Z, Vasconcelos N (2018) Cascade r-cnn: Delving into high quality object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6154–6162
Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2017) Flownet 2.0: Evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462–2470
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241. Springer
Mao X, Li Q, Xie H, Lau RY, Wang Z, Paul Smolley S (2017) Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2794–2802
Li W, Mahadevan V, Vasconcelos N (2013) Anomaly detection and localization in crowded scenes. IEEE Trans Pattern Anal Mach Intell 36(1):18–32
Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 fps in matlab. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2720–2727
Flaborea A, D’Amely G, D’Arrigo S, Sterpa MA, Sampieri A, Galasso F (2023) Contracting skeletal kinematics for human-related video anomaly detection. arXiv preprint arXiv:2301.09489
Barbalau A, Ionescu RT, Georgescu M-I, Dueholm J, Ramachandra B, Nasrollahi K, Khan FS, Moeslund TB, Shah M (2023) Ssmtl++: revisiting self-supervised multi-task learning for video anomaly detection. Comput Vis Image Underst 229:103656
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No.41971343).
Author information
Authors and Affiliations
Contributions
Jiaqi Wang contributed to conceptualization, methodology, software, validation, formal analysis, writing—original draft. Genlin Ji helped in methodology, validation, investigation, writing—review & editing, supervision, project administration, funding acquisition. Bin Zhao helped in methodology, validation, writing—review & editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, J., Ji, G. & Zhao, B. Video anomaly detection using diverse motion-conditioned adversarial predictive network. Neural Comput & Applic 36, 18645–18659 (2024). https://doi.org/10.1007/s00521-024-10173-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-024-10173-7