Abstract
High Efficiency Video Coding is one of the most widely used Video Coding standards. It could encode videos to bitstream with a high compression rate for transporting, and videos would be reconstructed by decoding bitstream. However, decoded videos with compression artifacts would degrade the viewing experience of users. At present, many methods based on deep learning have been proposed to reduce compression artifacts for quality enhancement. Most methods train their models assuming that quantization parameters (QP) are known. But the QPs of videos are obtained from bitstream, which means QPs are probably unknown if there are only decoded videos. Without QP, training a network for compression artifacts reduction would become difficult. To this end, we propose a video compression artifacts reduction approach combined with quantization parameters estimation (VRQPE). In light of the fact that decoded videos with different QPs have different artifacts, we extract representative sample blocks in video frames to estimate the QPs of these videos. Then, we propose a majority voting to achieve better estimation results. Afterward, we proposed a quality enhancement network to reduce compressed artifacts. Extensive experiments demonstrate that our proposed VRQPE achieves superior performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhang X, Xiong R, Lin W, Zhang J, Wang S, Ma S, Gao W (2016) Low-rank-based nonlocal adaptive loop filter for high-efficiency video compression. IEEE Trans Circuits Systems Video Technol 27(10):2177–2188
Li F, Tan W, Yan B (2018) Deep residual network for enhancing quality of the decoded intra frames of hevc. In: 2018 25th IEEE International Conferenceon Image Processing (ICIP) IEEE 3918–3922 39183922
Ma L, Tian Y, Huang T (2018) Residual-based video restoration for hevc intracoding. In: 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), IEEE, pp 1–7
Tong J, Wu X, Ding D, Zhu Z, Liu Z (2019) Learning-based multi-frame video quality enhancement. In: 2019 IEEE International Conference on Image Processing (ICIP), IEEE, pp 929–933
Dong C, Deng Y, Loy CC, Tang X (2015) Compression artifacts reduction by a deep convolutional network. In:Proceedings of the IEEE International Conference on Computer Vision, pp 576–584
Dai Y, Liu D, Wu F (2017) A convolutional neural network approach for post-processing in hevc intra coding, in: International Conference on Multimedia Modeling, Springer, pp 28–39
Yang R, Xu M, Wang Z (2017) Decoder-side hevc quality enhancement with scalable convolutional neural network, in: 2017 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp 817–822
Yang R, Xu M, Wang Z, Li T (2018) Multi-frame quality enhancement for compressed video. In: Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition, pp 6664–6673
Deng J, Wang L, Pu S, Zhuo C (2020) Spatio-temporal deformable convolutionfor compressed video quality enhancement, In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 10696–10703
Claus M, van Gemert J (2019) Videnn: Deep blind video denoising, In: Pro-ceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition (CVPR) Workshops,
Ehret T, Davy A, Morel J-M, Facciolo G, Arias P (2019) Model-blind video denoising via frame-to-frame training, in: Proceedings of the IEEE/CVFConference on Computer Vision and Pattern Recognition, pp 11369–11378
Amaranageswarao G, Deivalakshmi S, Ko SB (2020) Blind compression artifact reduction using dense parallel convolutional neural network. Signal Process Image Commun 89:116009
Caballero J, Ledig C, Aitken A, Acosta A, Totz J, Wang Z, Shi W (2017) Real-time video super-resolution with spatio-temporal networks and motioncompensation. In: Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition, pp 4778–4787
Guo D, Xia Y, Xu L, Li W, Luo X (2021) Remote sensing image super-resolution using cascade generative adversarial nets. Neurocomputing 443:117–130.19
Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring, in: Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (CVPR)
Zhang H, Wu Y, Zhang L, Zhang Z, Li Y (2020) Image deblurring using tri-segment intensity prior. Neurocomputing 398:265–279
Xu M, Li T, Wang Z, Deng X, Yang R, Guan Z (2018) Reducing complexity of hevc: A deep learning approach. IEEE Trans Image Process 27(10):5044–5059. https://doi.org/10.1109/TIP.2018.2847035.
Sengar SS, Mukhopadhyay S (2020) Motion segmentation-based surveillance video compression using adaptive particle swarm optimization. Neural Comput Appl 32:11443–11457
Huang YL, Chang RF (2000) Error concealment using adaptive multilayer perceptrons (MLPs) for block-based image coding. Neural Comput Appl 9(2):83–92
Shi Z , Feng Y , Zhao M et al (2020)A joint deep neural networks-based method for single nighttime rainy image enhancement. Neural Comput Appl, 32(9)
Paraschiv EG, Ruiz-Coll D, Pantoja M et al (2019) Parallelization and improvement of the MDV-SW algorithm for HEVC intra-prediction coding. J Supercomput 75:1150–1162. https://doi.org/10.1007/s11227-018-2329-2
Piñol P, Migallón H, López-Granado O et al (2015) Slice-based parallel approach for HEVC encoder. J Supercomput 71:1882–1892. https://doi.org/10.1007/s11227-014-1371-y
Lu J, Liong VE, Zhou J (2017) Deep hashing for scalable image search. IEEE Trans Image Process 26(5):2352–2367. https://doi.org/10.1109/TIP.2017.2678163
Duan Y, Lu J, Wang Z, Feng J, Zhou J (2019) Learning deep binary descriptor with multi quantization. IEEE Trans Pattern Anal Mach Intell 41(8):1924–1938. https://doi.org/10.1109/TPAMI.2018.2858760
Sun W, He X, Chen H, Sheriff RE, Xiong S (2020) A quality enhancement framework with noise distribution characteristics for high efficiency videocoding. Neurocomputing 41:428–441.20
Lu CT, Wang LL, Shen JH et al (2021) Image enhancement using deep-learning fully connected neural network mean filter. J Supercomput 77:3144–3164. https://doi.org/10.1007/s11227-020-03389-6
Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network forimage restoration. In: Proceedings of the IEEE International Conferenceon Computer Vision (ICCV)
Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155. https://doi.org/10.1109/TIP.2017.2662206
Wang T, Chen M, Chao H (2017) A novel deep learning-based method of improving coding efficiency from the decoder-end for hevc. Data Compression Conference (DCC) 2017:410–419. https://doi.org/10.1109/DCC.2017.42
Liu M, Tang L, Zhong S, Luo H, Peng J (2021) Learning noise-decoupled affine models for extreme low-light image enhancement. Neurocomputing 448:21–29
Yang R, Xu M, Liu T, Wang Z, Guan Z (2018) Enhancing quality for hevc compressed videos. IEEE Trans Circuits Syst Video Technol 29(7):2039–2054
Guan Z, Xing Q, Xu M, Yang R, Liu T, Wang Z Mfqe 2.0: A newapproach for multi-frame quality enhancement on compressed video. IEEE Trans Patt Anal Mach Intell
Chen CC, Zhang N, Ye YY et al (2021) A new framework based on spatio-temporal information for enhancing compressed video. In: 2021 4th International Conference on Information Communication and Signal Processing (ICICSP). IEEE, pp 571–576
Li T, Xu M, Zhu C, Yang R, Wang Z, Guan Z (2019) A deep learning approach for multi-frame in-loop filter of hevc. IEEE Trans Image Process 28(11):5663–5678
Yang R, Sun X, Xu M, Zeng W (2019) Quality-gated convolutional lstm for enhancing compressed video, in: 2019 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp 532–537.21
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conf Comput Vis Pattern Recog (CVPR) 2016:770–778. https://doi.org/10.1109/CVPR.2016.90
Xue T, Chen B, Wu J, Wei D, Freeman WT (2019) Video enhancement with task-oriented flow. Int J Comput Vis 127(8):1106–1125
Kim Y, Soh JW, Park J, Ahn B, Lee HS, Moon YS, Cho NI (2019) A pseudo-blind convolutional neural network for the reduction of compression artifacts. IEEE Trans Circuits Syst Video Technol 30(4):1121–1135
Acknowledgements
This work is supported by the National Natural Science Foundation of China (Grant No. 61871279 and No. 62081330105).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shuai, X., Qing, L., Zhang, M. et al. A video compression artifact reduction approach combined with quantization parameters estimation. J Supercomput 78, 13564–13582 (2022). https://doi.org/10.1007/s11227-022-04412-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-022-04412-8