Skip to main content

Advertisement

Springer Nature Link
Log in

VNE solution for network differentiated QoS and security requirements: from the perspective of deep reinforcement learning

  • Special Issue Article
  • Published:
Computing Aims and scope Submit manuscript

Abstract

The rapid development and deployment of network services has brought a series of challenges to researchers. On the one hand, the needs of Internet end users/applications reflect the characteristics of travel alienation, and they pursue different perspectives of service quality. On the other hand, with the explosive growth of information in the era of big data, a lot of private information is stored in the network. End users/applications naturally start to pay attention to network security. In order to solve the requirements of differentiated quality of service (QoS) and security, this paper proposes a virtual network embedding (VNE) algorithm based on deep reinforcement learning (DRL), aiming at the CPU, bandwidth, delay and security attributes of substrate network. DRL agent is trained in the network environment constructed by the above attributes. The purpose is to deduce the mapping probability of each substrate node and map the virtual node according to this probability. Finally, the breadth first strategy (BFS) is used to map the virtual links. In the experimental stage, the algorithm based on DRL is compared with other representative algorithms in three aspects: long term average revenue, long term revenue consumption ratio and acceptance rate. The results show that the algorithm proposed in this paper has achieved good experimental results, which proves that the algorithm can be effectively applied to solve the end user/application differentiated QoS and security requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Zhang P, Yao H, Qiu C, Liu Y (2018) Virtual network embedding using node multiple metrics based on simplified ELECTRE method. IEEE Access 6:37314–37327

    Article  Google Scholar 

  2. Kumar N, Aujla GS, Garg S, Kaur K, Ranjan R, Garg SK (2019) Renewable energy-based multi-indexed job classification and container management scheme for sustainability of cloud data centers. IEEE Trans Ind Inform 15(5):2947–2957

    Article  Google Scholar 

  3. Ning Z, Dong P, Wang X et al (2020) When deep reinforcement learning meets 5g-enabled vehicular networks: a distributed offloading framework for traffic big data. IEEE Trans Ind Inform 16(2):1352–1361

    Article  Google Scholar 

  4. Du J, Jiang C, Zhang H, Ren Y, Guizani M (2018) Auction design and analysis for SDN-based traffic offloading in hybrid satellite-terrestrial networks. IEEE J Sel Areas Commun 36(10):2202–2217

    Article  Google Scholar 

  5. Munoz R et al (2015) Integrated SDN/NFV management and orchestration architecture for dynamic deployment of virtual SDN control instances for virtual tenant networks [invited]. IEEE/OSA J Opt Commun Netw 7(11):B62–B70

    Article  Google Scholar 

  6. Aujla GS, Chaudhary R, Kumar N, Rodrigues JJPC, Vinel A (2017) Data offloading in 5g-enabled software-defined vehicular networks: a Stackelberg-game-based approach. IEEE Commun Mag 55(8):100–108

    Article  Google Scholar 

  7. Batth RS, Nayyar A, Nagpal A (2018) Internet of robotic things: driving intelligent robotics of future—concept, architecture, applications and technologies. In: 2018 4th International Conference on Computing Sciences (ICCS), Jalandhar, pp 151–160

  8. Martins G, Kopp LF, Genta J et al (2019) A Prediction-based multisensor heuristic for the internet of things. In: The 15th ACM International Symposium on QoS and Security for Wireless and Mobile Networks, pp 71–78

  9. Du J, Gelenbe E, Jiang C, Zhang H, Ren Y (2017) Contract design for traffic offloading and resource allocation in software defined ultra-dense networks. IEEE J Sel Areas Commun 35(11):2457–2467

    Article  Google Scholar 

  10. Zhang P, Yao H, Liu Y (2016) Virtual network embedding based on the degree and clustering coefficient information. IEEE Access 4:8572–8580

    Article  Google Scholar 

  11. Sandhu AK, Singh Batth R, Nagpal A (2019) Improved QoS Using Novel Fault Tolerant Shortest Path Algorithm in Virtual Software Defined Network (VSDN). In: 2019 International Conference on Automation, Computational and Technology Management (ICACTM), London, United Kingdom, pp 383–388

  12. Ning Z, Zhang K, Wang X, Obaidat MS et al (2020) Joint computing and caching in 5g-envisioned internet of vehicles: a deep reinforcement learning-based traffic control system. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2020.2970276

    Article  Google Scholar 

  13. Jindal A, Aujla GS, Kumar N, Chaudhary R, Obaidat MS, You I (2018) SeDaTiVe: SDN-enabled deep learning architecture for network traffic control in vehicular cyber-physical systems. IEEE Netw 32(6):66–73

    Article  Google Scholar 

  14. Aljeri N, Boukerche A (2019) An efficient handover trigger scheme for vehicular networks using recurrent neural networks. In: The 15th ACM international symposium on qos and security for wireless and mobile networks, pp 85–91

  15. Zhang Y, Ren SQ, Chen SB, Tan B, Lim ES, Yong KL (2013) DifferCloudStor: differentiated quality of service for cloud storage. IEEE Trans Magn 49(6):2451–2458

    Article  Google Scholar 

  16. Xiong B, Yang K, Zhao J, Li W, Li K (2016) Performance evaluation of OpenFlow-based software-defined networks based on queueing model. Comput Netw 102:172–185

    Article  Google Scholar 

  17. Zhang J, Wei W, Chaoquan L, Jin W, Arun KS (2020) Lightweight deep network for traffic sign classification. Ann Telecommun 75(7):369–379

    Article  Google Scholar 

  18. Zhao J, Zhigang H, Xiong B, Li K (2018) Accelerating packet classification with counting bloom filters for virtual openflow switching. China Commun 15(10):117–128

    Article  Google Scholar 

  19. Aujla GS, Jindal A, Kumar N (2018) EVaaS: electric vehicle-as-a-service for energy trading in SDN-enabled smart transportation system. Comput Netw 143:247–262

    Article  Google Scholar 

  20. Vishnu NS, Singh Batth R, Singh G (2019) Denial of service: types, techniques, defence mechanisms and safe guards. In: 2019 international conference on computational intelligence and knowledge economy (ICCIKE), Dubai, United Arab Emirates, pp 695–700

  21. Wang J, Gao Y, Liu W, Wenbing W, Lim S-J (2019) An asynchronous clustering and mobile data gathering schema based on timer mechanism in wireless sensor networks. Comput Mater Contin 58(3):711–725

    Article  Google Scholar 

  22. Wang J, Gao Y, Yin X, Li F, Kim H-J (2018) An enhanced PEGASIS algorithm with mobile sink support for wireless sensor networks. Wireless Communications and Mobile Computing 2018

  23. Ju C, Yu G, Arun KS, Gwang-jun K (2018) A PSO based energy efficient coverage control algorithm for wireless sensor networks. Comput Mater Contin 56(3):433–446

    Google Scholar 

  24. Abhishek NV, Lim TJ, Tandon A, Sikdar B (2018) Detecting forwarding misbehavior in clustered IoT networks. In: The 14th ACM international symposium on QoS and security for wireless and mobile networks, pp 1–6

  25. Ouferhat N, Mellouk A (2006) QoS dynamic routing for wireless sensor networks. In: The 2nd ACM international workshop on Quality of service & security for wireless and mobile networks, pp 45–50

  26. Ahmad I, Namal S, Ylianttila M, Gurtov A (2015) Security in software defined networks: a survey. IEEE Commun Surv Tutor 17(4):2317–2346

    Article  Google Scholar 

  27. Varadharajan V, Karmakar K, Tupakula U, Hitchens M (2019) A policy-based security architecture for software-defined networks. IEEE Trans Inf Forens Secur 14(4):897–912

    Article  Google Scholar 

  28. Aujla GS, Chaudhary R, Kaur K, Garg S, Kumar N, Ranjan R (2019) SAFE: SDN-assisted framework for edge-cloud interplay in secure healthcare ecosystem. IEEE Trans Ind Inform 15(1):469–480

    Article  Google Scholar 

  29. Ziane S, Mellouk A (2005) A swarm intelligent multi-path routing for multimedia traffic over mobile ad hoc networks. In: First ACM international workshop on quality of service and security in wireless and mobile networks, pp 55–62

  30. Zhang P, Yao H, Liu Y (2018) Virtual network embedding based on computing, network, and storage resource constraints. IEEE Internet Things J 5(5):3298–3304

    Article  Google Scholar 

  31. Aujla GS, Chaudhary R, Kumar N, Kumar R, Rodrigues JJ (2018) An ensembled scheme for QoS-aware traffic flow management in software defined networks. In: 2018 IEEE international conference on communications (ICC), pp 1–7. IEEE

  32. Parra OS, Garica G, Reyes B (2014) Traffic forecasting using a multi layer perceptron model. In: 10th ACM Symposium on QoS and security for wireless and mobile networks, pp 133–136

  33. Kaur K, Garg S, Aujla GS, Kumar N, Rodrigues JJPC, Guizani M (2018) Edge computing in the industrial internet of things environment: software-defined-networks-based edge-cloud interplay. IEEE Commun Mag 56(2):44–51

    Article  Google Scholar 

  34. Ning Z, Dong P, Wang X, Rodrigues J, Xia F (2019) Deep reinforcement learning for vehicular edge computing: an intelligent offloading system. ACM Trans Intell Syst Technol 10(6):60

    Article  Google Scholar 

  35. Jiang C, Zhang H, Ren Y, Han Z, Chen K, Hanzo L (2017) Machine learning paradigms for next-generation wireless networks. IEEE Wirel Commun 24(2):98–105

    Article  Google Scholar 

  36. Wang J, Jiang C, Zhang H, Ren Y, Chen K-C, Hanzo L (2020) Thirty years of machine learning: the road to pareto-optimal wireless networks. In: IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/COMST.2020.2965856, Early Access, pp 1–1

  37. Aujla GS, Chaudhary R, Kumar N, Das AK, Rodrigues JJPC (2018) SecSVA: secure storage, verification, and auditing of big data in the cloud environment. IEEE Commun Mag 56(1):78–85

    Article  Google Scholar 

  38. Zhang P, Yao H, Li M, Liu Y (2017) Virtual network embedding based on modified genetic algorithm. Peer-to-Peer Netw Appl 2:1–12

    Google Scholar 

  39. Cao H, Wu S, Aujla GS, Wang Q, Yang L, Zhu H (2020) Dynamic embedding and quality of service-driven adjustment for cloud networks. IEEE Trans Ind Inform 16(2):1406–1416

    Article  Google Scholar 

  40. Pham M, Hoang DB, Chaczko Z (2020) Congestion-aware and energy-aware virtual network embedding. IEEE/ACM Trans Netw 28(1):210–223

    Article  Google Scholar 

  41. Xu S et al (2019) Load-balancing and QoS based dynamic resource allocation method for smart gird fiber-wireless networks. Chin J Electron 28(6):1234–1243

    Article  Google Scholar 

  42. Li M, Chen C, Hua C, Guan X (2019) Intelligent latency-aware virtual network embedding for industrial wireless networks. IEEE Internet Things J 6(5):7484–7496

    Article  Google Scholar 

  43. Ning Z, Zhang K, Wang X, Guo L et al (2020) Intelligent edge computing in internet of vehicles: a joint computation offloading and caching solution. In: IEEE transactions on intelligent transportation systems. https://doi.org/10.1109/TITS.2020.2997832, pp 1–14

  44. Doriguzzi-Corin R, Scott-Hayward S, Siracusa D, Savi M, Salvadori E (2020) Dynamic and application-aware provisioning of chained virtual security network functions. IEEE Trans Netw Serv Manag 17(1):294–307

    Article  Google Scholar 

  45. Zhang P, Li H, Ni Y, Gong F, Li M, Wang F (2019) Security aware virtual network embedding algorithm using information entropy TOPSIS. J Netw Syst Manag 5:1–23

    Google Scholar 

  46. Gong S, Chen J, Huang C, Zhu Q (2015) Trust-aware secure virtual network embedding algorithm. J Commun 36(11):1–10

    Google Scholar 

  47. Liu X, Wang B, Liu S, Yang Z, Zhao Z (2018) Heuristic algorithm for secure virtual network embedding. Syst Eng Electron 40(3):1–6

    Google Scholar 

  48. Du J, Jiang C, Han Z, Zhang H, Mumtaz S, Ren Y (2019) Contract mechanism and performance analysis for data transaction in mobile social networks. IEEE Trans Netw Sci Eng 6(2):103–115

    Article  Google Scholar 

  49. Zhang P (2018) Incorporating energy and load balance into virtual network embedding process. Comput Commun 129:80–88

    Article  Google Scholar 

  50. Aujla GS, Singh A, Kumar N (2020) AdaptFlow: adaptive flow forwarding scheme for software-defined industrial networks. IEEE Internet Things J 7(7):5843–5851

    Article  Google Scholar 

  51. Aujla GS, Singh A, Singh M, Sharma S, Kumar N, Choo KR (2020) BloCkEd: blockchain-based secure data processing framework in edge envisioned V2X environment. IEEE Trans Veh Technol 69(6):5850–5863

    Article  Google Scholar 

  52. Reverdy P, Leonard NE (2016) Parameter estimation in softmax decision-making models with linear objective functions. IEEE Trans Autom Sci Eng 13(1):54–67

    Article  Google Scholar 

  53. Yu M, Yi Y, Rexford J, Chiang M (2008) Rethinking virtual network embedding: substrate support for path splitting and migration. ACM SIGCOMM Comput Commun Rev 38:17–29

    Article  Google Scholar 

  54. Chowdhury NMMK, Rahman MR, Boutaba R (2009) Virtual network embedding with coordinated node and link mapping. In: Proceedings of the IEEE INFOCOM[C]. Rio de Janeiro, pp 783–791

Download references

Acknowledgements

This work is partially supported by the Major Scientific and Technological Projects of CNPC under Grant ZD2019-183-006, partially supported by Shandong Provincial Natural Science Foundation under Grant ZR2020MF006, and partially supported by “the Fundamental Research Funds for the Central Universities” of China University of Petroleum (East China) under Grant 20CX05017A.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, China University of Petroleum (East China), Qingdao, China

    Chao Wang, Peiying Zhang & Youxiang Duan

  2. School of Computer Science and Engineering, Lovely Profession University, Phagwara, Punjab, India

    Ranbir Singh Batth

  3. Department of Computer Science, Durham University, Durham, UK

    Gagangeet Singh Aujla

  4. School of Geosciences, China University of Petroleum (East China), Qingdao, China

    Lihua Ren

Authors
  1. Chao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ranbir Singh Batth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peiying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gagangeet Singh Aujla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Youxiang Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lihua Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ranbir Singh Batth or Peiying Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Batth, R.S., Zhang, P. et al. VNE solution for network differentiated QoS and security requirements: from the perspective of deep reinforcement learning. Computing 103, 1061–1083 (2021). https://doi.org/10.1007/s00607-020-00883-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-020-00883-w

Keywords

Mathematics Subject Classification

Search

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy