Abstract
With the continuous increase of data, the data center that plays the role of backup is facing the problem of energy hunger. In practice, to reduce the bandwidth, the local data is synchronized to the data center based on incremental synchronization. In this process, the data center will generate a huge CPU load. To solve this pressure of the data center, first, we analyze the process of the Rsync algorithm, the most commonly used in incremental synchronization, and CDC algorithms, another way of chunking algorithm. Then we propose a data structure called Shadow Data, which greatly reduces the CPU load of the data center by sacrificing part of the hard disk space in the local node.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Elahi, B., Malik, A.W., Rahman, A.U., Khan, M.A.: Toward scalable cloud data center simulation using high-level architecture. Softw. Pract. Exp. 50(6), 827-843 (2020)
Tang, X., Wang, F., Tong, L.I., Zhang, P.: Research and implementation of real-time exchange system in data center. Comput. Sci. 70, 104–125 (2017)
Nizam, K.K., Sanja, S., Tapio, N., Nurminen, J.K., Sebastian, V.A., Olli-Pekka, L.: Analyzing the power consumption behavior of a large scale data center. Comput. Sci. Res. Dev. 34, 61–70 (2018)
Zhi, C., Huang, G.: Saving energy in data center networks with traffic-aware virtual machine placement. Inf. Technol. J. 12(19), 5064–5069 (2013)
Tridgell, A.: Effcient algorithms for sorting and synchronization. https://www.samba.org//tridge/phd_thesis.pdf. Accessed February 1999
Chao, Y., Ye, T., Di, M., Shen, S., Wei, M.: A server friendly file synchronization mechanism for cloud storage. In: IEEE International Conference on Green Computing & Communications, IEEE & Internet of Things(2013)
Won, Y., Lim, K., Min, J.: Much: multithreaded content-based file chunking. IEEE Trans. Comput. 64(5), 1375–1388 (2015)
Ma, J., Bi, C., Bai, Y., Zhang, L.: UCDC: unlimited content-defined chunking, a file-differing method apply to file-synchronization among multiple hosts. In: 2016 12th International Conference on Semantics, Knowledge and Grids (SKG), pp. 76–82 (August 2016)
Bjørner, N., Blass, A., Gurevich, Y.: Content-dependent chunking for differential compression, the local maximum approach. J. Comput. Syst. Sci. 76(3–4), 154–203 (2010)
Zhang, Y., Feng, D., Jiang, H., Xia, W., Fu, M., Huang, F., Zhou, Y.: A fast asymmetric extremum content defined chunking algorithm for data deduplication in backup storage systems. IEEE Trans. Comput. 66(2), 199–211 (2017)
Widodo, R.N.S., Lim, H., Atiquzzaman, M.: A new content-defined chunking algorithm for data deduplication in cloud storage. Futur. Gener. Comput. Syst. 71, 145–156 (2017)
Zhang, C., et al.: MII: a novel content defined chunking algorithm for finding incremental data in data synchronization. IEEE Access 7, 86932–86945 (2019)
Zhang, C., Qi, D., Li, W., Guo, J.: Function of content defined chunking algorithms in incremental synchronization. IEEE Access 8, 5316–5330 (2020)
Matsumoto, M., Nishimura, T.: Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8(1), 3–30 (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 IFIP International Federation for Information Processing
About this paper
Cite this paper
Zhang, C., Qi, D., Huang, W. (2021). Shadow Data: A Method to Optimize Incremental Synchronization in Data Center. In: He, X., Shao, E., Tan, G. (eds) Network and Parallel Computing. NPC 2020. Lecture Notes in Computer Science(), vol 12639. Springer, Cham. https://doi.org/10.1007/978-3-030-79478-1_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-79478-1_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79477-4
Online ISBN: 978-3-030-79478-1
eBook Packages: Computer ScienceComputer Science (R0)
