Abstract
Today, applications that require large memory footprint prevail in cloud computing fields from both industry and academia. They impose great stress on the memory management of operating system, spend quite a substantial proportion of time dealing with TLB misses and excessively reduce consolidation ratio in term of server consolidation and virtualization. There are two methods to address these problems: main memory compression and large page support. However, to the best of our knowledge, there is no existing practical research on combination of these two methods since the combination in commodity operating system requires lots of low-level design modifications.
We propose a new memory management framework that is decoupled and flexible for easy developments and is able to run simultaneously with the original memory management. On top of the new framework, we implement Hzmem, a completely new large page memory management redesign with features of main memory compression to address the aforementioned problems once for all but requires only minor modifications to the other subsystems of the underlying operating system. Our method achieves competitive performance compared with native large page support, increases effective memory size and impacts little on other subsystems of operating system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yelp Dataset Challenge. https://www.yelp.com/dataset_challenge/
Basu, A., Gandhi, J., Chang, J., Hill, M.D., Swift, M.M.: Efficient virtual memory for big memory servers. In: Proceedings of the International Symposium on Computer Architecture, pp. 237–248 (2013)
Clements, A.T., Kaashoek, M.F., Zeldovich, N.: RadixVM: scalable address spaces for multithreaded applications. In: EuroSys, pp. 211–224 (2013)
Corbet, J.: Huge pages part 1 (Introduction) (2010). https://lwn.net/Articles/374424/
Ekman, M., Stenstrom, P.: A robust main-memory compression scheme. In: Proceedings of the International Symposium on Computer Architecture 00(C), pp. 74–85 (2005)
Ferdman, M., Adileh, A., Kocberber, O., Volos, S., Alisafaee, M., Jevdjic, D., Kaynak, C., Popescu, A.D., Ailamaki, A., Falsafi, B.: Clearing the clouds: a study of emerging scale-out workloads on modern hardware. ACM SIGPLAN Not. 47, 37–48 (2012). ACM
Gerber, S., Zellweger, G., Achermann, R., Kourtis, K., Roscoe, T., Milojicic, D.: Not your parents’ physical address space. In: HotOS (2015)
Henning, J.L.: Spec cpu2006 benchmark descriptions. ACM SIGARCH Comput. Archit. News 34(4), 1–17 (2006)
Huang, J., Qureshi, M.K., Schwan, K.: An evolutionary study of linux memory management for fun and profit methodology. In: Review-ATC 2016 (2016)
Kwon, Y., Yu, H., Peter, S., Rossbach, C.J., Witchel, E.: Coordinated and efficient huge page management with ingens. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 705–721 (2016)
McCalpin: STREAM benchmark (2002). http://www.cs.virginia.edu/stream/
Navarro, J., Iyer, S., Druschel, P., Cox, A.: Practical, transparent operating system support for superpages. ACM SIGOPS Oper. Syst. Rev. 36, 89 (2002)
Ousterhout, J., Agrawal, P., Erickson, D., Kozyrakis, C., Leverich, J., Mazières, D., Mitra, S., Narayanan, A., Ongaro, D., Parulkar, G., et al.: The case for RAMCloud. Commun. ACM 54(7), 121–130 (2011)
Pekhimenko, G., Mowry, T.C., Mutlu, O.: Linearly compressed pages: a low-complexity, low-latency main memory compression framework. In: MICRO-46 Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), p. 489 (2013)
Tuduce, I.C., Gross, T.R.: Adaptive main memory compression. In: USENIX Annual Technical Conference, General Track, pp. 237–250 (2005)
Zaitsev, P., Tkachenko, V.: Evaluating Database Compression Methods: Update (2016). https://www.percona.com/blog/2016/04/13/evaluating-database-compression-methods-update/
Acknowledgments
Many thanks to members of ARC Lab of Zhejiang University for their constructive comments and helps during the project. We would like to thank the anonymous reviewers for their feedbacks. This research is funded by National Key Technologies R&D Program of Ministry of Science and Technology of the People’s Republic of China under Grant NO. 2016YFB0800201.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Li, G., Chen, W., Su, K., Lu, Z., Wang, Z. (2017). Hzmem: New Huge Page Allocator with Main Memory Compression. In: Ibrahim, S., Choo, KK., Yan, Z., Pedrycz, W. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2017. Lecture Notes in Computer Science(), vol 10393. Springer, Cham. https://doi.org/10.1007/978-3-319-65482-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-65482-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65481-2
Online ISBN: 978-3-319-65482-9
eBook Packages: Computer ScienceComputer Science (R0)