skip to main content
research-article

Toward Virtual Machine Image Management for Persistent Memory

Published:16 August 2021Publication History
Skip Abstract Section

Abstract

Persistent memory’s (PM) byte-addressability and high capacity will also make it emerging for virtualized environment. Modern virtual machine monitors virtualize PM using either I/O virtualization or memory virtualization. However, I/O virtualization will sacrifice PM’s byte-addressability, and memory virtualization does not get the chance of PM image management. In this article, we enhance QEMU’s memory virtualization mechanism. The enhanced system can achieve both PM’s byte-addressability inside virtual machines and PM image management outside the virtual machines. We also design pcow, a virtual machine image format for PM, which is compatible with our enhanced memory virtualization and supports storage virtualization features including thin-provisioning, base image, snapshot, and striping. Address translation is performed with the help of the Extended Page Table, thus much faster than image formats implemented in I/O virtualization. We also optimize pcow considering PM’s characteristics. We perform exhaustive performance evaluations on an x86 server equipping with Intel’s Optane DC persistent memory. The evaluation demonstrates that our scheme boosts the overall performance by up to 50× compared with qcow2, an image format implemented in I/O virtualization, and brings almost no performance overhead compared with the native memory virtualization. The striping feature can also scale-out the virtual PM’s bandwidth performance.

References

  1. Redis Labs. 2009. redis. Retrieved from https://redis.io/.Google ScholarGoogle Scholar
  2. Redis Labs. 2009. redis-benchmark. Retrieved from https://redis.io/topics/benchmarks.Google ScholarGoogle Scholar
  3. VMware. 2011. Virtual Disk Format 5.0. Retrieved from https://www.vmware.com/support/developer/vddk/vmdk_50_technote.pdf.Google ScholarGoogle Scholar
  4. Jens Axboe, et al. 2014. Flexible I/O Tester. Retrieved from https://github.com/axboe/fio.Google ScholarGoogle Scholar
  5. Intel. 2014. Persistent Memory Development Kit. Retrieved from https://pmem.io/pmdk/.Google ScholarGoogle Scholar
  6. Intel. 2017. BTT - Block Translation Table, Linux Kernel Documentation. Retrieved from https://www.kernel.org/doc/Documentation/nvdimm/btt.txt.Google ScholarGoogle Scholar
  7. Kevin Wolf, et al. 2017. Qcow2 Image Format, QEMU Documentation. Retrieved from https://github.com/qemu/qemu/blob/master/docs/interop/qcow2.txt.Google ScholarGoogle Scholar
  8. VMware. 2018. PMem (Persistant Memory) NVDIMM support in vSphere. Retrieved from https://storagehub.vmware.com/t/vsphere-storage/vsphere-6-7-core-storage-1/pmem-persistant-memory-nvdimm-support-in-vsphere/.Google ScholarGoogle Scholar
  9. Intel. 2018. pmem-redis. Retrieved from https://github.com/pmem/pmem-redis.Google ScholarGoogle Scholar
  10. Intel. 2018. Using Persistent Memory Devices with the Linux Device Mapper. Retrieved from https://pmem.io/2018/05/15/using_persistent_memory_devices_with_the_linux_device_mapper.html.Google ScholarGoogle Scholar
  11. Intel. 2019. Intel Optane DC Persistent Memory. Retrieved from https://www.intel.com/content/www/us/en/architecture-and-technology/optane-dc-persistent-memory.html.Google ScholarGoogle Scholar
  12. Avi Kivity, Anthony Liguori, et al. 2019. The memory API, QEMU Documentation. Retrieved from https://github.com/qemu/qemu/blob/master/docs/devel/memory.rst.Google ScholarGoogle Scholar
  13. Pankaj Gupta. 2019. Virtio-pmem, patchwork. Retrieved from https://patchwork.kernel.org/cover/11010055.Google ScholarGoogle Scholar
  14. Dmytro Apalkov, Alexey Khvalkovskiy, Steven Watts, Vladimir Nikitin, Xueti Tang, Daniel Lottis, Kiseok Moon, Xiao Luo, Eugene Chen, Adrian Ong et al. 2013. Spin-transfer torque magnetic random access memory (STT-MRAM). ACM J. Emerg. Technol. Comput. Syst. 9, 2 (2013), 13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Qingshu Chen, Liang Liang, Yubin Xia, Haibo Chen, and Hyunsoo Kim. 2016. Mitigating sync amplification for copy-on-write virtual disk. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). 241–247. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Deukyeon Hwang, Wook-Hee Kim, Youjip Won, and Beomseok Nam. 2018. Endurable transient inconsistency in byte-addressable persistent B+-tree. In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST). 187. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Arpit Joshi, Vijay Nagarajan, Marcelo Cintra, and Stratis Viglas. 2018. DHTM: Durable hardware transactional memory. In Proceedings of the International Symposium on Computer Architecture. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sudarsun Kannan, Nitish Bhat, Ada Gavrilovska, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. Redesigning LSMs for nonvolatile memory with NoveLSM. In Proceedings of the USENIX Annual Technical Conference (ATC’18). 993–1005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Liang Liang, Rong Chen, Haibo Chen, Yubin Xia, KwanJong Park, Binyu Zang, and Haibing Guan. 2016. A case for virtualizing persistent memory. In Proceedings of the ACM Symposium on Cloud Computing (SOCC’16). 126–140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Youyou Lu, Jiwu Shu, Long Sun, and Onur Mutlu. 2014. Loose-ordering consistency for persistent memory. In Proceedings of the 32nd International Conference on Computer Design (ICCD’14). IEEE, 216–223.Google ScholarGoogle ScholarCross RefCross Ref
  21. Richard McDougall and Jennifer Anderson. 2010. Virtualization performance: Perspectives and challenges ahead. ACM SIGOPS Oper. Syst. Rev. 44, 4 (2010), 40–56. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Timothy Merrifield and H. Reza Taheri. 2016. Performance implications of extended page tables on virtualized x86 processors. ACM SIGPLAN Notices 51, 7 (2016), 25–35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Bo Peng, Haozhong Zhang, Jianguo Yao, Yaozu Dong, Yu Xu, and Haibing Guan. 2018. MDev-NVMe: A NVMe storage virtualization solution with mediated pass-through. In Proceedings of the USENIX Annual Technical Conference (ATC’18). 665–676. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Ramnatthan Alagappan, Samer Al-Kiswany, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2015. Crash consistency. Queue 13, 7 (2015), 20.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Simone Raoux, Geoffrey W. Burr, Matthew J. Breitwisch, Charles T. Rettner, Y.-C. Chen, Robert M. Shelby, Martin Salinga, Daniel Krebs, S.-H. Chen, H.-L. Lung, et al. 2008. Phase-change random access memory: A scalable technology. IBM J. Res. Dev. 52, 4.5 (2008), 465–479. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Andy Rudoff. 2017. Persistent memory programming. Login: Usenix Mag. 42 (2017), 34–40.Google ScholarGoogle Scholar
  27. Rusty Russell. 2008. virtio: Towards a de-facto standard for virtual I/O devices. ACM SIGOPS Oper. Syst. Rev. 42, 5 (2008), 95–103. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. SNIA NVM Programming Technical Working Group. 2017. NVM Programming Model (Version 1.2).Google ScholarGoogle Scholar
  29. Xiang Song, Jian Yang, and Haibo Chen. 2014. Architecting flash-based solid-state drive for high-performance I/O virtualization. Comput. Architect. Lett. 13, 2 (2014), 61–64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Chunqiang Tang. 2011. FVD: A high-performance virtual machine image format for cloud. In Proceedings of the USENIX Annual Technical Conference (ATC’11). 16–24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Zev Weiss, Sriram Subramanian, Swaminathan Sundararaman, Nisha Talagala, Andrea C. Arpaci-Dusseau, and Remzi H Arpaci-Dusseau. 2015. ANViL: Advanced virtualization for modern non-volatile memory devices. In Proceedings of the 13rd USENIX Conference on File and Storage Technologies (FAST’15). 111–118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Cong Xu, Dimin Niu, Naveen Muralimanohar, Rajeev Balasubramonian, Tao Zhang, Shimeng Yu, and Yuan Xie. 2015. Overcoming the challenges of crossbar resistive memory architectures. In Proceedings of the 21st International Symposium on High Performance Computer Architecture (HPCA’15). IEEE, 476–488.Google ScholarGoogle ScholarCross RefCross Ref
  33. Jian Xu, Lu Zhang, Amirsaman Memaripour, Akshatha Gangadharaiah, Amit Borase, Tamires Brito Da Silva, Steven Swanson, and Andy Rudoff. 2017. NOVA-fortis: A fault-tolerant non-volatile main memory file system. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP’17). 478–496. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Jiwei Xu, Wenbo Zhang, Zhenyu Zhang, Tao Wang, and Tao Huang. 2016. Clustering-based acceleration for virtual machine image deduplication in the cloud environment. J. Syst. Softw. 121 (2016), 144–156. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. J. Zhang, L. Cui, P. Li, X. Liu, and G. Wang. 2019. Towards virtual machine image management for persistent memory. In Proceedings of the 35th Symposium on Mass Storage Systems and Technologies (MSST’19). 116–125.Google ScholarGoogle Scholar
  36. Jiachen Zhang, Peng Li, Bo Liu, Trent G. Marbach, Xiaoguang Liu, and Gang Wang. 2018. Performance analysis of 3D XPoint SSDs in virtualized and non-virtualized environments. In Proceedings of the 24th International Conference on Parallel and Distributed Systems (ICPADS’18). IEEE, 51–60.Google ScholarGoogle ScholarCross RefCross Ref
  37. Yiying Zhang and Steven Swanson. 2015. A study of application performance with non-volatile main memory. In Proceedings of the 31st Symposium on Mass Storage Systems and Technologies (MSST’15). IEEE, 1–10.Google ScholarGoogle ScholarCross RefCross Ref
  38. Pengfei Zuo and Yu Hua. 2017. A write-friendly hashing scheme for non-volatile memory systems. In Proceedings of the 33rd Symposium on Mass Storage Systems and Technologies (MSST’17).Google ScholarGoogle Scholar

Index Terms

  1. Toward Virtual Machine Image Management for Persistent Memory

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Storage
            ACM Transactions on Storage  Volume 17, Issue 3
            August 2021
            227 pages
            ISSN:1553-3077
            EISSN:1553-3093
            DOI:10.1145/3477268
            • Editor:
            • Sam H. Noh
            Issue’s Table of Contents

            Copyright © 2021 Association for Computing Machinery.

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 16 August 2021
            • Accepted: 1 February 2021
            • Revised: 1 November 2020
            • Received: 1 July 2020
            Published in tos Volume 17, Issue 3

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!