Abstract
Persistent memory’s (PM) byte-addressability and high capacity will also make it emerging for virtualized environment. Modern virtual machine monitors virtualize PM using either I/O virtualization or memory virtualization. However, I/O virtualization will sacrifice PM’s byte-addressability, and memory virtualization does not get the chance of PM image management. In this article, we enhance QEMU’s memory virtualization mechanism. The enhanced system can achieve both PM’s byte-addressability inside virtual machines and PM image management outside the virtual machines. We also design pcow, a virtual machine image format for PM, which is compatible with our enhanced memory virtualization and supports storage virtualization features including thin-provisioning, base image, snapshot, and striping. Address translation is performed with the help of the Extended Page Table, thus much faster than image formats implemented in I/O virtualization. We also optimize pcow considering PM’s characteristics. We perform exhaustive performance evaluations on an x86 server equipping with Intel’s Optane DC persistent memory. The evaluation demonstrates that our scheme boosts the overall performance by up to 50× compared with qcow2, an image format implemented in I/O virtualization, and brings almost no performance overhead compared with the native memory virtualization. The striping feature can also scale-out the virtual PM’s bandwidth performance.
- Redis Labs. 2009. redis. Retrieved from https://redis.io/.Google Scholar
- Redis Labs. 2009. redis-benchmark. Retrieved from https://redis.io/topics/benchmarks.Google Scholar
- VMware. 2011. Virtual Disk Format 5.0. Retrieved from https://www.vmware.com/support/developer/vddk/vmdk_50_technote.pdf.Google Scholar
- Jens Axboe, et al. 2014. Flexible I/O Tester. Retrieved from https://github.com/axboe/fio.Google Scholar
- Intel. 2014. Persistent Memory Development Kit. Retrieved from https://pmem.io/pmdk/.Google Scholar
- Intel. 2017. BTT - Block Translation Table, Linux Kernel Documentation. Retrieved from https://www.kernel.org/doc/Documentation/nvdimm/btt.txt.Google Scholar
- Kevin Wolf, et al. 2017. Qcow2 Image Format, QEMU Documentation. Retrieved from https://github.com/qemu/qemu/blob/master/docs/interop/qcow2.txt.Google Scholar
- VMware. 2018. PMem (Persistant Memory) NVDIMM support in vSphere. Retrieved from https://storagehub.vmware.com/t/vsphere-storage/vsphere-6-7-core-storage-1/pmem-persistant-memory-nvdimm-support-in-vsphere/.Google Scholar
- Intel. 2018. pmem-redis. Retrieved from https://github.com/pmem/pmem-redis.Google Scholar
- Intel. 2018. Using Persistent Memory Devices with the Linux Device Mapper. Retrieved from https://pmem.io/2018/05/15/using_persistent_memory_devices_with_the_linux_device_mapper.html.Google Scholar
- Intel. 2019. Intel Optane DC Persistent Memory. Retrieved from https://www.intel.com/content/www/us/en/architecture-and-technology/optane-dc-persistent-memory.html.Google Scholar
- Avi Kivity, Anthony Liguori, et al. 2019. The memory API, QEMU Documentation. Retrieved from https://github.com/qemu/qemu/blob/master/docs/devel/memory.rst.Google Scholar
- Pankaj Gupta. 2019. Virtio-pmem, patchwork. Retrieved from https://patchwork.kernel.org/cover/11010055.Google Scholar
- Dmytro Apalkov, Alexey Khvalkovskiy, Steven Watts, Vladimir Nikitin, Xueti Tang, Daniel Lottis, Kiseok Moon, Xiao Luo, Eugene Chen, Adrian Ong et al. 2013. Spin-transfer torque magnetic random access memory (STT-MRAM). ACM J. Emerg. Technol. Comput. Syst. 9, 2 (2013), 13. Google Scholar
Digital Library
- Qingshu Chen, Liang Liang, Yubin Xia, Haibo Chen, and Hyunsoo Kim. 2016. Mitigating sync amplification for copy-on-write virtual disk. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). 241–247. Google Scholar
Digital Library
- Deukyeon Hwang, Wook-Hee Kim, Youjip Won, and Beomseok Nam. 2018. Endurable transient inconsistency in byte-addressable persistent B+-tree. In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST). 187. Google Scholar
Digital Library
- Arpit Joshi, Vijay Nagarajan, Marcelo Cintra, and Stratis Viglas. 2018. DHTM: Durable hardware transactional memory. In Proceedings of the International Symposium on Computer Architecture. Google Scholar
Digital Library
- Sudarsun Kannan, Nitish Bhat, Ada Gavrilovska, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. Redesigning LSMs for nonvolatile memory with NoveLSM. In Proceedings of the USENIX Annual Technical Conference (ATC’18). 993–1005. Google Scholar
Digital Library
- Liang Liang, Rong Chen, Haibo Chen, Yubin Xia, KwanJong Park, Binyu Zang, and Haibing Guan. 2016. A case for virtualizing persistent memory. In Proceedings of the ACM Symposium on Cloud Computing (SOCC’16). 126–140. Google Scholar
Digital Library
- Youyou Lu, Jiwu Shu, Long Sun, and Onur Mutlu. 2014. Loose-ordering consistency for persistent memory. In Proceedings of the 32nd International Conference on Computer Design (ICCD’14). IEEE, 216–223.Google Scholar
Cross Ref
- Richard McDougall and Jennifer Anderson. 2010. Virtualization performance: Perspectives and challenges ahead. ACM SIGOPS Oper. Syst. Rev. 44, 4 (2010), 40–56. Google Scholar
Digital Library
- Timothy Merrifield and H. Reza Taheri. 2016. Performance implications of extended page tables on virtualized x86 processors. ACM SIGPLAN Notices 51, 7 (2016), 25–35. Google Scholar
Digital Library
- Bo Peng, Haozhong Zhang, Jianguo Yao, Yaozu Dong, Yu Xu, and Haibing Guan. 2018. MDev-NVMe: A NVMe storage virtualization solution with mediated pass-through. In Proceedings of the USENIX Annual Technical Conference (ATC’18). 665–676. Google Scholar
Digital Library
- Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Ramnatthan Alagappan, Samer Al-Kiswany, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2015. Crash consistency. Queue 13, 7 (2015), 20.Google Scholar
Digital Library
- Simone Raoux, Geoffrey W. Burr, Matthew J. Breitwisch, Charles T. Rettner, Y.-C. Chen, Robert M. Shelby, Martin Salinga, Daniel Krebs, S.-H. Chen, H.-L. Lung, et al. 2008. Phase-change random access memory: A scalable technology. IBM J. Res. Dev. 52, 4.5 (2008), 465–479. Google Scholar
Digital Library
- Andy Rudoff. 2017. Persistent memory programming. Login: Usenix Mag. 42 (2017), 34–40.Google Scholar
- Rusty Russell. 2008. virtio: Towards a de-facto standard for virtual I/O devices. ACM SIGOPS Oper. Syst. Rev. 42, 5 (2008), 95–103. Google Scholar
Digital Library
- SNIA NVM Programming Technical Working Group. 2017. NVM Programming Model (Version 1.2).Google Scholar
- Xiang Song, Jian Yang, and Haibo Chen. 2014. Architecting flash-based solid-state drive for high-performance I/O virtualization. Comput. Architect. Lett. 13, 2 (2014), 61–64. Google Scholar
Digital Library
- Chunqiang Tang. 2011. FVD: A high-performance virtual machine image format for cloud. In Proceedings of the USENIX Annual Technical Conference (ATC’11). 16–24. Google Scholar
Digital Library
- Zev Weiss, Sriram Subramanian, Swaminathan Sundararaman, Nisha Talagala, Andrea C. Arpaci-Dusseau, and Remzi H Arpaci-Dusseau. 2015. ANViL: Advanced virtualization for modern non-volatile memory devices. In Proceedings of the 13rd USENIX Conference on File and Storage Technologies (FAST’15). 111–118. Google Scholar
Digital Library
- Cong Xu, Dimin Niu, Naveen Muralimanohar, Rajeev Balasubramonian, Tao Zhang, Shimeng Yu, and Yuan Xie. 2015. Overcoming the challenges of crossbar resistive memory architectures. In Proceedings of the 21st International Symposium on High Performance Computer Architecture (HPCA’15). IEEE, 476–488.Google Scholar
Cross Ref
- Jian Xu, Lu Zhang, Amirsaman Memaripour, Akshatha Gangadharaiah, Amit Borase, Tamires Brito Da Silva, Steven Swanson, and Andy Rudoff. 2017. NOVA-fortis: A fault-tolerant non-volatile main memory file system. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP’17). 478–496. Google Scholar
Digital Library
- Jiwei Xu, Wenbo Zhang, Zhenyu Zhang, Tao Wang, and Tao Huang. 2016. Clustering-based acceleration for virtual machine image deduplication in the cloud environment. J. Syst. Softw. 121 (2016), 144–156. Google Scholar
Digital Library
- J. Zhang, L. Cui, P. Li, X. Liu, and G. Wang. 2019. Towards virtual machine image management for persistent memory. In Proceedings of the 35th Symposium on Mass Storage Systems and Technologies (MSST’19). 116–125.Google Scholar
- Jiachen Zhang, Peng Li, Bo Liu, Trent G. Marbach, Xiaoguang Liu, and Gang Wang. 2018. Performance analysis of 3D XPoint SSDs in virtualized and non-virtualized environments. In Proceedings of the 24th International Conference on Parallel and Distributed Systems (ICPADS’18). IEEE, 51–60.Google Scholar
Cross Ref
- Yiying Zhang and Steven Swanson. 2015. A study of application performance with non-volatile main memory. In Proceedings of the 31st Symposium on Mass Storage Systems and Technologies (MSST’15). IEEE, 1–10.Google Scholar
Cross Ref
- Pengfei Zuo and Yu Hua. 2017. A write-friendly hashing scheme for non-volatile memory systems. In Proceedings of the 33rd Symposium on Mass Storage Systems and Technologies (MSST’17).Google Scholar
Index Terms
Toward Virtual Machine Image Management for Persistent Memory
Recommendations
A Case for Virtualizing Persistent Memory
SoCC '16: Proceedings of the Seventh ACM Symposium on Cloud ComputingWith the proliferation of software and hardware support for persistent memory (PM) like PCM and NV-DIMM, we envision that PM will soon become a standard component of commodity cloud, especially for those applications demanding high performance and low ...
Efficient memory virtualization for Cross-ISA system mode emulation
VEE '14Cross-ISA system-mode emulation has many important applications. For example, Cross-ISA system-mode emulation helps computer architects and OS developers trace and debug kernel execution-flow efficiently by emulating a slower platform (such as ARM) on a ...
Efficient memory virtualization for Cross-ISA system mode emulation
VEE '14: Proceedings of the 10th ACM SIGPLAN/SIGOPS international conference on Virtual execution environmentsCross-ISA system-mode emulation has many important applications. For example, Cross-ISA system-mode emulation helps computer architects and OS developers trace and debug kernel execution-flow efficiently by emulating a slower platform (such as ARM) on a ...






Comments