Abstract
Persistent memory provides data persistence at main memory with emerging non-volatile main memories (NVMMs). Recent persistent memory file systems aggressively use direct access, which directly copy data between user buffer and the storage layer, to avoid the double-copy overheads through the OS page cache. However, we observe they all suffer from slow writes due to NVMMs’ asymmetric read-write performance and much slower performance than DRAM.
In this article, we propose HiNFS, a high-performance file system for non-volatile main memory, to combine both buffering and direct access for fine-grained file system operations. HiNFS uses an NVMM-aware Write Buffer to buffer the lazy-persistent file writes in DRAM, while performing direct access to NVMM for eager-persistent file writes. It directly reads file data from both DRAM and NVMM, by ensuring read consistency with a combination of the DRAM Block Index and Cacheline Bitmap to track the latest data between DRAM and NVMM. HiNFS also employs a Buffer Benefit Model to identify the eager-persistent file writes before issuing I/Os. Evaluations show that HiNFS significantly improves throughput by up to 184% and reduces execution time by up to 64%comparing with state-of-the-art persistent memory file systems PMFS and EXT4-DAX.
- Dbt2 test suite. Retrieved from http://sourceforge.net/apps/mediawiki/osdldbt.Google Scholar
- Filebench 1.4.9.1. Retrieved from http://sourceforge.net/projects/filebench/.Google Scholar
- FIU system call io trace. Retrieved from http://sylab-srv.cs.fiu.edu/dokuwiki/doku.php?id=projects:nbw:start.Google Scholar
- Flexible IO (fio) Tester. Retrieved from http://freecode.com/projects/fio.Google Scholar
- Lasr system call io trace. Retrieved from http://iotta.snia.org/historical_section?tracetype_id=1.Google Scholar
- 2014. Support ext4 on NV-DIMMs. Retrieved from http://lwn.net/Articles/588218/.Google Scholar
- 2014. Supporting file systems in persistent memory. Retrieved from https://lwn.net/Articles/610174/.Google Scholar
- Geoffrey W. Burr, Matthew J. Breitwisch, Michele Franceschini, Davide Garetto, Kailash Gopalakrishnan, Bryan Jackson, Bülent Kurdi, Chung Lam, Luis A. Lastras, Alvaro Padilla, Bipin Rajendran, Simone Raoux, and Rohit S. Shenoy. 2010. Phase change memory technology. J. Vacuum Sci. Technol. B 28, 2 (2010), 223--262.Google Scholar
Cross Ref
- Daniel Campello, Hector Lopez, Ricardo Koller, Raju Rangaswami, and Luis Useche. 2015. Non-blocking writes to files. In Proceedings of the 13th USENIX Conference on File and Storage Technologies (FAST’15). 151--165. Google Scholar
Digital Library
- Shimin Chen, Phillip B. Gibbons, and Suman Nath. 2011. Rethinking database algorithms for phase change memory. In Proceedings of the 5th Biennial Conference on Innovative Data Systems Research (CIDR’11). 21--31.Google Scholar
- Joel Coburn, Adrian M. Caulfield, Ameen Akel, Laura M. Grupp, Rajesh K. Gupta, Ranjit Jhala, and Steven Swanson. 2011. NV-heaps: Making persistent objects fast and safe with next-generation, non-volatile memories. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). 105--118. Google Scholar
Digital Library
- Edward Grady Coffman and Peter J. Denning. 1973. Operating Systems Theory. Vol. 973.Google Scholar
- Jeremy Condit, Edmund B. Nightingale, Christopher Frost, Engin Ipek, Benjamin Lee, Doug Burger, and Derrick Coetzee. 2009. Better I/O through byte-addressable, persistent memory. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles (SOSP’09). 133--146. Google Scholar
Digital Library
- Intel Cooperation. 2015. NVDIMM Namespace Specification. Retrieved from http://pmem.io/documents/NVDIMM_ Namespace_Spec.pdf.Google Scholar
- Intel Cooperation. 2016. Intel Architecture Instruction Set Extensions Programming Reference. Retrieved from https://software.intel.com/sites/default/files/managed/b4/3a/319433-024.pdf.Google Scholar
- Peter J. Denning. 1968. The working set model for program behavior. Commun. ACM 11, 5 (1968), 323--333. Google Scholar
Digital Library
- Edward Doller. 2009. Phase change memory and its impacts on memory hierarchy. Retrieved from http://www.pdl.cmu.edu/SDI/2009/slides/Numonyx.pdf.Google Scholar
- Subramanya R. Dulloor, Sanjay Kumar, Anil Keshavamurthy, Philip Lantz, Dheeraj Reddy, Rajesh Sankaran, and Jeff Jackson. 2014. System software for persistent memory. In Proceedings of the 9th European Conference on Computer Systems (EuroSys’14). 15:1--15:15. Google Scholar
Digital Library
- Tyler Harter, Chris Dragga, Michael Vaughn, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2011. A file is not a file: Understanding the I/O behavior of apple desktop applications. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP’11). 71--83. Google Scholar
Digital Library
- Jian Huang, Karsten Schwan, and Moinuddin K. Qureshi. 2014. NVRAM-aware logging in transaction systems. Proc. VLDB Endow. 8, 4 (Dec. 2014), 389--400. Google Scholar
Digital Library
- Taeho Hwang, Jaemin Jung, and Youjip Won. 2015. Heapo: Heap-based persistent object store. ACM Trans. Storage (TOS) 11, 1 (2015), 3. Google Scholar
Digital Library
- Lei Jiang, Bo Zhao, Youtao Zhang, Jun Yang, and B. R. Childers. 2012. Improving write operations in MLC phase change memory. In Proceedings of the 18th International Symposium on High Performance Computer Architecture (HPCA’12). 1--10. Google Scholar
Digital Library
- Heeseung Jo, Jeong-Uk Kang, Seon-Yeong Park, Jin-Soo Kim, and Joonwon Lee. 2006. FAB: Flash-aware buffer management policy for portable media players. IEEE Trans. Consum. Electron. 52, 2 (May 2006), 485--493. Google Scholar
Digital Library
- Theodore Johnson and Dennis Shasha. 1994. 2Q: A low-overhead high-performance buffer management replacement algorithm. In Proceedings of the 20th International Conference on Very Large Data Bases (VLDB’94). 439--450. Google Scholar
Digital Library
- William K. Josephson, Lars A. Bongo, David Flynn, and Kai Li. 2010. DFS: A file system for virtualized flash storage. In Proceedings of the 8th USENIX Conference on File and Storage Technologies (FAST’10). USENIX, Berkeley, CA. Google Scholar
Digital Library
- Myoungsoo Jung, John Shalf, and Mahmut Kandemir. 2013. Design of a large-scale storage-class RRAM system. In Proceedings of the 27th International ACM Conference on International Conference on Supercomputing (ICS’13). 103--114. Google Scholar
Digital Library
- Sooyong Kang, Sungmin Park, Hoyoung Jung, Hyoki Shim, and Jaehyuk Cha. 2009. Performance trade-offs in using NVRAM write buffer for flash memory-based storage devices. IEEE Trans. Comput. (TC) 58, 6 (June 2009), 744--758. Google Scholar
Digital Library
- Jeffrey Katcher. 1997. Postmark: A New File System Benchmark. Technical Report TR3022, Network Appliance.Google Scholar
- Hyojun Kim and Seongjun Ahn. 2008. BPLRU: A buffer management scheme for improving random writes in flash storage. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST’08). 16:1--16:14. Google Scholar
Digital Library
- ESOS LABORATORY. 2013. Mobibench benchmark tool. Retrieved from http://www.mobibench.co.kr/.Google Scholar
- B. C. Lee, Ping Zhou, Jun Yang, Youtao Zhang, Bo Zhao, E. Ipek, O. Mutlu, and D. Burger. 2010. Phase-change technology and the future of main memory. Micro, IEEE 30, 1 (Jan 2010), 143--143. Google Scholar
Digital Library
- Benjamin C. Lee, Engin Ipek, Onur Mutlu, and Doug Burger. 2009. Architecting phase change memory as a scalable dram alternative. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). 2--13. Google Scholar
Digital Library
- Changman Lee, Dongho Sim, Jooyoung Hwang, and Sangyeun Cho. 2015. F2FS: A new file system for flash storage. In Proceedings of the 13th USENIX Conference on File and Storage Technologies (FAST’15). USENIX, Santa Clara, CA. Retrieved from https://www.usenix.org/conference/fast15/technical-sessions/presentation/lee. Google Scholar
Digital Library
- Eunji Lee and Hyokyung Bahn. 2014. Caching strategies for high-performance storage media. ACM Trans. Storage (TOS) 10, 3 (Aug. 2014), 11:1--11:22. Google Scholar
Digital Library
- Youyou Lu, Jiwu Shu, Youmin Chen, and Tao Li. 2017. Octopus: An RDMA-enabled distributed persistent memory file system. In Proceedings of the 2017 USENIX Annual Technical Conference (USENIX ATC’17). USENIX Association, 773--785. Google Scholar
Digital Library
- Y. Lu, J. Shu, and L. Sun. 2015. Blurred persistence in transactional persistent memory. In Proceedings of the 31st Symposium on Mass Storage Systems and Technologies (MSST’15). 1--13.Google Scholar
- Youyou Lu, Jiwu Shu, and Long Sun. 2016. Blurred persistence: Efficient transactions in persistent memory. ACM Trans. Storage (TOS) 12, 1 (2016), 3. Google Scholar
Digital Library
- Y. Lu, J. Shu, L. Sun, and O. Mutlu. 2014. Loose-ordering consistency for persistent memory. In Proceedings of the 32nd IEEE International Conference on Computer Design (ICCD’14). 216--223.Google Scholar
- Youyou Lu, Jiwu Shu, Long Sun, and Onur Mutlu. 2017. Improving performance and endurance of persistent memory with loose-ordering consistency. IEEE Trans. Parall. Distrib. Syst. PP, 99 (2017), 1--1.Google Scholar
- Youyou Lu, Jiwu Shu, and Wei Wang. 2014. ReconFS: A reconstructable file system on flash storage. In Proceedings of the 12th USENIX Conference on File and Storage Technologies (FAST’14). USENIX, Berkeley, CA, 75--88. Google Scholar
Digital Library
- Youyou Lu, Jiwu Shu, and Weimin Zheng. 2013. Extending the lifetime of flash-based storage through reducing write amplification from file systems. In Proceedings of the 11th USENIX Conference on File and Storage Technologies (FAST’13). USENIX, Berkeley, CA. Google Scholar
Digital Library
- Nimrod Megiddo and Dharmendra S. Modha. 2003. ARC: A self-tuning, low overhead replacement cache. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST’03). 115--130. Google Scholar
Digital Library
- Changwoo Min, Kangnyeon Kim, Hyunjin Cho, Sang-Won Lee, and Young Ik Eom. 2012. SFS: Random write considered harmful in solid state drives. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12). San Jose, CA, 139--154. Google Scholar
Digital Library
- Iulian Moraru, David G. Andersen, Michael Kaminsky, Niraj Tolia, Parthasarathy Ranganathan, and Nathan Binkert. 2013. Consistent, durable, and safe memory management for byte-addressable nonvolatile main memory. In Proceedings of the First ACM SIGOPS Conference on Timely Results in Operating Systems (TRIOS’13). 1:1--1:17. Google Scholar
Digital Library
- Edmund B. Nightingale, Kaushik Veeraraghavan, Peter M. Chen, and Jason Flinn. 2006. Rethink the sync. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI’06). 1--14. Google Scholar
Digital Library
- Jiaxin Ou and Jiwu Shu. 2016. Fast and failure-consistent updates of application data in non-volatile main memory file system. In Proceedings of the 32st Symposium on Mass Storage Systems and Technologies (MSST’16).Google Scholar
- Jiaxin Ou, Jiwu Shu, and Youyou Lu. 2016. A high-performance file system for non-volatile main memory. In Proceedings of the 11th European Conference on Computer Systems. ACM, 12. Google Scholar
Digital Library
- Jiaxin Ou, Jiwu Shu, Youyou Lu, Letian Yi, and Wei Wang. 2014. EDM: An endurance-aware data migration scheme for load balancing in SSD storage clusters. In Proceedings of the 28th International Parallel and Distributed Processing Symposium (IPDPS’14). 787--796. Google Scholar
Digital Library
- Steven Pelley, Peter M. Chen, and Thomas F. Wenisch. 2014. Memory persistency. In Proceedings of the 41st Annual International Symposium on Computer Architecuture (ISCA’14). 265--276. Google Scholar
Digital Library
- Moinuddin K. Qureshi, Vijayalakshmi Srinivasan, and Jude A. Rivers. 2009. Scalable high-performance main memory system using phase-change memory technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). 24--33. Google Scholar
Digital Library
- Luiz E. Ramos, Eugene Gorbatov, and Ricardo Bianchini. 2011. Page placement in hybrid memory systems. In Proceedings of the International Conference on Supercomputing (ICS’11). 85--95. Google Scholar
Digital Library
- Drew Roselli, Jacob R. Lorch, and Thomas E. Anderson. 2000. A comparison of file system workloads. In Proceedings of the Annual Conference on USENIX Annual Technical Conference (ATEC’00). 41--54. Google Scholar
Digital Library
- Chris Ruemmler and John Wilkes. 1993. UNIX disk access patterns. In Proceedings of the USENIX Winter Conference, Vol. 93. 405--420.Google Scholar
- Long Sun, Youyou Lu, and Jiwu Shu. 2015. DP2: Reducing transaction overhead with differential and dual persistency in persistent memory. In Proceedings of the ACM International Conference on Computing Frontiers (CF’15). ACM. Google Scholar
Digital Library
- Kosuke Suzuki and Steven Swanson. 2015. The Non-Volatile Memory Technology Database (NVMDB). Technical Report CS2015-1011. Department of Computer Science 8 Engineering, University of California, San Diego.Google Scholar
- Shivaram Venkataraman, Niraj Tolia, Parthasarathy Ranganathan, and Roy H. Campbell. 2011. Consistent and durable data structures for non-volatile byte-addressable memory. In Proceedings of the 9th USENIX Conference on File and Storage Technologies (FAST’11). San Jose, CA, 61--75. Google Scholar
Digital Library
- Haris Volos, Guilherme Magalhaes, Ludmila Cherkasova, and Jun Li. 2015. Quartz: A lightweight performance emulator for persistent memory software. In Proceedings of the 16th Annual Middleware Conference. ACM, 37--49. Google Scholar
Digital Library
- Haris Volos, Sanketh Nalli, Sankarlingam Panneerselvam, Venkatanathan Varadarajan, Prashant Saxena, and Michael M. Swift. 2014. Aerie: Flexible file-system interfaces to storage-class memory. In Proceedings of the 9th European Conference on Computer Systems (EuroSys’14). 14:1--14:14. Google Scholar
Digital Library
- Haris Volos, Andres Jaan Tack, and Michael M. Swift. 2011. Mnemosyne: Lightweight persistent memory. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). 91--104. Google Scholar
Digital Library
- D. L. Willick, D. L. Eager, and R. B. Bunt. 1993. Disk cache replacement policies for network fileservers. In Proceedings the 13th International Conference on Distributed Computing Systems (ICDCS’93). 2--11.Google Scholar
- Xiaojian Wu and A. L. Narasimha Reddy. 2011. SCMFS: A file system for storage class memory. In Proceedings of the 2011 International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’11). 39:1--39:11. Google Scholar
Digital Library
- Jian Xu and Steven Swanson. 2016. NOVA: A log-structured file system for hybrid volatile/non-volatile main memories. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). 323--338. Google Scholar
Digital Library
- Jun Yang, Qingsong Wei, Cheng Chen, Chundong Wang, Khai Leong Yong, and Bingsheng He. 2015. NV-tree: Reducing consistency cost for NVM-based single level systems. In Proceedings of the 13th USENIX Conference on File and Storage Technologies (FAST’15). 167--181. Google Scholar
Digital Library
- J. Joshua Yang and R. Stanley Williams. 2013. Memristive devices in computing system: Promises and challenges. J. Emerg. Technol. Comput. Syst. (JETC’13) 9, 2 (May 2013), 11:1--11:20. Google Scholar
Digital Library
- Jiacheng Zhang, Jiwu Shu, and Youyou Lu. 2016. ParaFS: A log-structured file system to exploit the internal parallelism of flash devices. In Proceedings of the 2016 USENIX Annual Technical Conference (USENIX ATC’16). Google Scholar
Digital Library
- Yiying Zhang and Steven Swanson. 2015. A study of application performance with non-volatile main memory. In Proceedings of the 31st Symposium on Mass Storage Systems and Technologies (MSST’15). 1--10.Google Scholar
Cross Ref
- Yiying Zhang, Jian Yang, Amirsaman Memaripour, and Steven Swanson. 2015. Mojim: A reliable and highly-available non-volatile memory system. In Proceedings of the 20th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’15). 3--18. Google Scholar
Digital Library
- Jishen Zhao, Sheng Li, Doe Hyun Yoon, Yuan Xie, and Norman P. Jouppi. 2013. Kiln: Closing the performance gap between systems with and without persistence support. In Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’13). ACM, 421--432. Google Scholar
Digital Library
- Ping Zhou, Bo Zhao, Jun Yang, and Youtao Zhang. 2009. A durable and energy efficient main memory using phase change memory technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). 14--23. Google Scholar
Digital Library
Index Terms
HiNFS: A Persistent Memory File System with Both Buffering and Direct-Access
Recommendations
SplitFS: reducing software overhead in file systems for persistent memory
SOSP '19: Proceedings of the 27th ACM Symposium on Operating Systems PrinciplesWe present SplitFS, a file system for persistent memory (PM) that reduces software overhead significantly compared to state-of-the-art PM file systems. SplitFS presents a novel split of responsibilities between a user-space library file system and an ...
A file system bypassing volatile main memory: towards a single-level persistent store
CF '18: Proceedings of the 15th ACM International Conference on Computing FrontiersExisting persistent memory (PM) based file systems rely on a DRAM and PM hybrid store. Although a hybrid store does boost system performance while avoiding some current PM limitations like limited endurance, we envision that with more advances PM ...
NOVA-Fortis: A Fault-Tolerant Non-Volatile Main Memory File System
SOSP '17: Proceedings of the 26th Symposium on Operating Systems PrinciplesEmerging fast, persistent memories will enable systems that combine conventional DRAM with large amounts of non-volatile main memory (NVMM) and provide huge increases in storage performance. Fully realizing this potential requires fundamental changes in ...






Comments