Abstract
Although ubiquitous, modern filesystems have rather complex behaviours that are hardly understood by programmers and lead to severe software bugs such as data corruption. As a first step to ensure correctness of software performing file I/O, we formalize the semantics of the Linux ext4 filesystem, which we integrate with the weak memory consistency semantics of C/C++. We further develop an effective model checking approach for verifying programs that use the filesystem. In doing so, we discover and report bugs in commonly-used text editors such as vim, emacs and nano.
- Parosh Abdulla, Stavros Aronis, Bengt Jonsson, and Konstantinos Sagonas ( 2014 ). “Optimal dynamic partial order reduction.” In: POPL 2014. New York, NY, USA: ACM, pp. 373-384. doi: 10.1145/2535838.2535845.Google Scholar
Digital Library
- Advanced Format ( 2020 ). url: https://en.wikipedia.org/wiki/Advanced_Format (visited on May 20, 2020 ).Google Scholar
- Jade Alglave, Luc Maranget, and Michael Tautschnig ( July 2014 ). “Herding Cats: Modelling, Simulation, Testing, and Data Mining for Weak Memory.” In: ACM Trans. Program. Lang. Syst. 36.2, 7 : 1-7 : 74. doi: 10.1145/2627752.Google Scholar
Digital Library
- Jef Bonwick (Oct. 2005 ). ZFS: The Last Word in Filesystems. Library Catalog: blogs.oracle.com. url: https://blogs.oracle.com/ bonwick/zfs% 3A-the-last-word-in-filesystems (visited on June 17, 2020 ).Google Scholar
- James Bornholt, Antoine Kaufmann, Jialin Li, Arvind Krishnamurthy, Emina Torlak, and Xi Wang ( 2016 ). “Specifying and Checking File System Crash-Consistency Models.” In: ASPLOS 2016 44.2, pp. 83-98. doi: 10.1145/2980024.2872406.Google Scholar
Digital Library
- Haogang Chen, Daniel Ziegler, Tej Chajed, Adam Chlipala, M. Frans Kaashoek, and Nickolai Zeldovich ( 2015 ). “Using Crash Hoare logic for certifying the FSCQ file system.” In: SOSP 2015. the 25th Symposium. Monterey, California: ACM Press, pp. 18-37. doi: 10.1145/2815400.2815402.Google Scholar
Digital Library
- Ran Chen, Martin Clochard, and Claude Marché ( 2016 ). “A Formal Proof of a Unix Path Resolution Algorithm.” In: HAL hal-01406848. url: https://hal.inria.fr/hal-01406848/document (visited on Nov. 16, 2020 ).Google Scholar
- Copy-on-write ( 2020 ). url: https://en.wikipedia.org/wiki/Copy-on-write (visited on May 20, 2020 ).Google Scholar
- Heming Cui, Gang Hu, Jingyue Wu, and Junfeng Yang ( 2013 ). “Verifying Systems Rules Using Rule-Directed Symbolic Execution.” In: ASPLOS 2013. Houston, Texas, USA: ACM, pp. 329-342. doi: 10.1145/2451116.2451152.Google Scholar
Digital Library
- GNU Emacs ( 2019 ). GNU Emacs: An extensible, customizable, free/libre text editor-and more. url: https://www.gnu.org/ software/emacs/ (visited on June 15, 2020 ).Google Scholar
- ext4 benchmarks ( 2012 ). EXT4 File-System Tuning Benchmarks. url: https://www.phoronix.com/scan.php ?page=article& item=ext4_linux35_tuning&num=1 (visited on May 20, 2020 ).Google Scholar
- Ext4 data loss ( 2009 ). url: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781 (visited on May 20, 2020 ).Google Scholar
- ext4 Linux kernel ( 2020 ). ext4 Data Structures and Algorithms. url: https://www.kernel.org/doc/html/latest/filesystems/ext4/ index. html (visited on May 20, 2020 ).Google Scholar
- ext4 corruption ( 2015 ). ext4: Filesystem corruption on panic. url: https://bugs.chromium.org/p/chromium/issues/detail?id= 502898 (visited on May 20, 2020 ).Google Scholar
- Michalis Kokologiannakis ( July 2020 ). files: improve the backup procedure to ensure no data is lost. url: https://git.savannah. gnu.org/cgit/nano.git/commit/? id=a84cdaaa50a804a8b872f6d468412dadf105b3c5 (visited on July 9, 2020 ).Google Scholar
- Cormac Flanagan and Patrice Godefroid ( 2005 ). “Dynamic partial-order reduction for model checking software.” In: POPL 2005. New York, NY, USA: ACM, pp. 110-121. doi: 10.1145/1040305.1040315.Google Scholar
Digital Library
- Patrice Godefroid ( 1997 ). “Model Checking for Programming Languages using VeriSoft.” In: POPL 1997. Paris, France: ACM, pp. 174-186. doi: 10.1145/263699.263717.Google Scholar
Digital Library
- Patrice Godefroid (Mar. 2005 ). “Software Model Checking: The VeriSoft Approach.” In: Form. Meth. Syst. Des. 26.2, pp. 77-101. doi: 10.1007/s10703-005-1489-x.Google Scholar
Digital Library
- JOE ( 2018 ). JOE-Joe's Own Editor. url: https://joe-editor. sourceforge.io (visited on June 15, 2020 ).Google Scholar
- Rajeev Joshi and Gerard Holzmann (June 11, 2007 ). “A Mini Challenge: Build a Verifiable Filesystem.” In: Formal Asp. Comput. 19, pp. 269-272. doi: 10.1007/s00165-006-0022-3.Google Scholar
Digital Library
- Eunsuk Kang and Daniel Jackson ( 2008 ). “Formal Modeling and Analysis of a Flash Filesystem in Alloy.” In: ABZ 2008. Ed. by Egon Börger, Michael Butler, Jonathan P. Bowen, and Paul Boca. Vol. 5238. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 294-308. doi: 10.1007/978-3-540-87603-8_23.Google Scholar
Digital Library
- Gabriele Keller, Toby Murray, Sidney Amani, Liam O'Connor, Zilin Chen, Leonid Ryzhyk, Gerwin Klein, and Gernot Heiser ( 2013 ). “File systems deserve verification too!” In: PLOS 2013. Farmington, Pennsylvania: ACM Press, pp. 1-7. doi: 10.1145/2525528.2525530.Google Scholar
Digital Library
- Michalis Kokologiannakis, Ilya Kaysin, Azalea Raad, and Viktor Vafeiadis (Jan. 2021 ). “PerSeVerE: Persistency Semantics for Verification under Ext4 (Supplementary Material).” In: url: https://plv.mpi-sws.org/persevere.Google Scholar
- Michalis Kokologiannakis, Azalea Raad, and Viktor Vafeiadis ( 2019 ). “Model Checking for Weakly Consistent Libraries.” In: PLDI 2019. New York, NY, USA: ACM. doi: 10.1145/3314221.3314609.Google Scholar
Digital Library
- Ori Lahav, Viktor Vafeiadis, Jeehoon Kang, Chung-Kil Hur, and Derek Dreyer ( 2017 ). “Repairing Sequential Consistency in C/C++ 11.” In: PLDI 2017. Barcelona, Spain: ACM, pp. 618-632. doi: 10.1145/3062341.3062352.Google Scholar
Digital Library
- Linux man pages ( 2020 ). url: http://www.man7.org/linux/man-pages /index.html (visited on May 20, 2020 ).Google Scholar
- Richard Gooch ( 1999 ). Overview of the Linux Virtual File System. url: https://www.kernel.org/doc/html/latest/filesystems/ vfs. html (visited on May 20, 2020 ).Google Scholar
- Jayashree Mohan, Ashlie Martinez, Soujanya Ponnapalli, Pandian Raju, and Vijay Chidambaram ( 2018 ). “Finding CrashConsistency Bugs with Bounded Black-Box Crash Testing.” In: OSDI 2018. Carlsbad, CA, USA: USENIX Association, pp. 33-50. url: https://www.usenix.org/system/files/osdi18-mohan. pdf (visited on Nov. 16, 2020 ).Google Scholar
- Madanlal Musuvathi, Shaz Qadeer, Thomas Ball, Gérard Basler, Piramanayagam Arumuga Nainar, and Iulian Neamtiu ( 2008 ). “Finding and Reproducing Heisenbugs in Concurrent Programs.” In: OSDI 2008. USENIX Association, pp. 267-280. url: https://www.usenix.org/legacy/events/osdi08/tech/full_papers/musuvathi/musuvathi. pdf (visited on Nov. 16, 2020 ).Google Scholar
- GNU Nano ( 2019 ). The GNU Nano homepage. url: https://nano-editor. org (visited on June 15, 2020 ).Google Scholar
- Gian Ntzik and Philippa Gardner (Oct. 23, 2015 ). “Reasoning about the POSIX file system: local update and global pathnames.” In: OOPSLA 2015. Pittsburgh, PA, USA: Association for Computing Machinery, pp. 201-220. doi: 10.1145/2814270.2814306.Google Scholar
Digital Library
- Daejun Park and Dongkun Shin ( 2017 ). “iJournaling: Fine-Grained Journaling for Improving the Latency of Fsync System Call.” In: pp. 787-798. url: https://www.usenix.org/conference/atc17/technical-sessions/presentation/park.Google Scholar
- Thanumalayan Sankaranarayana Pillai, Ramnatthan Alagappan, Lanyue Lu, Vijay Chidambaram, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau (Oct. 27, 2017 ). “Application Crash Consistency and Performance with CCFS.” In: ACM Trans. Storage 13.3, pp. 1-29. doi: 10.1145/3119897.Google Scholar
Digital Library
- Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Ramnatthan Alagappan, Samer Al-Kiswany, Andrea C. ArpaciDusseau, and Remzi H. Arpaci-Dusseau (Oct. 2014 ). “All File Systems Are Not Created Equal: On the Complexity of Crafting Crash-Consistent Applications.” In: OSDI 2014. Broomfield, CO: USENIX Association, pp. 433-448. url: https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai.Google Scholar
- Anton Podkopaev, Ori Lahav, and Viktor Vafeiadis (Jan. 2019 ). “Bridging the Gap Between Programming Languages and Hardware Weak Memory Models.” In: Proc. ACM Program. Lang. 3.POPL, 69 : 1-69 : 31. doi: 10.1145/3290382.Google Scholar
- POSIX ( 2018 ). The Open Group Base Specifications Issue 7. url: https://pubs.opengroup.org/onlinepubs/9699919799/ (visited on May 20, 2020 ).Google Scholar
- Vijayan Prabhakaran, Andrea C Arpaci-Dusseau, and Remzi H Arpaci-Dusseau ( 2005 ). “Analysis and Evolution of Journaling File Systems.” In: p. 16. url: https://www.usenix.org/legacy/events/usenix05/tech/general/full_papers/prabhakaran/ prabhakaran.pdf.Google Scholar
- Azalea Raad and Viktor Vafeiadis (Oct. 2018 ). “Persistence Semantics for Weak Memory: Integrating Epoch Persistency with the TSO Memory Model.” In: Proc. ACM Program. Lang. 2.OOPSLA. doi: 10.1145/3276507.Google Scholar
Digital Library
- Azalea Raad, John Wickerson, Gil Neiger, and Viktor Vafeiadis (Dec. 20, 2019a ). “Persistency semantics of the Intel-x86 architecture.” In: Proc. ACM Program. Lang. 4 (POPL), 11 : 1-11 : 31. doi: 10.1145/3371079.Google Scholar
- Azalea Raad, John Wickerson, and Viktor Vafeiadis (Oct. 10, 2019b ). “Weak Persistency Semantics from the Ground Up.” In: Proc. ACM Program. Lang. 3 (OOPSLA), 135 : 1-135 : 27. doi: 10.1145/3360561.Google Scholar
- renameio ( 2020 ). url: https://github.com/google/renameio (visited on May 20, 2020 ).Google Scholar
- Tom Ridge, David Sheets, Thomas Tuerk, Andrea Giugliano, Anil Madhavapeddy, and Peter Sewell ( 2015 ). “SibylFS: formal specification and oracle-based testing for POSIX and real-world file systems.” In: SOSP 2015. Monterey, California: ACM Press, pp. 38-53. doi: 10.1145/2815400.2815411.Google Scholar
Digital Library
- Ohad Rodeh, Josef Bacik, and Chris Mason (Aug. 1, 2013 ). “ BTRFS: The Linux B-Tree Filesystem.” In: ACM Trans. Storage 9.3, 9 : 1-9 : 32. doi: 10.1145/2501620.2501623.Google Scholar
Digital Library
- Cindy Rubio-González, Haryadi S. Gunawi, Ben Liblit, Remzi H. Arpaci-Dusseau, and Andrea C. Arpaci-Dusseau ( June 15, 2009 ). “Error propagation analysis for file systems.” In: SIGPLAN Not. 44.6, pp. 270-280. doi: 10.1145/1543135.1542506.Google Scholar
Digital Library
- Gerhard Schellhorn, Gidon Ernst, Jörg Pfähler, Dominik Haneberg, and Wolfgang Reif ( 2014 ). “Development of a Verified Flash File System.” In: ABZ 2014. Vol. 8477. Berlin, Heidelberg, pp. 9-24. doi: 10.1007/978-3-662-43652-3_2.Google Scholar
Digital Library
- Helgi Sigurbjarnarson, James Bornholt, Emina Torlak, and Xi Wang ( 2016 ). “Push-Button Verification of File Systems via Crash Refinement.” In: OSDI 2016. Savannah, GA, USA: USENIX Association, pp. 1-16. url: https://www.usenix.org/ system/files/conference/osdi16/osdi16-sigurbjarnarson.pdf.Google Scholar
- Seongbae Son, Jinsoo Yoo, and Youjip Won ( 2017 ). “Guaranteeing the Metadata Update Atomicity in EXT4 File system.” In: APSys 2017, pp. 1-8. doi: 10.1145/3124680.3124722.Google Scholar
- SQLite ( 2020 ). url: https://sqlite.org/index. html (visited on May 20, 2020 ).Google Scholar
- Atomic Commit In SQLite ( 2020 ). url: https://sqlite.org/atomiccommit. html (visited on May 20, 2020 ).Google Scholar
- Adam Sweeney ( 1996 ). “Scalability in the XFS file system. ” In: USENIX ATC 1996, pp. 1-14. url: https://www.usenix.org/ legacy/publications/library/proceedings/sd96/sweeney.html.Google Scholar
- Theodore Y Ts'o and Stephen Tweedie ( 2002 ). “Planned Extensions to the Linux Ext2/Ext3 Filesystem.” In: pp. 235-243. url: http://www.usenix.org/publications/library/proceedings/usenix02/tech/freenix/tso.html.Google Scholar
- Stephen C Tweedie ( 1998 ). “Journaling the Linux ext2fs Filesystem.” In: LinuxExpo 1998. url: http://e2fsprogs.sourceforge. net/journal-design. pdf (visited on Nov. 16, 2020 ).Google Scholar
- Vim ( 2019 ). Vim-the ubiquitous text editor. url: https://vim. org (visited on June 15, 2020 ).Google Scholar
- Junfeng Yang, Can Sar, and Dawson Engler (Nov. 6, 2006 ). “EXPLODE: a lightweight, general system for finding serious storage system errors.” In: OSDI 2006. Seattle, Washington: USENIX Association, pp. 131-146. url: https://www.usenix. org/legacy/event/osdi06/tech/full_papers/yang_junfeng/yang_junfeng. pdf (visited on June 17, 2020 ).Google Scholar
- Mai Zheng, Joseph Tucek, Dachuan Huang, Elizabeth S Yang, Bill W Zhao, Feng Qin, Mark Lillibridge, and Shashank Singh ( 2014 ). “Torturing Databases for Fun and Profit.” In: OSDI 2014. Broomfield, CO: USENIX Association, pp. 449-464. url: https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-zheng_mai. pdf (visited on Nov. 16, 2020 ).Google Scholar
Index Terms
PerSeVerE: persistency semantics for verification under ext4
Recommendations
Taming x86-TSO persistency
We study the formal semantics of non-volatile memory in the x86-TSO architecture. We show that while the explicit persist operations in the recent model of Raad et al. from POPL'20 only enforce order between writes to the non-volatile memory, it is ...
Safe replication through bounded concurrency verification
High-level data types are often associated with semantic invariants that must be preserved by any correct implementation. While having implementations enforce strong guarantees such as linearizability or serializability can often be used to prevent ...
Evolving Ext4 for shingled disks
FAST'17: Proceedings of the 15th Usenix Conference on File and Storage TechnologiesDrive-Managed SMR (Shingled Magnetic Recording) disks offer a plug-compatible higher-capacity replacement for conventional disks. For non-sequential workloads, these disks show bimodal behavior: After a short period of high throughput they enter a ...






Comments