skip to main content

PerSeVerE: persistency semantics for verification under ext4

Published:04 January 2021Publication History
Skip Abstract Section

Abstract

Although ubiquitous, modern filesystems have rather complex behaviours that are hardly understood by programmers and lead to severe software bugs such as data corruption. As a first step to ensure correctness of software performing file I/O, we formalize the semantics of the Linux ext4 filesystem, which we integrate with the weak memory consistency semantics of C/C++. We further develop an effective model checking approach for verifying programs that use the filesystem. In doing so, we discover and report bugs in commonly-used text editors such as vim, emacs and nano.

References

  1. Parosh Abdulla, Stavros Aronis, Bengt Jonsson, and Konstantinos Sagonas ( 2014 ). “Optimal dynamic partial order reduction.” In: POPL 2014. New York, NY, USA: ACM, pp. 373-384. doi: 10.1145/2535838.2535845.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Advanced Format ( 2020 ). url: https://en.wikipedia.org/wiki/Advanced_Format (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  3. Jade Alglave, Luc Maranget, and Michael Tautschnig ( July 2014 ). “Herding Cats: Modelling, Simulation, Testing, and Data Mining for Weak Memory.” In: ACM Trans. Program. Lang. Syst. 36.2, 7 : 1-7 : 74. doi: 10.1145/2627752.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Jef Bonwick (Oct. 2005 ). ZFS: The Last Word in Filesystems. Library Catalog: blogs.oracle.com. url: https://blogs.oracle.com/ bonwick/zfs% 3A-the-last-word-in-filesystems (visited on June 17, 2020 ).Google ScholarGoogle Scholar
  5. James Bornholt, Antoine Kaufmann, Jialin Li, Arvind Krishnamurthy, Emina Torlak, and Xi Wang ( 2016 ). “Specifying and Checking File System Crash-Consistency Models.” In: ASPLOS 2016 44.2, pp. 83-98. doi: 10.1145/2980024.2872406.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Haogang Chen, Daniel Ziegler, Tej Chajed, Adam Chlipala, M. Frans Kaashoek, and Nickolai Zeldovich ( 2015 ). “Using Crash Hoare logic for certifying the FSCQ file system.” In: SOSP 2015. the 25th Symposium. Monterey, California: ACM Press, pp. 18-37. doi: 10.1145/2815400.2815402.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Ran Chen, Martin Clochard, and Claude Marché ( 2016 ). “A Formal Proof of a Unix Path Resolution Algorithm.” In: HAL hal-01406848. url: https://hal.inria.fr/hal-01406848/document (visited on Nov. 16, 2020 ).Google ScholarGoogle Scholar
  8. Copy-on-write ( 2020 ). url: https://en.wikipedia.org/wiki/Copy-on-write (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  9. Heming Cui, Gang Hu, Jingyue Wu, and Junfeng Yang ( 2013 ). “Verifying Systems Rules Using Rule-Directed Symbolic Execution.” In: ASPLOS 2013. Houston, Texas, USA: ACM, pp. 329-342. doi: 10.1145/2451116.2451152.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. GNU Emacs ( 2019 ). GNU Emacs: An extensible, customizable, free/libre text editor-and more. url: https://www.gnu.org/ software/emacs/ (visited on June 15, 2020 ).Google ScholarGoogle Scholar
  11. ext4 benchmarks ( 2012 ). EXT4 File-System Tuning Benchmarks. url: https://www.phoronix.com/scan.php ?page=article& item=ext4_linux35_tuning&num=1 (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  12. Ext4 data loss ( 2009 ). url: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781 (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  13. ext4 Linux kernel ( 2020 ). ext4 Data Structures and Algorithms. url: https://www.kernel.org/doc/html/latest/filesystems/ext4/ index. html (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  14. ext4 corruption ( 2015 ). ext4: Filesystem corruption on panic. url: https://bugs.chromium.org/p/chromium/issues/detail?id= 502898 (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  15. Michalis Kokologiannakis ( July 2020 ). files: improve the backup procedure to ensure no data is lost. url: https://git.savannah. gnu.org/cgit/nano.git/commit/? id=a84cdaaa50a804a8b872f6d468412dadf105b3c5 (visited on July 9, 2020 ).Google ScholarGoogle Scholar
  16. Cormac Flanagan and Patrice Godefroid ( 2005 ). “Dynamic partial-order reduction for model checking software.” In: POPL 2005. New York, NY, USA: ACM, pp. 110-121. doi: 10.1145/1040305.1040315.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Patrice Godefroid ( 1997 ). “Model Checking for Programming Languages using VeriSoft.” In: POPL 1997. Paris, France: ACM, pp. 174-186. doi: 10.1145/263699.263717.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Patrice Godefroid (Mar. 2005 ). “Software Model Checking: The VeriSoft Approach.” In: Form. Meth. Syst. Des. 26.2, pp. 77-101. doi: 10.1007/s10703-005-1489-x.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. JOE ( 2018 ). JOE-Joe's Own Editor. url: https://joe-editor. sourceforge.io (visited on June 15, 2020 ).Google ScholarGoogle Scholar
  20. Rajeev Joshi and Gerard Holzmann (June 11, 2007 ). “A Mini Challenge: Build a Verifiable Filesystem.” In: Formal Asp. Comput. 19, pp. 269-272. doi: 10.1007/s00165-006-0022-3.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Eunsuk Kang and Daniel Jackson ( 2008 ). “Formal Modeling and Analysis of a Flash Filesystem in Alloy.” In: ABZ 2008. Ed. by Egon Börger, Michael Butler, Jonathan P. Bowen, and Paul Boca. Vol. 5238. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 294-308. doi: 10.1007/978-3-540-87603-8_23.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Gabriele Keller, Toby Murray, Sidney Amani, Liam O'Connor, Zilin Chen, Leonid Ryzhyk, Gerwin Klein, and Gernot Heiser ( 2013 ). “File systems deserve verification too!” In: PLOS 2013. Farmington, Pennsylvania: ACM Press, pp. 1-7. doi: 10.1145/2525528.2525530.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Michalis Kokologiannakis, Ilya Kaysin, Azalea Raad, and Viktor Vafeiadis (Jan. 2021 ). “PerSeVerE: Persistency Semantics for Verification under Ext4 (Supplementary Material).” In: url: https://plv.mpi-sws.org/persevere.Google ScholarGoogle Scholar
  24. Michalis Kokologiannakis, Azalea Raad, and Viktor Vafeiadis ( 2019 ). “Model Checking for Weakly Consistent Libraries.” In: PLDI 2019. New York, NY, USA: ACM. doi: 10.1145/3314221.3314609.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Ori Lahav, Viktor Vafeiadis, Jeehoon Kang, Chung-Kil Hur, and Derek Dreyer ( 2017 ). “Repairing Sequential Consistency in C/C++ 11.” In: PLDI 2017. Barcelona, Spain: ACM, pp. 618-632. doi: 10.1145/3062341.3062352.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Linux man pages ( 2020 ). url: http://www.man7.org/linux/man-pages /index.html (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  27. Richard Gooch ( 1999 ). Overview of the Linux Virtual File System. url: https://www.kernel.org/doc/html/latest/filesystems/ vfs. html (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  28. Jayashree Mohan, Ashlie Martinez, Soujanya Ponnapalli, Pandian Raju, and Vijay Chidambaram ( 2018 ). “Finding CrashConsistency Bugs with Bounded Black-Box Crash Testing.” In: OSDI 2018. Carlsbad, CA, USA: USENIX Association, pp. 33-50. url: https://www.usenix.org/system/files/osdi18-mohan. pdf (visited on Nov. 16, 2020 ).Google ScholarGoogle Scholar
  29. Madanlal Musuvathi, Shaz Qadeer, Thomas Ball, Gérard Basler, Piramanayagam Arumuga Nainar, and Iulian Neamtiu ( 2008 ). “Finding and Reproducing Heisenbugs in Concurrent Programs.” In: OSDI 2008. USENIX Association, pp. 267-280. url: https://www.usenix.org/legacy/events/osdi08/tech/full_papers/musuvathi/musuvathi. pdf (visited on Nov. 16, 2020 ).Google ScholarGoogle Scholar
  30. GNU Nano ( 2019 ). The GNU Nano homepage. url: https://nano-editor. org (visited on June 15, 2020 ).Google ScholarGoogle Scholar
  31. Gian Ntzik and Philippa Gardner (Oct. 23, 2015 ). “Reasoning about the POSIX file system: local update and global pathnames.” In: OOPSLA 2015. Pittsburgh, PA, USA: Association for Computing Machinery, pp. 201-220. doi: 10.1145/2814270.2814306.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Daejun Park and Dongkun Shin ( 2017 ). “iJournaling: Fine-Grained Journaling for Improving the Latency of Fsync System Call.” In: pp. 787-798. url: https://www.usenix.org/conference/atc17/technical-sessions/presentation/park.Google ScholarGoogle Scholar
  33. Thanumalayan Sankaranarayana Pillai, Ramnatthan Alagappan, Lanyue Lu, Vijay Chidambaram, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau (Oct. 27, 2017 ). “Application Crash Consistency and Performance with CCFS.” In: ACM Trans. Storage 13.3, pp. 1-29. doi: 10.1145/3119897.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Ramnatthan Alagappan, Samer Al-Kiswany, Andrea C. ArpaciDusseau, and Remzi H. Arpaci-Dusseau (Oct. 2014 ). “All File Systems Are Not Created Equal: On the Complexity of Crafting Crash-Consistent Applications.” In: OSDI 2014. Broomfield, CO: USENIX Association, pp. 433-448. url: https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai.Google ScholarGoogle Scholar
  35. Anton Podkopaev, Ori Lahav, and Viktor Vafeiadis (Jan. 2019 ). “Bridging the Gap Between Programming Languages and Hardware Weak Memory Models.” In: Proc. ACM Program. Lang. 3.POPL, 69 : 1-69 : 31. doi: 10.1145/3290382.Google ScholarGoogle Scholar
  36. POSIX ( 2018 ). The Open Group Base Specifications Issue 7. url: https://pubs.opengroup.org/onlinepubs/9699919799/ (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  37. Vijayan Prabhakaran, Andrea C Arpaci-Dusseau, and Remzi H Arpaci-Dusseau ( 2005 ). “Analysis and Evolution of Journaling File Systems.” In: p. 16. url: https://www.usenix.org/legacy/events/usenix05/tech/general/full_papers/prabhakaran/ prabhakaran.pdf.Google ScholarGoogle Scholar
  38. Azalea Raad and Viktor Vafeiadis (Oct. 2018 ). “Persistence Semantics for Weak Memory: Integrating Epoch Persistency with the TSO Memory Model.” In: Proc. ACM Program. Lang. 2.OOPSLA. doi: 10.1145/3276507.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Azalea Raad, John Wickerson, Gil Neiger, and Viktor Vafeiadis (Dec. 20, 2019a ). “Persistency semantics of the Intel-x86 architecture.” In: Proc. ACM Program. Lang. 4 (POPL), 11 : 1-11 : 31. doi: 10.1145/3371079.Google ScholarGoogle Scholar
  40. Azalea Raad, John Wickerson, and Viktor Vafeiadis (Oct. 10, 2019b ). “Weak Persistency Semantics from the Ground Up.” In: Proc. ACM Program. Lang. 3 (OOPSLA), 135 : 1-135 : 27. doi: 10.1145/3360561.Google ScholarGoogle Scholar
  41. renameio ( 2020 ). url: https://github.com/google/renameio (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  42. Tom Ridge, David Sheets, Thomas Tuerk, Andrea Giugliano, Anil Madhavapeddy, and Peter Sewell ( 2015 ). “SibylFS: formal specification and oracle-based testing for POSIX and real-world file systems.” In: SOSP 2015. Monterey, California: ACM Press, pp. 38-53. doi: 10.1145/2815400.2815411.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Ohad Rodeh, Josef Bacik, and Chris Mason (Aug. 1, 2013 ). “ BTRFS: The Linux B-Tree Filesystem.” In: ACM Trans. Storage 9.3, 9 : 1-9 : 32. doi: 10.1145/2501620.2501623.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Cindy Rubio-González, Haryadi S. Gunawi, Ben Liblit, Remzi H. Arpaci-Dusseau, and Andrea C. Arpaci-Dusseau ( June 15, 2009 ). “Error propagation analysis for file systems.” In: SIGPLAN Not. 44.6, pp. 270-280. doi: 10.1145/1543135.1542506.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Gerhard Schellhorn, Gidon Ernst, Jörg Pfähler, Dominik Haneberg, and Wolfgang Reif ( 2014 ). “Development of a Verified Flash File System.” In: ABZ 2014. Vol. 8477. Berlin, Heidelberg, pp. 9-24. doi: 10.1007/978-3-662-43652-3_2.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Helgi Sigurbjarnarson, James Bornholt, Emina Torlak, and Xi Wang ( 2016 ). “Push-Button Verification of File Systems via Crash Refinement.” In: OSDI 2016. Savannah, GA, USA: USENIX Association, pp. 1-16. url: https://www.usenix.org/ system/files/conference/osdi16/osdi16-sigurbjarnarson.pdf.Google ScholarGoogle Scholar
  47. Seongbae Son, Jinsoo Yoo, and Youjip Won ( 2017 ). “Guaranteeing the Metadata Update Atomicity in EXT4 File system.” In: APSys 2017, pp. 1-8. doi: 10.1145/3124680.3124722.Google ScholarGoogle Scholar
  48. SQLite ( 2020 ). url: https://sqlite.org/index. html (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  49. Atomic Commit In SQLite ( 2020 ). url: https://sqlite.org/atomiccommit. html (visited on May 20, 2020 ).Google ScholarGoogle Scholar
  50. Adam Sweeney ( 1996 ). “Scalability in the XFS file system. ” In: USENIX ATC 1996, pp. 1-14. url: https://www.usenix.org/ legacy/publications/library/proceedings/sd96/sweeney.html.Google ScholarGoogle Scholar
  51. Theodore Y Ts'o and Stephen Tweedie ( 2002 ). “Planned Extensions to the Linux Ext2/Ext3 Filesystem.” In: pp. 235-243. url: http://www.usenix.org/publications/library/proceedings/usenix02/tech/freenix/tso.html.Google ScholarGoogle Scholar
  52. Stephen C Tweedie ( 1998 ). “Journaling the Linux ext2fs Filesystem.” In: LinuxExpo 1998. url: http://e2fsprogs.sourceforge. net/journal-design. pdf (visited on Nov. 16, 2020 ).Google ScholarGoogle Scholar
  53. Vim ( 2019 ). Vim-the ubiquitous text editor. url: https://vim. org (visited on June 15, 2020 ).Google ScholarGoogle Scholar
  54. Junfeng Yang, Can Sar, and Dawson Engler (Nov. 6, 2006 ). “EXPLODE: a lightweight, general system for finding serious storage system errors.” In: OSDI 2006. Seattle, Washington: USENIX Association, pp. 131-146. url: https://www.usenix. org/legacy/event/osdi06/tech/full_papers/yang_junfeng/yang_junfeng. pdf (visited on June 17, 2020 ).Google ScholarGoogle Scholar
  55. Mai Zheng, Joseph Tucek, Dachuan Huang, Elizabeth S Yang, Bill W Zhao, Feng Qin, Mark Lillibridge, and Shashank Singh ( 2014 ). “Torturing Databases for Fun and Profit.” In: OSDI 2014. Broomfield, CO: USENIX Association, pp. 449-464. url: https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-zheng_mai. pdf (visited on Nov. 16, 2020 ).Google ScholarGoogle Scholar

Index Terms

  1. PerSeVerE: persistency semantics for verification under ext4

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!