skip to main content
research-article
Public Access

Pallas: Semantic-Aware Checking for Finding Deep Bugs in Fast Path

Authors Info & Claims
Published:04 April 2017Publication History
Skip Abstract Section

Abstract

Software optimization is constantly a serious concern for developing high-performance systems. To accelerate the workflow execution of a specific functionality, software developers usually define and implement a fast path to speed up the critical and commonly executed functions in the workflow. However, producing a bug-free fast path is nontrivial. Our study on the Linux kernel discloses that a committed fast path can have up to 19 follow-up patches for bug fixing, and most of them are deep semantic bugs, which are difficult to be pinpointed by existing bug-finding tools.

In this paper, we present such a new category of software bugs based on our fast-path bug study across various system software including virtual memory manager, file systems, network, and device drivers. We investigate their root causes and identify five error-prone aspects in a fast path: path state, trigger condition, path output, fault handling, and assistant data structure. We find that many of the deep bugs can be prevented by applying static analysis incorporating simple semantic information. We extract a set of rules based on our findings and build a toolkit PALLAS to check fast-path bugs. The evaluation results show that PALLAS can effectively reveal fast-path bugs in a variety of systems including Linux kernel, mobile operating system, software-defined networking system, and web browser.

References

  1. S. Amani, A. Hixon, Z. Chen, C. Rizkallah, P. Chubb, L. O'Connor, J. Beeren, Y. Nagashima, J. Lim, T. Sewell, J. Tuong, G. Keller, T. Murray, G. Klein, and G. Heiser. COGENT: Verifying High-Assurance File System Implementations. In ASPLOS'16, Atlanta, GA, Apr. 2016.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. www-androidAndroid Open Source Project. https://source.android.com/index.html.Google ScholarGoogle Scholar
  3. www-chromeChromium: An Open-Source Browser Project. https://www.chromium.org/Home.Google ScholarGoogle Scholar
  4. www-clangclang: a C language family frontend for LLVM. http://clang.llvm.org/.Google ScholarGoogle Scholar
  5. D. Engler and M. Musuvathi. Static Analysis Versus Software Model Checking for Bug Finding. In VMCAI'04, 2004. Google ScholarGoogle ScholarCross RefCross Ref
  6. fastpath-wikiFast Path. https://en.wikipedia.org/wiki/Fast_path.Google ScholarGoogle Scholar
  7. D. Fryer, K. Sun, R. Mahmood, T. Cheng, S. Benjamin, A. Goel, and A. D. Brown. Recon: Verifying file system consistency at runtime. Trans. Storage, 8 (4), Dec. 2012.Google ScholarGoogle Scholar
  8. inode-patchfs: Remove i_cindex from struct inode. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers?id=9fd5746fd3d7838bf6ff991d50f1257057d1156f.Google ScholarGoogle Scholar
  9. H. S. Gunawi, C. Rubio-Gonzalez, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, and B. Liblit. EIO: Error Handling is Occasionally Correct. In FAST'08, 2008.Google ScholarGoogle Scholar
  10. H. S. Gunawi, M. Hao, T. Leesatapornwongsa, T. Patana-anake, T. Do, J. Adityatama, K. J. Eliazar, A. Laksono, J. F. Lukman, V. Martin, and A. D. Satria. What Bugs Live in the Cloud? A Study of 3000Google ScholarGoogle Scholar
  11. Issues in Cloud Systems. In SOCC'14, Seattle, WA, Nov. 2014.Google ScholarGoogle Scholar
  12. C. Hawblitzel, J. Howell, J. R. Lorch, A. Narayan, B. Parno, D. Zhang, and B. Zill. Ironclad Apps: End-to-End Security via Automated Full-System Verification. In OSDI'14, Broomfield, CO, Oct. 2014.Google ScholarGoogle Scholar
  13. J. Huang, X. Zhang, and K. Schwan. Understanding Issue Correlations: A Case Study of the Hadoop System. In SOCC'15, Kohala Coast, HI, Aug. 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. J. Huang, M. K. Qureshi, and K. Schwan. An Evolutionary Study of Linux Memory Management for Fun and Profit. In USENIX ATC'16, Denver, CO, June 2016.Google ScholarGoogle Scholar
  15. A. Hunter. A Brief Introduction to the Design of UBIFS. Technical Report.Google ScholarGoogle Scholar
  16. K. Kelsey, T. Bai, C. Ding, and C. Zhang. Fast Track: A Software System for Speculative Program Optimization. In CGO'09, Seattle, WA, Mar. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. G. Klein, K. Elphinstone, G. Heiser, J. Andronick, D. Cock, P. Derrin, D. Elkaduwe, K. Engelhardt, M. Norrish, R. Kolanski, T. Sewell, H. Tuch, and S. Winwood. seL4: Formal Verification of an OS Kernel. In SOSP'09, Big Sky, Montana, Oct. 2009.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Kogan and E. Petrank. A Methodology for Creating Fast Wait-Free Data Structures. In PPoPP'12, New Orleans, Louisiana, USA, Feb. 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. L. Kuhtz. Model Checking Finite Paths and Trees. PhD thesis, Saarland University, 2010.Google ScholarGoogle Scholar
  20. T. Leesatapornwongsa, M. Hao, P. Joshi, J. F. Lukman, and H. S. Gunawi. SAMC: Semantic-Aware Model Checking for Fast Discovery of Deep Bugs in Cloud Systems. In OSDI'14, Broomfield, CO, Oct. 2014.Google ScholarGoogle Scholar
  21. T. Leesatapornwongsa, J. F. Lukman, S. Lu, and H. S. Gunawi. TaxDC: A Taxonomy of Non-Deterministic Concurrency Bugs in Datacenter Distributed Systems. In ASPLOS'16, Atlanta, GA, Apr. 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. D. Lie, A. Chou, D. Engler, and D. L. Dill. A Simple Method for Extracting Models from Protocol Code. In ISCA'01, 2001. Google ScholarGoogle ScholarCross RefCross Ref
  23. T. A. Limoncelli and D. Hughe. LISA'11 Theme -- DevOps: New Challenges, Proven Values. USENIX; login:, 36 (4), Aug. 2011.Google ScholarGoogle Scholar
  24. X. Liu, C. Kreitz, R. van Renesse, J. Hickey, M. Hayden, K. Birman, and R. Constable. Building Reliable, High-Performance Communication Systems from Components. In SOSP'99, Kiawah Island, SC, Dec. 1999.Google ScholarGoogle Scholar
  25. L. Lu, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, and S. Lu. A Study of Linux File System Evolution. In FAST'13, Feb. 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. S. Lu, S. Park, C. Hu, X. Ma, W. Jiang, Z. Li, R. A. Popa, and Y. Zhou. MUVI: Automatically Inferring Multi-Variable Access Correlations and Detecting Related Semantic and Concurrency Bugs. In SOSP'07, stevenson, Washington, Oct. 2007.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from Mistakes - A Comprehensive Study on Real World Concurrency Bug Characteristics. In ASPLOS'08, Seattle, WA, Mar. 2008.Google ScholarGoogle Scholar
  28. N. Markey and P. Schnoebelen. Model Checking a Path. Technical Report, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  29. D. McNamee, J. Walpole, C. Pu, C. Cowan, C. Krasic, A. Goel, and P. Wagle. Specialization Tools and Techniques for Systematic Optimization of System Software. ACM Transactions on Computer Systems, 19 (2). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. C. Min, S. Kashyap, B. Lee, C. Song, and T. Kim. Cross-checking Semantic Correctness: The Case of Finding File System Bugs. In SOSP'15, Monterey, CA, Oct. 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. mm-zonemm: page_alloc: spill to remote nodes before waking kswapd. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/mm/page_alloc.c?id=3a025760fc158b3726eac89ee95d7f29599e9dfa.Google ScholarGoogle Scholar
  32. prefer-patchmm:fix deferred congestion timeout if preferred zone is not allowed. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f33261d75b88f55a08e6a9648cef73509979bfba.Google ScholarGoogle Scholar
  33. memcontrol-patchmm/memcontrol.c: fix uninitialized variable use in mem_cgroup_move_parent(). https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/mm/memcontrol.c?id=8dba474f034c322d96ada39cb20cac711d80dcb2.Google ScholarGoogle Scholar
  34. D. Mosberger and L. L. Peterson. Making Paths Explicit in the Scout Operating System. In OSDI'96, Oct. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. net-corenet: Check rps\_flow\_table when RPS map length is 1. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/net?id=8587523640441a9ff2564ebc6efeb39497ad6709.Google ScholarGoogle Scholar
  36. key-patchnet: tcp: add key management to congestion control. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/net?id=c5c6a8ab45ec0f18733afb4aaade0d4a139d80b3.Google ScholarGoogle Scholar
  37. inode-structure-patchnfsd/create race fixes, infrastructure. http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/fs/inode.c?id=261bca86ed4f7f391d1938167624e78da61dcc6b.Google ScholarGoogle Scholar
  38. ocfs2OCFS2 - Oracle Cluster File System for Linux. http://www.oracle.com/us/technologies/linux/025995.htm.Google ScholarGoogle Scholar
  39. ocfs2-patchocfs2: fix disk file size and memory file size mismatch. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/fs?id=ce170828e24959c69e7a40364731edc0535c550f.Google ScholarGoogle Scholar
  40. P. Olivier, J. Boukhobza, and E. Senn. On Benchmarking Embedded Linux Flash File Systems. Technical Report.Google ScholarGoogle Scholar
  41. www-ovsProduction Quality, Multilayer Open Virtual Switch. http://openvswitch.org/.Google ScholarGoogle Scholar
  42. C. Pu, T. Autrey, A. Black, C. Consel, C. Cowan, J. Inouye, L. Kethana, J. Walpole, and K. Zhang. Optimistic Incremental Specialization: Streamlining a Commercial Operating System. In SOSP'95, CO, USA, Dec. 1995.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. frozen-patchslub: Add frozen check in_\_slab\_alloc. http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/mm/slub.c?id=507effeaba29bf724dfe38317fbd11d0fe25fa40.Google ScholarGoogle Scholar
  44. tcp-output-patchtcp: Fix slab corruption with ipv6 and tcp6fuzz. http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/net/ipv4/tcp_input.c?id=9ae27e0adbf471c7a6b80102e38e1d5a346b3b38.Google ScholarGoogle Scholar
  45. www-rfc793Transmission Control Protocol. https://tools.ietf.org/html/rfc793.Google ScholarGoogle Scholar
  46. W. Xu, S. Kumar, and K. Li. Fast Paths in Concurrent Programs. In PACT'04, 2004.Google ScholarGoogle Scholar
  47. J. Yang, P. Twohey, D. Engler, and M. Musuvathi. Using Model Checking to Find Serious File System Errors. In OSDI'04, San Francisco, CA, Dec. 2004.Google ScholarGoogle Scholar
  48. J. Yang, C. Sar, and D. Engler. EXPLODE: A Lightweight, General System for Finding Serious Storage System Errors. In OSDI'06, Seattle, WA, Nov. 2006.Google ScholarGoogle Scholar
  49. D. Yuan, Y. Luo, X. Zhuang, G. R. Rodrigues, X. Zhao, Y. Zhang, P. U. Jain, and M. Stumm. Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems. In OSDI'14, Broomfield, CO, Oct. 2014.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Pallas: Semantic-Aware Checking for Finding Deep Bugs in Fast Path

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 52, Issue 4
        ASPLOS '17
        April 2017
        811 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/3093336
        Issue’s Table of Contents
        • cover image ACM Conferences
          ASPLOS '17: Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems
          April 2017
          856 pages
          ISBN:9781450344654
          DOI:10.1145/3037697

        Copyright © 2017 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 4 April 2017

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!