skip to main content
research-article
Open Access

RacerD: compositional static race detection

Published:24 October 2018Publication History
Skip Abstract Section

Abstract

Automatic static detection of data races is one of the most basic problems in reasoning about concurrency. We present RacerD—a static program analysis for detecting data races in Java programs which is fast, can scale to large code, and has proven effective in an industrial software engineering scenario. To our knowledge, RacerD is the first inter-procedural, compositional data race detector which has been shown to have non-trivial precision and impact. Due to its compositionality, it can analyze code changes quickly, and this allows it to perform continuous reasoning about a large, rapidly changing codebase as part of deployment within a continuous integration ecosystem. In contrast to previous static race detectors, its design favors reporting high-confidence bugs over ensuring their absence. RacerD has been in deployment for over a year at Facebook, where it has flagged over 2500 issues that have been fixed by developers before reaching production. It has been important in enabling the development of new code as well as fixing old code: it helped support conversion of part of the main Facebook Android app from a single-threaded to a multi-threaded architecture. In this paper we describe RacerD’s design, implementation, deployment and impact.

Skip Supplemental Material Section

Supplemental Material

a144-blackshear.webm

References

  1. Robert Atkey and Donald Sannella. 2015. ThreadSafe: Static Analysis for Java Concurrency. ECEASST 72 (2015).Google ScholarGoogle Scholar
  2. Nathaniel Ayewah, David Hovemeyer, J. David Morgenthaler, John Penix, and William Pugh. 2008. Using Static Analysis to Find Bugs. IEEE Software 25, 5 (2008), 22–29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. David Bacon, Joshua Bloch, Jeff Bogda, Cliff Click, Paul Haahr, Doug Lea, Tom May, Jan-Willem Maessen, Jeremy Manson, John D. Mitchell, Kelvin Nilsen, Bill Pugh, and Emin Gün Sirer. 2012. The “Double-Checked Locking is Broken” Declaration. Available online: https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html . (July 2012).Google ScholarGoogle Scholar
  4. Al Bessey, Ken Block, Benjamin Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles-Henri Gros, Asya Kamsky, Scott McPeak, and Dawson R. Engler. 2010. A few billion lines of code later: using static analysis to find bugs in the real world. Commun. ACM 53, 2 (2010), 66–75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Pavol Bielik, Veselin Raychev, and Martin T. Vechev. 2015. Scalable race detection for Android applications. In OOPSLA. ACM, 332–348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Stephen M. Blackburn, Robin Garner, Chris Hoffmann, Asjad M. Khan, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony L. Hosking, Maria Jump, Han Bok Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanovic, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. 2006. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA. ACM, 169–190. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Stephen Brookes and Peter W. O’Hearn. 2016. Concurrent separation logic. SIGLOG News 3, 3 (2016), 47–65. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Cristiano Calcagno and Dino Distefano. 2011. Infer: An Automatic Program Verifier for Memory Safety of C Programs. In NASA Formal Methods (LNCS) , Vol. 6617. Springer, 459–465. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Cristiano Calcagno, Dino Distefano, Jérémy Dubreil, Dominik Gabi, Pieter Hooimeijer, Martino Luca, Peter W. O’Hearn, Irene Papakonstantinou, Jim Purbrick, and Dulma Rodriguez. 2015. Moving Fast with Software Verification. In NASA Formal Methods (LNCS) , Vol. 9058. Springer, 3–11.Google ScholarGoogle Scholar
  10. Cristiano Calcagno, Dino Distefano, Peter W. O’Hearn, and Hongseok Yang. 2011. Compositional Shape Analysis by Means of Bi-Abduction. J. ACM 58, 6 (2011), 26:1–26:66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Jong-Deok Choi, Manish Gupta, Mauricio J. Serrano, Vugranam C. Sreedhar, and Samuel P. Midkiff. 1999. Escape Analysis for Java. In OOPSLA. ACM, 1–19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Andy Chou. 2014. From the Trenches: Static Analysis in Industry. (2014). Invited keynote talk at POPL’14. Available at https://popl.mpi-sws.org/2014/andy.pdf .Google ScholarGoogle Scholar
  13. David G. Clarke and Sophia Drossopoulou. 2002. Ownership, encapsulation and the disjointness of type and effect. In OOPSLA . ACM, 292–310. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ernie Cohen, Markus Dahlweid, Mark A. Hillebrand, Dirk Leinenbach, Michal Moskal, Thomas Santen, Wolfram Schulte, and Stephan Tobies. 2009. VCC: A Practical System for Verifying Concurrent C. In TPHOLs (LNCS), Vol. 5674. Springer, 23–42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Ernie Cohen, Michal Moskal, Wolfram Schulte, and Stephan Tobies. 2008. A Practical Verification Methodology for Concurrent Programs. (2008).Google ScholarGoogle Scholar
  16. Coq Development Team. 2018. The Coq Proof Assistant Reference Manual - Version 8.8.Google ScholarGoogle Scholar
  17. Thomas Dinsdale-Young, Pedro da Rocha Pinto, Kristoffer Just Andersen, and Lars Birkedal. 2017. Caper - Automatic Verification for Fine-Grained Concurrency. In ESOP (LNCS), Vol. 10201. Springer, 420–447.Google ScholarGoogle Scholar
  18. Dawson R. Engler and Ken Ashcraft. 2003. RacerX: effective, static detection of race conditions and deadlocks. In SOSP. ACM, 237–252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Michael D. Ernst, Alberto Lovato, Damiano Macedonio, Fausto Spoto, and Javier Thaine. 2016. Locking discipline inference and checking. In ICSE. IEEE/ACM, 1133–1144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Pietro Ferrara. 2013. A generic static analyzer for multithreaded Java programs. Softw., Pract. Exper. 43, 6 (2013), 663–684.Google ScholarGoogle ScholarCross RefCross Ref
  21. Cormac Flanagan and Stephen N. Freund. 2009. FastTrack: efficient and precise dynamic race detection. In PLDI. ACM, 121–133. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Cormac Flanagan and Patrice Godefroid. 2005. Dynamic partial-order reduction for model checking software. In PLDI. ACM, 110–121. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea. 2006. Java Concurrency in Practice. Addison-Wesley.Google ScholarGoogle Scholar
  24. Mark Harman and Peter W. O’Hearn. 2018. From Start-ups to Scale-ups: Open Problems and Challenges in Static and Dynamic Program Analysis for Testing and Verification (keynote paper). In International Working Conference on Source Code Analysis and Manipulation .Google ScholarGoogle Scholar
  25. DeLesley Hutchins, Aaron Ballman, and Dean Sutherland. 2014. C/C++ Thread Safety Analysis. In SCAM. IEEE, 41–46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Bart Jacobs, Jan Smans, Pieter Philippaerts, Frédéric Vogels, Willem Penninckx, and Frank Piessens. 2011. VeriFast: A Powerful, Sound, Predictable, Fast Verifier for C and Java. In NASA Formal Methods (LNCS), Vol. 6617. Springer, 41–55. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Neil D. Jones and Steven S. Muchnick. 1979. Flow Analysis and Optimization of LISP-like Structures. In POPL. ACM, 244–256. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Robbert Krebbers, Amin Timany, and Lars Birkedal. 2017. Interactive proofs in higher-order concurrent separation logic. In POPL . ACM, 205–217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Bozhen Liu and Jeff Huang. 2018. D4: fast concurrency debugging with parallel differential analysis. In PLDI. ACM, 359–373. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Pallavi Maiya, Aditya Kanade, and Rupak Majumdar. 2014. Race detection for Android applications. In PLDI. ACM, 316–325. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: multi-objective automated testing for Android applications. In ISSTA. ACM, 94–105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Peter Müller, Malte Schwerhoff, and Alexander J. Summers. 2016. Viper: A Verification Infrastructure for Permission-Based Reasoning. In VMCAI (LNCS), Vol. 9583. Springer, 41–62. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Mayur Naik, Alex Aiken, and John Whaley. 2006. Effective static race detection for Java. In PLDI. ACM, 308–319. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Peter W. O’Hearn. 2018a. Continuous Reasoning: Scaling the impact of formal methods. In LICS. IEEE, 13–25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Peter W. O’Hearn. 2018b. Experience developing and deploying concurrency analysis at Facebook. In SAS (LNCS), Vol. 11002. Springer, 56–70.Google ScholarGoogle Scholar
  36. J. Aaron Pendergrass, Susan C. Lee, and C. Durward McDonell. 2013. Theory and Practice of Mechanized Software Analysis. In Johns Hopkins APL Technical Digest, Volume 32, Number 2. 499–508.Google ScholarGoogle Scholar
  37. Mukund Raghothaman, Sulekha Kulkarni, Kihong Heo, and Mayur Naik. 2018. Interactive Program Reasoning using Bayesian Inference. In PLDI. ACM, 722–735. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. 2018. Lessons from Building Static Analysis Tools at Google. Commun. ACM 61, 4 (March 2018), 58–66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Konstantin Serebryany and Timur Iskhodzhanov. 2009. ThreadSanitizer: data race detection in practice. Proceedings of the Workshop on Binary Instrumentation and Applications , 62–71. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Ilya Sergey, Aleksandar Nanevski, and Anindya Banerjee. 2015. Mechanized Verification of Fine-grained Concurrent Programs. In PLDI. ACM, 77–87. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yannis Smaragdakis and George Balatsouras. 2015. Pointer Analysis. Foundations and Trends in Programming Languages 2, 1 (2015), 1–69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Fausto Spoto. 2016. The Julia Static Analyzer for Java. In SAS (LNCS), Vol. 9837. Springer, 39–57.Google ScholarGoogle Scholar
  43. Aaron Turon. 2015. Fearless Concurrency with Rust. (10 April 2015). The Rust Programming Language Blog, available at https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.html .Google ScholarGoogle Scholar
  44. Matt Windsor, Mike Dodds, Ben Simner, and Matthew J. Parkinson. 2017. Starling: Lightweight Concurrency Verification with Views. In CAV (I) (LNCS), Vol. 10426. Springer, 544–569.Google ScholarGoogle Scholar

Index Terms

  1. RacerD: compositional static race detection

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!