Abstract
Automatic static detection of data races is one of the most basic problems in reasoning about concurrency. We present RacerD—a static program analysis for detecting data races in Java programs which is fast, can scale to large code, and has proven effective in an industrial software engineering scenario. To our knowledge, RacerD is the first inter-procedural, compositional data race detector which has been shown to have non-trivial precision and impact. Due to its compositionality, it can analyze code changes quickly, and this allows it to perform continuous reasoning about a large, rapidly changing codebase as part of deployment within a continuous integration ecosystem. In contrast to previous static race detectors, its design favors reporting high-confidence bugs over ensuring their absence. RacerD has been in deployment for over a year at Facebook, where it has flagged over 2500 issues that have been fixed by developers before reaching production. It has been important in enabling the development of new code as well as fixing old code: it helped support conversion of part of the main Facebook Android app from a single-threaded to a multi-threaded architecture. In this paper we describe RacerD’s design, implementation, deployment and impact.
Supplemental Material
- Robert Atkey and Donald Sannella. 2015. ThreadSafe: Static Analysis for Java Concurrency. ECEASST 72 (2015).Google Scholar
- Nathaniel Ayewah, David Hovemeyer, J. David Morgenthaler, John Penix, and William Pugh. 2008. Using Static Analysis to Find Bugs. IEEE Software 25, 5 (2008), 22–29. Google Scholar
Digital Library
- David Bacon, Joshua Bloch, Jeff Bogda, Cliff Click, Paul Haahr, Doug Lea, Tom May, Jan-Willem Maessen, Jeremy Manson, John D. Mitchell, Kelvin Nilsen, Bill Pugh, and Emin Gün Sirer. 2012. The “Double-Checked Locking is Broken” Declaration. Available online: https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html . (July 2012).Google Scholar
- Al Bessey, Ken Block, Benjamin Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles-Henri Gros, Asya Kamsky, Scott McPeak, and Dawson R. Engler. 2010. A few billion lines of code later: using static analysis to find bugs in the real world. Commun. ACM 53, 2 (2010), 66–75. Google Scholar
Digital Library
- Pavol Bielik, Veselin Raychev, and Martin T. Vechev. 2015. Scalable race detection for Android applications. In OOPSLA. ACM, 332–348. Google Scholar
Digital Library
- Stephen M. Blackburn, Robin Garner, Chris Hoffmann, Asjad M. Khan, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony L. Hosking, Maria Jump, Han Bok Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanovic, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. 2006. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA. ACM, 169–190. Google Scholar
Digital Library
- Stephen Brookes and Peter W. O’Hearn. 2016. Concurrent separation logic. SIGLOG News 3, 3 (2016), 47–65. Google Scholar
Digital Library
- Cristiano Calcagno and Dino Distefano. 2011. Infer: An Automatic Program Verifier for Memory Safety of C Programs. In NASA Formal Methods (LNCS) , Vol. 6617. Springer, 459–465. Google Scholar
Digital Library
- Cristiano Calcagno, Dino Distefano, Jérémy Dubreil, Dominik Gabi, Pieter Hooimeijer, Martino Luca, Peter W. O’Hearn, Irene Papakonstantinou, Jim Purbrick, and Dulma Rodriguez. 2015. Moving Fast with Software Verification. In NASA Formal Methods (LNCS) , Vol. 9058. Springer, 3–11.Google Scholar
- Cristiano Calcagno, Dino Distefano, Peter W. O’Hearn, and Hongseok Yang. 2011. Compositional Shape Analysis by Means of Bi-Abduction. J. ACM 58, 6 (2011), 26:1–26:66. Google Scholar
Digital Library
- Jong-Deok Choi, Manish Gupta, Mauricio J. Serrano, Vugranam C. Sreedhar, and Samuel P. Midkiff. 1999. Escape Analysis for Java. In OOPSLA. ACM, 1–19. Google Scholar
Digital Library
- Andy Chou. 2014. From the Trenches: Static Analysis in Industry. (2014). Invited keynote talk at POPL’14. Available at https://popl.mpi-sws.org/2014/andy.pdf .Google Scholar
- David G. Clarke and Sophia Drossopoulou. 2002. Ownership, encapsulation and the disjointness of type and effect. In OOPSLA . ACM, 292–310. Google Scholar
Digital Library
- Ernie Cohen, Markus Dahlweid, Mark A. Hillebrand, Dirk Leinenbach, Michal Moskal, Thomas Santen, Wolfram Schulte, and Stephan Tobies. 2009. VCC: A Practical System for Verifying Concurrent C. In TPHOLs (LNCS), Vol. 5674. Springer, 23–42. Google Scholar
Digital Library
- Ernie Cohen, Michal Moskal, Wolfram Schulte, and Stephan Tobies. 2008. A Practical Verification Methodology for Concurrent Programs. (2008).Google Scholar
- Coq Development Team. 2018. The Coq Proof Assistant Reference Manual - Version 8.8.Google Scholar
- Thomas Dinsdale-Young, Pedro da Rocha Pinto, Kristoffer Just Andersen, and Lars Birkedal. 2017. Caper - Automatic Verification for Fine-Grained Concurrency. In ESOP (LNCS), Vol. 10201. Springer, 420–447.Google Scholar
- Dawson R. Engler and Ken Ashcraft. 2003. RacerX: effective, static detection of race conditions and deadlocks. In SOSP. ACM, 237–252. Google Scholar
Digital Library
- Michael D. Ernst, Alberto Lovato, Damiano Macedonio, Fausto Spoto, and Javier Thaine. 2016. Locking discipline inference and checking. In ICSE. IEEE/ACM, 1133–1144. Google Scholar
Digital Library
- Pietro Ferrara. 2013. A generic static analyzer for multithreaded Java programs. Softw., Pract. Exper. 43, 6 (2013), 663–684.Google Scholar
Cross Ref
- Cormac Flanagan and Stephen N. Freund. 2009. FastTrack: efficient and precise dynamic race detection. In PLDI. ACM, 121–133. Google Scholar
Digital Library
- Cormac Flanagan and Patrice Godefroid. 2005. Dynamic partial-order reduction for model checking software. In PLDI. ACM, 110–121. Google Scholar
Digital Library
- Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea. 2006. Java Concurrency in Practice. Addison-Wesley.Google Scholar
- Mark Harman and Peter W. O’Hearn. 2018. From Start-ups to Scale-ups: Open Problems and Challenges in Static and Dynamic Program Analysis for Testing and Verification (keynote paper). In International Working Conference on Source Code Analysis and Manipulation .Google Scholar
- DeLesley Hutchins, Aaron Ballman, and Dean Sutherland. 2014. C/C++ Thread Safety Analysis. In SCAM. IEEE, 41–46. Google Scholar
Digital Library
- Bart Jacobs, Jan Smans, Pieter Philippaerts, Frédéric Vogels, Willem Penninckx, and Frank Piessens. 2011. VeriFast: A Powerful, Sound, Predictable, Fast Verifier for C and Java. In NASA Formal Methods (LNCS), Vol. 6617. Springer, 41–55. Google Scholar
Digital Library
- Neil D. Jones and Steven S. Muchnick. 1979. Flow Analysis and Optimization of LISP-like Structures. In POPL. ACM, 244–256. Google Scholar
Digital Library
- Robbert Krebbers, Amin Timany, and Lars Birkedal. 2017. Interactive proofs in higher-order concurrent separation logic. In POPL . ACM, 205–217. Google Scholar
Digital Library
- Bozhen Liu and Jeff Huang. 2018. D4: fast concurrency debugging with parallel differential analysis. In PLDI. ACM, 359–373. Google Scholar
Digital Library
- Pallavi Maiya, Aditya Kanade, and Rupak Majumdar. 2014. Race detection for Android applications. In PLDI. ACM, 316–325. Google Scholar
Digital Library
- Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: multi-objective automated testing for Android applications. In ISSTA. ACM, 94–105. Google Scholar
Digital Library
- Peter Müller, Malte Schwerhoff, and Alexander J. Summers. 2016. Viper: A Verification Infrastructure for Permission-Based Reasoning. In VMCAI (LNCS), Vol. 9583. Springer, 41–62. Google Scholar
Digital Library
- Mayur Naik, Alex Aiken, and John Whaley. 2006. Effective static race detection for Java. In PLDI. ACM, 308–319. Google Scholar
Digital Library
- Peter W. O’Hearn. 2018a. Continuous Reasoning: Scaling the impact of formal methods. In LICS. IEEE, 13–25. Google Scholar
Digital Library
- Peter W. O’Hearn. 2018b. Experience developing and deploying concurrency analysis at Facebook. In SAS (LNCS), Vol. 11002. Springer, 56–70.Google Scholar
- J. Aaron Pendergrass, Susan C. Lee, and C. Durward McDonell. 2013. Theory and Practice of Mechanized Software Analysis. In Johns Hopkins APL Technical Digest, Volume 32, Number 2. 499–508.Google Scholar
- Mukund Raghothaman, Sulekha Kulkarni, Kihong Heo, and Mayur Naik. 2018. Interactive Program Reasoning using Bayesian Inference. In PLDI. ACM, 722–735. Google Scholar
Digital Library
- Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. 2018. Lessons from Building Static Analysis Tools at Google. Commun. ACM 61, 4 (March 2018), 58–66. Google Scholar
Digital Library
- Konstantin Serebryany and Timur Iskhodzhanov. 2009. ThreadSanitizer: data race detection in practice. Proceedings of the Workshop on Binary Instrumentation and Applications , 62–71. Google Scholar
Digital Library
- Ilya Sergey, Aleksandar Nanevski, and Anindya Banerjee. 2015. Mechanized Verification of Fine-grained Concurrent Programs. In PLDI. ACM, 77–87. Google Scholar
Digital Library
- Yannis Smaragdakis and George Balatsouras. 2015. Pointer Analysis. Foundations and Trends in Programming Languages 2, 1 (2015), 1–69. Google Scholar
Digital Library
- Fausto Spoto. 2016. The Julia Static Analyzer for Java. In SAS (LNCS), Vol. 9837. Springer, 39–57.Google Scholar
- Aaron Turon. 2015. Fearless Concurrency with Rust. (10 April 2015). The Rust Programming Language Blog, available at https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.html .Google Scholar
- Matt Windsor, Mike Dodds, Ben Simner, and Matthew J. Parkinson. 2017. Starling: Lightweight Concurrency Verification with Views. In CAV (I) (LNCS), Vol. 10426. Springer, 544–569.Google Scholar
Index Terms
RacerD: compositional static race detection
Recommendations
A true positives theorem for a static race detector
RacerD is a static race detector that has been proven to be effective in engineering practice: it has seen thousands of data races fixed by developers before reaching production, and has supported the migration of Facebook's Android app rendering ...
Data-Race-Freedom of Concurrent Programs
APSEC '13: Proceedings of the 2013 20th Asia-Pacific Software Engineering Conference (APSEC) - Volume 01Reasoning about access isolation in a program that uses locks, transactions or both to coordinate accesses to shared memory is complex and error-prone. The programmer must understand when accesses issued to the same memory by distinct threads, under ...
Effective static race detection for Java
Proceedings of the 2006 PLDI ConferenceWe present a novel technique for static race detection in Java programs, comprised of a series of stages that employ a combination of static analyses to successively reduce the pairs of memory accesses potentially involved in a race. We have implemented ...






Comments