skip to main content
research-article
Open Access

Fast, sound, and effectively complete dynamic race prediction

Published:20 December 2019Publication History
Skip Abstract Section

Abstract

Writing concurrent programs is highly error-prone due to the nondeterminism in interprocess communication. The most reliable indicators of errors in concurrency are data races, which are accesses to a shared resource that can be executed concurrently. We study the problem of predicting data races in lock-based concurrent programs. The input consists of a concurrent trace t, and the task is to determine all pairs of events of t that constitute a data race. The problem lies at the heart of concurrent verification and has been extensively studied for over three decades. However, existing polynomial-time sound techniques are highly incomplete and can miss simple races.

In this work we develop M2: a new polynomial-time algorithm for this problem, which has no false positives. In addition, our algorithm is complete for input traces that consist of two processes, i.e., it provably detects all races in the trace. We also develop sufficient criteria for detecting completeness dynamically in cases of more than two processes. We make an experimental evaluation of our algorithm on a challenging set of benchmarks taken from recent literature on the topic. Our algorithm soundly reports hundreds of real races, many of which are missed by existing methods. In addition, using our dynamic completeness criteria, M2 concludes that it has detected all races in the benchmark set, hence the reports are both sound and complete. Finally, its running times are comparable, and often smaller than the theoretically fastest, yet highly incomplete, existing methods. To our knowledge, M2 is the first sound algorithm that achieves such a level of performance on both running time and completeness of the reported races.

Skip Supplemental Material Section

Supplemental Material

a17-pavlogiannis.webm

References

  1. Utpal Banerjee, Brian Bliss, Zhiqiang Ma, and Paul Petersen. 2006. A Theory of Data Race Detection. In Proceedings of the 2006 Workshop on Parallel and Distributed Systems: Testing and Debugging (PADTAD ’06). ACM, New York, NY, USA, 69–78. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Stephen M. Blackburn, Robin Garner, Chris Hoffmann, Asjad M. Khang, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony Hosking, Maria Jump, Han Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanović, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. 2006. The DaCapo Benchmarks: Java Benchmarking Development and Analysis. In OOPSLA.Google ScholarGoogle Scholar
  3. Michael D. Bond, Katherine E. Coons, and Kathryn S. McKinley. 2010. PACER: Proportional Detection of Data Races. In Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’10). ACM, New York, NY, USA, 255–268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Feng Chen and Grigore Roşu. 2007. Parametric and Sliced Causality. In Proceedings of the 19th International Conference on Computer Aided Verification (CAV’07). Springer-Verlag, Berlin, Heidelberg, 240–253. http://dl.acm.org/citation.cfm?id= 1770351.1770387Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Feng Chen, Traian Florin Serbanuta, and Grigore Rosu. 2008. jPredictor: A Predictive Runtime Analysis Tool for Java. In Proceedings of the 30th International Conference on Software Engineering (ICSE ’08). ACM, New York, NY, USA, 221–230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jong-Deok Choi, Keunwoo Lee, Alexey Loginov, Robert O’Callahan, Vivek Sarkar, and Manu Sridharan. 2002. Efficient and Precise Datarace Detection for Multithreaded Object-oriented Programs. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI ’02). ACM, New York, NY, USA, 258–269. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Mark Christiaens and Koenraad De Bosschere. 2001. TRaDe: Data Race Detection for Java. In Proceedings of the International Conference on Computational Science-Part II (ICCS ’01). Springer-Verlag, London, UK, UK, 761–770. http://dl.acm.org/ citation.cfm?id=645456.654536Google ScholarGoogle ScholarCross RefCross Ref
  8. Anne Dinning and Edith Schonberg. 1991. Detecting Access Anomalies in Programs with Critical Sections. In Proceedings of the 1991 ACM/ONR Workshop on Parallel and Distributed Debugging (PADD ’91). ACM, New York, NY, USA, 85–96. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Tayfun Elmas, Shaz Qadeer, and Serdar Tasiran. 2007. Goldilocks: A Race and Transaction-aware Java Runtime. In Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’07). ACM, New York, NY, USA, 245–255. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Eitan Farchi, Yarden Nir, and Shmuel Ur. 2003. Concurrent Bug Patterns and How to Test Them. In Proceedings of the 17th International Symposium on Parallel and Distributed Processing (IPDPS ’03). IEEE Computer Society, Washington, DC, USA, 286.2–. http://dl.acm.org/citation.cfm?id=838237.838485Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Azadeh Farzan, P. Madhusudan, and Francesco Sorrentino. 2009. Meta-analysis for Atomicity Violations Under Nested Locking. In Proceedings of the 21st International Conference on Computer Aided Verification (CAV ’09). Springer-Verlag, Berlin, Heidelberg, 248–262. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Peter M. Fenwick. 1994. A new data structure for cumulative frequency tables. Software: Practice and Experience 24, 3 (1994), 327–336. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cormac Flanagan and Stephen N. Freund. 2009. FastTrack: Efficient and Precise Dynamic Race Detection. In Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’09). ACM, New York, NY, USA, 121–133. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Kaan Genç, Jake Roemer, Yufan Xu, and Michael D. Bond. 2019. Dependence-Aware, Unbounded Sound Predictive Race Detection (OOPSLA 2019). To appear.Google ScholarGoogle Scholar
  15. Jim Gray. 1985. Why Do Computers Stop And What Can Be Done About It? Büroautomation (1985), 128–145.Google ScholarGoogle Scholar
  16. D. P. Helmbold, C. E. McDowell, and Jian-Zhong Wang. 1991. Detecting data races from sequential traces. In Proceedings of the Twenty-Fourth Annual Hawaii International Conference on System Sciences, Vol. ii. 408–417 vol.2. Google ScholarGoogle ScholarCross RefCross Ref
  17. Jeff Huang, Patrick O’Neil Meredith, and Grigore Rosu. 2014. Maximal Sound Predictive Race Detection with Control Flow Abstraction. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14). ACM, New York, NY, USA, 337–348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jeff Huang and Lawrence Rauchwerger. 2015. Finding Schedule-sensitive Branches. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). ACM, New York, NY, USA, 439–449. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Vineet Kahlon, Franjo Ivančić, and Aarti Gupta. 2005. Reasoning About Threads Communicating via Locks. In Proceedings of the 17th International Conference on Computer Aided Verification (CAV’05). Springer-Verlag, Berlin, Heidelberg, 505–518. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. KyungHee Kim, Tuba Yavuz-Kahveci, and Beverly A. Sanders. 2009. Precise Data Race Detection in a Relaxed Memory Model Using Heuristic-Based Model Checking. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering (ASE ’09). IEEE Computer Society, Washington, DC, USA, 495–499. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Dileep Kini, Umang Mathur, and Mahesh Viswanathan. 2017. Dynamic Race Prediction in Linear Time. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2017). ACM, New York, NY, USA, 157–170. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM 21, 7 (July 1978), 558–565. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Christopher Lidbury and Alastair F. Donaldson. 2017. Dynamic Race Detection for C++11. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL 2017). ACM, New York, NY, USA, 443–457. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Peng Liu, Omer Tripp, and Xiangyu Zhang. 2016. IPA: Improving Predictive Analysis with Pointer Analysis. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). ACM, New York, NY, USA, 59–69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Shan Lu, Soyeon Park, Eunsoo Seo, and Yuanyuan Zhou. 2008. Learning from Mistakes: A Comprehensive Study on Real World Concurrency Bug Characteristics. SIGOPS Oper. Syst. Rev. 42, 2 (March 2008), 329–339. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Daniel Marino, Madanlal Musuvathi, and Satish Narayanasamy. 2009. LiteRace: Effective Sampling for Lightweight Data-race Detection. In Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’09). ACM, New York, NY, USA, 134–143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Umang Mathur, Dileep Kini, and Mahesh Viswanathan. 2018. What Happens-after the First Race? Enhancing the Predictive Power of Happens-before Based Dynamic Race Detection. Proc. ACM Program. Lang. 2, OOPSLA, Article 145 (Oct. 2018), 29 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Friedemann Mattern. 1989. Virtual Time and Global States of Distributed Systems. In Parallel and Distributed Algorithms: proceedings of the International Workshop on Parallel & Distributed Algorithms, M. Cosnard et. al. (Ed.). Elsevier Science Publishers B. V., 215–226.Google ScholarGoogle Scholar
  29. A Mazurkiewicz. 1987. Trace Theory. In Advances in Petri Nets 1986, Part II on Petri Nets: Applications and Relationships to Other Models of Concurrency. Springer-Verlag New York, Inc., 279–324.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Madanlal Musuvathi, Shaz Qadeer, Thomas Ball, Gerard Basler, Piramanayagam Arumuga Nainar, and Iulian Neamtiu. 2008. Finding and Reproducing Heisenbugs in Concurrent Programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI’08). USENIX Association, Berkeley, CA, USA, 267–280. http: //dl.acm.org/citation.cfm?id=1855741.1855760Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Mayur Naik, Alex Aiken, and John Whaley. 2006. Effective Static Race Detection for Java. In Proceedings of the 27th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’06). ACM, New York, NY, USA, 308–319. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Robert O’Callahan and Jong-Deok Choi. 2003. Hybrid Dynamic Data Race Detection. SIGPLAN Not. 38, 10 (June 2003), 167–178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Andreas Pavlogiannis. 2019. Fast, Sound and Effectively Complete Dynamic Race Detection. (2019). arXiv: arXiv:1901.08857Google ScholarGoogle Scholar
  34. Eli Pozniansky and Assaf Schuster. 2003. Efficient On-the-fly Data Race Detection in Multithreaded C++ Programs. SIGPLAN Not. 38, 10 (June 2003), 179–190. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Polyvios Pratikakis, Jeffrey S. Foster, and Michael Hicks. 2011. LOCKSMITH: Practical Static Race Detection for C. ACM Trans. Program. Lang. Syst. 33, 1, Article 3 (Jan. 2011), 55 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Raghavan Raman, Jisheng Zhao, Vivek Sarkar, Martin Vechev, and Eran Yahav. 2012. Scalable and Precise Dynamic Datarace Detection for Structured Parallelism. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’12). ACM, New York, NY, USA, 531–542. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Jake Roemer, Kaan Genç, and Michael D. Bond. 2018. High-coverage, Unbounded Sound Predictive Race Detection. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2018). ACM, New York, NY, USA, 374–389. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Mahmoud Said, Chao Wang, Zijiang Yang, and Karem Sakallah. 2011. Generating Data Race Witnesses by an SMT-based Analysis. In Proceedings of the Third International Conference on NASA Formal Methods (NFM’11). Springer-Verlag, Berlin, Heidelberg, 313–327. http://dl.acm.org/citation.cfm?id=1986308.1986334Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Stefan Savage, Michael Burrows, Greg Nelson, Patrick Sobalvarro, and Thomas Anderson. 1997. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. ACM Trans. Comput. Syst. 15, 4 (Nov. 1997), 391–411. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. D. Schonberg. 1989. On-the-fly Detection of Access Anomalies. In Proceedings of the ACM SIGPLAN 1989 Conference on Programming Language Design and Implementation (PLDI ’89). ACM, New York, NY, USA, 285–297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Koushik Sen. 2008. Race Directed Random Testing of Concurrent Programs. In Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’08). ACM, New York, NY, USA, 11–21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Koushik Sen, Grigore Roşu, and Gul Agha. 2005. Detecting Errors in Multithreaded Programs by Generalized Predictive Analysis of Executions. In Formal Methods for Open Object-Based Distributed Systems, Martin Steffen and Gianluigi Zavattaro (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 211–226.Google ScholarGoogle Scholar
  43. Yao Shi, Soyeon Park, Zuoning Yin, Shan Lu, Yuanyuan Zhou, Wenguang Chen, and Weimin Zheng. 2010. Do I Use the Wrong Definition?: DeFuse: Definition-use Invariants for Detecting Concurrency and Sequential Bugs. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA ’10). ACM, New York, NY, USA, 160–174. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Yannis Smaragdakis, Jacob Evans, Caitlin Sadowski, Jaeheon Yi, and Cormac Flanagan. 2012. Sound Predictive Race Detection in Polynomial Time. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’12). ACM, New York, NY, USA, 387–400. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. L. A. Smith, J. M. Bull, and J. Obdrzálek. 2001. A Parallel Java Grande Benchmark Suite. In Proceedings of the 2001 ACM/IEEE Conference on Supercomputing (SC ’01). ACM, New York, NY, USA, 8–8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Francesco Sorrentino, Azadeh Farzan, and P. Madhusudan. 2010. PENELOPE: Weaving Threads to Expose Atomicity Violations. In Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE ’10). ACM, New York, NY, USA, 37–46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. SPEC. 2008. SPEC releases free SPECjvm2008 benchmark. (2008). https://www.spec.org/jvm2008/press/release.htmlGoogle ScholarGoogle Scholar
  48. Christoph von Praun and Thomas R. Gross. 2001. Object Race Detection. In Proceedings of the 16th ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA ’01). ACM, New York, NY, USA, 70–82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Jan Wen Voung, Ranjit Jhala, and Sorin Lerner. 2007. RELAY: Static Race Detection on Millions of Lines of Code. In Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering (ESEC-FSE ’07). ACM, New York, NY, USA, 205–214. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Chao Wang, Sudipta Kundu, Malay Ganai, and Aarti Gupta. 2009. Symbolic Predictive Analysis for Concurrent Programs. In Proceedings of the 2Nd World Congress on Formal Methods (FM ’09). Springer-Verlag, Berlin, Heidelberg, 256–272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Misun Yu, Joon-Sang Lee, and Doo-Hwan Bae. 2018. AdaptiveLock: Efficient Hybrid Data Race Detection Based on Real-World Locking Patterns. International Journal of Parallel Programming (04 Jun 2018). Google ScholarGoogle ScholarCross RefCross Ref
  52. Yuan Yu, Tom Rodeheffer, and Wei Chen. 2005. RaceTrack: Efficient Detection of Data Race Conditions via Adaptive Tracking. SIGOPS Oper. Syst. Rev. 39, 5 (Oct. 2005), 221–234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Ke Zhai, Boni Xu, W. K. Chan, and T. H. Tse. 2012. CARISMA: A Context-sensitive Approach to Race-condition Sampleinstance Selection for Multithreaded Applications. In Proceedings of the 2012 International Symposium on Software Testing and Analysis (ISSTA 2012). ACM, New York, NY, USA, 221–231. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Fast, sound, and effectively complete dynamic race prediction

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image Proceedings of the ACM on Programming Languages
          Proceedings of the ACM on Programming Languages  Volume 4, Issue POPL
          January 2020
          1984 pages
          EISSN:2475-1421
          DOI:10.1145/3377388
          Issue’s Table of Contents

          Copyright © 2019 Owner/Author

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 20 December 2019
          Published in pacmpl Volume 4, Issue POPL

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!