skip to main content
research-article
Public Access

D4: fast concurrency debugging with parallel differential analysis

Published:11 June 2018Publication History
Skip Abstract Section

Abstract

We present D4, a fast concurrency analysis framework that detects concurrency bugs (e.g., data races and deadlocks) interactively in the programming phase. As developers add, modify, and remove statements, the code changes are sent to D4 to detect concurrency bugs in real time, which in turn provides immediate feedback to the developer of the new bugs. The cornerstone of D4 includes a novel system design and two novel parallel differential algorithms that embrace both change and parallelization for fundamental static analyses of concurrent programs. Both algorithms react to program changes by memoizing the analysis results and only recomputing the impact of a change in parallel. Our evaluation on an extensive collection of large real-world applications shows that D4 efficiently pinpoints concurrency bugs within 100ms on average after a code change, several orders of magnitude faster than both the exhaustive analysis and the state-of-the-art incremental techniques.

Skip Supplemental Material Section

Supplemental Material

p359-liu.webm

References

  1. Akka Cluster Usage. http://http://doc.akka.io/docs/akka/current/java/ cluster-usage.html .Google ScholarGoogle Scholar
  2. Titan Graph Partitioning. http://s3.thinkaurelius.com/docs/titan/0.5. 0/graph-partitioning.html .Google ScholarGoogle Scholar
  3. T. J. Watson Libraries for Analysis (WALA). http://wala.sourceforge. net/ .Google ScholarGoogle Scholar
  4. RacerD. http://fbinfer.com/docs/racerd.html .Google ScholarGoogle Scholar
  5. D4 website. https://github.com/parasol-aser/D4 .Google ScholarGoogle Scholar
  6. The Language Server protocol. https://langserver.org/ .Google ScholarGoogle Scholar
  7. Zhan, Sheng and Huang, Jeff ECHO: Instantaneous In Situ Race Detection in the IDE. In Proceedings of the International Symposium on the Foundations of Software Engineering (2016), pp. 775–786. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Blackburn, S. M. and Garner, R. and Hoffman, C. and Khan, A. M. and McKinley, K. S. and Bentzur, R. and Diwan, A. and Feinberg, D. and Frampton, D. and Guyer, S. Z. and Hirzel, M. and Hosking, A. and Jump, M. and Lee, H. and Moss, J. E. B. and Phansalkar, A. and Stefanović, D. and VanDrunen, T. and von Dincklage, D. and Wiedermann, B. The DaCapo Benchmarks: Java Benchmarking Development and Analysis. In Proceedings of the 21st annual ACM SIG-PLAN conference on Object-Oriented Programing, Systems, Languages, and Applications (2006), pp. 169–190. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Acar, U. A. Self-adjusting Computation. PhD thesis, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Acar, U. A., Blelloch, G. E., Blume, M., and Tangwongsan, K. An experimental analysis of self-adjusting computation. In Proceedings of the 27th ACM SIGPLAN Conference on Programming Language Design and Implementation (2006), pp. 96–107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Arzt, S., and Bodden, E. Reviser: Efficiently updating ide-/ifds-based data-flow analyses in response to incremental program changes. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, ACM, pp. 288–298. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Bhatotia, P., Fonseca, P., Acar, U. A., Brandenburg, B. B., and Rodrigues, R. ithreads: A threading library for parallel incremental computation. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (2015), pp. 645–659. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Burckhardt, S., Kothari, P., Musuvathi, M., and Nagarakatte, S. A randomized scheduler with probabilistic guarantees of finding bugs. In Proceedings of the Fifteenth Edition of ASPLOS on Architectural Support for Programming Languages and Operating Systems, ASPLOS XV, ACM, pp. 167–178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Chen, Y., Dunfield, J., and Acar, U. A. Type-directed automatic incrementalization. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (2012), pp. 299–310. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Edvinsson, M., Lundberg, J., and Löwe, W. Parallel points-to analysis for multi-core machines. In Proceedings of the 6th International Conference on High Performance and Embedded Architectures and Compilers, HiPEAC ’11, ACM, pp. 45–54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Engler, D., and Ashcraft, K. Racerx: Effective, static detection of race conditions and deadlocks. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP ’03, ACM, pp. 237–252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Flanagan, C., and Freund, S. N. Fasttrack: Efficient and precise dynamic race detection. In Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, ACM, pp. 121–133. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Fonseca, P., Li, C., and Rodrigues, R. Finding complex concurrency bugs in large multi-threaded applications. In Proceedings of the Sixth Conference on Computer Systems, EuroSys ’11, ACM, pp. 215–228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Grove, D., and Chambers, C. A framework for call graph construction algorithms. ACM Trans. Program. Lang. Syst. 23, 6 (2001), 685–746. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Sridharan, M., Chandra, S., Dolby, J., and Fink, S. J., and Yahav, E. Alias Analysis for Object-oriented Programs. Aliasing in ObjectOriented Programming (2013), pp. 196–232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hardekopf, B., and Lin, C. The ant and the grasshopper: Fast and accurate pointer analysis for millions of lines of code. In Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation (2007), pp. 290–299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Jin, G., Zhang, W., Deng, D., Liblit, B., and Lu, S. Automated concurrency-bug fixing. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI’12, USENIX Association, pp. 221–236. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Kastrinis, G., and Smaragdakis, Y. Efficient and effective handling of exceptions in java points-to analysis. In Proceedings of the 22Nd International Conference on Compiler Construction, CC’13, SpringerVerlag, pp. 41–60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Mendez-Lojo, M., Burtscher, M., and Pingali, K. A gpu implementation of inclusion-based points-to analysis. In Proceedings of the 17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP ’12, ACM, pp. 107–116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Méndez-Lojo, M., Mathew, A., and Pingali, K. Parallel inclusionbased points-to analysis. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’10, ACM, pp. 428–443. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Musuvathi, M., Qadeer, S., Ball, T., Basler, G., Nainar, P. A., and Neamtiu, I. Finding and reproducing heisenbugs in concurrent programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, USENIX Association, pp. 267–280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Nagaraj, V., and Govindarajan, R. Parallel flow-sensitive pointer analysis by graph-rewriting. In Proceedings of the 22Nd International Conference on Parallel Architectures and Compilation Techniques, PACT ’13, IEEE Press, pp. 19–28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Naik, M., Aiken, A., and Whaley, J. Effective static race detection for java. In Proceedings of the 27th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’06, ACM, pp. 308–319. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Putta, S., and Nasre, R. Parallel replication-based points-to analysis. In Proceedings of the 21st International Conference on Compiler Construction, CC’12, Springer-Verlag, pp. 61–80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Saha, D., and Ramakrishnan, C. R. Incremental and demand-driven points-to analysis using logic programming. In Proceedings of the 7th ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming, PPDP ’05, ACM, pp. 117–128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Savage, S., Burrows, M., Nelson, G., Sobalvarro, P., and Anderson, T. Eraser: A dynamic data race detector for multithreaded programs. ACM Trans. Comput. Syst. 15, 4 (Nov. 1997), 391–411. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Sen, K. Race directed random testing of concurrent programs. In Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’08, ACM, pp. 11–21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Serebryany, K., and Iskhodzhanov, T. Threadsanitizer: Data race detection in practice. In Proceedings of the Workshop on Binary Instrumentation and Applications, WBIA ’09, ACM, pp. 62–71. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Shang, L., Lu, Y., and Xue, J. Fast and precise points-to analysis with incremental cfl-reachability summarisation: Preliminary experience. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, ASE 2012, ACM, pp. 270–273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sridharan, M., Gopan, D., Shan, L., and Bodík, R. Demand-driven points-to analysis for java. In Proceedings of the 20th Annual ACM SIG-PLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’05, ACM, pp. 59–76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Su, Y., Ye, D., and Xue, J. Parallel pointer analysis with cfl-reachability. In Proceedings of the 2014 Brazilian Conference on Intelligent Systems, BRACIS ’14, IEEE Computer Society, pp. 451–460. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Szabó, T., Erdweg, S., and Voelter, M. Inca: A dsl for the definition of incremental program analyses. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, ACM, pp. 320–331. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Terragni, V., Cheung, S.-C., and Zhang, C. Recontest: Effective regression testing of concurrent programs. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (2015), pp. 246–256. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Voung, J. W., Jhala, R., and Lerner, S. Relay: Static race detection on millions of lines of code. In Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, ESEC-FSE ’07, ACM, pp. 205–214. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Yu, J., Narayanasamy, S., Pereira, C., and Pokam, G. Maple: A coverage-driven testing tool for multithreaded programs. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’12, ACM, pp. 485–502. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Zhang, W., Lim, J., Olichandran, R., Scherpelz, J., Jin, G., Lu, S., and Reps, T. Conseq: Detecting concurrency bugs through sequential errors. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, ACM, pp. 251–264. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Zhou, P., Teodorescu, R., and Zhou, Y. Hard: Hardware-assisted lockset-based race detection. In Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture, HPCA ’07, IEEE Computer Society, pp. 121–132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Do, Lisa Nguyen Quang and Ali, Karim and Livshits, Benjamin and Bodden, Eric and Smith, Justin and Murphy-Hill, Emerson Just-in-time Static Analysis. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’17, ACM, pp. 307–317. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Camil Demetrescu Fully Dynamic Algorithms for Path Problems on Directed Graphs. PhD thesis, 2001.Google ScholarGoogle Scholar
  45. Bender, Michael A. and Fineman, Jeremy T. and Gilbert, Seth and Tarjan, Robert E. A New Approach to Incremental Cycle Detection and Related Problems. In ACM Trans. Algorithms (2016), pp. 14:1–14:22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Italiano, Giuseppe F. and Nussbaum, Yahav and Sankowski, Piotr and Wulff-Nilsen, Christian Improved Algorithms for Min Cut and Max Flow in Undirected Planar Graphs. In Proceedings of the Forty-third Annual ACM Symposium on Theory of Computing (2011), pp. 313–322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Yu, T., Srisa-an, W., and Rothermel, G. SimRT: An automated framework to support regression testing for data races. In Proceedings of the 36th International Conference on Software Engineering (2014), pp. 48–59. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. D4: fast concurrency debugging with parallel differential analysis

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!