Abstract
We present D4, a fast concurrency analysis framework that detects concurrency bugs (e.g., data races and deadlocks) interactively in the programming phase. As developers add, modify, and remove statements, the code changes are sent to D4 to detect concurrency bugs in real time, which in turn provides immediate feedback to the developer of the new bugs. The cornerstone of D4 includes a novel system design and two novel parallel differential algorithms that embrace both change and parallelization for fundamental static analyses of concurrent programs. Both algorithms react to program changes by memoizing the analysis results and only recomputing the impact of a change in parallel. Our evaluation on an extensive collection of large real-world applications shows that D4 efficiently pinpoints concurrency bugs within 100ms on average after a code change, several orders of magnitude faster than both the exhaustive analysis and the state-of-the-art incremental techniques.
Supplemental Material
- Akka Cluster Usage. http://http://doc.akka.io/docs/akka/current/java/ cluster-usage.html .Google Scholar
- Titan Graph Partitioning. http://s3.thinkaurelius.com/docs/titan/0.5. 0/graph-partitioning.html .Google Scholar
- T. J. Watson Libraries for Analysis (WALA). http://wala.sourceforge. net/ .Google Scholar
- RacerD. http://fbinfer.com/docs/racerd.html .Google Scholar
- D4 website. https://github.com/parasol-aser/D4 .Google Scholar
- The Language Server protocol. https://langserver.org/ .Google Scholar
- Zhan, Sheng and Huang, Jeff ECHO: Instantaneous In Situ Race Detection in the IDE. In Proceedings of the International Symposium on the Foundations of Software Engineering (2016), pp. 775–786. Google Scholar
Digital Library
- Blackburn, S. M. and Garner, R. and Hoffman, C. and Khan, A. M. and McKinley, K. S. and Bentzur, R. and Diwan, A. and Feinberg, D. and Frampton, D. and Guyer, S. Z. and Hirzel, M. and Hosking, A. and Jump, M. and Lee, H. and Moss, J. E. B. and Phansalkar, A. and Stefanović, D. and VanDrunen, T. and von Dincklage, D. and Wiedermann, B. The DaCapo Benchmarks: Java Benchmarking Development and Analysis. In Proceedings of the 21st annual ACM SIG-PLAN conference on Object-Oriented Programing, Systems, Languages, and Applications (2006), pp. 169–190. Google Scholar
Digital Library
- Acar, U. A. Self-adjusting Computation. PhD thesis, 2005. Google Scholar
Digital Library
- Acar, U. A., Blelloch, G. E., Blume, M., and Tangwongsan, K. An experimental analysis of self-adjusting computation. In Proceedings of the 27th ACM SIGPLAN Conference on Programming Language Design and Implementation (2006), pp. 96–107. Google Scholar
Digital Library
- Arzt, S., and Bodden, E. Reviser: Efficiently updating ide-/ifds-based data-flow analyses in response to incremental program changes. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, ACM, pp. 288–298. Google Scholar
Digital Library
- Bhatotia, P., Fonseca, P., Acar, U. A., Brandenburg, B. B., and Rodrigues, R. ithreads: A threading library for parallel incremental computation. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (2015), pp. 645–659. Google Scholar
Digital Library
- Burckhardt, S., Kothari, P., Musuvathi, M., and Nagarakatte, S. A randomized scheduler with probabilistic guarantees of finding bugs. In Proceedings of the Fifteenth Edition of ASPLOS on Architectural Support for Programming Languages and Operating Systems, ASPLOS XV, ACM, pp. 167–178. Google Scholar
Digital Library
- Chen, Y., Dunfield, J., and Acar, U. A. Type-directed automatic incrementalization. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (2012), pp. 299–310. Google Scholar
Digital Library
- Edvinsson, M., Lundberg, J., and Löwe, W. Parallel points-to analysis for multi-core machines. In Proceedings of the 6th International Conference on High Performance and Embedded Architectures and Compilers, HiPEAC ’11, ACM, pp. 45–54. Google Scholar
Digital Library
- Engler, D., and Ashcraft, K. Racerx: Effective, static detection of race conditions and deadlocks. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP ’03, ACM, pp. 237–252. Google Scholar
Digital Library
- Flanagan, C., and Freund, S. N. Fasttrack: Efficient and precise dynamic race detection. In Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, ACM, pp. 121–133. Google Scholar
Digital Library
- Fonseca, P., Li, C., and Rodrigues, R. Finding complex concurrency bugs in large multi-threaded applications. In Proceedings of the Sixth Conference on Computer Systems, EuroSys ’11, ACM, pp. 215–228. Google Scholar
Digital Library
- Grove, D., and Chambers, C. A framework for call graph construction algorithms. ACM Trans. Program. Lang. Syst. 23, 6 (2001), 685–746. Google Scholar
Digital Library
- Sridharan, M., Chandra, S., Dolby, J., and Fink, S. J., and Yahav, E. Alias Analysis for Object-oriented Programs. Aliasing in ObjectOriented Programming (2013), pp. 196–232. Google Scholar
Digital Library
- Hardekopf, B., and Lin, C. The ant and the grasshopper: Fast and accurate pointer analysis for millions of lines of code. In Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation (2007), pp. 290–299. Google Scholar
Digital Library
- Jin, G., Zhang, W., Deng, D., Liblit, B., and Lu, S. Automated concurrency-bug fixing. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI’12, USENIX Association, pp. 221–236. Google Scholar
Digital Library
- Kastrinis, G., and Smaragdakis, Y. Efficient and effective handling of exceptions in java points-to analysis. In Proceedings of the 22Nd International Conference on Compiler Construction, CC’13, SpringerVerlag, pp. 41–60. Google Scholar
Digital Library
- Mendez-Lojo, M., Burtscher, M., and Pingali, K. A gpu implementation of inclusion-based points-to analysis. In Proceedings of the 17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP ’12, ACM, pp. 107–116. Google Scholar
Digital Library
- Méndez-Lojo, M., Mathew, A., and Pingali, K. Parallel inclusionbased points-to analysis. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’10, ACM, pp. 428–443. Google Scholar
Digital Library
- Musuvathi, M., Qadeer, S., Ball, T., Basler, G., Nainar, P. A., and Neamtiu, I. Finding and reproducing heisenbugs in concurrent programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, USENIX Association, pp. 267–280. Google Scholar
Digital Library
- Nagaraj, V., and Govindarajan, R. Parallel flow-sensitive pointer analysis by graph-rewriting. In Proceedings of the 22Nd International Conference on Parallel Architectures and Compilation Techniques, PACT ’13, IEEE Press, pp. 19–28. Google Scholar
Digital Library
- Naik, M., Aiken, A., and Whaley, J. Effective static race detection for java. In Proceedings of the 27th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’06, ACM, pp. 308–319. Google Scholar
Digital Library
- Putta, S., and Nasre, R. Parallel replication-based points-to analysis. In Proceedings of the 21st International Conference on Compiler Construction, CC’12, Springer-Verlag, pp. 61–80. Google Scholar
Digital Library
- Saha, D., and Ramakrishnan, C. R. Incremental and demand-driven points-to analysis using logic programming. In Proceedings of the 7th ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming, PPDP ’05, ACM, pp. 117–128. Google Scholar
Digital Library
- Savage, S., Burrows, M., Nelson, G., Sobalvarro, P., and Anderson, T. Eraser: A dynamic data race detector for multithreaded programs. ACM Trans. Comput. Syst. 15, 4 (Nov. 1997), 391–411. Google Scholar
Digital Library
- Sen, K. Race directed random testing of concurrent programs. In Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’08, ACM, pp. 11–21. Google Scholar
Digital Library
- Serebryany, K., and Iskhodzhanov, T. Threadsanitizer: Data race detection in practice. In Proceedings of the Workshop on Binary Instrumentation and Applications, WBIA ’09, ACM, pp. 62–71. Google Scholar
Digital Library
- Shang, L., Lu, Y., and Xue, J. Fast and precise points-to analysis with incremental cfl-reachability summarisation: Preliminary experience. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, ASE 2012, ACM, pp. 270–273. Google Scholar
Digital Library
- Sridharan, M., Gopan, D., Shan, L., and Bodík, R. Demand-driven points-to analysis for java. In Proceedings of the 20th Annual ACM SIG-PLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’05, ACM, pp. 59–76. Google Scholar
Digital Library
- Su, Y., Ye, D., and Xue, J. Parallel pointer analysis with cfl-reachability. In Proceedings of the 2014 Brazilian Conference on Intelligent Systems, BRACIS ’14, IEEE Computer Society, pp. 451–460. Google Scholar
Digital Library
- Szabó, T., Erdweg, S., and Voelter, M. Inca: A dsl for the definition of incremental program analyses. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, ACM, pp. 320–331. Google Scholar
Digital Library
- Terragni, V., Cheung, S.-C., and Zhang, C. Recontest: Effective regression testing of concurrent programs. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (2015), pp. 246–256. Google Scholar
Digital Library
- Voung, J. W., Jhala, R., and Lerner, S. Relay: Static race detection on millions of lines of code. In Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, ESEC-FSE ’07, ACM, pp. 205–214. Google Scholar
Digital Library
- Yu, J., Narayanasamy, S., Pereira, C., and Pokam, G. Maple: A coverage-driven testing tool for multithreaded programs. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’12, ACM, pp. 485–502. Google Scholar
Digital Library
- Zhang, W., Lim, J., Olichandran, R., Scherpelz, J., Jin, G., Lu, S., and Reps, T. Conseq: Detecting concurrency bugs through sequential errors. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, ACM, pp. 251–264. Google Scholar
Digital Library
- Zhou, P., Teodorescu, R., and Zhou, Y. Hard: Hardware-assisted lockset-based race detection. In Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture, HPCA ’07, IEEE Computer Society, pp. 121–132. Google Scholar
Digital Library
- Do, Lisa Nguyen Quang and Ali, Karim and Livshits, Benjamin and Bodden, Eric and Smith, Justin and Murphy-Hill, Emerson Just-in-time Static Analysis. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’17, ACM, pp. 307–317. Google Scholar
Digital Library
- Camil Demetrescu Fully Dynamic Algorithms for Path Problems on Directed Graphs. PhD thesis, 2001.Google Scholar
- Bender, Michael A. and Fineman, Jeremy T. and Gilbert, Seth and Tarjan, Robert E. A New Approach to Incremental Cycle Detection and Related Problems. In ACM Trans. Algorithms (2016), pp. 14:1–14:22. Google Scholar
Digital Library
- Italiano, Giuseppe F. and Nussbaum, Yahav and Sankowski, Piotr and Wulff-Nilsen, Christian Improved Algorithms for Min Cut and Max Flow in Undirected Planar Graphs. In Proceedings of the Forty-third Annual ACM Symposium on Theory of Computing (2011), pp. 313–322. Google Scholar
Digital Library
- Yu, T., Srisa-an, W., and Rothermel, G. SimRT: An automated framework to support regression testing for data races. In Proceedings of the 36th International Conference on Software Engineering (2014), pp. 48–59. Google Scholar
Digital Library
Index Terms
D4: fast concurrency debugging with parallel differential analysis
Recommendations
D4: fast concurrency debugging with parallel differential analysis
PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and ImplementationWe present D4, a fast concurrency analysis framework that detects concurrency bugs (e.g., data races and deadlocks) interactively in the programming phase. As developers add, modify, and remove statements, the code changes are sent to D4 to detect ...
Sound and efficient concurrency bug prediction
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringConcurrency bugs are extremely difficult to detect. Recently, several dynamic techniques achieve sound analysis. M2 is even complete for two threads. It is designed to decide whether two events can occur consecutively. However, real-world concurrency ...
Concurrency Control in Distributed Databases Through Time Intervals and Short-Term Locks
A method for concurrency control in distributed database management systems that increases the level of concurrent execution of transactions, called ordering by serialization numbers (OSN), is proposed. The OSN method works in the certifier model and ...







Comments