Abstract
Program analyses detect errors in code, but when code changes frequently as in an IDE, repeated re-analysis from-scratch is unnecessary: It leads to poor performance unless we give up on precision and recall. Incremental program analysis promises to deliver fast feedback without giving up on precision or recall by deriving a new analysis result from the previous one. However, Datalog and other existing frameworks for incremental program analysis are limited in expressive power: They only support the powerset lattice as representation of analysis results, whereas many practically relevant analyses require custom lattices and aggregation over lattice values. To this end, we present a novel algorithm called DRedL that supports incremental maintenance of recursive lattice-value aggregation in Datalog. The key insight of DRedL is to dynamically recognize increasing replacements of old lattice values by new ones, which allows us to avoid the expensive deletion of the old value. We integrate DRedL into the analysis framework IncA and use IncA to realize incremental implementations of strong-update points-to analysis and string analysis for Java. As our performance evaluation demonstrates, both analyses react to code changes within milliseconds.
Supplemental Material
- Steven Arzt and Eric Bodden. 2014. Reviser: Efficiently Updating IDE-/IFDS-based Data-flow Analyses in Response to Incremental Program Changes. In Proceedings of the 36th International Conference on Software Engineering (ICSE 2014). ACM, New York, NY, USA, 288–298. Google Scholar
Digital Library
- Michael Francis Atiyah and Ian Grant Macdonald. 1994. Introduction to commutative algebra. Westview press.Google Scholar
- Pavel Avgustinov, Oege de Moor, Michael Peyton Jones, and Max Schäfer. 2016. QL: Object-oriented Queries on Relational Data. In LIPIcs-Leibniz International Proceedings in Informatics, Vol. 56. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google Scholar
- Pramod Bhatotia, Alexander Wieder, Rodrigo Rodrigues, Umut A. Acar, and Rafael Pasquin. 2011. Incoop: MapReduce for Incremental Computations. In Proceedings of the 2Nd ACM Symposium on Cloud Computing (SOCC ’11). ACM, New York, NY, USA, Article 7, 14 pages. Google Scholar
Digital Library
- S. Ceri, G. Gottlob, and L. Tanca. 1989. What You Always Wanted to Know About Datalog (And Never Dared to Ask). IEEE Trans. on Knowl. and Data Eng. 1, 1 (March 1989), 146–166. Google Scholar
Digital Library
- Neil Conway, William R. Marczak, Peter Alvaro, Joseph M. Hellerstein, and David Maier. 2012. Logic and Lattices for Distributed Programming. In Proceedings of the Third ACM Symposium on Cloud Computing (SoCC ’12). ACM, New York, NY, USA, Article 1, 14 pages. Google Scholar
Digital Library
- Keith D. Cooper and Ken Kennedy. 1984. Efficient Computation of Flow Insensitive Interprocedural Summary Information. In Proceedings of the 1984 SIGPLAN Symposium on Compiler Construction (SIGPLAN ’84). ACM, New York, NY, USA, 247–258. Google Scholar
Digital Library
- Giulia Costantini, Pietro Ferrara, and Agostino Cortesi. 2011. Static Analysis of String Values. In Proceedings of the 13th International Conference on Formal Methods and Software Engineering (ICFEM’11). Springer-Verlag, Berlin, Heidelberg, 505–521. http://dl.acm.org/citation.cfm?id=2075089.2075132 Google Scholar
Digital Library
- Patrick Cousot and Radhia Cousot. 2004. Basic Concepts of Abstract Interpretation. Springer US, Boston, MA, 359–366.Google Scholar
- Georg Dotzler and Michael Philippsen. 2016. Move-optimized Source Code Tree Differencing. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016). ACM, New York, NY, USA, 660–671. Google Scholar
Digital Library
- Michael Eichberg, Matthias Kahl, Diptikalyan Saha, Mira Mezini, and Klaus Ostermann. 2007. Automatic Incrementalization of Prolog Based Static Analyses. In Proceedings of the 9th International Conference on Practical Aspects of Declarative Languages (PADL’07). Springer-Verlag, Berlin, Heidelberg, 109–123. Google Scholar
Digital Library
- Charles L. Forgy. 1982. Rete: A Fast Algorithm for the Many Pattern/Many Object Pattern Match Problem. Artif. Intell. 19, 1 (Sept. 1982), 17–37. Google Scholar
Digital Library
- Todd J. Green, Shan Shan Huang, Boon Thau Loo, and Wenchao Zhou. 2013. Datalog and Recursive Query Processing. Found. Trends databases 5, 2 (Nov. 2013), 105–195. Google Scholar
Digital Library
- Ashish Gupta and Inderpal Singh Mumick. 1995. Maintenance of Materialized Views: Problems, Techniques, and Applications. IEEE Data Eng. Bull. 18, 2 (1995), 3–18. http://sites.computer.org/debull/95JUN- CD.pdfGoogle Scholar
- Ashish Gupta, Inderpal Singh Mumick, and V. S. Subrahmanian. 1993. Maintaining Views Incrementally. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data (SIGMOD ’93). ACM, New York, NY, USA, 157–166. Google Scholar
Digital Library
- Eric N. Hanson and Mohammed S. Hasan. 1993. Gator: An Optimized Discrimination Network for Active Database Rule Condition Testing. Technical Report TR93-036. Univ. of Florida.Google Scholar
- Manuel Hermenegildo, German Puebla, Kim Marriott, and Peter J. Stuckey. 2000. Incremental Analysis of Constraint Logic Programs. ACM Trans. Program. Lang. Syst. 22, 2 (March 2000), 187–223. Google Scholar
Digital Library
- Usman Ismail. 2009. Incremental call graph construction for the Eclipse IDE. Technical Report CS-2009-07. University of Waterloo.Google Scholar
- Uday Khedker. 1995. A Generalised Theory of Bit Vector Data Flow Analysis. Ph.D. Dissertation. Department of Computer Science and Engineering, IIT Bombay.Google Scholar
- Patrick Lam, Eric Bodden, Ondrej Lhoták, and Laurie Hendren. 2011. The Soot framework for Java program analysis: a retrospective. In Cetus Users and Compiler Infastructure Workshop (CETUS 2011), Vol. 15. 35.Google Scholar
- Ondrej Lhoták and Kwok-Chiang Andrew Chung. 2011. Points-to Analysis with Efficient Strong Updates. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’11). ACM, New York, NY, USA, 3–16. Google Scholar
Digital Library
- M. Liu, N. E. Taylor, W. Zhou, Z. G. Ives, and B. T. Loo. 2009. Recursive Computation of Regions and Connectivity in Networks. In 2009 IEEE 25th International Conference on Data Engineering. 1108–1119. Google Scholar
Digital Library
- Yi Lu, Lei Shang, Xinwei Xie, and Jingling Xue. 2013. An Incremental Points-to Analysis with CFL-Reachability. In Proceedings of the 22Nd International Conference on Compiler Construction (CC’13). Springer-Verlag, Berlin, Heidelberg, 61–81. Google Scholar
Digital Library
- Magnus Madsen, Ming-Ho Yee, and Ondřej Lhoták. 2016. From Datalog to Flix: A Declarative Language for Fixed Points on Lattices. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’16). ACM, New York, NY, USA, 194–208. Google Scholar
Digital Library
- Mirjana Mazuran, Edoardo Serra, and Carlo Zaniolo. 2013. Extending the power of datalog recursion. 22 (08 2013). Google Scholar
Digital Library
- Daniel P. Miranker. 1987. TREAT: A Better Match Algorithm for AI Production Systems; Long Version. (1987).Google Scholar
Digital Library
- Ralf Mitschke, Sebastian Erdweg, Mirko Köhler, Mira Mezini, and Guido Salvaneschi. 2014. i3QL: language-integrated live data views. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2014, part of SPLASH 2014, Portland, OR, USA, October 20-24, 2014, Andrew P. Black and Todd D. Millstein (Eds.). ACM, 417–432. Google Scholar
Digital Library
- Boris Motik, Yavor Nenov, Robert Piro, and Ian Horrocks. 2015. Incremental Update of Datalog Materialisation: The Backward/Forward Algorithm. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI’15). AAAI Press, 1560–1568. http://dl.acm.org/citation.cfm?id=2886521.2886537 Google Scholar
Digital Library
- Flemming Nielson, Hanne R. Nielson, and Chris Hankin. 2010. Principles of Program Analysis. Springer Publishing Company, Incorporated. Google Scholar
Digital Library
- L. L. Pollock and M. L. Soffa. 1989. An Incremental Version of Iterative Data Flow Analysis. IEEE Trans. Softw. Eng. 15, 12 (Dec. 1989), 1537–1549. Google Scholar
Digital Library
- Germán Puebla and Manuel Hermenegildo. 1996. Optimized algorithms for incremental analysis of logic programs. In Static Analysis: Third International Symposium, SAS ’96 Aachen, Germany, September 24–26, 1996 Proceedings, Radhia Cousot and David A. Schmidt (Eds.). Springer Berlin Heidelberg, 270–284. Google Scholar
Digital Library
- G. Ramalingam and Thomas Reps. 1993. A Categorized Bibliography on Incremental Computation. In Proceedings of the 20th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’93). ACM, New York, NY, USA, 502–510. Google Scholar
Digital Library
- Thomas Reps, Susan Horwitz, and Mooly Sagiv. 1995. Precise Interprocedural Dataflow Analysis via Graph Reachability. In Proceedings of the 22Nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’95). ACM, New York, NY, USA, 49–61. Google Scholar
Digital Library
- Kenneth A. Ross and Yehoshua Sagiv. 1992. Monotonic Aggregation in Deductive Databases. In Proceedings of the Eleventh ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS ’92). ACM, New York, NY, USA, 114–126. Google Scholar
Digital Library
- Grzegorz Rozenberg (Ed.). 1997. Handbook of Graph Grammars and Computing by Graph Transformation: Volume I. Foundations. World Scientific Publishing Co., Inc., River Edge, NJ, USA. Google Scholar
Digital Library
- Diptikalyan Saha and C. R. Ramakrishnan. 2005. Incremental and Demand-driven Points-to Analysis Using Logic Programming. In Proceedings of the 7th ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming (PPDP ’05). ACM, New York, NY, USA, 117–128. Google Scholar
Digital Library
- Robert Sedgewick and Kevin Wayne. 2011. Algorithms (4th ed.). Addison-Wesley Professional. Google Scholar
Digital Library
- A. Shkapsky, M. Yang, and C. Zaniolo. 2015. Optimizing recursive queries with monotonic aggregates in DeALS. In 2015 IEEE 31st International Conference on Data Engineering. 867–878.Google Scholar
- Yannis Smaragdakis and Martin Bravenboer. 2011. Using Datalog for Fast and Easy Program Analysis. In Proceedings of the First International Conference on Datalog Reloaded (Datalog’10). Springer-Verlag, Berlin, Heidelberg, 245–251. Google Scholar
Digital Library
- Amie L. Souter and Lori L. Pollock. 2001. Incremental Call Graph Reanalysis for Object-Oriented Software Maintenance. In Proceedings of the IEEE International Conference on Software Maintenance (ICSM’01) (ICSM ’01). IEEE Computer Society, Washington, DC, USA, 682–. Google Scholar
Digital Library
- Johannes Späth, Lisa Nguyen Quang Do, Karim Ali, and Eric Bodden. 2016. Boomerang: Demand-Driven Flow- and Context-Sensitive Pointer Analysis for Java. In 30th European Conference on Object-Oriented Programming (ECOOP 2016) (Leibniz International Proceedings in Informatics (LIPIcs)), Shriram Krishnamurthi and Benjamin S. Lerner (Eds.), Vol. 56. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 22:1–22:26.Google Scholar
- Ozgur Sumer, Umut Acar, Alexander T Ihler, and Ramgopal R Mettu. 2008. Efficient bayesian inference for dynamically changing graphs. In Advances in Neural Information Processing Systems. 1441–1448.Google Scholar
- Tamás Szabó, Sebastian Erdweg, and Markus Voelter. 2016. IncA: A DSL for the Definition of Incremental Program Analyses. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016). ACM, New York, NY, USA, 320–331. Google Scholar
Digital Library
- Zoltán Ujhelyi, Gábor Bergmann, Ábel Hegedüs, Ákos Horváth, Benedek Izsó, István Ráth, Zoltán Szatmári, and Dániel Varró. 2015. EMF-IncQuery: An integrated development environment for live model queries. Science of Computer Programming 98, Part 1, 0 (2015), 80 – 99. Google Scholar
Digital Library
- Markus Voelter, Janet Siegmund, Thorsten Berger, and Bernd Kolb. 2014. Towards User-Friendly Projectional Editors. In Software Language Engineering, Benoît Combemale, DavidJ. Pearce, Olivier Barais, and JurgenJ. Vinju (Eds.). Lecture Notes in Computer Science, Vol. 8706. Springer International Publishing, 41–61.Google Scholar
- Markus Voelter, Tamás Szabó, Sascha Lisson, Bernd Kolb, Sebastian Erdweg, and Thorsten Berger. 2016. Efficient Development of Consistent Projectional Editors Using Grammar Cells. In Proceedings of the 2016 ACM SIGPLAN International Conference on Software Language Engineering (SLE 2016). ACM, New York, NY, USA, 28–40. Google Scholar
Digital Library
- John Whaley and Monica S. Lam. 2004. Cloning-based Context-sensitive Pointer Alias Analysis Using Binary Decision Diagrams. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation (PLDI ’04). ACM, New York, NY, USA, 131–144. Google Scholar
Digital Library
- Jyh-shiarn Yur, Barbara G. Ryder, and William A. Landi. 1999. An Incremental Flow- and Context-sensitive Pointer Aliasing Analysis. In Proceedings of the 21st International Conference on Software Engineering (ICSE ’99). ACM, New York, NY, USA, 442–451. Google Scholar
Digital Library
- Frank Kenneth Zadeck. 1984. Incremental data flow analysis in a structured program editor. Vol. 19. ACM.Google Scholar
Index Terms
Incrementalizing lattice-based program analyses in Datalog
Recommendations
IncA: a DSL for the definition of incremental program analyses
ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software EngineeringProgram analyses support software developers, for example, through error detection, code-quality assurance, and by enabling compiler optimizations and refactorings. To provide real-time feedback to developers within IDEs, an analysis must run ...
Incremental whole-program analysis in Datalog with lattices
PLDI 2021: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and ImplementationIncremental static analyses provide up-to-date analysis results in time proportional to the size of a code change, not the entire code base. This promises fast feedback to programmers in IDEs and when checking in commits. However, existing incremental ...
From Datalog to flix: a declarative language for fixed points on lattices
PLDI '16We present Flix, a declarative programming language for specifying and solving least fixed point problems, particularly static program analyses. Flix is inspired by Datalog and extends it with lattices and monotone functions. Using Flix, implementors ...






Comments