skip to main content
research-article
Free Access

From datalog rules to efficient programs with time and space guarantees

Authors Info & Claims
Published:26 August 2009Publication History
Skip Abstract Section

Abstract

This article describes a method for transforming any given set of Datalog rules into an efficient specialized implementation with guaranteed worst-case time and space complexities, and for computing the complexities from the rules. The running time is optimal in the sense that only useful combinations of facts that lead to all hypotheses of a rule being simultaneously true are considered, and each such combination is considered exactly once in constant time. The associated space usage may sometimes be reduced using scheduling optimizations to eliminate some summands in the space usage formula. The transformation is based on a general method for algorithm design that exploits fixed-point computation, incremental maintenance of invariants, and combinations of indexed and linked data structures. We apply the method to a number of analysis problems, some with improved algorithm complexities and all with greatly improved algorithm understanding and greatly simplified complexity analysis.

References

  1. Abiteboul, S. 1997. Querying semi-structured data. In Proceedings of the International Conference on Database Theory. 1--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Abiteboul, S., Hull, R., and Vianu, V. 1995. Foundations of Databases. Addison-Wesley, Reading, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Aho, A. V., Hopcroft, J. E., and Ullman, J. D. 1983. Data Structures and Algorithms. Addison-Wesley, Reading, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Aiken, A. 1999. Introduction to set constraint-based program analysis. Sci. Comput. Program. 35, 2-3, 79--111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Andersen, L. O. 1994. Program analysis and specialization for the C programming language. Ph.D. thesis, DIKU, University of Copenhagen.Google ScholarGoogle Scholar
  6. Avgustinov, P., Hajiyev, E., Ongkingco, N., de Moor, O., Sereni, D., Tibble, J., and Verbaere, M. 2007. Semantics of static pointcuts in AspectJ. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL'07). ACM Press, New York, 11--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Bancilhon, F., Maier, D., Sagiv, Y., and Ullman, J. D. 1986. Magic sets and other strange ways to implement logic programs. In Proceedings of the 5th ACM SIGACT-SIGMOD Symposium on Principles of Database Systems. 1--16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Beeri, C. and Bernstein, P. A. 1979. Computational problems related to the design of normal form relational schemas. ACM Trans. Datab. Syst 4, 1, 30--59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Cai, J., Facon, P., Henglein, F., Paige, R., and Schonberg, E. 1991. Type analysis and data structure selection. In Constructing Programs from Specifications, B. Möller, Ed. North-Holland, Amsterdam, 126--164.Google ScholarGoogle Scholar
  10. Cai, J. and Paige, R. 1988. Program derivation by fixed point computation. Sci. Comput. Program. 11, 197--261. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Calvanese, D., DeGiacomo, G., Lenzerini, M., and Vardi, M. 2000. Answering regular path queries using views. In Proceedings of the 16th IEEE International Conference on Data Engineering. 389--398. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Ceri, S., Gottlob, G., and Tanca, L. 1990. Logic Programming and Databases. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Chen, W. and Warren, D. S. 1996. Tabled evaluation with delaying for general logic programs. J. ACM 43, 1, 20--74. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Clarke, D. E., Elien, J.-E., Ellison, C. M., Fredette, M., Morcos, A., and Rivest, R. L. 2001. Certificate chain discovery in SPKI/SDSI. J. Comput. Secur. 9, 4, 285--322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Clarke Jr., E. M., Grumberg, O., and Peled, D. A. 1999. Model Checking. MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Cousot, P. and Cousot, R. 1977. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Conference Record of the 4th Annual ACM Symposium on Principles of Programming Languages. 238--252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Dantsin, E., Eiter, T., Gottlob, G., and Voronkov, A. 2001. Complexity and expressive power of logic programming. ACM Comput. Surv. 33, 3, 374--425. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. de Moor, O., Lacey, D., and Wyk, E. V. 2003. Universal regular path queries. High.-Ord. Symbol. Comput. 16, 1-2, 15--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Ellison, C., Frantz, B., Lampson, B., Rivest, R. L., Thomas, B., and Ylonen, T. 1999. RFC 2693: SPKI Certificate Theory. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Esparza, J., Hansel, D., Rossmanith, P., and Schwoon, S. 2000. Efficient algorithms for model checking pushdown systems. In Proceedings of the 12th International Conference on Computer-Aided Verification. Lecture Notes in Computer Science, vol. 1855. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Ganzinger, H. and McAllester, D. A. 2001. A new meta-complexity theorem for bottom-up logic programs. In Proceedings of the 1st International Joint Conference on Automated Reasoning. Springer-Verlag, Berlin, 514--528. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Ganzinger, H. and McAllester, D. A. 2002. Logical algorithms. In Proceedings of the 18th International Conference on Logic Programming (ICLP'02). Springer-Verlag, 209--223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Gelfond, M. and Lifschitz, V. 1988. The stable model semantics for logic programming. In Proceedings of the 5th International Conference on Logic Programming, R. A. Kowalski and K. Bowen, Eds. MIT Press, Cambridge, MA, 1070--1080.Google ScholarGoogle Scholar
  24. Gottlob, G., Koch, C., and Schulz, K. U. 2006. Conjunctive queries over trees. J. ACM 53, 2, 238--272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Gottlob, G. and Papadimitriou, C. 2003. On the complexity of single-rule datalog queries. Inform. Comput. 183, 1, 104--122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Heintze, N. and Jaffar, J. 1994. Set constraints and set-based analysis. In Proceedings of the 2nd International Workshop on Principles and Practice of Constraint Programming. Lecture Notes in Computer Science, vol. 874. Springer-Verlag, Berlin, 281--298. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Heintze, N. and Tardieu, O. 2001. Ultra-fast aliasing analysis using CLA: A million lines of C code in a second. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. 254--263. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Holzbaur, C., Banda, M. G. D. L., Stuckey, P. J., and Duck, G. J. 2005. Optimizing compilation of constraint handling rules in hal. Theor. Prac. Logic Program. 5, 4-5, 503--531. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Hristova, K. and Liu, Y. A. 2006. Improved algorithm complexities for linear temporal logic model checking of push down systems. In Proceedings of the 7th International Conference on Verification, Model Checking and Abstract Interpretation. Springer-Verlag, Berlin, 190--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Hristova, K., Rothamel, T., Liu, Y. A., and Stoller, S. D. 2007a. Efficient type inference for secure information flow. Tech. rep. DAR 07-35, Computer Science Department, SUNY Stony Brook. A preliminary version of this work appeared in Proceedings of the 2006 ACM SIGPLAN Workshop on Programming Languages and Analysis for Security (PLAS'06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Hristova, K., Tekle, K. T., and Liu, Y. A. 2007b. Efficient trust management policy analysis from rules. In Proceedings of the 9th ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming. 211--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Jaffar, J. and Maher, M. J. 1994. Constraint logic programming: A survey. J. Logic Program. 19/20, 503--581.Google ScholarGoogle ScholarCross RefCross Ref
  33. Jha, S. and Reps, T. W. 2004. Model checking SPKI/SDSI. J. Comput. Secur. 12, 3-4, 317--353. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kifer, M. and Lozinskii, E. L. 1986. A framework for an efficient implementation of deductive databases. In Proceedings of the 6th Advanced Database Symposium. 109--116.Google ScholarGoogle Scholar
  35. Kifer, M. and Lozinskii, E. L. 1990. On compile-time query optimization in deductive databases by means of static filtering. ACM Trans. Datab. Syst. 15, 3, 385--426. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Kolaitis, P. G. and Vardi, M. Y. 1995. On the expressive power of datalog: Tools and a case study. J. Comput. Syst. Sci. 51, 1, 110--134. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Lam, M. S., Whaley, J., Livshits, V. B., Martin, M. C., Avots, D., Carbin, M., and Unkel, C. 2005. Context-sensitive program analysis as database queries. In Proceedings of the 24th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'05). ACM Press, New York, 1--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Leuschel, M. 1998. Logic program specialisation. In Partial Evaluation, J. Hatcliff, T. Æ. Mogensen, and P. Thiemann, Eds. Lecture Notes in Computer Science, vol. 1706. Springer, 155--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Li, N. and Mitchell, J. C. 2003. Datalog with constraints: A foundation for trust management languages. In Proceedings of the 5th International Symposium on Practical Aspects of Declarative Languages. V. Dahl and P. Wadler, Eds. Lecture Notes in Computer Science, vol. 2562. Springer-Verlag, 58--73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Li, Q. and Moon, B. 2001. Indexing and querying XML data for regular path expressions. In Proceedings of the 27th International Conference on Very Large Databases. 361--370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Liu, Y. A. 2000. Efficiency by incrementalization: An introduction. High.-Ord. Symbol. Comput. 13, 4, 289--313. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Liu, Y. A., Li, N., and Stoller, S. D. 2001a. Solving regular tree grammar-based constraints. In Proceedings of the 8th International Static Analysis Symposium. Lecture Notes in Computer Science, vol. 2126. Springer-Verlag, Berlin, 213--233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Liu, Y. A., Rothamel, T., Yu, F., Stoller, S., and Hu, N. 2004. Parametric regular path queries. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. 219--230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Liu, Y. A. and Stoller, S. D. 2002. Program optimization using indexed and recursive data structures. In Proceedings of the ACM SIGPLAN Workshop on Partial Evaluation and Semantics-Based Program Manipulation. 108--118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Liu, Y. A. and Stoller, S. D. 2003. Dynamic programming via static incrementalization. High.-Ord. Symbol. Comput. 16, 1-2, 37--62 (Special issue in Memory of Bob Paige.) Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Liu, Y. A. and Stoller, S. D. 2006. Querying complex graphs. In Proceedings of the 8th International Symposium on Practical Aspects of Declarative Languages. Lecture Notes in Computer Science, vol. 3819. Springer-Verlag, Berlin, 199--214. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Liu, Y. A., Stoller, S. D., and Teitelbaum, T. 1998. Static caching for incremental computation. ACM Trans. Program. Lang. Syst. 20, 3, 546--585. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Liu, Y. A., Stoller, S. D., and Teitelbaum, T. 2001b. Strengthening invariants for efficient computation. Sci. Comput. Program. 41, 2, 139--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Liu, Y. A. and Yu, F. 2002. Solving regular path queries. In Proceedings of the 6th International Conference on Mathematics of Program Construction. Lecture Notes in Computer Science, vol. 2386. Springer-Verlag, Berlin, 195--208. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Lloyd, J. W. and Shepherdson, J. C. 1991. Partial evaluation in logic programming. J. Log. Program. 11, 3&4, 217--242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. McAllester, D. A. 1999. On the complexity analysis of static analyses. In Proceedings of the 6th International Static Analysis Symposium. Lecture Notes in Computer Science, vol. 1694. Springer-Verlag, Berlin, 312--329. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Naughton, J. F. and Ramakrishnan, R. 1991. Bottom-up evaluation of logic programs. In Computational Logic: Essays in Honor of Alan Robinson, J.-L. Lassez and G. Plotkin, Eds. MIT Press, Cambridge, MA, 640--700.Google ScholarGoogle Scholar
  53. Nielson, F., Nielson, H. R., and Seidl, H. 2002. Automatic complexity analysis. In Proceedings of the 11th European Symposium on Programming. Lecture Notes in Computer Science, vol. 2305. Springer-Verlag, Berlin, 243--261. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Paige, R. 1981. Formal Differentiation: A Program Synthesis Technique. Computer Science and Artificial Intelligence, vol. 6. UMI Research Press, Ann Arbor, MI.Google ScholarGoogle Scholar
  55. Paige, R. 1986. Programming with invariants. IEEE Softw. 3, 1, 56--69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Paige, R. 1989. Real-time simulation of a set machine on a RAM. In Proceedings of the International Conference on Computing and Information (ICCI'89). Computing and Information, vol. II Canadian Scholars Press, 69--73.Google ScholarGoogle Scholar
  57. Paige, R. and Koenig, S. 1982. Finite differencing of computable expressions. ACM Trans. Program. Lang. Syst. 4, 3, 402--454. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Ramakrishnan, I. V., Sekar, R. C., and Voronkov, A. 2001. Term indexing. In Handbook of Automated Reasoning, J. A. Robinson and A. Voronkov, Eds. Elsevier and MIT Press, Chap. 26, 1853--1964. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Reps, T. 1998. Program analysis via graph reachability. Inform. Softw. Technol. 40, 11-12, 701--726. Special issue on program slicing.Google ScholarGoogle ScholarCross RefCross Ref
  60. Sagonas, K., Swift, T., and Warren, D. S. 1994. XSB as a deductive database. In Proceedings of the 5th ACM SIGACT-SIGMOD Symposium on Principles of Database Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Schrijvers, T. 2005. Analyses, optimizations and extensions of constraint handling rules. Ph.D. thesis, Katholieke Universiteit Leuven, Belgium.Google ScholarGoogle Scholar
  62. Schwartz, J. T., Dewar, R. B. K., Dubinsky, E., and Schonberg, E. 1986. Programming with Sets: An Introduction to SETL. Springer-Verlag, Berlin. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Snyder, W. K. 1990. The SETL2 Programming Language. Tech. rep. 490, Courant Institute of Mathematical Sciences, New York University.Google ScholarGoogle Scholar
  64. Stoller, S. D. and Liu, Y. A. 2007. Generating efficient security software from policies. In Department of Defense Sponsored Information Security Research: New Methods for Protecting Against Cyber Threats. John Wiley & Sons, New York, 416--424.Google ScholarGoogle Scholar
  65. Tamaki, H. and Sato, T. 1986. OLD resolution with tabulation. In Proceedings of the 3rd International Conference on Logic Programming, E. Shapiro, Ed. Springer-Verlag, Berlin, 84--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Tekle, K. T., Hristova, K., and Liu, Y. A. 2008. Generating specialized rules and programs for demand-driven analysis. In Proceedings of the 12th International Conference on Algebraic Methodology and Software Technology. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Vardi, M. Y. 1982. The complexity of relational query languages (extended abstract). In Proceedings of the 14th Annual ACM Symposium on Theory of Computing (STOC'82). ACM Press, New York, 137--146. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Whaley, J. and Lam, M. S. 2004. Cloning-based context-sensitive pointer alias analysis using binary decision diagrams. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. 131--144. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. From datalog rules to efficient programs with time and space guarantees

                  Recommendations

                  Comments

                  Login options

                  Check if you have access through your login credentials or your institution to get full access on this article.

                  Sign in

                  Full Access

                  PDF Format

                  View or Download as a PDF file.

                  PDF

                  eReader

                  View online with eReader.

                  eReader
                  About Cookies On This Site

                  We use cookies to ensure that we give you the best experience on our website.

                  Learn more

                  Got it!