skip to main content
research-article

i3QL: language-integrated live data views

Authors Info & Claims
Published:15 October 2014Publication History
Skip Abstract Section

Abstract

An incremental computation updates its result based on a change to its input, which is often an order of magnitude faster than a recomputation from scratch. In particular, incrementalization can make expensive computations feasible for settings that require short feedback cycles, such as interactive systems, IDEs, or (soft) real-time systems.

This paper presents i3QL, a general-purpose programming language for specifying incremental computations. i3QL provides a declarative SQL-like syntax and is based on incremental versions of operators from relational algebra, enriched with support for general recursion. We integrated i3QL into Scala as a library, which enables programmers to use regular Scala code for non-incremental subcomputations of an i3QL query and to easily integrate incremental computations into larger software projects. To improve performance, i3QL optimizes user-defined queries by applying algebraic laws and partial evaluation. We describe the design and implementation of i3QL and its optimizations, demonstrate its applicability, and evaluate its performance.

Skip Supplemental Material Section

Supplemental Material

References

  1. U. A. Acar, G. E. Blelloch, M. Blume, R. Harper, and K. Tangwongsan. An experimental analysis of self-adjusting computation. ACM Trans. Program. Lang. Syst., 32(1):3:1--3:53, Nov. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. P. Bhatotia, A. Wieder, R. Rodrigues, U. A. Acar, and R. Pasquin. Incoop: Mapreduce for incremental computations. In Proceedings of the 2nd ACM Symposium on Cloud Computing, SOCC '11, pages 7:1--7:14, New York, NY, USA, 2011. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. J. A. Blakeley, N. Coburn, and P.-V. Larson. Updating derived relations: detecting irrelevant and autonomously computable updates. ACM Trans. Database Syst., 14(3):369--400, Sept. 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. J. A. Blakeley, P.-A. Larson, and F. W. Tompa. Efficiently updating materialized views. SIGMOD Rec., 15(2):61--71, June 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Ceri and J. Widom. Deriving production rules for incremental view maintenance. In Proceedings of the 17th International Conference on Very Large Data Bases, VLDB '91, pages 577--589, San Francisco, CA, USA, 1991. Morgan Kaufmann Publishers Inc. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Chaudhuri, R. Krishnamurthy, S. Potamianos, and K. Shim. Optimizing queries with materialized views. In Data Engineering, 1995. Proceedings of the Eleventh International Conference on, pages 190--200, mar 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. N. Chomsky. Three models for the description of language. Information Theory, IRE Transactions on, 2(3):113--124, September 1956.Google ScholarGoogle Scholar
  8. G. Cooper and S. Krishnamurthi. Embedding dynamic dataflow in a call-by-value language. In P. Sestoft, editor, Programming Languages and Systems, volume 3924 of Lecture Notes in Computer Science, pages 294--308. Springer Berlin Heidelberg, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Demetrescu, I. Finocchi, and A. Ribichini. Reactive imperative programming with dataflow constraints. In Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications, OOPSLA '11, pages 407--426, New York, NY, USA, 2011. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Eichberg, M. Kahl, D. Saha, M. Mezini, and K. Ostermann. Automatic incrementalization of prolog based static analyses. In Proceedings of the 9th International Conference on Practical Aspects of Declarative Languages, PADL'07, pages 109--123, Berlin, Heidelberg, 2007. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. B. N. Freeman-Benson. Kaleidoscope: Mixing objects, constraints, and imperative programming. In Proceedings of the European Conference on Object-oriented Programming on Object-oriented Programming Systems, Languages, and Applications, OOPSLA/ECOOP '90, pages 77--88, New York, NY, USA, 1990. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Garcia, A. Izmaylova, and S. Schupp. Extending Scala with database query capability. Journal of Object Technology, 9(4):45--68, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  13. P. G. Giarrusso, K. Ostermann, M. Eichberg, R. Mitschke, T. Rendel, and C. Kästner. Reify your collection queries for modularity and speed! In Proceedings of the 12th Annual International Conference on Aspect-oriented Software Development, AOSD '13, pages 1--12, New York, NY, USA, 2013. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. P. M. D. Gray, L. Kerschberg, P. J. H. King, and A. Poulovassilis. The Functional Approach to Data Management: Modeling, Analyzing and Integrating Heterogeneous Data. Springer Publishing Company, Incorporated, 1st edition, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. T. Grust, M. Mayr, J. Rittinger, and T. Schreiber. Ferry: database-supported program execution. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, SIGMOD '09, pages 1063--1066, New York, NY, USA, 2009. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Gupta and I. Mumick. Materialized Views: Techniques, Implementations, and Applications. MIT Press, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Gupta, I. S. Mumick, and K. A. Ross. Adapting materialized views after redefinitions. SIGMOD Rec., 24(2):211--222, May 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Gupta, I. S. Mumick, and V. S. Subrahmanian. Maintaining views incrementally. In Proceedings of the 1993 ACM SIGMOD international conference on Management of data, SIGMOD '93, pages 157--166, New York, NY, USA, 1993. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. D. Hovemeyer and W. Pugh. Finding bugs is easy. SIGPLAN Not., 39(12):92--106, Dec. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. P. Hudak. Modular domain specific languages and tools. In Proceedings of International Conference on Software Reuse (ICSR), pages 134--142. IEEE, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. Kay. Readings in natural language processing. chapter Algorithm schemata and data structures in syntactic processing, pages 35--70. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. D. Leijen and E. Meijer. Domain specific embedded compilers. In Proceedings of the 2nd conference on Domain-specific languages, DSL '99, pages 109--122, New York, NY, USA, 1999. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. V. Lifschitz. Principles of knowledge representation. chapter Foundations of Logic Programming, pages 69--127. Center for the Study of Language and Information, Stanford, CA, USA, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Y. A. Liu, S. D. Stoller, M. Gorbovitski, T. Rothamel, and Y. E. Liu. Incrementalization across object abstraction. In Proceedings of the 20th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA '05, pages 473--486, New York, NY, USA, 2005. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. LiveLinq Web site. http://www.componentone.com/SuperProducts/LiveLinq/.Google ScholarGoogle Scholar
  26. I. Maier and M. Odersky. Higher-order reactive programming with incremental lists. In Proceedings of the 27th European conference on Object-Oriented Programming, ECOOP'13, pages 707--731, Berlin, Heidelberg, 2013. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. S. Melnik, A. Adya, and P. A. Bernstein. Compiling mappings to bridge applications and databases. ACM Trans. Database Syst., 33(4):22:1--22:50, Dec. 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. L. A. Meyerovich, A. Guha, J. Baskin, G. H. Cooper, M. Greenberg, A. Bromfield, and S. Krishnamurthi. Flapjax: A programming language for Ajax applications. SIGPLAN Not., 44(10):1--20, Oct. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. R. Mitschke, M. Eichberg, M. Mezini, A. Garcia, and I. Macia. Modular specification and checking of structural dependencies. In Proceedings of the 12th Annual International Conference on Aspect-oriented Software Development, AOSD '13, pages 85--96, New York, NY, USA, 2013. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. B. A. Myers, R. G. McDaniel, R. C. Miller, A. S. Ferrency, A. Faulring, B. D. Kyle, A. Mickish, A. Klimovitski, and P. Doane. The Amulet environment: New models for effective user interface software development. IEEE Trans. Softw. Eng., 23(6):347--365, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. V. Nerella, S. Surapaneni, S. Madria, and T. Weigert. Exploring optimization and caching for efficient collection operations. Automated Software Engineering, 21(1):3--40, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. R. Paige. Applications of finite differencing to database integrity control and query/transaction optimization. In Advances in Data Base Theory, pages 171--209, 1982.Google ScholarGoogle Scholar
  33. T. Palpanas, R. Sidle, R. Cochrane, and H. Pirahesh. Incremental maintenance for non-distributive aggregate functions. In Proceedings of the 28th international conference on Very Large Data Bases, VLDB '02, pages 802--813. VLDB Endowment, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. F. Pfenning and C. Elliot. Higher-order abstract syntax. SIGPLAN Not., 23(7):199--208, June 1988. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. X. Qian and G.Wiederhold. Incremental recomputation of active relational expressions. IEEE Transactions on Knowledge and Data Engineering, 3(3):337--341, 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. G. Ramalingam and T. Reps. A categorized bibliography on incremental computation. In Proceedings of the 20th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, POPL '93, pages 502--510, New York, NY, USA, 1993. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. T. Rompf and M. Odersky. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled dsls. SIGPLAN Not., 46(2):127--136, Oct. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. T. Rompf and M. Odersky. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled dsls. Commun. ACM, 55(6):121--130, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. T. Rothamel and Y. A. Liu. Generating incremental implementations of object-set queries. In Proceedings of the 7th International Conference on Generative Programming and Component Engineering, GPCE '08, pages 55--66, New York, NY, USA, 2008. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. N. Roussopoulos. An incremental access method for view-cache: concept, algorithms, and cost analysis. ACM Trans. Database Syst., 16(3):535--563, Sept. 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. K. Sagonas, T. Swift, and D. S. Warren. Xsb as an efficient deductive database engine. SIGMOD Rec., 23(2):442--453, May 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. D. Saha and C. Ramakrishnan. Incremental evaluation of tabled logic programs. In C. Palamidessi, editor, Logic Programming, volume 2916 of Lecture Notes in Computer Science, pages 392--406. Springer Berlin Heidelberg, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  43. D. Saha and C. Ramakrishnan. Symbolic support graph: A space efficient data structure for incremental tabled evaluation. In M. Gabbrielli and G. Gupta, editors, Logic Programming, volume 3668 of Lecture Notes in Computer Science, pages 235--249. Springer Berlin Heidelberg, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. G. Salvaneschi, G. Hintz, and M. Mezini. REScala: Bridging between object-oriented and functional style in reactive applications. In Proceedings of the 13th International Conference on Modularity, MODULARITY '14, pages 25--36, New York, NY, USA, 2014. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. A. Shankar and R. Bodík. Ditto: automatic incrementalization of data structure invariant checks (in Java). In Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation, PLDI '07, pages 310--319, New York, NY, USA, 2007. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. O. Shmueli and A. Itai. Maintenance of views. In Proceedings of the 1984 ACM SIGMOD international conference on Management of data, SIGMOD '84, pages 240--255, New York, NY, USA, 1984. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. O. Sumer, U. Acar, A. T. Ihler, and R. R. Mettu. Efficient bayesian inference for dynamically changing graphs. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 1441--1448. MIT Press, Cambridge, MA, 2008.Google ScholarGoogle Scholar
  48. J. D. Ullman, H. Garcia-Molina, and J. Widom. Database Systems: The Complete Book. Prentice Hall PTR, Upper Saddle River, NJ, USA, 1st edition, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. G.Wachsmuth, G. D. P. Konat, V. A. Vergu, D. M. Groenewegen, and E. Visser. A language independent task engine for incremental name and type analysis. In SLE, volume 8225 of Lecture Notes in Computer Science, pages 260--280. Springer, 2013.Google ScholarGoogle ScholarCross RefCross Ref
  50. T. A. Wagner and S. L. Graham. Efficient and flexible incremental parsing. ACM Trans. Program. Lang. Syst., 20(5):980--1013, Sept. 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. D.Willis, D. J. Pearce, and J. Noble. Efficient object querying for Java. In Proceedings of the 20th European conference on Object-Oriented Programming, ECOOP'06, pages 28--49, Berlin, Heidelberg, 2006. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. D. Willis, D. J. Pearce, and J. Noble. Caching and incrementalisation in the Java query language. SIGPLAN Not., 43(10):1--18, Oct. 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. L. Wong. Kleisli, a functional query system. Journal of Functional Programming, 10:19--56, 0 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. i3QL: language-integrated live data views

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 49, Issue 10
      OOPSLA '14
      October 2014
      907 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2714064
      • Editor:
      • Andy Gill
      Issue’s Table of Contents
      • cover image ACM Conferences
        OOPSLA '14: Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications
        October 2014
        946 pages
        ISBN:9781450325851
        DOI:10.1145/2660193

      Copyright © 2014 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 October 2014

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!