skip to main content
10.1145/1265530.1265561acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
Article

Optimization of continuous queries with shared expensive filters

Published:11 June 2007Publication History

ABSTRACT

We consider the problem of optimizing and executing multiple continuous queries, where each query is a conjunction of filters and each filter may occur in multiple queries. When filters are expensive, significant performance gains are achieved by sharing filter evaluations across queries. A shared execution strategy in our scenario can either be fixed, in which filters are evaluated in the same predetermined order for all input, or adaptive, in which the next filter to be evaluated is chosen at runtime based on the results of the filters evaluated so far. We show that as filter costs increase, the best adaptive strategy is superior to any fixed strategy, despite the overhead of adaptivity. We show that itis NP-hard to find the optimal adaptive strategy, even if we are willing to approximate within any factor smaller than m where m is the number of queries. We then present a greedy adaptive execution strategy and show that it approximates the best adaptive strategy to within a factor O(log2m log n) where n is the number of distinct filters. We also give a precomputation technique that can reduce the execution overhead of adaptive strategies.

References

  1. A. Arasu and J. Widom. Resource sharing in continuous sliding-window aggregates. In Proc. of the 2004 Intl. Conf. on Very Large Data Bases, pages 336--347, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. Avnur and J. Hellerstein. Eddies: Continuously adaptive query processing. In Proc. of the 2000 ACM SIGMOD Intl. Conf. on Management of Data, pages 261--272, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom. Models and issues in data stream systems. In Proc. of the 2002 ACM Symp. on Principles of Database Systems, pages 1--16, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. S. Babu et al. Adaptive ordering of pipelined stream filters. In Proc. of the 2004 ACM SIGMOD Intl. Conf. on Management of Data, pages 407--418, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. P. Bizarro, S. Babu, D. DeWitt, and J. Widom. Content-based routing: Different plans for different data. In Proc. of the 2005 Intl. Conf. on Very Large Data Bases, pages 757--768, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Chaudhuri and K. Shim. Optimization of queries with user-defined predicates. ACM Trans. on Database Systems, 24(2):177--228, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. J. Chen, D. J. DeWitt, F. Tian, and Y. Wang. NiagaraCQ: A scalable continuous query system for internet databases. In Proc. of the 2000 ACM SIGMOD Intl. Conf. on Management of Data, pages 379--390, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. N. Dalvi, S. Sanghai, P. Roy, and S. Sudarshan. Pipelining in multi-query optimization. In Proc. of the 2001 ACM Symp. on Principles of Database Systems, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. B. Dean, M. Goemans, and J. Vondrák. Approximating the stochastic knapsack problem: The benefit of adaptivity. In Proc. of the 2004 Annual IEEE Symp. on Foundations of Computer Science, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A. Deshpande, C. Guestrin, S. Madden, J. M. Hellerstein, and W. Hong. Model-driven data acquisition in sensor networks. In Proc. of the 2004 Intl. Conf. on Very Large Data Bases, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. O. Etzioni et al. Efficient information gathering on the internet. In Proc. of the 1996 Annual IEEE Symp. on Foundations of Computer Science, pages 234--243, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Garey and D. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., 1979. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. E. Hanson. The Interval Skip List: A data structure for finding all intervals that overlap a point. In WADS, pages 153--164, 1991.Google ScholarGoogle ScholarCross RefCross Ref
  14. E. Hanson. Rule condition testing and action execution in Ariel. In Proc. of the 1992 ACM SIGMOD Intl. Conf. on Management of Data, pages 49--58, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J. Hellerstein and M. Stonebraker. Predicate migration: Optimizing queries with expensive predicates. In Proc. of the 1993 ACM SIGMOD Intl. Conf. on Management of Data, pages 267--276, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. S. Madden, M. Shah, J. Hellerstein, and V. Raman. Continuously adaptive continuous queries over streams. In Proc. of the 2002 ACM SIGMOD Intl. Conf. on Management of Data, pages 49--60, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. K. Munagala, U. Srivastava, and J. Widom. Optimization of continuous queries with shared expensive filters. Technical report, Stanford University, 2005. Available at http://dbpubs.stanford.edu/pub/2005-36.Google ScholarGoogle Scholar
  19. Open source computer vision library. http://sourceforge.net/projects/ opencvlibrary.Google ScholarGoogle Scholar
  20. M. Skutella and M. Uetz. Scheduling precedence-constrained jobs with stochastic processing times on parallel machines. In Proc. of the 2001 Annual ACM-SIAM Symp. on Discrete Algorithms, pages 589--590, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. R. Strom et al. Gryphon: An information flow based approach to message brokering. In Intl. Symp. on Software Reliability Engineering, 1998.Google ScholarGoogle Scholar
  22. V. Vazirani. Approximation Algorithms. Springer, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. T. Yan and H. Garcia-Molina. The SIFT information dissemination system. ACM Trans. on Database Systems, 24(4):529--565, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Optimization of continuous queries with shared expensive filters

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        PODS '07: Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
        June 2007
        328 pages
        ISBN:9781595936851
        DOI:10.1145/1265530

        Copyright © 2007 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 11 June 2007

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • Article

        Acceptance Rates

        Overall Acceptance Rate476of1,835submissions,26%

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!