skip to main content
research-article

Efficient load balancing in partitioned queries under random perturbations

Published:04 May 2012Publication History
Skip Abstract Section

Abstract

This work investigates a particular instance of the problem of designing efficient adaptive systems, under the condition that each adaptation decision incurs some nonnegligible cost when enacted. More specifically, we deal with the problem of dynamic, intraquery load balancing in parallel database queries across heterogeneous nodes in a way that takes into account the inherent cost of adaptations and thus avoids both overreacting and deciding when to adapt in a completely heuristic manner. The latter may lead to serious performance degradation in several cases, such as periodic and random imbalances. We follow a control theoretical approach to this problem; more specifically, we propose a multiple-input multiple-output feedback linear quadratic regulation (LQR) controller, which captures the tradeoff between reaching a balanced state and the cost inherent in such adaptations. Our approach, apart from benefitting from and being characterized by a solid theoretical foundation, exhibits better performance than state-of-the-art heuristics in realistic situations, as verified by thorough evaluation.

References

  1. Aström, K. J. and Wittenmark, B. 1995. Adaptive Control. Addison-Wesley, Reading, MA.Google ScholarGoogle Scholar
  2. Balazinska, M., Balakrishnan, H., and Stonebraker, M. 2004. Contract-based load management in federated distributed systems. In Proceedings of the ACM Symposium on Networked Systems Design and Implementation (NSDI). 197--210. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Birdwell, J., Zhong, T., Chiasson, J., Abdallah, C., and Hayat, M. 2006. Resource-constrained load balancing controller for a parallel database. In Proceedings of the American Control Conference.Google ScholarGoogle Scholar
  4. Camponogara, E., Jia, D., Krogh, B., and Talukdar, S. 2002. Distributed model predictive control. IEEE Control Syst. Mag. 22, 1, 44--52.Google ScholarGoogle ScholarCross RefCross Ref
  5. Deshpande, A., Ives, Z. G., and Raman, V. 2007. Adaptive query processing. Found. Trends Datab. 1, 1, 1--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. DeWitt, D. 1992. Parallel database systems: The future of high performance database systems. Comm. ACM 35, 6, 85--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Diao, Y., Hellerstein, J. L., Parekh, S. S., Griffith, R., Kaiser, G. E., and Phung, D. B. 2005a. Self-managing systems: A control theory foundation. Proc. IEEE, 441--448. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Diao, Y., Hellerstein, J. L., Storm, A. J., Surendra, M., Lightstone, S., Parekh, S. S., and Garcia-Arellano, C. 2004. Incorporating cost of control into the design of a load balancing controller. In Proceedings ofthe IEEE Real-Time and Embedded Technology and Applications Symposium. 376--387. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Diao, Y., Wu, C. W., Hellerstein, J. L., Storm, A. J., Surendra, M., Lightstone, S., Parekh, S., Garcia-Arellano, C., Carroll, M., Chu, L., and Colaco, J. 2005b. Comparative studies of load balancing with control and optimization techniques. In Proceedings of the American Control Conference. 1484--1490.Google ScholarGoogle Scholar
  10. Dumont, G. and Huzmezan, M. 2002. Concepts, methods and techniques in adaptive control. In Proceedings of the American Control Conference.Google ScholarGoogle Scholar
  11. Franklin, G., Powell, J., and Workman, M. 1998. Digital Control of Dynamic Systems 3rd Ed. Addison-Wesley, Reading, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Fu, Y., Wang, H., Lu, C., and Chandra, R. S. 2006. Distributed utilization control for real-time clusters with load balancing. In Proceedings of the 27th IEEE International Real-Time Systems Symposium (RTSS). 137--146. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Gedik, B. and Liu, L. 2003. Peercq: A decentralized and self-configuring peer-to-peer information monitoring system. In Proceedings of the IEEE International Conference on Distributed Computer Systems (ICDCS). 490--499. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Gounaris, A., Yfoulis, C., Sakellariou, R., and Dikaiakos, M. D. 2008a. A control theoretical approach to self-optimizing block transfer in web service grids. ACM Trans. Auton. Adapt. Syst. 3, 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Gounaris, A., Yfoulis, C., Sakellariou, R., and Dikaiakos, M. D. 2008b. Robust runtime optimization of data transfer in queries over web services. In Proceedings of the International Conference on Data Engineering (ICDE). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Gounaris, A., Yfoulis, C. A., and Paton, N. W. 2009. An efficient load balancing LQR controller in parallel databases queries under random perturbations. In Proceedings ofthe 3rd IEEE Multi-Conference on Systems and Control (2009).Google ScholarGoogle Scholar
  17. Graefe, G. 1990. Encapsulation of parallelism in the volcano query processing system. In Proceedings of the ACM SIGMOD International Conference on Management of Data. H. Garcia-Molina and H. V. Jagadish, Eds., 102--111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Graefe, G. 1996. Iterators, schedulers, and distributed memory parallelism. Softw. Pract. Exper. 26, 4, 427--452. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Hellerstein, J. L., Diao, Y., Parekh, S., and Tilbury, D. M. 2004. Feedback Control of Computing Systems. John Wiley & Sons. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Ives, Z., Florescu, D., Friedman, M., Levy, A., and Weld, D. 1999. An adaptive query execution system for data integration. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 299--310. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Kephart, J. and Chess, D. 2003. The vision of autonomic computing. IEEE Computer 36, 1, 41--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Kephart, J. and Das, R. 2007. Achieving self-management via utility functions. IEEE Internet Comput. 11, 1, 40--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Lightstone, S., Schiefer, B., Zilio, D., and Kleewein, J. 2003. Autonomic computing for relational databases: The ten-year vision. In Proceedings of the IEEE Workshop Autonomic Computing Principles and Architectures (AUCOPA). 419--424.Google ScholarGoogle Scholar
  24. Lightstone, S., Surendra, M., Diao, Y., Parekh, S. S., Hellerstein, J. L., Rose, K., Storm, A. J., and Garcia-Arellano, C. 2007. Control theory: A foundational technique for self managing databases. In Proceedings of the International Conference on Data Engineering Workshops (ICDE). 395--403. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Lu, H. and Tan, K.-L. 1992. Dynamic and load-balanced task-oriented datbase query processing in parallel systems. In Proceedings ofthe 3rd International Conference on Extending Database Technology (EDBT). 357--372. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Markl, V., Raman, V., Simmen, D., Lohman, G., and Pirahesh, H. 2004. Robust query processing through progressive optimization. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 659--670. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Mehta, M. and DeWitt, D. J. 1995. Managing intra-operator parallelism in parallel database systems. In Proceedings of the 21th International Conference on Very Large Data Bases (VLDB). 382--394. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ng, K. W., Wang, Z., Muntz, R. R., and Nittel, S. 1999. Dynamic query re-optimization. In Proceedings of the International Conference on Statistical and Scientific Database Management (SSDBM). 264--273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Paton, N. W., Chávez, J. B., Chen, M., Raman, V., Swart, G., Narang, I., Yellin, D. M., and Fernandes, A. A. A. 2009. Autonomic query parallelization using non-dedicated computers: an evaluation of adaptivity options. VLDB J. 18, 1, 119--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Rahm, E. and Marek, R. 1995. Dynamic multi-resource load balancing in parallel database systems. In Proceedings of the International Conference on Very Large Data Bases (VLDB). 395--406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Raman, V., Han, W., and Narang, I. 2005. Parallel querying with non-dedicated computers. In Proceedings of the International Conference on Very Large Data Bases (VLDB). 61--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Sampaio, S., Paton, N., Smith, J., and Watson, P. 2006. Measuring and Modelling the Performance of a Parallel ODMG Compliant Object Database Server. Concurr. Computat. Pract. Exper. 18, 1, 63--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Shah, M. A., Hellerstein, J. M., Chandrasekaran, S., and Franklin, M. J. 2003. Flux: An adaptive partitioning operator for continuous query systems. In Proceedings of the International Conference on Data Engineering Workshops (ICDE). 25--36.Google ScholarGoogle Scholar
  34. Wolf, J. L., Yu, P. S., Turek, J., and Dias, D. M. 1993. A parallel hash join algorithm for managing data skew. IEEE Trans. Parall. Distrib. Syst. 4, 12, 1355--1371. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Xing, Y., Zdonik, S. B., and Hwang, J.-H. 2005. Dynamic load distribution in the borealis stream processor. In Proceedings of the International Conference on Data Engineering Workshops (ICDE). 791--802. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Efficient load balancing in partitioned queries under random perturbations

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Autonomous and Adaptive Systems
        ACM Transactions on Autonomous and Adaptive Systems  Volume 7, Issue 1
        Special section on formal methods in pervasive computing, pervasive adaptation, and self-adaptive systems: Models and algorithms
        April 2012
        365 pages
        ISSN:1556-4665
        EISSN:1556-4703
        DOI:10.1145/2168260
        Issue’s Table of Contents

        Copyright © 2012 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 4 May 2012
        • Accepted: 1 November 2010
        • Revised: 1 September 2010
        • Received: 1 May 2009
        Published in taas Volume 7, Issue 1

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!