skip to main content
research-article

Online learning of timeout policies for dynamic power management

Published:10 March 2014Publication History
Skip Abstract Section

Abstract

Dynamic power management (DPM) refers to strategies which selectively change the operational states of a device during runtime to reduce the power consumption based on the past usage pattern, the current workload, and the given performance constraint. The power management problem becomes more challenging when the workload exhibits nonstationary behavior which may degrade the performance of any single or static DPM policy.

This article presents a reinforcement learning (RL)-based DPM technique for optimal selection of timeout values in the different device states. Each timeout period determines how long the device will remain in a particular state before the transition decision is taken. The timeout selection is based on workload estimates derived from a Multilayer Artificial Neural Network (ML-ANN) and an objective function given by weighted performance and power parameters. Our DPM approach is further able to adapt the power-performance weights online to meet user-specified power and performance constraints, respectively. We have completely implemented our DPM algorithm on our embedded traffic surveillance platform and performed long-term experiments using real traffic data to demonstrate the effectiveness of the DPM. Our results show that the proposed learning algorithm not only adequately explores the power-performance trade-off with nonstationary workload but can also successfully perform online adjustment of the trade-off parameter in order to meet the user-specified constraint.

References

  1. ACPI. 2011. Advanced Configuration and Power Interface Specification (ACPI). ACPI Specification Document 5. http://www.acpi.info.Google ScholarGoogle Scholar
  2. F. Barrero, S. Toral, M. Vargas, F. Cortés, and J. Milla. 2010. Internet in the development of future road-traffic control systems. Int. Res. 20, 2, 154--168.Google ScholarGoogle Scholar
  3. L. Benini, A. Bogliolo, and G. De Micheli. 1999. Policy optimization for dynamic power management. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 18, 6, 813--833. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. L. Benini, A. Bogliolo, and G. De Micheli. 2000. A survey of design techniques for system-level dynamic power management. IEEE Trans. VLSI Syst. 8, 3, 299--316. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. K. Bhatti, C. Belleudy, and M. Auguin. 2010. Power management in real time embedded systems through online and adaptive interplay of dpm and dvfs policies. In Proceedings of the IEEE/IFIP 8th International Conference on Embedded and Ubiquitous Computing (EUC). 184--191. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. H. Bischof, M. Godec, C. Leistner, B. Rinner, and A. Starzacher. 2010. Autonomous audio-supported learning of visual classifiers for traffic monitoring. IEEE Intell. Syst. 25, 3, 15--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. P. Bogdan and R. Marculescu. 2011a. Cyberphysical systems: Workload modeling and design optimization. IEEE Trans. Des. Test Comput. 28, 4, 78--87. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. P. Bogdan and R. Marculescu. 2011b. Non-stationary traffic analysis and its implications on multicore platform design. IEEE Trans. Comput.-Aid. Des. Integr. Circuits Syst. 30, 4, 508--519. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. P. Bogdan and R. Marculescu. 2011c. Towards a science of cyber-physical systems design. In Proceedings of the IEEE/ACM International Conference on Cyber-Physical Systems (ICCPS). 99--108. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. P. Bogdan, R. Marculescu, S. Jain, and R. T. Gavila. 2012. An optimal control approach to power management for multi-voltage and frequency islands multiprocessor platforms under highly variable workloads. In Proceedings of the IEEE/ACM 6th International Symposium on Networks on Chip (NoCS). 35--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. E. Y. Chung, L. Benini, and G. De Micheli. 1999. Dynamic power management using adaptive learning tree. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 274--279. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. R. David, P. Bogdan, and R. Marculescu. 2012. Dynamic power management for multicores: Case study using the intel SCC. In Proceedings of the IEEE/IFIP 20th International Conference on VLSI and System-on-Chip (VLSI-SoC). 147--152.Google ScholarGoogle Scholar
  13. G. Dhiman and T. S. Rosing. 2006. Dynamic power management using machine learning. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 747--754. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. F. Douglis, P. Krishnan, and B. Bershad. 1995. Adaptive disk spin-down policies for mobile computers. Comput. Syst. 8, 4.Google ScholarGoogle Scholar
  15. J. Durand, S. Girard, and V. Ciriza. 2012. Optimization of power consumption and device availability based on point process modelling of the request sequence. J. Royal Stat. Soci. 3.Google ScholarGoogle Scholar
  16. C. Eui-Young, L. Benini, A. Bogliolo, L. Yung-Hsiang, and G. De-Micheli. 2002. Dynamic power management for nonstationary service requests. IEEE Trans. Comput. 51, 11, 1345--1361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Fallahi and E. Hossain. 2007. QoS provisioning in wireless video sensor networks: A dynamic power management framework. IEEE Trans. Wirel. Commun. 14, 6, 40--49. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. K. Fei, T. Pin, and Q. Shi. 2006. Genetic algorithm based idle length prediction scheme for dynamic power management. In Proceedings of the IMACS Multiconference on Computational Engineering in Systems Applications. 1437--1443.Google ScholarGoogle Scholar
  19. M. Ghasemazar, H. Goudarzi, and M. Pedram. 2012. Robust optimization of a chip multiprocessor's performance under power and thermal constraints. In Proceedings of the IEEE 30th Conference on Computer Design (ICCD). IEEE, 108--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. C. H. Hwang and A. C. H. Wu. 2000. A predictive system shutdown method for energy saving of event-driven computation. ACM Trans. Des. Autom. Electron. Syst. 5, 2, 226--241. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. Hwisung and M. Pedram. 2010. Supervised learning based power management for multicore processors. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 29, 9, 1395--1408. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. S. Irani, S. Shukla, and R. Gupta. 2003. Online strategies for dynamic power management in systems with multiple power-saving states. ACM Trans. Embed. Comput. Syst. 2, 3, 325--346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. N. Kandasamy and S. Abdelwahed. 2004. A control-theoretic approach to power management in embedded processors. In Proceedings of Informatics 9, 1--12.Google ScholarGoogle Scholar
  24. U. A. Khan. 2013. Online learning of timeout policies for dynamic power management. Ph.D. dissertation. Alpen-Adria Universität Klagenfurt, Austria.Google ScholarGoogle Scholar
  25. U. A. Khan, M. Godec, M. Quaritsch, M. Hennecke, H. Bischof, and B. Rinner. 2012. MobiTrick--Mobile traffic checker. In Proceedings of the ITS World Congress.Google ScholarGoogle Scholar
  26. U. A. Khan, M. Quaritsch, and B. Rinner. 2011. Design of a heterogeneous, energy-aware, stereo-vision based sensing platform for traffic surveillance. In Proceedings of the 9th Workshop on Intelligent Solutions in Embedded Systems. 47--52.Google ScholarGoogle Scholar
  27. U. A. Khan and B. Rinner. 2012a. Dynamic power management for portable, multi-camera traffic monitoring. In Proceedings of the IEEE Real Time and Embedded Technology and Applications Symposium.Google ScholarGoogle Scholar
  28. U. A. Khan and B. Rinner. 2012b. A reinforcement learning framework for dynamic power management of a portable, multi-camera traffic monitoring system. In Proceedings of the IEEE Conference on Green Computing and Communications. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Y. H. Lu and G. De Micheli. 2001. Comparing system-level power management policies. IEEE Trans. Des. Test Comput. 18, 2, 10--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. S. Mannor, B. Kveton, S. Siddiqi, and C. H. Yu. 2006. Machine learning for adaptive power management. Auton. Comput. 10, 4, 299--312.Google ScholarGoogle Scholar
  31. C. Moser, J. Chen, and L. Thiele. 2010. Dynamic power management in environmentally powered systems. In Proceedings of the Design Automation Conference (ASP-DAC). IEEE, 81--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. S. Natarajan and P. Tadepalli. 2005. Dynamic preferences in multi-criteria reinforcement learning. In Proceedings of the International Conference on Machine Learning. 601--608. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. U. Y. Ogras, R. Marculescu, D. Marculescu, and E. G. Jung. 2009. Design and management of voltage-frequency island partitioned networks-on-chip. IEEE Trans. VLSI Syst. 17, 3, 330--341. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. C. M. Olsen and C. Narayanaswarni. 2006. PowerNap: An efficient power management scheme for mobile devices. IEEE Trans. Mobile Comput. 5, 7, 816--828. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. A. Paul. 2013. Dynamic power management for ubiquitous network devices. Adv. Sci. Lett. 19, 7, 2046--2049.Google ScholarGoogle ScholarCross RefCross Ref
  36. T. Phit and K. Abe. 2006. Packet inter-arrival time estimation using neural network models. In Proceedings of the Internet Conference.Google ScholarGoogle Scholar
  37. F. Pletzer, R. Tusch, L. Böszörmenyi, and B. Rinner. 2012. Robust traffic state estimation on smart cameras. In Proceedings of the IEEE Conference on Advanced Video and Signal-Based Surveillance. 434--439. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. V. L. Prabha and E. C. Monie. 2007. Hardware architecture of reinforcement learning scheme for dynamic power management in embedded systems. EURASIP J. Embed. Syst. 07, 1, 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Q. Qiu and M. Pedram. 1999. Dynamic power management based on continuous-time Markov decision processes. In Proceedings of the 36th Annual ACM/IEEE Design Automation Conference. 555--561. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Z. Ren, B. H. Krogh, and R. Marculescu. 2005. Hierarchical adaptive dynamic power management. IEEE Trans. Comput. 54, 4, 409--420. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. H. C. Shih and K. Wang. 2012. An adaptive hybrid dynamic power management algorithm for mobile devices. Comput. Netw. 56, 2, 548--565. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. T. Simunic, L. Benini, P. Glynn, and G. De Micheli. 2000. Dynamic power management for portable systems. In Proceedings of the 6th International Conference on Mobile Computing and Networking. 11--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. T. Simunic, L. Benini, P. Glynn, and G. De Micheli. 2001. Event-driven power management. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 20, 7, 840--857. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. M. B. Srivastava, A. P. Chandrakasan, and R. W. Brodersen. 1996. Predictive system shutdown and other architectural techniques for energy efficient programmable computation. IEEE Trans. Integ. VLSI Syst. 4, 1, 42--55. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. R. S. Sutton. 1990. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the International Conference on Machine Learning. 216--224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Y. Tan, W. Liu, and Q. Qiu. 2009. Adaptive power management using reinforcement learning. In Proceedings of the International Conference on Computer-Aided Design. 461--467. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Y. Wang, Q. Xie, A. Ammari, and M. Pedram. 2011b. Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classification. In Proceedings of the Design Automation Conference. 41--46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Z. Wang, H. Wang, X. Chen, and J. Lin. 2011a. Cyber physical systems. J. Chinese Comput. Syst. 32, 5, 881--886.Google ScholarGoogle Scholar
  49. C. J. C. H. Watkins and P. Dayan. 2011. Q-Learning. Machine Learn. 8, 3--4, 279--292. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. H. Young, K. Sung, and C. Ki-Seok. 2010. A predictive dynamic power management technique for embedded mobile devices. IEEE Trans. Consumer Electron. 56, 2, 713--719. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. A. Zafra, E. L. Gibaja, and S. Ventura. 2011. Multiple instance learning with multiple objective genetic programming for Web mining. Appl. Soft Comput. 11, 1, 93--102. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Online learning of timeout policies for dynamic power management

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!