Abstract
Dynamic power management (DPM) refers to strategies which selectively change the operational states of a device during runtime to reduce the power consumption based on the past usage pattern, the current workload, and the given performance constraint. The power management problem becomes more challenging when the workload exhibits nonstationary behavior which may degrade the performance of any single or static DPM policy.
This article presents a reinforcement learning (RL)-based DPM technique for optimal selection of timeout values in the different device states. Each timeout period determines how long the device will remain in a particular state before the transition decision is taken. The timeout selection is based on workload estimates derived from a Multilayer Artificial Neural Network (ML-ANN) and an objective function given by weighted performance and power parameters. Our DPM approach is further able to adapt the power-performance weights online to meet user-specified power and performance constraints, respectively. We have completely implemented our DPM algorithm on our embedded traffic surveillance platform and performed long-term experiments using real traffic data to demonstrate the effectiveness of the DPM. Our results show that the proposed learning algorithm not only adequately explores the power-performance trade-off with nonstationary workload but can also successfully perform online adjustment of the trade-off parameter in order to meet the user-specified constraint.
- ACPI. 2011. Advanced Configuration and Power Interface Specification (ACPI). ACPI Specification Document 5. http://www.acpi.info.Google Scholar
- F. Barrero, S. Toral, M. Vargas, F. Cortés, and J. Milla. 2010. Internet in the development of future road-traffic control systems. Int. Res. 20, 2, 154--168.Google Scholar
- L. Benini, A. Bogliolo, and G. De Micheli. 1999. Policy optimization for dynamic power management. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 18, 6, 813--833. Google Scholar
Digital Library
- L. Benini, A. Bogliolo, and G. De Micheli. 2000. A survey of design techniques for system-level dynamic power management. IEEE Trans. VLSI Syst. 8, 3, 299--316. Google Scholar
Digital Library
- K. Bhatti, C. Belleudy, and M. Auguin. 2010. Power management in real time embedded systems through online and adaptive interplay of dpm and dvfs policies. In Proceedings of the IEEE/IFIP 8th International Conference on Embedded and Ubiquitous Computing (EUC). 184--191. Google Scholar
Digital Library
- H. Bischof, M. Godec, C. Leistner, B. Rinner, and A. Starzacher. 2010. Autonomous audio-supported learning of visual classifiers for traffic monitoring. IEEE Intell. Syst. 25, 3, 15--23. Google Scholar
Digital Library
- P. Bogdan and R. Marculescu. 2011a. Cyberphysical systems: Workload modeling and design optimization. IEEE Trans. Des. Test Comput. 28, 4, 78--87. Google Scholar
Digital Library
- P. Bogdan and R. Marculescu. 2011b. Non-stationary traffic analysis and its implications on multicore platform design. IEEE Trans. Comput.-Aid. Des. Integr. Circuits Syst. 30, 4, 508--519. Google Scholar
Digital Library
- P. Bogdan and R. Marculescu. 2011c. Towards a science of cyber-physical systems design. In Proceedings of the IEEE/ACM International Conference on Cyber-Physical Systems (ICCPS). 99--108. Google Scholar
Digital Library
- P. Bogdan, R. Marculescu, S. Jain, and R. T. Gavila. 2012. An optimal control approach to power management for multi-voltage and frequency islands multiprocessor platforms under highly variable workloads. In Proceedings of the IEEE/ACM 6th International Symposium on Networks on Chip (NoCS). 35--42. Google Scholar
Digital Library
- E. Y. Chung, L. Benini, and G. De Micheli. 1999. Dynamic power management using adaptive learning tree. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 274--279. Google Scholar
Digital Library
- R. David, P. Bogdan, and R. Marculescu. 2012. Dynamic power management for multicores: Case study using the intel SCC. In Proceedings of the IEEE/IFIP 20th International Conference on VLSI and System-on-Chip (VLSI-SoC). 147--152.Google Scholar
- G. Dhiman and T. S. Rosing. 2006. Dynamic power management using machine learning. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 747--754. Google Scholar
Digital Library
- F. Douglis, P. Krishnan, and B. Bershad. 1995. Adaptive disk spin-down policies for mobile computers. Comput. Syst. 8, 4.Google Scholar
- J. Durand, S. Girard, and V. Ciriza. 2012. Optimization of power consumption and device availability based on point process modelling of the request sequence. J. Royal Stat. Soci. 3.Google Scholar
- C. Eui-Young, L. Benini, A. Bogliolo, L. Yung-Hsiang, and G. De-Micheli. 2002. Dynamic power management for nonstationary service requests. IEEE Trans. Comput. 51, 11, 1345--1361. Google Scholar
Digital Library
- A. Fallahi and E. Hossain. 2007. QoS provisioning in wireless video sensor networks: A dynamic power management framework. IEEE Trans. Wirel. Commun. 14, 6, 40--49. Google Scholar
Digital Library
- K. Fei, T. Pin, and Q. Shi. 2006. Genetic algorithm based idle length prediction scheme for dynamic power management. In Proceedings of the IMACS Multiconference on Computational Engineering in Systems Applications. 1437--1443.Google Scholar
- M. Ghasemazar, H. Goudarzi, and M. Pedram. 2012. Robust optimization of a chip multiprocessor's performance under power and thermal constraints. In Proceedings of the IEEE 30th Conference on Computer Design (ICCD). IEEE, 108--114. Google Scholar
Digital Library
- C. H. Hwang and A. C. H. Wu. 2000. A predictive system shutdown method for energy saving of event-driven computation. ACM Trans. Des. Autom. Electron. Syst. 5, 2, 226--241. Google Scholar
Digital Library
- J. Hwisung and M. Pedram. 2010. Supervised learning based power management for multicore processors. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 29, 9, 1395--1408. Google Scholar
Digital Library
- S. Irani, S. Shukla, and R. Gupta. 2003. Online strategies for dynamic power management in systems with multiple power-saving states. ACM Trans. Embed. Comput. Syst. 2, 3, 325--346. Google Scholar
Digital Library
- N. Kandasamy and S. Abdelwahed. 2004. A control-theoretic approach to power management in embedded processors. In Proceedings of Informatics 9, 1--12.Google Scholar
- U. A. Khan. 2013. Online learning of timeout policies for dynamic power management. Ph.D. dissertation. Alpen-Adria Universität Klagenfurt, Austria.Google Scholar
- U. A. Khan, M. Godec, M. Quaritsch, M. Hennecke, H. Bischof, and B. Rinner. 2012. MobiTrick--Mobile traffic checker. In Proceedings of the ITS World Congress.Google Scholar
- U. A. Khan, M. Quaritsch, and B. Rinner. 2011. Design of a heterogeneous, energy-aware, stereo-vision based sensing platform for traffic surveillance. In Proceedings of the 9th Workshop on Intelligent Solutions in Embedded Systems. 47--52.Google Scholar
- U. A. Khan and B. Rinner. 2012a. Dynamic power management for portable, multi-camera traffic monitoring. In Proceedings of the IEEE Real Time and Embedded Technology and Applications Symposium.Google Scholar
- U. A. Khan and B. Rinner. 2012b. A reinforcement learning framework for dynamic power management of a portable, multi-camera traffic monitoring system. In Proceedings of the IEEE Conference on Green Computing and Communications. Google Scholar
Digital Library
- Y. H. Lu and G. De Micheli. 2001. Comparing system-level power management policies. IEEE Trans. Des. Test Comput. 18, 2, 10--19. Google Scholar
Digital Library
- S. Mannor, B. Kveton, S. Siddiqi, and C. H. Yu. 2006. Machine learning for adaptive power management. Auton. Comput. 10, 4, 299--312.Google Scholar
- C. Moser, J. Chen, and L. Thiele. 2010. Dynamic power management in environmentally powered systems. In Proceedings of the Design Automation Conference (ASP-DAC). IEEE, 81--88. Google Scholar
Digital Library
- S. Natarajan and P. Tadepalli. 2005. Dynamic preferences in multi-criteria reinforcement learning. In Proceedings of the International Conference on Machine Learning. 601--608. Google Scholar
Digital Library
- U. Y. Ogras, R. Marculescu, D. Marculescu, and E. G. Jung. 2009. Design and management of voltage-frequency island partitioned networks-on-chip. IEEE Trans. VLSI Syst. 17, 3, 330--341. Google Scholar
Digital Library
- C. M. Olsen and C. Narayanaswarni. 2006. PowerNap: An efficient power management scheme for mobile devices. IEEE Trans. Mobile Comput. 5, 7, 816--828. Google Scholar
Digital Library
- A. Paul. 2013. Dynamic power management for ubiquitous network devices. Adv. Sci. Lett. 19, 7, 2046--2049.Google Scholar
Cross Ref
- T. Phit and K. Abe. 2006. Packet inter-arrival time estimation using neural network models. In Proceedings of the Internet Conference.Google Scholar
- F. Pletzer, R. Tusch, L. Böszörmenyi, and B. Rinner. 2012. Robust traffic state estimation on smart cameras. In Proceedings of the IEEE Conference on Advanced Video and Signal-Based Surveillance. 434--439. Google Scholar
Digital Library
- V. L. Prabha and E. C. Monie. 2007. Hardware architecture of reinforcement learning scheme for dynamic power management in embedded systems. EURASIP J. Embed. Syst. 07, 1, 1--6. Google Scholar
Digital Library
- Q. Qiu and M. Pedram. 1999. Dynamic power management based on continuous-time Markov decision processes. In Proceedings of the 36th Annual ACM/IEEE Design Automation Conference. 555--561. Google Scholar
Digital Library
- Z. Ren, B. H. Krogh, and R. Marculescu. 2005. Hierarchical adaptive dynamic power management. IEEE Trans. Comput. 54, 4, 409--420. Google Scholar
Digital Library
- H. C. Shih and K. Wang. 2012. An adaptive hybrid dynamic power management algorithm for mobile devices. Comput. Netw. 56, 2, 548--565. Google Scholar
Digital Library
- T. Simunic, L. Benini, P. Glynn, and G. De Micheli. 2000. Dynamic power management for portable systems. In Proceedings of the 6th International Conference on Mobile Computing and Networking. 11--19. Google Scholar
Digital Library
- T. Simunic, L. Benini, P. Glynn, and G. De Micheli. 2001. Event-driven power management. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 20, 7, 840--857. Google Scholar
Digital Library
- M. B. Srivastava, A. P. Chandrakasan, and R. W. Brodersen. 1996. Predictive system shutdown and other architectural techniques for energy efficient programmable computation. IEEE Trans. Integ. VLSI Syst. 4, 1, 42--55. Google Scholar
Digital Library
- R. S. Sutton. 1990. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the International Conference on Machine Learning. 216--224. Google Scholar
Digital Library
- Y. Tan, W. Liu, and Q. Qiu. 2009. Adaptive power management using reinforcement learning. In Proceedings of the International Conference on Computer-Aided Design. 461--467. Google Scholar
Digital Library
- Y. Wang, Q. Xie, A. Ammari, and M. Pedram. 2011b. Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classification. In Proceedings of the Design Automation Conference. 41--46. Google Scholar
Digital Library
- Z. Wang, H. Wang, X. Chen, and J. Lin. 2011a. Cyber physical systems. J. Chinese Comput. Syst. 32, 5, 881--886.Google Scholar
- C. J. C. H. Watkins and P. Dayan. 2011. Q-Learning. Machine Learn. 8, 3--4, 279--292. Google Scholar
Digital Library
- H. Young, K. Sung, and C. Ki-Seok. 2010. A predictive dynamic power management technique for embedded mobile devices. IEEE Trans. Consumer Electron. 56, 2, 713--719. Google Scholar
Digital Library
- A. Zafra, E. L. Gibaja, and S. Ventura. 2011. Multiple instance learning with multiple objective genetic programming for Web mining. Appl. Soft Comput. 11, 1, 93--102. Google Scholar
Digital Library
Index Terms
Online learning of timeout policies for dynamic power management
Recommendations
A Reinforcement Learning Framework for Dynamic Power Management of a Portable, Multi-camera Traffic Monitoring System
GREENCOM '12: Proceedings of the 2012 IEEE International Conference on Green Computing and CommunicationsDynamic Power Management (DPM) refers to a set of strategies that achieves efficient power consumption by selectively turning off (or reducing the performance of) a system components when they are idle or are serving light workloads. This paper presents ...
Reinforcement learning based dynamic power management with a hybrid power supply
ICCD '12: Proceedings of the 2012 IEEE 30th International Conference on Computer Design (ICCD 2012)Dynamic power management (DPM) in battery-powered mobile systems attempts to achieve higher energy efficiency by selectively setting idle components to a sleep state. However, re-activating these components at a later time consumes a large amount of ...
Co-Manage Power Delivery and Consumption for Manycore Systems Using Reinforcement Learning
2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD)Maintaining high energy efficiency has become a critical design issue for high-performance systems. Many power management techniques have been proposed for the processor cores such as dynamic voltage and frequency scaling (DVFS). However, very few ...






Comments