Abstract

Conventional off-chip voltage regulators are typically bulky and slow, and are inefficient at exploiting system and workload variability using Dynamic Voltage and Frequency Scaling (DVFS). On-die integration of voltage regulators has the potential to increase the energy efficiency of computer systems by enabling power control at a fine granularity in both space and time. The energy conversion efficiency of on-chip regulators, however, is typically much lower than off-chip regulators, which results in significant energy losses. Fine-grained power control and high voltage regulator efficiency are difficult to achieve simultaneously, with either emerging on-chip or conventional off-chip regulators.
A voltage conversion framework that relies on a hierarchy of off-chip switching regulators and on-chip linear regulators is proposed to enable fine-grained power control with a regulator efficiency greater than 90%. A DVFS control policy that is based on a reinforcement learning (RL) approach is developed to exploit the proposed framework. Per-core RL agents learn and improve their control policies independently, while retaining the ability to coordinate their actions to accomplish system level power management objectives. When evaluated on a mix of 14 parallel and 13 multiprogrammed workloads, the proposed voltage conversion framework achieves 18% greater energy efficiency than a conventional framework that uses on-chip switching regulators. Moreover, when the RL based DVFS control policy is used to control the proposed voltage conversion framework, the system achieves a 21% higher energy efficiency over a baseline oracle policy with coarse-grained power control capability.
- Martin Allen and Phil Fritzsche. Reinforcement learning with adaptive kanerva coding for xpilot game ai. In Evolutionary Computation (CEC), 2011 IEEE Congress on, pages 1521--1528. IEEE, 2011.Google Scholar
Cross Ref
- David H Bailey, Eric Barszcz, John T Barton, David S Browning, Russell L Carter, Leonardo Dagum, Rod A Fatoohi, Paul O Frederickson, Thomas A Lasinski, and Rob S Schreiber. NAS parallel benchmarks. Technical report, NASA Ames Research Center, March 1994. Tech. Rep. RNR-94-007.Google Scholar
- Andrew G Barto. Reinforcement learning: An introduction. MIT press, 1998.Google Scholar
- Hee Rak Beom and Kyung Suck Cho. A sensor-based navigation for a mobile robot using fuzzy logic and reinforcement learning. Systems, Man and Cybernetics, IEEE Transactions on, 25(3):464--477, 1995.Google Scholar
- R. Bitirgen, E. Ipek, and J. F. Martinez. Coordinated management of multiple interacting resources in chip multiprocessors: A machine learning approach. In International Symposium on Microarchitecture, Lake Como, Italy, Nov 2008. Google Scholar
Digital Library
- Incisive enterprise simulator. http://www.cadence.com/products/fv/enterprise_simulator.Google Scholar
- Yu-Han Chang, Tracey Ho, and Leslie Pack Kaelbling. Mobilized ad-hoc networks: A reinforcement learning approach. In Autonomic Computing, 2004. Proceedings. International Conference on, pages 240--247. IEEE, 2004.Google Scholar
Cross Ref
- C.K. Chava and J. Silva-Martinez. A robust frequency compensation scheme for ldo regulators. In Circuits and Systems, 2002. ISCAS 2002. IEEE International Symposium on, volume 5, pages V-825-V-828 vol.5, 2002. Google Scholar
Cross Ref
- Chia-Min Chen and Chung-Chih Hung. A capacitor-free cmos low-dropout voltage regulator. In Circuits and Systems, 2009. ISCAS 2009. IEEE International Symposium on, pages 2525--2528. IEEE, 2009. Google Scholar
Cross Ref
- Design Compiler. Synopsys inc, 2000.Google Scholar
- L. Dagum and R. Menon. OpenMP: An industry-standard API for shared-memory programming. IEEE Computational Science and Engineering, 5:46--55, 1998. Google Scholar
Digital Library
- Qingyuan Deng, David Meisner, Abhishek Bhattacharjee, Thomas F Wenisch, and Ricardo Bianchini. Coscale: Coordinating cpu and memory system dvfs in server systems. In Microarchitecture (MICRO), 2012 45th Annual IEEE/ACM International Symposium on, pages 143--154. IEEE, 2012.Google Scholar
Digital Library
- Marco Dorigo and LM Gambardella. Ant-q: A reinforcement learning approach to the traveling salesman problem. In Proceedings of ML-95, Twelfth Intern. Conf. on Machine Learning, pages 252--260, 2014.Google Scholar
- Hadi Esmaeilzadeh, Emily Blem, Renee St Amant, Karthikeyan Sankaralingam, and Doug Burger. Dark silicon and the end of multicore scaling. In Computer Architecture (ISCA), 2011 38th Annual International Symposium on, pages 365--376. IEEE, 2011. Google Scholar
Digital Library
- Waclaw Godycki, Christopher Torng, Ivan Bukreyev, Alyssa Apsel, and Christopher Batten. Enabling realistic fine-grain voltage scaling with reconfigurable power distribution networks. In Microarchitecture (MICRO), 2014 47th Annual IEEE/ACM International Symposium on, pages 381--393. IEEE, 2014. Google Scholar
Digital Library
- Inc. Gurobi Optimization. Gurobi optimizer reference manual, 2015.Google Scholar
- Mark A Hall. Correlation-based feature selection for machine learning. PhD thesis, The University of Waikato, 1999.Google Scholar
- Per Hammarlund, Rajesh Kumar, Randy B Osborne, Ravi Rajwar, Ronak Singhal, Reynold D'Sa, Robert Chappell, Shiv Kaushik, Srinivas Chennupaty, and Stephan Jourdan. Haswell: The fourth-generation intel core processor. IEEE Micro, (2):6--20, 2014. Google Scholar
Cross Ref
- John L. Henning. SPEC CPU2006 benchmark descriptions. SIGARCH Comput. Archit. News, 34(4):1--17, September 2006. Google Scholar
Digital Library
- E. Ipek, O. Mutlu, J. Martinez, and R. Caruana. Self-optimizing memory controllers : A reinforcement learning approach. In International Symposium on Computer Architecture, Beijing, China, Jun 2008. Google Scholar
Digital Library
- Engin Ipek, Onur Mutlu, José F Martínez, and Rich Caruana. Self-optimizing memory controllers: A reinforcement learning approach. In Computer Architecture, 2008. ISCA'08. 35th International Symposium on, pages 39--50. IEEE, 2008.Google Scholar
Digital Library
- Canturk Isci and Margaret Martonosi. Runtime power monitoring in high-end processors: Methodology and empirical data. In Proceedings of the 36th annual IEEE/ACM International Symposium on Microarchitecture, page 93. IEEE Computer Society, 2003.Google Scholar
Digital Library
- Rinkle Jain, Bibiche M Geuskens, Stephen T Kim, Muhammad M Khellah, Jaydeep Kulkarni, James W Tschanz, and Vivek De. A 0.45--1 v fully-integrated distributed switched capacitor dc-dc converter with high density mim capacitor in 22 nm tri-gate cmos. IEEE Journal of Solid-State Circuits, 49(4):917--927, 2014. Google Scholar
Cross Ref
- Wonyoung Kim, D. Brooks, and Gu-Yeon Wei. A fully-integrated 3-level dc-dc converter for nanosecond-scale dvfs. Solid-State Circuits, IEEE Journal of, 47(1):206--219, Jan 2012.Google Scholar
- Wonyoung Kim, Meeta Sharma Gupta, Gu-Yeon Wei, and David Brooks. System level analysis of fast, per-core dvfs using on-chip switching regulators. In High Performance Computer Architecture, 2008. HPCA 2008. IEEE 14th International Symposium on, pages 123--134. IEEE, 2008.Google Scholar
- Jens Kober and Jan Peters. Reinforcement learning in robotics: A survey. In Reinforcement Learning, pages 579--610. Springer, 2012.Google Scholar
Cross Ref
- S. Kose, E.G. Friedman, S. Tarn, S. Pinzon, and B. McDermott. An area efficient on-chip hybrid voltage regulator. In Quality Electronic Design (ISQED), 2012 13th International Symposium on, pages 398--403, March 2012. Google Scholar
Cross Ref
- S. Kose, S. Tam, S. Pinzon, B. McDermott, and E.G. Friedman. Active filter-based hybrid on-chip dc-dc converter for point-of-load voltage regulation. Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, 21(4):680--691, April 2013.Google Scholar
- Selcuk Kose and Eby G. Friedman. On-chip point-of-load voltage regulator for distributed power supplies. In Proceedings of the 20th Symposium on Great Lakes Symposium on VLSI, GLSVLSI '10, pages 377--380, New York, NY, USA, 2010. ACM. Google Scholar
Digital Library
- R Matthew Kretchmar. Reinforcement learning algorithms for homogenous multi-agent systems. In Workshop on Agent and Swarm Programming, 2003.Google Scholar
- Hanh-Phuc Le, J. Crossley, S.R. Sanders, and E. Alon. A sub-ns response fully integrated battery-connected switched-capacitor voltage regulator delivering 0.19w/mm2 at 73% efficiency. In Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2013 IEEE International, pages 372--373, Feb 2013.Google Scholar
- Ka Nang Leung and Philip KT Mok. A capacitor-free cmos low-dropout regulator with damping-factor-control frequency compensation. Solid-State Circuits, IEEE Journal of, 38(10):1691--1702, 2003.Google Scholar
- Sheng Li, Jung Ho Ahn, Richard D. Strong, Jay B. Brockman, Dean M. Tullsen, and Norman P. Jouppi. McPAT: An integrated power, area, and timing modeling framework for multicore and manycore architectures. In International Symposium on Computer Architecture, 2009.Google Scholar
Digital Library
- Amy McGovern, Eliot Moss, and Andrew G Barto. Scheduling straight-line code using reinforcement learning and rollouts. 1999.Google Scholar
- Micron Technology, Inc., http://www.micron.com//get-document/documentId=416. 8Gb DDR3 SDRAM, 2009.Google Scholar
- NanGate FreePDK45 Open Cell Library. http://www.nangate.com.Google Scholar
- Junhong Nie and Simon Haykin. A Q-learning-based dynamic channel assignment technique for mobile communication systems. Vehicular Technology, IEEE Transactions on, 48(5):1676--1687, 1999.Google Scholar
- Venkatesh Pallipadi and Alexey Starikovskiy. The ondemand governor. In Proceedings of the Linux Symposium, volume 2, pages 215--230. sn, 2006.Google Scholar
- G. Patounakis, Y.W. Li, and Kenneth L. Shepard. A fully integrated on-chip dc-dc conversion and power management system. Solid-State Circuits, IEEE Journal of, 39(3):443--451, March 2004.Google Scholar
- J. Pisharath, Y. Liu, W. Liao, A. Choudhary, G. Memik, and J. Parhi. NU-MineBench 2.0. Technical report, Northwestern University, August 2005. Tech. Rep. CUCIS-2005-08-01.Google Scholar
- Krishna K. Rangan, Gu-Yeon Wei, and David Brooks. Thread motion: Fine-grained power management for multi-core systems. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA '09, pages 302--313, New York, NY, USA, 2009. ACM. Google Scholar
Digital Library
- Jose Renau, Basilio Fraguela, James Tuck, Wei Liu, Milos Prvulovic, Luis Ceze, Smruti Sarangi, Paul Sack, Karin Strauss, and Pablo Montesinos. SESC simulator, Jan 2005.Google Scholar
- Emre Salman and Eby G.Friedman. High Performance Integrated Circuit Design. McGraw-Hill Professional, 2012.Google Scholar
- Karan Singh, Major Bhadauria, and Sally A McKee. Real time power estimation and thread scheduling via performance counters. ACM SIGARCH Computer Architecture News, 37(2):46--55, 2009. Google Scholar
Digital Library
- A.A. Sinkar, H.R. Ghasemi, M.J. Schulte, U.R. Karpuzcu, and Nam Sung Kim. Low-cost per-core voltage domain support for power-constrained high-performance processors. Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, 22(4):747--758, April 2014.Google Scholar
- Abhishek A Sinkar, Hamid Reza Ghasemi, Michael J Schulte, Ulya R Karpuzcu, and Nam Sung Kim. Low-cost per-core voltage domain support for power-constrained high-performance processors. Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, 22(4):747--758, 2014.Google Scholar
- Vasileios Spiliopoulos, Stefanos Kaxiras, and Georgios Keramidas. Green governors: A framework for continuously adaptive dvfs. In Green Computing Conference and Workshops (IGCC), 2011 International, pages 1--8. IEEE, 2011. Google Scholar
Digital Library
- A Stafylopatis and K Blekas. Autonomous vehicle navigation using evolutionary reinforcement learning. European Journal of Operational Research, 108(2):306--318, 1998. Google Scholar
Cross Ref
- N. Sturcken, E. O'Sullivan, Naigang Wang, P. Herget, B. Webb, L. Romankiw, M. Petracca, R. Davies, R. Fontana, G. Decad, I. Kymissis, A. Peterchev, L. Carloni, W. Gallagher, and K. Shepard. A 2.5d integrated voltage regulator using coupled-magnetic-core inductors on silicon interposer delivering 10.8a/mm2. In Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2012 IEEE International, Feb 2012.Google Scholar
Cross Ref
- Bo Su, Junli Gu, Li Shen, Wei Huang, Joseph L Greathouse, and Zhiying Wang. PPEP: Online performance, power, and energy prediction framework and dvfs space exploration. In Microarchitecture (MICRO), 2014 47th Annual IEEE/ACM International Symposium on, pages 445--457. IEEE, 2014.Google Scholar
Digital Library
- Gerald Tesauro. Online resource allocation using decompositional reinforcement learning. In AAAI, volume 5, pages 886--891, 2005.Google Scholar
- Chun-Yen Tseng, Li-Wen Wang, and Po-Chiun Huang. An integrated linear regulator with fast output voltage transition for dual-supply srams in dvfs systems. Solid-State Circuits, IEEE Journal of, 45(11):2239--2249, Nov 2010.Google Scholar
- Inna Vaisband, Burt Price, Selçuk Köse, Yesh Kolla, EbyG. Friedman, and Jeff Fischer. Distributed ldo regulators in a 28 nm power delivery system. Analog Integrated Circuits and Signal Processing, 83(3):295--309, 2015. Google Scholar
Digital Library
- Xiaodong Wang and J.F. Martinez. Xchange: A market-based approach to scalable dynamic multi-resource allocation in multicore architectures. In High Performance Computer Architecture (HPCA), 2015 IEEE 21st International Symposium on, pages 113--125, Feb 2015. Google Scholar
Cross Ref
- Samuel Williams, Andrew Waterman, and David Patterson. Roofline: An insightful visual performance model for multicore architectures. Commun. ACM, 52(4):65--76, April 2009. Google Scholar
Digital Library
- Ian H. Witten, Eibe Frank, Len Trigg, Mark Hall, Geoffrey Holmes, and Sally Jo Cunningham. Weka: Practical machine learning tools and techniques with java implementations, 1999.Google Scholar
- Steven Cameron Woo, Moriyoshi Ohara, Evan Torrie, Jaswinder Pal Singh, and Anoop Gupta. The SPLASH-2 programs: Characterization and methodological considerations. In ISCA, 1995.Google Scholar
- Lei Xu, Pingfan Yan, and Tong Chang. Best first strategy for feature selection. In Pattern Recognition, 1988., 9th International Conference on, 1988.Google Scholar
- Guihai Yan, Yingmin Li, Yinhe Han, Xiaowei Li, Minyi Guo, and Xiaoyao Liang. Agileregulator: A hybrid voltage regulator scheme redeeming dark silicon for power efficiency in a multicore architecture. In High Performance Computer Architecture (HPCA), 2012 IEEE 18th International Symposium on, pages 1--12. IEEE, 2012. Google Scholar
Digital Library
- Richard M Yoo, Anthony Romano, and Christos Kozyrakis. Phoenix rebirth: Scalable MapReduce on a large-scale shared-memory system. In Proceedings of IEEE International Symposium on Workload Characterization, 2009.Google Scholar
Index Terms
Voltage Regulator Efficiency Aware Power Management
Recommendations
Voltage Regulator Efficiency Aware Power Management
ASPLOS '17: Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating SystemsConventional off-chip voltage regulators are typically bulky and slow, and are inefficient at exploiting system and workload variability using Dynamic Voltage and Frequency Scaling (DVFS). On-die integration of voltage regulators has the potential to ...
Voltage Regulator Efficiency Aware Power Management
Asplos'17Conventional off-chip voltage regulators are typically bulky and slow, and are inefficient at exploiting system and workload variability using Dynamic Voltage and Frequency Scaling (DVFS). On-die integration of voltage regulators has the potential to ...
Hybrid power management in real time embedded systems: an interplay of DVFS and DPM techniques
Energy-aware scheduling of real time applications over multiprocessor systems is considered in this paper. Early research reports that while various energy-saving policies, for instance Dynamic Power Management (DPM) and Dynamic Voltage & Frequency ...







Comments