skip to main content
research-article

Adaptive Power Management in Solar Energy Harvesting Sensor Node Using Reinforcement Learning

Published:27 September 2017Publication History
Skip Abstract Section

Abstract

In this paper, we present an adaptive power manager for solar energy harvesting sensor nodes. We use a simplified model consisting of a solar panel, an ideal battery and a general sensor node with variable duty cycle. Our power manager uses Reinforcement Learning (RL), specifically SARSA(λ) learning, to train itself from historical data. Once trained, we show that our power manager is capable of adapting to changes in weather, climate, device parameters and battery degradation while ensuring near-optimal performance without depleting or overcharging its battery. Our approach uses a simple but novel general reward function and leverages the use of weather forecast data to enhance performance. We show that our method achieves near perfect energy neutral operation (ENO) with less than 6% root mean square deviation from ENO as compared to more than 23% deviation that occur when using other approaches.

References

  1. Luca Benini, Giuliano Castelli, Alberto Macii, and Riccardo Scarsi. 2001. Battery-driven dynamic power management. IEEE Design 8 Test of Computers 18, 2 (2001), 53--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Pol Blasco et al. 2013. A learning theoretic approach to energy harvesting communication system optimization. IEEE Tr. on Wireless Communications 12, 4 (2013), 1872--1882.Google ScholarGoogle ScholarCross RefCross Ref
  3. Wai Hong Ronald Chan et al. 2015. Adaptive duty cycling in sensor networks with energy harvesting using continuous-time Markov chain and fluid models. IEEE Journal on Selected Areas in Communications 33, 12 (2015), 2687--2700.Google ScholarGoogle ScholarCross RefCross Ref
  4. Deniz Gunduz, Kostas Stamatiou, Nicolo Michelusi, and Michele Zorzi. 2014. Designing intelligent energy harvesting communication systems. IEEE Communications Magazine 52, 1 (2014), 210--216.Google ScholarGoogle ScholarCross RefCross Ref
  5. Jason Hsu et al. 2006. Adaptive duty cycling for energy harvesting systems. In Proc. of the 2006 ISLPED. 180--185. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Roy Chaoming Hsu et al. 2014. A Reinforcement Learning-Based ToD Provisioning Dynamic Power Management for Sustainable Operation of Energy Harvesting Wireless Sensor Node. IEEE Tr. on Emerging Topics in Computing 2, 2 (2014), 181--191.Google ScholarGoogle ScholarCross RefCross Ref
  7. Roy Chaoming Hsu et al. 2015. Dynamic energy management of energy harvesting wireless sensor nodes using fuzzy inference system with reinforcement learning. In IEEE 13th INDIN. 116--120.Google ScholarGoogle Scholar
  8. Xiaofan Jiang, Joseph Polastre, and David Culler. 2005. Perpetual environmentally powered sensor networks. In Proceedings of the 4th International Symposium on Information Processing in Sensor Networks. IEEE Press, 65. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Aman Kansal et al. 2007. Power management in energy harvesting sensor networks. ACM Tr. on Embedded Computing Systems 6, 4 (2007), 32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Aman Kansal, Dunny Potter, and Mani B. Srivastava. 2004. Performance aware tasking for environmentally powered sensor networks. ACM SIGMETRICS Performance Evaluation Review 32, 1 (2004), 223--234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Junaid Ahmed Khan et al. 2015. Energy management in wireless sensor networks: a survey. Computers 8 Electrical Engineering 41 (2015), 159--176.Google ScholarGoogle Scholar
  12. Cheng-Ting Liu and Roy Chaoming Hsu. 2011. Dynamic power management utilizing reinforcement learning with fuzzy reward for energy harvesting wireless sensor nodes. In 37th Annual Conference on IEEE Industrial Electronics Society. 2365--2369.Google ScholarGoogle ScholarCross RefCross Ref
  13. S. Danish Maqbool et al. 2011. Analysis of adaptability of Reinforcement Learning approach. In IEEE 14th INMIC. 45--49.Google ScholarGoogle Scholar
  14. Nicolò Michelusi et al. 2013. Energy management policies for harvesting-based wireless sensor devices with battery degradation. IEEE Tr. on Communications 61, 12 (2013), 4934--4947.Google ScholarGoogle ScholarCross RefCross Ref
  15. Andrea Ortiz et al. 2016. Reinforcement learning for energy harvesting point-to-point communications. In 2016 IEEE International Conference on Communications. 1--6.Google ScholarGoogle Scholar
  16. Vijay Raghunathan, Aman Kansal, Jason Hsu, Jonathan Friedman, and Mani Srivastava. 2005. Design considerations for solar energy harvesting wireless embedded systems. In Proceedings of the 4th International Symposium on Information Processing in Sensor Networks. IEEE Press, 64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Luigi Rucco et al. 2013. A bird’s eye view on reinforcement learning approaches for power management in WSNs. In 6th WMNC. 1--8.Google ScholarGoogle Scholar
  18. Navin Sharma et al. 2010. Cloudy computing: Leveraging weather forecasts in energy harvesting sensor systems. In 7th IEEE SECON. 1--9.Google ScholarGoogle Scholar
  19. Sujesha Sudevalayam and Purushottam Kulkarni. 2011. Energy harvesting sensor nodes: Survey and implications. IEEE Communications Surveys 8 Tutorials 13, 3 (2011), 443--461.Google ScholarGoogle Scholar
  20. Srikanth Sundaresan et al. 2009. Event-driven adaptive duty-cycling in sensor networks. International Journal of Sensor Networks 6, 2 (2009), 89--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Richard S. Sutton et al. 1992. Reinforcement learning is direct adaptive optimal control. IEEE Control Systems 12, 2 (1992), 19--22.Google ScholarGoogle ScholarCross RefCross Ref
  22. Emre Ünsal, Taner Akkan, L. Özlem Akkan, and Yalçın Çebi. 2016. Power management for Wireless Sensor Networks in underground mining. In Signal Processing and Communication Application Conference (SIU), 2016 24th. IEEE, 1053--1056.Google ScholarGoogle ScholarCross RefCross Ref
  23. Christopher M. Vigorito et al. 2007. Adaptive control of duty cycling in energy-harvesting wireless sensor networks. In 4th IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks. 21--30.Google ScholarGoogle Scholar

Index Terms

  1. Adaptive Power Management in Solar Energy Harvesting Sensor Node Using Reinforcement Learning

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!