skip to main content
research-article
Open Access

Bregman-style Online Convex Optimization with EnergyHarvesting Constraints

Published:30 November 2020Publication History
Skip Abstract Section

Abstract

This paper considers online convex optimization (OCO) problems where decisions are constrained by available energy resources. A key scenario is optimal power control for an energy harvesting device with a finite capacity battery. The goal is to minimize a time-average loss function while keeping the used energy less than what is available. In this setup, the distribution of the randomly arriving harvestable energy (which is assumed to be i.i.d.) is unknown, the current loss function is unknown, and the controller is only informed by the history of past observations. A prior algorithm is known to achieve $O(\sqrtT )$ regret by using a battery with an $O(\sqrtT )$ capacity. This paper develops a new algorithm that maintains this asymptotic trade-off with the number of time steps T while improving dependency on the dimension of the decision vector from $O(\sqrtn )$ to $O(\sqrtłog(n) )$. The proposed algorithm introduces a separation of the decision vector into amplitude and direction components. It uses two distinct types of Bregman divergence, together with energy queue information, to make decisions for each component.

Skip Supplemental Material Section

Supplemental Material

Bregman style Online Convex Optimization with Energy Harvesting Constraints-v1.mp4

Presentation video of Bregman-style Online Convex Optimization with EnergyHarvesting Constraints

References

  1. Jacob D Abernethy, Elad Hazan, and Alexander Rakhlin. 2009. Competing in the dark: An efficient algorithm for bandit linear optimization. (2009).Google ScholarGoogle Scholar
  2. Ahmed Arafa, Abdulrahman Baknina, and Sennur Ulukus. 2017. Energy harvesting networks with general utility functions: Near optimal online policies. In 2017 IEEE International Symposium on Information Theory (ISIT). IEEE, 809--813.Google ScholarGoogle ScholarCross RefCross Ref
  3. Amir Beck and Marc Teboulle. 2003. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters 31, 3 (2003), 167--175.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Pol Blasco, Deniz Gunduz, and Mischa Dohler. 2013. A learning theoretic approach to energy harvesting communication system optimization. IEEE Transactions on Wireless Communications 12, 4 (2013), 1872--1882.Google ScholarGoogle ScholarCross RefCross Ref
  5. Sébastien Bubeck. 2011. Introduction to online optimization. Lecture Notes 2 (2011).Google ScholarGoogle Scholar
  6. Sébastien Bubeck and Nicolo Cesa-Bianchi. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. arXiv preprint arXiv:1204.5721 (2012).Google ScholarGoogle ScholarCross RefCross Ref
  7. Ying Cao, Bo Sun, and Danny HK Tsang. 2020. Optimal Online Algorithms for One-Way Trading and Online Knapsack Problems: A Unified Competitive Analysis. arXiv preprint arXiv:2004.10358 (2020).Google ScholarGoogle Scholar
  8. Nicolo Cesa-Bianchi, Philip M Long, and Manfred K Warmuth. 1996. Worst-case quadratic loss bounds for prediction using linear functions and gradient descent. IEEE Transactions on Neural Networks 7, 3 (1996), 604--619.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Nicolo Cesa-Bianchi and Gábor Lugosi. 2006. Prediction, learning, and games. Cambridge university press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Chi-Kin Chau, Guanglin Zhang, and Minghua Chen. 2016. Cost minimizing online algorithms for energy storage management with worst-case guarantee. IEEE Transactions on Smart Grid 7, 6 (2016), 2691--2702.Google ScholarGoogle ScholarCross RefCross Ref
  11. G. Chen and M. Teboulle. 1993. Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman Functions. SIAM Journal on Optimization 3, 3 (1993), 538--543.Google ScholarGoogle ScholarCross RefCross Ref
  12. Tianyi Chen and Georgios B Giannakis. 2018. Bandit convex optimization for scalable and dynamic IoT management. IEEE Internet of Things Journal 6, 1 (2018), 1276--1286.Google ScholarGoogle ScholarCross RefCross Ref
  13. Ran El-Yaniv, Amos Fiat, Richard M Karp, and Gordon Turpin. 2001. Optimal search and one-way trading online algorithms. Algorithmica 30, 1 (2001), 101--139.Google ScholarGoogle ScholarCross RefCross Ref
  14. M. Gatzianas, L. Georgiadis, and L. Tassiulas. Feb. 2010. Control of Wireless Networks with Rechargeable Batteries. IEEE Transactions on Wireless Communications vol. 9, no. 2, pp. 581--593 (Feb. 2010).Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Elad Hazan. 2019. Introduction to online convex optimization. arXiv preprint arXiv:1909.05207 (2019).Google ScholarGoogle Scholar
  16. Elad Hazan, Amit Agarwal, and Satyen Kale. 2007. Logarithmic regret algorithms for online convex optimization. Machine Learning 69, 2--3 (2007), 169--192.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Elad Hazan and Satyen Kale. 2010. Extracting certainty from uncertainty: Regret bounded by variation in costs. Machine learning 80, 2--3 (2010), 165--188.Google ScholarGoogle Scholar
  18. L. Huang and M. J. Neely. Aug. 2013. Utility Optimal Scheduling in Energy Harvesting Networks. IEEE/ACM Transactions on Networking vol. 21, no. 4, pp. 1117--1130 (Aug. 2013).Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Rodolphe Jenatton, Jim Huang, and Cédric Archambeau. 2016. Adaptive algorithms for online convex optimization with long-term constraints. In International Conference on Machine Learning. 402--411.Google ScholarGoogle Scholar
  20. Nikolaos Liakopoulos, Apostolos Destounis, Georgios Paschos, Thrasyvoulos Spyropoulos, and Panayotis Mertikopoulos. 2019. Cautious regret minimization: Online optimization with long-term budget constraints. In International Conference on Machine Learning. 3944--3952.Google ScholarGoogle Scholar
  21. Qiulin Lin, Hanling Yi, John Pang, Minghua Chen, Adam Wierman, Michael Honig, and Yuanzhang Xiao. 2019. Competitive online optimization under inventory constraints. Proceedings of the ACM on Measurement and Analysis of Computing Systems 3, 1 (2019), 1--28.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Mehrdad Mahdavi, Rong Jin, and Tianbao Yang. 2012. Trading regret for efficiency: online convex optimization with long term constraints. The Journal of Machine Learning Research 13, 1 (2012), 2503--2528.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Shie Mannor, John N Tsitsiklis, and Jia Yuan Yu. 2009. Online Learning with Sample Path Constraints. Journal of Machine Learning Research 10, 3 (2009).Google ScholarGoogle Scholar
  24. Nicolo Michelusi, Kostas Stamatiou, and Michele Zorzi. 2013. Transmission policies for energy harvesting sensors with time-correlated energy supply. IEEE Transactions on Communications 61, 7 (2013), 2988--3001.Google ScholarGoogle ScholarCross RefCross Ref
  25. M. J. Neely. 2010. Stochastic Network Optimization with Application to Communication and Queueing Systems. Morgan & Claypool.Google ScholarGoogle Scholar
  26. M. J. Neely and L. Huang. Dec. 2010. Dynamic Product Assembly and Inventory Control for Maximum Profit. Proc. IEEE Conf. on Decision and Control (CDC) Atlanta, GA (Dec. 2010).Google ScholarGoogle ScholarCross RefCross Ref
  27. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. 2009. Robust Stochastic Approximation Approach to Stochastic Programming. SIAM Journal on Optimization 19, 4 (2009), 1574--1609.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Problem complexity and method efficiency in optimization. ([n. d.]).Google ScholarGoogle Scholar
  29. Michael Rossol, Bri-Mathias Hodge, Caroline Draxl, Andrew Clifton, Jim McCaa, Tarek Elgindy, Manajit Sengupta, Yu Xie, Anthony Lopez, and Aron Habte. [n.d.]. NREL Renewable Energy Resource Data. ([n. d.]). https://doi.org/10. 17041/drp/1473618Google ScholarGoogle Scholar
  30. Shai Shalev-Shwartz et al. 2011. Online learning and online convex optimization. Foundations and trends in Machine Learning 4, 2 (2011), 107--194.Google ScholarGoogle Scholar
  31. Shai Shalev-Shwartz and Yoram Singer. 2007. A primal-dual perspective of online learning algorithms. Machine Learning 69, 2--3 (2007), 115--142.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Dor Shaviv and Ayfer Özgür. 2016. Universally near optimal online power control for energy harvesting nodes. IEEE Journal on Selected Areas in Communications 34, 12 (2016), 3620--3631.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Alexander A Titov, Fedor S Stonyakin, Alexander V Gasnikov, and Mohammad S Alkousa. 2018. Mirror descent and constrained online optimization problems. In International Conference on Optimization and Applications. Springer, 64--78.Google ScholarGoogle Scholar
  34. Paul Tseng. 2008. On accelerated proximal gradient methods for convex-concave optimization. submitted to SIAM Journal on Optimization 2, 3 (2008).Google ScholarGoogle Scholar
  35. Kaya Tutuncuoglu and Aylin Yener. 2012. Optimum transmission policies for battery limited energy harvesting nodes. IEEE Transactions on Wireless Communications 11, 3 (2012), 1180--1189.Google ScholarGoogle ScholarCross RefCross Ref
  36. Xiaohan Wei, Hao Yu, and Michael J Neely. 2020. Online primal-dual mirror descent under stochastic constraints. In Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems. 3--4.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Weiwei Wu, Jianping Wang, Xiumin Wang, Feng Shan, and Junzhou Luo. 2016. Online throughput maximization for energy harvesting communication systems with battery overflow. IEEE Transactions on Mobile Computing 16, 1 (2016), 185--197.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Jing Yang and Sennur Ulukus. 2011. Optimal packet scheduling in an energy harvesting communication system. IEEE Transactions on Communications 60, 1 (2011), 220--230.Google ScholarGoogle ScholarCross RefCross Ref
  39. Lin Yang, Mohammad H Hajiesmaili, Ramesh Sitaraman, Adam Wierman, Enrique Mallada, and Wing S Wong. 2020. Online Linear Optimization with Inventory Management Constraints. Proceedings of the ACM on Measurement and Analysis of Computing Systems 4, 1 (2020), 1--29.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Hao Yu, Michael Neely, and Xiaohan Wei. 2017. Online convex optimization with stochastic constraints. In Advances in Neural Information Processing Systems. 1428--1438.Google ScholarGoogle Scholar
  41. Hao Yu and Michael J Neely. 2019. Learning-Aided Optimization for Energy-Harvesting Devices With Outdated State Information. IEEE/ACM Transactions on Networking 27, 4 (2019), 1501--1514.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Hao Yu and Michael J. Neely. 2020. A Low Complexity Algorithm with ??( ? ?? ) Regret and ??(1) Constraint Violations for Online Convex Optimization with Long Term Constraints. Journal of Machine Learning Research 21, 1 (2020), 1--24. http://jmlr.org/papers/v21/16--494.htmlGoogle ScholarGoogle Scholar
  43. Jianjun Yuan and Andrew Lamperski. 2018. Online convex optimization for cumulative constraints. In Advances in Neural Information Processing Systems. 6137--6146.Google ScholarGoogle Scholar
  44. Martin Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03). 928--936.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Bregman-style Online Convex Optimization with EnergyHarvesting Constraints

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Article Metrics

        • Downloads (Last 12 months)107
        • Downloads (Last 6 weeks)14

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!