Abstract
This paper considers online convex optimization (OCO) problems where decisions are constrained by available energy resources. A key scenario is optimal power control for an energy harvesting device with a finite capacity battery. The goal is to minimize a time-average loss function while keeping the used energy less than what is available. In this setup, the distribution of the randomly arriving harvestable energy (which is assumed to be i.i.d.) is unknown, the current loss function is unknown, and the controller is only informed by the history of past observations. A prior algorithm is known to achieve $O(\sqrtT )$ regret by using a battery with an $O(\sqrtT )$ capacity. This paper develops a new algorithm that maintains this asymptotic trade-off with the number of time steps T while improving dependency on the dimension of the decision vector from $O(\sqrtn )$ to $O(\sqrtłog(n) )$. The proposed algorithm introduces a separation of the decision vector into amplitude and direction components. It uses two distinct types of Bregman divergence, together with energy queue information, to make decisions for each component.
Supplemental Material
- Jacob D Abernethy, Elad Hazan, and Alexander Rakhlin. 2009. Competing in the dark: An efficient algorithm for bandit linear optimization. (2009).Google Scholar
- Ahmed Arafa, Abdulrahman Baknina, and Sennur Ulukus. 2017. Energy harvesting networks with general utility functions: Near optimal online policies. In 2017 IEEE International Symposium on Information Theory (ISIT). IEEE, 809--813.Google Scholar
Cross Ref
- Amir Beck and Marc Teboulle. 2003. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters 31, 3 (2003), 167--175.Google Scholar
Digital Library
- Pol Blasco, Deniz Gunduz, and Mischa Dohler. 2013. A learning theoretic approach to energy harvesting communication system optimization. IEEE Transactions on Wireless Communications 12, 4 (2013), 1872--1882.Google Scholar
Cross Ref
- Sébastien Bubeck. 2011. Introduction to online optimization. Lecture Notes 2 (2011).Google Scholar
- Sébastien Bubeck and Nicolo Cesa-Bianchi. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. arXiv preprint arXiv:1204.5721 (2012).Google Scholar
Cross Ref
- Ying Cao, Bo Sun, and Danny HK Tsang. 2020. Optimal Online Algorithms for One-Way Trading and Online Knapsack Problems: A Unified Competitive Analysis. arXiv preprint arXiv:2004.10358 (2020).Google Scholar
- Nicolo Cesa-Bianchi, Philip M Long, and Manfred K Warmuth. 1996. Worst-case quadratic loss bounds for prediction using linear functions and gradient descent. IEEE Transactions on Neural Networks 7, 3 (1996), 604--619.Google Scholar
Digital Library
- Nicolo Cesa-Bianchi and Gábor Lugosi. 2006. Prediction, learning, and games. Cambridge university press.Google Scholar
Digital Library
- Chi-Kin Chau, Guanglin Zhang, and Minghua Chen. 2016. Cost minimizing online algorithms for energy storage management with worst-case guarantee. IEEE Transactions on Smart Grid 7, 6 (2016), 2691--2702.Google Scholar
Cross Ref
- G. Chen and M. Teboulle. 1993. Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman Functions. SIAM Journal on Optimization 3, 3 (1993), 538--543.Google Scholar
Cross Ref
- Tianyi Chen and Georgios B Giannakis. 2018. Bandit convex optimization for scalable and dynamic IoT management. IEEE Internet of Things Journal 6, 1 (2018), 1276--1286.Google Scholar
Cross Ref
- Ran El-Yaniv, Amos Fiat, Richard M Karp, and Gordon Turpin. 2001. Optimal search and one-way trading online algorithms. Algorithmica 30, 1 (2001), 101--139.Google Scholar
Cross Ref
- M. Gatzianas, L. Georgiadis, and L. Tassiulas. Feb. 2010. Control of Wireless Networks with Rechargeable Batteries. IEEE Transactions on Wireless Communications vol. 9, no. 2, pp. 581--593 (Feb. 2010).Google Scholar
Digital Library
- Elad Hazan. 2019. Introduction to online convex optimization. arXiv preprint arXiv:1909.05207 (2019).Google Scholar
- Elad Hazan, Amit Agarwal, and Satyen Kale. 2007. Logarithmic regret algorithms for online convex optimization. Machine Learning 69, 2--3 (2007), 169--192.Google Scholar
Digital Library
- Elad Hazan and Satyen Kale. 2010. Extracting certainty from uncertainty: Regret bounded by variation in costs. Machine learning 80, 2--3 (2010), 165--188.Google Scholar
- L. Huang and M. J. Neely. Aug. 2013. Utility Optimal Scheduling in Energy Harvesting Networks. IEEE/ACM Transactions on Networking vol. 21, no. 4, pp. 1117--1130 (Aug. 2013).Google Scholar
Digital Library
- Rodolphe Jenatton, Jim Huang, and Cédric Archambeau. 2016. Adaptive algorithms for online convex optimization with long-term constraints. In International Conference on Machine Learning. 402--411.Google Scholar
- Nikolaos Liakopoulos, Apostolos Destounis, Georgios Paschos, Thrasyvoulos Spyropoulos, and Panayotis Mertikopoulos. 2019. Cautious regret minimization: Online optimization with long-term budget constraints. In International Conference on Machine Learning. 3944--3952.Google Scholar
- Qiulin Lin, Hanling Yi, John Pang, Minghua Chen, Adam Wierman, Michael Honig, and Yuanzhang Xiao. 2019. Competitive online optimization under inventory constraints. Proceedings of the ACM on Measurement and Analysis of Computing Systems 3, 1 (2019), 1--28.Google Scholar
Digital Library
- Mehrdad Mahdavi, Rong Jin, and Tianbao Yang. 2012. Trading regret for efficiency: online convex optimization with long term constraints. The Journal of Machine Learning Research 13, 1 (2012), 2503--2528.Google Scholar
Digital Library
- Shie Mannor, John N Tsitsiklis, and Jia Yuan Yu. 2009. Online Learning with Sample Path Constraints. Journal of Machine Learning Research 10, 3 (2009).Google Scholar
- Nicolo Michelusi, Kostas Stamatiou, and Michele Zorzi. 2013. Transmission policies for energy harvesting sensors with time-correlated energy supply. IEEE Transactions on Communications 61, 7 (2013), 2988--3001.Google Scholar
Cross Ref
- M. J. Neely. 2010. Stochastic Network Optimization with Application to Communication and Queueing Systems. Morgan & Claypool.Google Scholar
- M. J. Neely and L. Huang. Dec. 2010. Dynamic Product Assembly and Inventory Control for Maximum Profit. Proc. IEEE Conf. on Decision and Control (CDC) Atlanta, GA (Dec. 2010).Google Scholar
Cross Ref
- A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. 2009. Robust Stochastic Approximation Approach to Stochastic Programming. SIAM Journal on Optimization 19, 4 (2009), 1574--1609.Google Scholar
Digital Library
- Problem complexity and method efficiency in optimization. ([n. d.]).Google Scholar
- Michael Rossol, Bri-Mathias Hodge, Caroline Draxl, Andrew Clifton, Jim McCaa, Tarek Elgindy, Manajit Sengupta, Yu Xie, Anthony Lopez, and Aron Habte. [n.d.]. NREL Renewable Energy Resource Data. ([n. d.]). https://doi.org/10. 17041/drp/1473618Google Scholar
- Shai Shalev-Shwartz et al. 2011. Online learning and online convex optimization. Foundations and trends in Machine Learning 4, 2 (2011), 107--194.Google Scholar
- Shai Shalev-Shwartz and Yoram Singer. 2007. A primal-dual perspective of online learning algorithms. Machine Learning 69, 2--3 (2007), 115--142.Google Scholar
Digital Library
- Dor Shaviv and Ayfer Özgür. 2016. Universally near optimal online power control for energy harvesting nodes. IEEE Journal on Selected Areas in Communications 34, 12 (2016), 3620--3631.Google Scholar
Digital Library
- Alexander A Titov, Fedor S Stonyakin, Alexander V Gasnikov, and Mohammad S Alkousa. 2018. Mirror descent and constrained online optimization problems. In International Conference on Optimization and Applications. Springer, 64--78.Google Scholar
- Paul Tseng. 2008. On accelerated proximal gradient methods for convex-concave optimization. submitted to SIAM Journal on Optimization 2, 3 (2008).Google Scholar
- Kaya Tutuncuoglu and Aylin Yener. 2012. Optimum transmission policies for battery limited energy harvesting nodes. IEEE Transactions on Wireless Communications 11, 3 (2012), 1180--1189.Google Scholar
Cross Ref
- Xiaohan Wei, Hao Yu, and Michael J Neely. 2020. Online primal-dual mirror descent under stochastic constraints. In Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems. 3--4.Google Scholar
Digital Library
- Weiwei Wu, Jianping Wang, Xiumin Wang, Feng Shan, and Junzhou Luo. 2016. Online throughput maximization for energy harvesting communication systems with battery overflow. IEEE Transactions on Mobile Computing 16, 1 (2016), 185--197.Google Scholar
Digital Library
- Jing Yang and Sennur Ulukus. 2011. Optimal packet scheduling in an energy harvesting communication system. IEEE Transactions on Communications 60, 1 (2011), 220--230.Google Scholar
Cross Ref
- Lin Yang, Mohammad H Hajiesmaili, Ramesh Sitaraman, Adam Wierman, Enrique Mallada, and Wing S Wong. 2020. Online Linear Optimization with Inventory Management Constraints. Proceedings of the ACM on Measurement and Analysis of Computing Systems 4, 1 (2020), 1--29.Google Scholar
Digital Library
- Hao Yu, Michael Neely, and Xiaohan Wei. 2017. Online convex optimization with stochastic constraints. In Advances in Neural Information Processing Systems. 1428--1438.Google Scholar
- Hao Yu and Michael J Neely. 2019. Learning-Aided Optimization for Energy-Harvesting Devices With Outdated State Information. IEEE/ACM Transactions on Networking 27, 4 (2019), 1501--1514.Google Scholar
Digital Library
- Hao Yu and Michael J. Neely. 2020. A Low Complexity Algorithm with ??( ? ?? ) Regret and ??(1) Constraint Violations for Online Convex Optimization with Long Term Constraints. Journal of Machine Learning Research 21, 1 (2020), 1--24. http://jmlr.org/papers/v21/16--494.htmlGoogle Scholar
- Jianjun Yuan and Andrew Lamperski. 2018. Online convex optimization for cumulative constraints. In Advances in Neural Information Processing Systems. 6137--6146.Google Scholar
- Martin Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03). 928--936.Google Scholar
Digital Library
Index Terms
Bregman-style Online Convex Optimization with EnergyHarvesting Constraints
Recommendations
Bregman-style Online Convex Optimization with Energy Harvesting Constraints
SIGMETRICS '21: Abstract Proceedings of the 2021 ACM SIGMETRICS / International Conference on Measurement and Modeling of Computer SystemsThis paper considers online convex optimization (OCO) problems where decisions are constrained by available energy resources. A key scenario is optimal power control for an energy harvesting device with a finite capacity battery. The goal is to minimize ...
Bregman-style Online Convex Optimization with Energy Harvesting Constraints
SIGMETRICS '21This paper considers online convex optimization (OCO) problems where decisions are constrained by available energy resources. A key scenario is optimal power control for an energy harvesting device with a finite capacity battery. The goal is to minimize ...
TTS: a two-tiered scheduling mechanism for energy conservation in wireless sensor networks
In this paper, we present a two-tiered scheduling approach for effective energy conservation in wireless sensor networks. The effectiveness of this mechanism relies on dynamically updated two-tiered scheduling architecture. We aim to prolong network ...






Comments