Abstract
We study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be non-convex, while incurring a movement cost when changing actions between rounds. We ask:under what general conditions is it possible for an online learner to leverage predictions of future cost functions in order to achieve near-optimal costs? Prior work has provided near-optimal online algorithms for specific combinations of assumptions about hitting and switching costs, but no general results are known. In this work, we give two general sufficient conditions that specify a relationship between the hitting and movement costs which guarantees that a new algorithm, Synchronized Fixed Horizon Control (SFHC), achieves a 1+O(1/w) competitive ratio, where w is the number of predictions available to the learner. Our conditions do not require the cost functions to be convex, and we also derive competitive ratio results for non-convex hitting and movement costs. Our results provide the first constant, dimension-free competitive ratio for online non-convex optimization with movement costs. We also give an example of a natural problem, Convex Body Chasing (CBC), where the sufficient conditions are not satisfied and prove that no online algorithm can have a competitive ratio that converges to 1.
- Lachlan Andrew, Siddharth Barman, Katrina Ligett, Minghong Lin, Adam Meyerson, Alan Roytman, and Adam Wierman. 2013. A tale of two metrics: Simultaneous bounds on competitiveness and regret. In Conference on Learning Theory. 741--763.Google Scholar
Digital Library
- Antonios Antoniadis, Neal Barcelo, Michael Nugent, Kirk Pruhs, Kevin Schewior, and Michele Scquizzato. 2016. Chasing convex bodies and functions. In LATIN 2016: Theoretical Informatics. Springer, 68--81.Google Scholar
- Antonios Antoniadis and Kevin Schewior. 2017. A tight lower bound for online convex optimization with switching costs. In Proceedings of the International Workshop on Approximation and Online Algorithms. Springer, 164--175.Google Scholar
- David Ardia, Kris Boudt, Peter Carl, Katharine M Mullen, and Brian Peterson. 2010. Differential evolution (deoptim) for non-convex portfolio optimization. (2010).Google Scholar
- CJ Argue, Sébastien Bubeck, Michael B Cohen, Anupam Gupta, and Yin Tat Lee. 2019. A nearly-linear bound for chasing nested convex bodies. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA). 117--122.Google Scholar
Cross Ref
- CJ Argue, Anupam Gupta, Guru Guruganesh, and Ziye Tang. 2020. Chasing convex bodies with linear competitive ratio. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, 1519--1524.Google Scholar
Cross Ref
- Sanjeev Arora, Elad Hazan, and Satyen Kale. 2012. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing , Vol. 8, 1 (2012), 121--164.Google Scholar
Cross Ref
- Jean-Yves Audibert and Sébastien Bubeck. 2010. Regret bounds and minimax policies under partial monitoring. Journal of Machine Learning Research , Vol. 11, Oct (2010), 2785--2836.Google Scholar
Digital Library
- Masoud Badiei, Na Li, and Adam Wierman. 2015a. Online convex optimization with ramp constraints. In 2015 54th IEEE Conference on Decision and Control (CDC). IEEE, 6730--6736.Google Scholar
Cross Ref
- Masoud Badiei, Na Li, and Adam Wierman. 2015b. Online convex optimization with ramp constraints. In IEEE Conference on Decision and Control (CDC) . 6730--6736. https://doi.org/10.1109/CDC.2015.7403279Google Scholar
Cross Ref
- Nikhil Bansal and Anupam Gupta. 2017. Potential-function proofs for first-order methods. arXiv preprint arXiv:1712.04581 (2017).Google Scholar
- Nikhil Bansal, Anupam Gupta, Ravishankar Krishnaswamy, Kirk Pruhs, Kevin Schewior, and Cliff Stein. 2015. A 2-competitive algorithm for online convex optimization with switching costs. In Proceedings of the Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google Scholar
- Yair Bartal, Avrim Blum, Carl Burch, and Andrew Tomkins. 1997. A Polylog(N)-competitive Algorithm for Metrical Task Systems. In Proceedings of the ACM Symposium on Theory of Computing (STOC). 711--719. https://doi.org/10.1145/258533.258667Google Scholar
Digital Library
- Avrim Blum and Carl Burch. 2000. On-line learning and the metrical task system problem. Machine Learning , Vol. 39, 1 (2000), 35--58.Google Scholar
Digital Library
- Allan Borodin, Nathan Linial, and Michael E Saks. 1992. An optimal on-line algorithm for metrical task system. J. ACM , Vol. 39, 4 (1992), 745--763.Google Scholar
Digital Library
- Sébastien Bubeck, Nicolo Cesa-Bianchi, et almbox. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning , Vol. 5, 1 (2012), 1--122.Google Scholar
- Sebastien Bubeck, Michael B Cohen, Yin Tat Lee, James R Lee, and Aleksander Mka dry. 2018a. k-server via multiscale entropic regularization. In Proceedings of the ACM SIGACT Symposium on Theory of Computing (STOC). 3--16.Google Scholar
Digital Library
- Sébastien Bubeck, Yin Tat Lee, Yuanzhi Li, and Mark Sellke. 2018b. Chasing nested convex bodies nearly optimally. arXiv preprint arXiv:1811.00999 (2018).Google Scholar
- Sébastien Bubeck, Yuanzhi Li, Haipeng Luo, and Chen-Yu Wei. 2019. Improved path-length regret bounds for bandits. arXiv preprint arXiv:1901.10604 (2019).Google Scholar
- Niv Buchbinder, Anupam Gupta, Marco Molinaro, and Joseph Naor. 2019. k-servers with a smile: online algorithms via projections. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA). 98--116.Google Scholar
Cross Ref
- Niangjun Chen, Anish Agarwal, Adam Wierman, Siddharth Barman, and Lachlan LH Andrew. 2015. Online convex optimization using predictions. ACM SIGMETRICS Performance Evaluation Review , Vol. 43, 1 (2015), 191--204.Google Scholar
Digital Library
- Niangjun Chen, Joshua Comden, Zhenhua Liu, Anshul Gandhi, and Adam Wierman. 2016. Using predictions in online optimization: Looking forward with an eye on the past. ACM SIGMETRICS Performance Evaluation Review , Vol. 44, 1 (2016), 193--206.Google Scholar
Digital Library
- Niangjun Chen, Gautam Goel, and Adam Wierman. 2018. Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent. In Proceedings of Conference On Learning Theory (COLT). 1574--1594.Google Scholar
- Joshua Comden, Sijie Yao, Niangjun Chen, Haipeng Xing, and Zhenhua Liu. 2019. Online Optimization in Cloud Resource Provisioning: Predictions, Regrets, and Algorithms. Proceedings of the ACM on Measurement and Analysis of Computing Systems , Vol. 3, 1 (2019), 16.Google Scholar
Digital Library
- Seyda Ertekin, Leon Bottou, and C Lee Giles. 2010. Nonconvex online support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 33, 2 (2010), 368--381.Google Scholar
Digital Library
- Gautam Goel, Niangjun Chen, and Adam Wierman. 2017. Thinking fast and slow: Optimization decomposition across timescales. In Proceedings of the IEEE Conference on Decision and Control (CDC). 1291--1298.Google Scholar
Digital Library
- Gautam Goel, Yiheng Lin, Haoyuan Sun, and Adam Wierman. 2019. Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. dtextquotesingle Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 1873--1883. http://papers.nips.cc/paper/8463-beyond-online-balanced-descent-an-optimal-algorithm-for-smoothed-online-optimization.pdfGoogle Scholar
- Gautam Goel and Adam Wierman. 2019. An Online Algorithm for Smoothed Regression and LQR Control. In Proceedings of the Machine Learning Research, Vol. 89. 2504--2513. http://proceedings.mlr.press/v89/goel19a.htmlGoogle Scholar
- Elad Hazan et almbox. 2016. Introduction to online convex optimization. Foundations and Trends in Optimization , Vol. 2, 3--4 (2016), 157--325.Google Scholar
- Prateek Jain, Purushottam Kar, et almbox. 2017. Non-convex optimization for machine learning. Foundations and Trends® in Machine Learning , Vol. 10, 3--4 (2017), 142--336.Google Scholar
- Vinay Joseph and Gustavo de Veciana. 2012. Jointly optimizing multi-user rate adaptation for video transport over wireless systems: Mean-fairness-variability tradeoffs. In Proceedings of the IEEE INFOCOM . 567--575.Google Scholar
Cross Ref
- S. Kim and G. B. Giannakis. 2017. An Online Convex Optimization Approach to Real-Time Energy Pricing for Demand Response. IEEE Transactions on Smart Grid , Vol. 8, 6 (2017), 2784--2793. https://doi.org/10.1109/TSG.2016.2539948Google Scholar
Cross Ref
- Taehwan Kim, Yisong Yue, Sarah Taylor, and Iain Matthews. 2015. A decision tree framework for spatiotemporal sequence prediction. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 577--586.Google Scholar
Digital Library
- Pavlo Krokhmal, Jonas Palmquist, and Stanislav Uryasev. 2002. Portfolio optimization with conditional value-at-risk objective and constraints. Journal of risk , Vol. 4 (2002), 43--68.Google Scholar
Cross Ref
- Yingying Li, Guannan Qu, and Na Li. 2018a. Online optimization with predictions and switching costs: Fast algorithms and the fundamental limit. arXiv preprint arXiv:1801.07780 (2018).Google Scholar
- Yingying Li, Guannan Qu, and Na Li. 2018b. Using predictions in online optimization with switching costs: A fast algorithm and a fundamental limit. In Proceedings of the American Control Conference (ACC). IEEE, 3008--3013.Google Scholar
Cross Ref
- Minghong Lin, Zhenhua Liu, Adam Wierman, and Lachlan LH Andrew. 2012. Online algorithms for geographical load balancing. In Proceedings of the International Green Computing Conference (IGCC). 1--10.Google Scholar
Digital Library
- Minghong Lin, Adam Wierman, Lachlan LH Andrew, and Eno Thereska. 2013. Dynamic right-sizing for power-proportional data centers. IEEE/ACM Transactions on Networking (TON) , Vol. 21, 5 (2013), 1378--1391.Google Scholar
Digital Library
- Zhenhua Liu, Minghong Lin, Adam Wierman, Steven H Low, and Lachlan LH Andrew. 2011. Geographical load balancing with renewables. ACM SIGMETRICS Performance Evaluation Review , Vol. 39, 3 (2011), 62--66.Google Scholar
Digital Library
- Steven H Low. 2014a. Convex relaxation of optimal power flow-Part I: Formulations and equivalence. IEEE Transactions on Control of Network Systems , Vol. 1, 1 (2014), 15--27.Google Scholar
Cross Ref
- Steven H Low. 2014b. Convex relaxation of optimal power flow-Part II: Exactness. IEEE Transactions on Control of Network Systems , Vol. 1, 2 (2014), 177--189.Google Scholar
Cross Ref
- Mark S Manasse, Lyle A McGeoch, and Daniel D Sleator. 1990. Competitive algorithms for server problems. Journal of Algorithms , Vol. 11, 2 (1990), 208--230.Google Scholar
Cross Ref
- Llew Mason, Peter L Bartlett, and Jonathan Baxter. 2000. Improved generalization through explicit optimization of margins. Machine Learning , Vol. 38, 3 (2000), 243--255.Google Scholar
Digital Library
- Arkadii Semenovich Nemirovsky and David Borisovich Yudin. 1983. Problem complexity and method efficiency in optimization. (1983).Google Scholar
- Mark Sellke. 2020. Chasing convex bodies optimally. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, 1509--1518.Google Scholar
Cross Ref
- Ming Shi, Xiaojun Lin, and Lei Jiao. 2019. On the Value of Look-Ahead in Competitive Online Convex Optimization. Proceedings of the ACM on Measurement and Analysis of Computing Systems , Vol. 3, 2 (2019), 22.Google Scholar
Digital Library
- Lin Yang, Lei Deng, Mohammad H Hajiesmaili, Cheng Tan, and Wing Shing Wong. 2018. An optimal algorithm for online non-convex learning. Proceedings of the ACM on Measurement and Analysis of Computing Systems , Vol. 2, 2 (2018), 25.Google Scholar
Digital Library
- Francesco Zanini, David Atienza, Giovanni De Micheli, and Stephen P Boyd. 2010. Online convex optimization-based algorithm for thermal management of MPSoCs. In Proceedings of the 20th symposium on Great lakes symposium on VLSI. ACM, 203--208.Google Scholar
Digital Library
- Martin Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning (ICML-03) . 928--936.Google Scholar
Digital Library
- Sanjo Zlobec. 2004. Jensen's inequality for nonconvex functions. Mathematical Communications , Vol. 9, 2 (2004), 119--124.Google Scholar
Index Terms
Online Optimization with Predictions and Non-convex Losses
Recommendations
Online Optimization with Predictions and Non-convex Losses
We study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be nonconvex, while incurring a movement cost when changing actions between rounds. We ask: under what general conditions is it ...
Online Optimization with Predictions and Non-convex Losses
SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsWe study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be non-convex, while incurring a movement cost when changing actions between rounds. We ask: under what general conditions is it ...
On the Value of Look-Ahead in Competitive Online Convex Optimization
SIGMETRICS '19: Abstracts of the 2019 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsAlthough using look-ahead information is known to improve the competitive ratios of online convex optimization (OCO) problems with switching costs, the competitive ratios obtained from existing results often depend on the cost coefficients of the ...






Comments