skip to main content
research-article
Public Access

Online Optimization with Predictions and Non-convex Losses

Published:27 May 2020Publication History
Skip Abstract Section

Abstract

We study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be non-convex, while incurring a movement cost when changing actions between rounds. We ask:under what general conditions is it possible for an online learner to leverage predictions of future cost functions in order to achieve near-optimal costs? Prior work has provided near-optimal online algorithms for specific combinations of assumptions about hitting and switching costs, but no general results are known. In this work, we give two general sufficient conditions that specify a relationship between the hitting and movement costs which guarantees that a new algorithm, Synchronized Fixed Horizon Control (SFHC), achieves a 1+O(1/w) competitive ratio, where w is the number of predictions available to the learner. Our conditions do not require the cost functions to be convex, and we also derive competitive ratio results for non-convex hitting and movement costs. Our results provide the first constant, dimension-free competitive ratio for online non-convex optimization with movement costs. We also give an example of a natural problem, Convex Body Chasing (CBC), where the sufficient conditions are not satisfied and prove that no online algorithm can have a competitive ratio that converges to 1.

References

  1. Lachlan Andrew, Siddharth Barman, Katrina Ligett, Minghong Lin, Adam Meyerson, Alan Roytman, and Adam Wierman. 2013. A tale of two metrics: Simultaneous bounds on competitiveness and regret. In Conference on Learning Theory. 741--763.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Antonios Antoniadis, Neal Barcelo, Michael Nugent, Kirk Pruhs, Kevin Schewior, and Michele Scquizzato. 2016. Chasing convex bodies and functions. In LATIN 2016: Theoretical Informatics. Springer, 68--81.Google ScholarGoogle Scholar
  3. Antonios Antoniadis and Kevin Schewior. 2017. A tight lower bound for online convex optimization with switching costs. In Proceedings of the International Workshop on Approximation and Online Algorithms. Springer, 164--175.Google ScholarGoogle Scholar
  4. David Ardia, Kris Boudt, Peter Carl, Katharine M Mullen, and Brian Peterson. 2010. Differential evolution (deoptim) for non-convex portfolio optimization. (2010).Google ScholarGoogle Scholar
  5. CJ Argue, Sébastien Bubeck, Michael B Cohen, Anupam Gupta, and Yin Tat Lee. 2019. A nearly-linear bound for chasing nested convex bodies. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA). 117--122.Google ScholarGoogle ScholarCross RefCross Ref
  6. CJ Argue, Anupam Gupta, Guru Guruganesh, and Ziye Tang. 2020. Chasing convex bodies with linear competitive ratio. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, 1519--1524.Google ScholarGoogle ScholarCross RefCross Ref
  7. Sanjeev Arora, Elad Hazan, and Satyen Kale. 2012. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing , Vol. 8, 1 (2012), 121--164.Google ScholarGoogle ScholarCross RefCross Ref
  8. Jean-Yves Audibert and Sébastien Bubeck. 2010. Regret bounds and minimax policies under partial monitoring. Journal of Machine Learning Research , Vol. 11, Oct (2010), 2785--2836.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Masoud Badiei, Na Li, and Adam Wierman. 2015a. Online convex optimization with ramp constraints. In 2015 54th IEEE Conference on Decision and Control (CDC). IEEE, 6730--6736.Google ScholarGoogle ScholarCross RefCross Ref
  10. Masoud Badiei, Na Li, and Adam Wierman. 2015b. Online convex optimization with ramp constraints. In IEEE Conference on Decision and Control (CDC) . 6730--6736. https://doi.org/10.1109/CDC.2015.7403279Google ScholarGoogle ScholarCross RefCross Ref
  11. Nikhil Bansal and Anupam Gupta. 2017. Potential-function proofs for first-order methods. arXiv preprint arXiv:1712.04581 (2017).Google ScholarGoogle Scholar
  12. Nikhil Bansal, Anupam Gupta, Ravishankar Krishnaswamy, Kirk Pruhs, Kevin Schewior, and Cliff Stein. 2015. A 2-competitive algorithm for online convex optimization with switching costs. In Proceedings of the Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google ScholarGoogle Scholar
  13. Yair Bartal, Avrim Blum, Carl Burch, and Andrew Tomkins. 1997. A Polylog(N)-competitive Algorithm for Metrical Task Systems. In Proceedings of the ACM Symposium on Theory of Computing (STOC). 711--719. https://doi.org/10.1145/258533.258667Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Avrim Blum and Carl Burch. 2000. On-line learning and the metrical task system problem. Machine Learning , Vol. 39, 1 (2000), 35--58.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Allan Borodin, Nathan Linial, and Michael E Saks. 1992. An optimal on-line algorithm for metrical task system. J. ACM , Vol. 39, 4 (1992), 745--763.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Sébastien Bubeck, Nicolo Cesa-Bianchi, et almbox. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning , Vol. 5, 1 (2012), 1--122.Google ScholarGoogle Scholar
  17. Sebastien Bubeck, Michael B Cohen, Yin Tat Lee, James R Lee, and Aleksander Mka dry. 2018a. k-server via multiscale entropic regularization. In Proceedings of the ACM SIGACT Symposium on Theory of Computing (STOC). 3--16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sébastien Bubeck, Yin Tat Lee, Yuanzhi Li, and Mark Sellke. 2018b. Chasing nested convex bodies nearly optimally. arXiv preprint arXiv:1811.00999 (2018).Google ScholarGoogle Scholar
  19. Sébastien Bubeck, Yuanzhi Li, Haipeng Luo, and Chen-Yu Wei. 2019. Improved path-length regret bounds for bandits. arXiv preprint arXiv:1901.10604 (2019).Google ScholarGoogle Scholar
  20. Niv Buchbinder, Anupam Gupta, Marco Molinaro, and Joseph Naor. 2019. k-servers with a smile: online algorithms via projections. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA). 98--116.Google ScholarGoogle ScholarCross RefCross Ref
  21. Niangjun Chen, Anish Agarwal, Adam Wierman, Siddharth Barman, and Lachlan LH Andrew. 2015. Online convex optimization using predictions. ACM SIGMETRICS Performance Evaluation Review , Vol. 43, 1 (2015), 191--204.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Niangjun Chen, Joshua Comden, Zhenhua Liu, Anshul Gandhi, and Adam Wierman. 2016. Using predictions in online optimization: Looking forward with an eye on the past. ACM SIGMETRICS Performance Evaluation Review , Vol. 44, 1 (2016), 193--206.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Niangjun Chen, Gautam Goel, and Adam Wierman. 2018. Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent. In Proceedings of Conference On Learning Theory (COLT). 1574--1594.Google ScholarGoogle Scholar
  24. Joshua Comden, Sijie Yao, Niangjun Chen, Haipeng Xing, and Zhenhua Liu. 2019. Online Optimization in Cloud Resource Provisioning: Predictions, Regrets, and Algorithms. Proceedings of the ACM on Measurement and Analysis of Computing Systems , Vol. 3, 1 (2019), 16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Seyda Ertekin, Leon Bottou, and C Lee Giles. 2010. Nonconvex online support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 33, 2 (2010), 368--381.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Gautam Goel, Niangjun Chen, and Adam Wierman. 2017. Thinking fast and slow: Optimization decomposition across timescales. In Proceedings of the IEEE Conference on Decision and Control (CDC). 1291--1298.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Gautam Goel, Yiheng Lin, Haoyuan Sun, and Adam Wierman. 2019. Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. dtextquotesingle Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 1873--1883. http://papers.nips.cc/paper/8463-beyond-online-balanced-descent-an-optimal-algorithm-for-smoothed-online-optimization.pdfGoogle ScholarGoogle Scholar
  28. Gautam Goel and Adam Wierman. 2019. An Online Algorithm for Smoothed Regression and LQR Control. In Proceedings of the Machine Learning Research, Vol. 89. 2504--2513. http://proceedings.mlr.press/v89/goel19a.htmlGoogle ScholarGoogle Scholar
  29. Elad Hazan et almbox. 2016. Introduction to online convex optimization. Foundations and Trends in Optimization , Vol. 2, 3--4 (2016), 157--325.Google ScholarGoogle Scholar
  30. Prateek Jain, Purushottam Kar, et almbox. 2017. Non-convex optimization for machine learning. Foundations and Trends® in Machine Learning , Vol. 10, 3--4 (2017), 142--336.Google ScholarGoogle Scholar
  31. Vinay Joseph and Gustavo de Veciana. 2012. Jointly optimizing multi-user rate adaptation for video transport over wireless systems: Mean-fairness-variability tradeoffs. In Proceedings of the IEEE INFOCOM . 567--575.Google ScholarGoogle ScholarCross RefCross Ref
  32. S. Kim and G. B. Giannakis. 2017. An Online Convex Optimization Approach to Real-Time Energy Pricing for Demand Response. IEEE Transactions on Smart Grid , Vol. 8, 6 (2017), 2784--2793. https://doi.org/10.1109/TSG.2016.2539948Google ScholarGoogle ScholarCross RefCross Ref
  33. Taehwan Kim, Yisong Yue, Sarah Taylor, and Iain Matthews. 2015. A decision tree framework for spatiotemporal sequence prediction. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 577--586.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Pavlo Krokhmal, Jonas Palmquist, and Stanislav Uryasev. 2002. Portfolio optimization with conditional value-at-risk objective and constraints. Journal of risk , Vol. 4 (2002), 43--68.Google ScholarGoogle ScholarCross RefCross Ref
  35. Yingying Li, Guannan Qu, and Na Li. 2018a. Online optimization with predictions and switching costs: Fast algorithms and the fundamental limit. arXiv preprint arXiv:1801.07780 (2018).Google ScholarGoogle Scholar
  36. Yingying Li, Guannan Qu, and Na Li. 2018b. Using predictions in online optimization with switching costs: A fast algorithm and a fundamental limit. In Proceedings of the American Control Conference (ACC). IEEE, 3008--3013.Google ScholarGoogle ScholarCross RefCross Ref
  37. Minghong Lin, Zhenhua Liu, Adam Wierman, and Lachlan LH Andrew. 2012. Online algorithms for geographical load balancing. In Proceedings of the International Green Computing Conference (IGCC). 1--10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Minghong Lin, Adam Wierman, Lachlan LH Andrew, and Eno Thereska. 2013. Dynamic right-sizing for power-proportional data centers. IEEE/ACM Transactions on Networking (TON) , Vol. 21, 5 (2013), 1378--1391.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Zhenhua Liu, Minghong Lin, Adam Wierman, Steven H Low, and Lachlan LH Andrew. 2011. Geographical load balancing with renewables. ACM SIGMETRICS Performance Evaluation Review , Vol. 39, 3 (2011), 62--66.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Steven H Low. 2014a. Convex relaxation of optimal power flow-Part I: Formulations and equivalence. IEEE Transactions on Control of Network Systems , Vol. 1, 1 (2014), 15--27.Google ScholarGoogle ScholarCross RefCross Ref
  41. Steven H Low. 2014b. Convex relaxation of optimal power flow-Part II: Exactness. IEEE Transactions on Control of Network Systems , Vol. 1, 2 (2014), 177--189.Google ScholarGoogle ScholarCross RefCross Ref
  42. Mark S Manasse, Lyle A McGeoch, and Daniel D Sleator. 1990. Competitive algorithms for server problems. Journal of Algorithms , Vol. 11, 2 (1990), 208--230.Google ScholarGoogle ScholarCross RefCross Ref
  43. Llew Mason, Peter L Bartlett, and Jonathan Baxter. 2000. Improved generalization through explicit optimization of margins. Machine Learning , Vol. 38, 3 (2000), 243--255.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Arkadii Semenovich Nemirovsky and David Borisovich Yudin. 1983. Problem complexity and method efficiency in optimization. (1983).Google ScholarGoogle Scholar
  45. Mark Sellke. 2020. Chasing convex bodies optimally. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, 1509--1518.Google ScholarGoogle ScholarCross RefCross Ref
  46. Ming Shi, Xiaojun Lin, and Lei Jiao. 2019. On the Value of Look-Ahead in Competitive Online Convex Optimization. Proceedings of the ACM on Measurement and Analysis of Computing Systems , Vol. 3, 2 (2019), 22.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Lin Yang, Lei Deng, Mohammad H Hajiesmaili, Cheng Tan, and Wing Shing Wong. 2018. An optimal algorithm for online non-convex learning. Proceedings of the ACM on Measurement and Analysis of Computing Systems , Vol. 2, 2 (2018), 25.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Francesco Zanini, David Atienza, Giovanni De Micheli, and Stephen P Boyd. 2010. Online convex optimization-based algorithm for thermal management of MPSoCs. In Proceedings of the 20th symposium on Great lakes symposium on VLSI. ACM, 203--208.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Martin Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning (ICML-03) . 928--936.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Sanjo Zlobec. 2004. Jensen's inequality for nonconvex functions. Mathematical Communications , Vol. 9, 2 (2004), 119--124.Google ScholarGoogle Scholar

Index Terms

  1. Online Optimization with Predictions and Non-convex Losses

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!