skip to main content
research-article
Open Access

Nudge: Stochastically Improving upon FCFS

Published:04 June 2021Publication History
Skip Abstract Section

Abstract

The First-Come First-Served (FCFS) scheduling policy is the most popular scheduling algorithm used in practice. Furthermore, its usage is theoretically validated: for light-tailed job size distributions, FCFS has weakly optimal asymptotic tail of response time. But what if we don't just care about the asymptotic tail? What if we also care about the 99th percentile of response time, or the fraction of jobs that complete in under one second? Is FCFS still best? Outside of the asymptotic regime, only loose bounds on the tail of FCFS are known, and optimality is completely open.

In this paper, we introduce a new policy, Nudge, which is the first policy to provably stochastically improve upon FCFS. We prove that Nudge simultaneously improves upon FCFS at every point along the tail, for light-tailed job size distributions. As a result, Nudge outperforms FCFS for every moment and every percentile of response time. Moreover, Nudge provides a multiplicative improvement over FCFS in the asymptotic tail. This resolves a long-standing open problem by showing that, counter to previous conjecture, FCFS is not strongly asymptotically optimal.

References

  1. Joseph Abate, Gagan L Choudhury, and Ward Whitt. 1994. Waiting-time tail probabilities in queues with long-tail service-time distributions. Queueing systems, Vol. 16, 3--4 (1994), 311--338.Google ScholarGoogle Scholar
  2. Joseph Abate, Gagan L. Choudhury, and Ward Whitt. 1995. Exponential Approximations for Tail Probabilities in Queues, I: Waiting Times., Vol. 43, 5 (1995), 885--901.Google ScholarGoogle Scholar
  3. Joseph Abate and Ward Whitt. 1997. Asymptotics for M/G/1 low-priority waiting-time tail probabilities. Queueing Systems, Vol. 25, 1--4 (1997), 173--233.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. John T. Blake and Michael W. Carter. 1996. An analysis of emergency room wait time issues via computer simulation. INFOR, Vol. 34, 4 (November 1996), 263--273.Google ScholarGoogle Scholar
  5. Onno Boxma and Bert Zwart. 2007. Tails in Scheduling. ACM SIGMETRICS Performance Evaluation Review, Vol. 34, 4 (March 2007), 13--20.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Percy H Brill. 2000. A brief outline of the level crossing method in stochastic models. CORS Bulletin, Vol. 34, 4 (2000), 9--21.Google ScholarGoogle Scholar
  7. Patrick Chareka. 2007. A Finite-Interval Uniqueness Theorem for Bilateral Laplace Transforms. International Journal of Mathematics and Mathematical Sciences, Vol. 2007 (2007), 6 pages.Google ScholarGoogle ScholarCross RefCross Ref
  8. Y. Chen, S. Iyer, X. Liu, D. Milojicic, and A. Sahai. 2007. SLA Decomposition: Translating Service Level Objectives to System Level Thresholds. In Fourth International Conference on Autonomic Computing (ICAC'07). 10 pages.Google ScholarGoogle Scholar
  9. Mark E. Crovella, Murad S. Taqqu, and Azer Bestavros. 1998. Heavy-Tailed Probability Distributions in the World Wide Web. In A Practical Guide To Heavy Tails. Chapman & Hall, New York, Chapter 1, 1--23.Google ScholarGoogle Scholar
  10. Mark M. Davis. 1991. How Long Should a Customer Wait for Service? Decision Sciences, Vol. 22, 2 (1991), 421--434.Google ScholarGoogle ScholarCross RefCross Ref
  11. Eric J. Friedman and Shane G. Henderson. 2003. Fairness and Efficiency in Web Server Protocols. In Proceedings of the 2003 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (San Diego, CA, USA) (SIGMETRICS '03). Association for Computing Machinery, New York, NY, USA, 229--237.Google ScholarGoogle Scholar
  12. Eric J Friedman and Gavin Hurley. 2003. Protective scheduling. Technical Report. Cornell University Operations Research and Industrial Engineering.Google ScholarGoogle Scholar
  13. S. W. Fuhrmann and Robert B. Cooper. 1985. Stochastic Decompositions in the M /G /1 Queue with Generalized Vacations. Operations Research, Vol. 33, 5 (Oct. 1985), 1117--1129.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Mor Harchol-Balter. 1999. The Effect of Heavy-Tailed Job Size Distributions on Computer System Design. In Proceedings of the ASA-IMS Conference on Applications of Heavy Tailed Distributions in Economics, Engineering and Statistics. Washington, DC.Google ScholarGoogle Scholar
  15. Mor Harchol-Balter. 2002. Task Assignment with Unknown Duration. J. ACM, Vol. 49, 2 (March 2002), 260--288.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mor Harchol-Balter. 2013. Performance Modeling and Design of Computer Systems: Queueing Theory in Action .Cambridge University Press, Cambridge. QA76.545 .H37 2013Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Mor Harchol-Balter. 2021. Open poblems in queueing theory inspired by datacenter computing. Queueing Systems: Theory and Applications, Vol. 97, 1 (2021), 3--37.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Mor Harchol-Balter, Mark Crovella, and Cristina Murta. 1999. On Choosing a Task Assignment Policy for a Distributed Server System. IEEE Journal of Parallel and Distributed Computing, Vol. 59 (1999), 204--228.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Leora I. Horwitz, Jeremy Green, and Elizabeth H. Bradley. 2010. US Emergency Department Performance on Wait Time and Length of Visit. Annals of Emergency Medicine, Vol. 55, 2 (2010), 133 -- 141.Google ScholarGoogle ScholarCross RefCross Ref
  20. John F. C. Kingman. 1962. On Queues in Which Customers Are Served in Random Order. Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 58, 1 (Jan. 1962), 79--91.Google ScholarGoogle ScholarCross RefCross Ref
  21. J. F. C. Kingman. 1964. A martingale inequality in the theory of queues. Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 60, 2 (1964), 359--361.Google ScholarGoogle ScholarCross RefCross Ref
  22. J. F. C. Kingman. 1970. Inequalities in the Theory of Queues. Journal of the Royal Statistical Society: Series B (Methodological), Vol. 32, 1 (1970), 102--110.Google ScholarGoogle ScholarCross RefCross Ref
  23. Leonard Kleinrock. 1976. Queueing Systems, Volume 2: Computer Applications .Wiley, New York, NY .Google ScholarGoogle Scholar
  24. Jeffrey C. Mogul and John Wilkes. 2019. Nines are not enough: Meaningful metrics for clouds. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS19). USA, 136 -- 141.Google ScholarGoogle Scholar
  25. Jayakrishnan Nair, Adam Wierman, and Bert Zwart. 2010. Tail-robust scheduling via limited processor sharing. Performance Evaluation, Vol. 67, 11 (2010), 978 -- 995.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Misja Nuyens, Adam Wierman, and Bert Zwart. 2008. Preventing Large Sojourn Times Using SMART Scheduling. Operations Research, Vol. 56, 1 (2008), 88--101.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Natalia Osipova, Urtzi Ayesta, and Konstantin Avrachenkov. 2009. Optimal Policy for Multi-Class Scheduling in a Single Server Queue. In 2009 21st International Teletraffic Congress. IEEE, Paris, France, 1--8.Google ScholarGoogle Scholar
  28. T. Sakurai. 2004. Approximating M/G/1 Waiting Time Tail Probabilities. Stochastic Models, Vol. 20, 2 (2004), 173--191.Google ScholarGoogle ScholarCross RefCross Ref
  29. Linus Schrage. 1968. A proof of the optimality of the shortest remaining processing time discipline. Operations Research, Vol. 16 (1968), 687--690.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Linus E. Schrage. 1967. The Queue M /G /1 with Feedback to Lower Priority Queues. Management Science, Vol. 13, 7 (March 1967), 466--474.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Linus E. Schrage and Louis W. Miller. 1966. The Queue M /G /1 with the Shortest Remaining Processing Time Discipline. Operations Research, Vol. 14, 4 (Aug. 1966), 670--684.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Ziv Scully, Mor Harchol-Balter, and Alan Scheller-Wolf. 2018. SOAP: One Clean Analysis of All Age-Based Scheduling Policies. Proc. ACM Meas. Anal. Comput. Syst., Vol. 2, 1, Article 16 (April 2018), 30 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Ziv Scully, Lucas van Kreveld, Onno J. Boxma, Jan-Pieter Dorsman, and Adam Wierman. 2020. Characterizing Policies with Optimal Response Time Tails under Heavy-Tailed Job Sizes. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 4, 2, Article 30 (June 2020), 33 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kut C. So and Jing-Sheng Song. 1998. Price, delivery time guarantees and capacity selection. European Journal of Operational Research, Vol. 111, 1 (1998), 28 -- 49.Google ScholarGoogle ScholarCross RefCross Ref
  35. David A Stanford, Peter Taylor, and Ilze Ziedins. 2014. Waiting time distributions in the accumulating priority queue. Queueing Systems, Vol. 77, 3 (2014), 297--330.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Alexander L. Stolyar and Kavita Ramanan. 2001. Largest Weighted Delay First Scheduling: Large Deviations and Optimality. Annals of Applied Probability, Vol. 11, 1 (2001), 1--48.Google ScholarGoogle ScholarCross RefCross Ref
  37. Muhammad Tirmazi, Adam Barker, Nan Deng, MD E. Haque, Zhijing Gene Qin, Steven Hand, Mor Harchol-Balter, and John Wilkes. 2020. Borg: The next generation. In Proceedings of the Fifteenth European Conference on Computer Systems (EuroSys '20). Greece, 1--14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Timothy L. Urban. 2009. Establishing delivery guarantee policies. European Journal of Operational Research, Vol. 196, 3 (2009), 959 -- 967.Google ScholarGoogle ScholarCross RefCross Ref
  39. Peter D. Welch. 1964. On a Generalized M /G /1 Queuing Process in Which the First Customer of Each Busy Period Receives Exceptional Service. Operations Research, Vol. 12, 5 (Oct. 1964), 736--752.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Adam Wierman and Bert Zwart. 2012. Is Tail-Optimal Scheduling Possible? Operations Research, Vol. 60, 5 (Oct. 2012), 1249--1257.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Ronald W. Wolff. 1982. Poisson Arrivals See Time Averages. Operations Research, Vol. 30, 2 (1982), 223--231.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Nudge: Stochastically Improving upon FCFS

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!