Abstract
The First-Come First-Served (FCFS) scheduling policy is the most popular scheduling algorithm used in practice. Furthermore, its usage is theoretically validated: for light-tailed job size distributions, FCFS has weakly optimal asymptotic tail of response time. But what if we don't just care about the asymptotic tail? What if we also care about the 99th percentile of response time, or the fraction of jobs that complete in under one second? Is FCFS still best? Outside of the asymptotic regime, only loose bounds on the tail of FCFS are known, and optimality is completely open.
In this paper, we introduce a new policy, Nudge, which is the first policy to provably stochastically improve upon FCFS. We prove that Nudge simultaneously improves upon FCFS at every point along the tail, for light-tailed job size distributions. As a result, Nudge outperforms FCFS for every moment and every percentile of response time. Moreover, Nudge provides a multiplicative improvement over FCFS in the asymptotic tail. This resolves a long-standing open problem by showing that, counter to previous conjecture, FCFS is not strongly asymptotically optimal.
- Joseph Abate, Gagan L Choudhury, and Ward Whitt. 1994. Waiting-time tail probabilities in queues with long-tail service-time distributions. Queueing systems, Vol. 16, 3--4 (1994), 311--338.Google Scholar
- Joseph Abate, Gagan L. Choudhury, and Ward Whitt. 1995. Exponential Approximations for Tail Probabilities in Queues, I: Waiting Times., Vol. 43, 5 (1995), 885--901.Google Scholar
- Joseph Abate and Ward Whitt. 1997. Asymptotics for M/G/1 low-priority waiting-time tail probabilities. Queueing Systems, Vol. 25, 1--4 (1997), 173--233.Google Scholar
Digital Library
- John T. Blake and Michael W. Carter. 1996. An analysis of emergency room wait time issues via computer simulation. INFOR, Vol. 34, 4 (November 1996), 263--273.Google Scholar
- Onno Boxma and Bert Zwart. 2007. Tails in Scheduling. ACM SIGMETRICS Performance Evaluation Review, Vol. 34, 4 (March 2007), 13--20.Google Scholar
Digital Library
- Percy H Brill. 2000. A brief outline of the level crossing method in stochastic models. CORS Bulletin, Vol. 34, 4 (2000), 9--21.Google Scholar
- Patrick Chareka. 2007. A Finite-Interval Uniqueness Theorem for Bilateral Laplace Transforms. International Journal of Mathematics and Mathematical Sciences, Vol. 2007 (2007), 6 pages.Google Scholar
Cross Ref
- Y. Chen, S. Iyer, X. Liu, D. Milojicic, and A. Sahai. 2007. SLA Decomposition: Translating Service Level Objectives to System Level Thresholds. In Fourth International Conference on Autonomic Computing (ICAC'07). 10 pages.Google Scholar
- Mark E. Crovella, Murad S. Taqqu, and Azer Bestavros. 1998. Heavy-Tailed Probability Distributions in the World Wide Web. In A Practical Guide To Heavy Tails. Chapman & Hall, New York, Chapter 1, 1--23.Google Scholar
- Mark M. Davis. 1991. How Long Should a Customer Wait for Service? Decision Sciences, Vol. 22, 2 (1991), 421--434.Google Scholar
Cross Ref
- Eric J. Friedman and Shane G. Henderson. 2003. Fairness and Efficiency in Web Server Protocols. In Proceedings of the 2003 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (San Diego, CA, USA) (SIGMETRICS '03). Association for Computing Machinery, New York, NY, USA, 229--237.Google Scholar
- Eric J Friedman and Gavin Hurley. 2003. Protective scheduling. Technical Report. Cornell University Operations Research and Industrial Engineering.Google Scholar
- S. W. Fuhrmann and Robert B. Cooper. 1985. Stochastic Decompositions in the M /G /1 Queue with Generalized Vacations. Operations Research, Vol. 33, 5 (Oct. 1985), 1117--1129.Google Scholar
Digital Library
- Mor Harchol-Balter. 1999. The Effect of Heavy-Tailed Job Size Distributions on Computer System Design. In Proceedings of the ASA-IMS Conference on Applications of Heavy Tailed Distributions in Economics, Engineering and Statistics. Washington, DC.Google Scholar
- Mor Harchol-Balter. 2002. Task Assignment with Unknown Duration. J. ACM, Vol. 49, 2 (March 2002), 260--288.Google Scholar
Digital Library
- Mor Harchol-Balter. 2013. Performance Modeling and Design of Computer Systems: Queueing Theory in Action .Cambridge University Press, Cambridge. QA76.545 .H37 2013Google Scholar
Digital Library
- Mor Harchol-Balter. 2021. Open poblems in queueing theory inspired by datacenter computing. Queueing Systems: Theory and Applications, Vol. 97, 1 (2021), 3--37.Google Scholar
Digital Library
- Mor Harchol-Balter, Mark Crovella, and Cristina Murta. 1999. On Choosing a Task Assignment Policy for a Distributed Server System. IEEE Journal of Parallel and Distributed Computing, Vol. 59 (1999), 204--228.Google Scholar
Digital Library
- Leora I. Horwitz, Jeremy Green, and Elizabeth H. Bradley. 2010. US Emergency Department Performance on Wait Time and Length of Visit. Annals of Emergency Medicine, Vol. 55, 2 (2010), 133 -- 141.Google Scholar
Cross Ref
- John F. C. Kingman. 1962. On Queues in Which Customers Are Served in Random Order. Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 58, 1 (Jan. 1962), 79--91.Google Scholar
Cross Ref
- J. F. C. Kingman. 1964. A martingale inequality in the theory of queues. Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 60, 2 (1964), 359--361.Google Scholar
Cross Ref
- J. F. C. Kingman. 1970. Inequalities in the Theory of Queues. Journal of the Royal Statistical Society: Series B (Methodological), Vol. 32, 1 (1970), 102--110.Google Scholar
Cross Ref
- Leonard Kleinrock. 1976. Queueing Systems, Volume 2: Computer Applications .Wiley, New York, NY .Google Scholar
- Jeffrey C. Mogul and John Wilkes. 2019. Nines are not enough: Meaningful metrics for clouds. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS19). USA, 136 -- 141.Google Scholar
- Jayakrishnan Nair, Adam Wierman, and Bert Zwart. 2010. Tail-robust scheduling via limited processor sharing. Performance Evaluation, Vol. 67, 11 (2010), 978 -- 995.Google Scholar
Digital Library
- Misja Nuyens, Adam Wierman, and Bert Zwart. 2008. Preventing Large Sojourn Times Using SMART Scheduling. Operations Research, Vol. 56, 1 (2008), 88--101.Google Scholar
Digital Library
- Natalia Osipova, Urtzi Ayesta, and Konstantin Avrachenkov. 2009. Optimal Policy for Multi-Class Scheduling in a Single Server Queue. In 2009 21st International Teletraffic Congress. IEEE, Paris, France, 1--8.Google Scholar
- T. Sakurai. 2004. Approximating M/G/1 Waiting Time Tail Probabilities. Stochastic Models, Vol. 20, 2 (2004), 173--191.Google Scholar
Cross Ref
- Linus Schrage. 1968. A proof of the optimality of the shortest remaining processing time discipline. Operations Research, Vol. 16 (1968), 687--690.Google Scholar
Digital Library
- Linus E. Schrage. 1967. The Queue M /G /1 with Feedback to Lower Priority Queues. Management Science, Vol. 13, 7 (March 1967), 466--474.Google Scholar
Digital Library
- Linus E. Schrage and Louis W. Miller. 1966. The Queue M /G /1 with the Shortest Remaining Processing Time Discipline. Operations Research, Vol. 14, 4 (Aug. 1966), 670--684.Google Scholar
Digital Library
- Ziv Scully, Mor Harchol-Balter, and Alan Scheller-Wolf. 2018. SOAP: One Clean Analysis of All Age-Based Scheduling Policies. Proc. ACM Meas. Anal. Comput. Syst., Vol. 2, 1, Article 16 (April 2018), 30 pages.Google Scholar
Digital Library
- Ziv Scully, Lucas van Kreveld, Onno J. Boxma, Jan-Pieter Dorsman, and Adam Wierman. 2020. Characterizing Policies with Optimal Response Time Tails under Heavy-Tailed Job Sizes. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 4, 2, Article 30 (June 2020), 33 pages.Google Scholar
Digital Library
- Kut C. So and Jing-Sheng Song. 1998. Price, delivery time guarantees and capacity selection. European Journal of Operational Research, Vol. 111, 1 (1998), 28 -- 49.Google Scholar
Cross Ref
- David A Stanford, Peter Taylor, and Ilze Ziedins. 2014. Waiting time distributions in the accumulating priority queue. Queueing Systems, Vol. 77, 3 (2014), 297--330.Google Scholar
Digital Library
- Alexander L. Stolyar and Kavita Ramanan. 2001. Largest Weighted Delay First Scheduling: Large Deviations and Optimality. Annals of Applied Probability, Vol. 11, 1 (2001), 1--48.Google Scholar
Cross Ref
- Muhammad Tirmazi, Adam Barker, Nan Deng, MD E. Haque, Zhijing Gene Qin, Steven Hand, Mor Harchol-Balter, and John Wilkes. 2020. Borg: The next generation. In Proceedings of the Fifteenth European Conference on Computer Systems (EuroSys '20). Greece, 1--14.Google Scholar
Digital Library
- Timothy L. Urban. 2009. Establishing delivery guarantee policies. European Journal of Operational Research, Vol. 196, 3 (2009), 959 -- 967.Google Scholar
Cross Ref
- Peter D. Welch. 1964. On a Generalized M /G /1 Queuing Process in Which the First Customer of Each Busy Period Receives Exceptional Service. Operations Research, Vol. 12, 5 (Oct. 1964), 736--752.Google Scholar
Digital Library
- Adam Wierman and Bert Zwart. 2012. Is Tail-Optimal Scheduling Possible? Operations Research, Vol. 60, 5 (Oct. 2012), 1249--1257.Google Scholar
Digital Library
- Ronald W. Wolff. 1982. Poisson Arrivals See Time Averages. Operations Research, Vol. 30, 2 (1982), 223--231.Google Scholar
Digital Library
Index Terms
Nudge: Stochastically Improving upon FCFS
Recommendations
Nudge: Stochastically Improving upon FCFS
SIGMETRICS '21: Abstract Proceedings of the 2021 ACM SIGMETRICS / International Conference on Measurement and Modeling of Computer SystemsThe First-Come First-Served (FCFS) scheduling policy is the most popular scheduling algorithm used in practice. Furthermore, its usage is theoretically validated: for light-tailed job size distributions, FCFS has weakly optimal asymptotic tail of ...
Nudge: Stochastically Improving upon FCFS
SIGMETRICS '21The First-Come First-Served (FCFS) scheduling policy is the most popular scheduling algorithm used in practice. Furthermore, its usage is theoretically validated: for light-tailed job size distributions, FCFS has weakly optimal asymptotic tail of ...
Response times in a two-node queueing network with feedback
The study presented in this paper is motivated by the performance analysis of response times in distributed information systems, where transactions are handled by iterative server and database actions. We model system response times as sojourn times in ...






Comments