skip to main content

Executing Microservice Applications on Serverless, Correctly

Published:11 January 2023Publication History
Skip Abstract Section

Abstract

While serverless platforms substantially simplify the provisioning, configuration, and management of cloud applications, implementing correct services on top of these platforms can present significant challenges to programmers. For example, serverless infrastructures introduce a host of failure modes that are not present in traditional deployments. Individual serverless instances can fail while others continue to make progress, correct but slow instances can be killed by the cloud provider as part of resource management, and providers will often respond to such failures by re-executing requests. For functions with side-effects, these scenarios can create behaviors that are not observable in serverful deployments.

In this paper, we propose mu2sls, a framework for implementing microservice applications on serverless using standard Python code with two extra primitives: transactions and asynchronous calls. Our framework orchestrates user-written services to address several challenges, such as failures and re-executions, and provides formal guarantees that the generated serverless implementations are correct. To that end, we present a novel service specification abstraction and formalization of serverless implementations that facilitate reasoning about the correctness of a given application’s serverless implementation. This formalization forms the basis of the mu2sls prototype, which we then use to develop a few real-world microservice applications and show that the performance of the generated serverless implementations achieves significant scalability (3-5× the throughput of a sequential implementation) while providing correctness guarantees in the context of faults, re-execution, and concurrency.

References

  1. Martín Abadi, Andrew Birrell, Tim Harris, and Michael Isard. 2008. Semantics of Transactional Memory and Automatic Mutual Exclusion. In Proceedings of the 35th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’08). Association for Computing Machinery, New York, NY, USA. 63–74. isbn:9781595936899 https://doi.org/10.1145/1328438.1328449 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Kalev Alpernas, Aurojit Panda, Leonid Ryzhyk, and Mooly Sagiv. 2021. Cloud-Scale Runtime Verification of Serverless Applications. In Proceedings of the ACM Symposium on Cloud Computing (SoCC ’21). Association for Computing Machinery, New York, NY, USA. 92–107. isbn:9781450386388 https://doi.org/10.1145/3472883.3486977 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. 2020. AWS Step Functions. https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html Google ScholarGoogle Scholar
  4. 2022. FoundationDB. https://www.foundationdb.org/ Google ScholarGoogle Scholar
  5. Joe Armstrong. 1997. The development of Erlang. In ICFP. 97, 196–203. Google ScholarGoogle Scholar
  6. 2022. AWS Lambda. https://aws.amazon.com/lambda/ Google ScholarGoogle Scholar
  7. Jonas Bonér. 2020. Towards Stateful Serverless. https://www.youtube.com/watch?v=DVTf5WQlgB8 Google ScholarGoogle Scholar
  8. Sebastian Burckhardt, Chris Gillum, David Justo, Konstantinos Kallas, Connor McMahon, and Christopher S. Meiklejohn. 2021. Durable Functions: Semantics for Stateful Serverless. Proc. ACM Program. Lang., 5, OOPSLA (2021), Article 133, Oct, 27 pages. https://doi.org/10.1145/3485510 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Sergey Bykov, Alan Geller, Gabriel Kliot, James R. Larus, Ravi Pandya, and Jorgen Thelin. 2011. Orleans: Cloud Computing for Everyone. In Proceedings of the 2nd ACM Symposium on Cloud Computing (SOCC ’11). Association for Computing Machinery, New York, NY, USA. Article 16, 14 pages. isbn:9781450309769 https://doi.org/10.1145/2038916.2038932 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. 2020. Using Durable Objects, Cloudflare Docs. https://developers.cloudflare.com/workers/learning/using-durable-objects Google ScholarGoogle Scholar
  11. 2020. Workers Durable Objects Beta: A New Approach to Stateful Serverless. https://blog.cloudflare.com/introducing-workers-durable-objects/ Google ScholarGoogle Scholar
  12. Dmitry Duplyakin, Robert Ricci, Aleksander Maricq, Gary Wong, Jonathon Duerig, Eric Eide, Leigh Stoller, Mike Hibler, David Johnson, Kirk Webb, Aditya Akella, Kuangching Wang, Glenn Ricart, Larry Landweber, Chip Elliott, Michael Zink, Emmanuel Cecchet, Snigdhaswin Kar, and Prabodh Mishra. 2019. The Design and Operation of CloudLab. In Proceedings of the USENIX Annual Technical Conference (ATC). 1–14. https://www.flux.utah.edu/paper/duplyakin-atc19 Google ScholarGoogle Scholar
  13. Sadjad Fouladi, Francisco Romero, Dan Iter, Qian Li, Shuvo Chatterjee, Christos Kozyrakis, Matei Zaharia, and Keith Winstein. 2019. From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers. In Proceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC ’19). USENIX Association, USA. 475–488. isbn:9781939133038 https://dl.acm.org/doi/10.5555/3358807.3358848 Google ScholarGoogle Scholar
  14. Sadjad Fouladi, Riad S. Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads. In Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation (NSDI’17). USENIX Association, USA. 363–376. isbn:9781931971379 https://dl.acm.org/doi/10.5555/3154630.3154660 Google ScholarGoogle Scholar
  15. Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, and Brendon Jackson. 2019. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. 3–18. https://doi.org/10.1145/3297858.3304013 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. 2022. Google Cloud Functions. https://cloud.google.com/functions Google ScholarGoogle Scholar
  17. Philipp Haller. 2012. On the integration of the actor model in mainstream technologies: the Scala perspective. In Proceedings of the 2nd Edition on Programming Systems, Languages and Applications based on Actors, Agents, and Decentralized Control Abstractions. 1–6. https://doi.org/10.1145/2414639.2414641 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Maurice P Herlihy and Jeannette M Wing. 1990. Linearizability: A correctness condition for concurrent objects. ACM Transactions on Programming Languages and Systems (TOPLAS), 12, 3 (1990), 463–492. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. 2022. Hypercorn. https://pgjones.gitlab.io/hypercorn Google ScholarGoogle Scholar
  20. Suresh Jagannathan, Jan Vitek, Adam Welc, and Antony Hosking. 2005. A transactional object calculus. Science of Computer Programming, 57, 2 (2005), 164–186. https://doi.org/10.1016/j.scico.2005.03.001 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Abhinav Jangda, Donald Pinckney, Yuriy Brun, and Arjun Guha. 2019. Formal foundations of serverless computing. Proceedings of the ACM on Programming Languages, 3, OOPSLA (2019), 1–26. https://doi.org/10.1145/3360575 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Zhipeng Jia and Emmett Witchel. 2021. Boki: Stateful serverless computing with shared logs. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles. 691–707. https://doi.org/10.1145/3477132.3483541 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Zhipeng Jia and Emmett Witchel. 2021. Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 152–166. https://doi.org/10.1145/3445814.3446701 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the cloud: Distributed computing for the 99%. In Proceedings of the 2017 Symposium on Cloud Computing. 445–451. https://doi.org/10.1145/3127479.3128601 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, and Neeraja Yadwadkar. 2019. Cloud programming simplified: A berkeley view on serverless computing. arXiv preprint arXiv:1902.03383, https://doi.org/10.48550/arXiv.1902.03383 Google ScholarGoogle Scholar
  26. Ana Klimovic, Yawen Wang, Patrick Stuedi, Animesh Trivedi, Jonas Pfefferle, and Christos Kozyrakis. 2018. Pocket: Elastic ephemeral storage for serverless analytics. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 427–444. https://dl.acm.org/doi/10.5555/3291168.3291200 Google ScholarGoogle Scholar
  27. 2022. Knative. https://knative.dev Google ScholarGoogle Scholar
  28. Mohsen Lesani, Li-yao Xia, Anders Kaseorg, Christian J Bell, Adam Chlipala, Benjamin C Pierce, and Steve Zdancewic. 2022. C4: verified transactional objects. Proceedings of the ACM on Programming Languages, 6, OOPSLA (2022), 1–31. https://doi.org/10.1145/3527324 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Pedro García López, Aitor Arjona, Josep Sampé, Aleksander Slominski, and Lionel Villard. 2020. Triggerflow: Trigger-Based Orchestration of Serverless Workflows. In Proceedings of the 14th ACM International Conference on Distributed and Event-Based Systems (DEBS ’20). Association for Computing Machinery, New York, NY, USA. 3–14. isbn:9781450380287 https://doi.org/10.1145/3401025.3401731 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Christopher S. Meiklejohn, Andrea Estrada, Yiwen Song, Heather Miller, and Rohan Padhye. 2021. Service-Level Fault Injection Testing. In Proceedings of the ACM Symposium on Cloud Computing (SoCC ’21). Association for Computing Machinery, New York, NY, USA. 388–402. isbn:9781450386388 https://doi.org/10.1145/3472883.3487005 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. 2022. Azure Functions. https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview Google ScholarGoogle Scholar
  32. 2022. Minikube. https://minikube.sigs.k8s.io Google ScholarGoogle Scholar
  33. Katherine F Moore and Dan Grossman. 2008. High-level small-step operational semantics for transactions. In Proceedings of the 35th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. 51–62. https://doi.org/10.1145/1328897.1328448 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Atif Nazir, Saqib Raza, and Chen-Nee Chuah. 2008. Unveiling facebook: a measurement study of social network based applications. In Proceedings of the 8th ACM SIGCOMM Internet Measurement Conference. ACM. https://doi.org/10.1145/1452520.1452527 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Aurojit Panda, Mooly Sagiv, and Scott Shenker. 2017. Verification in the age of microservices. In Proceedings of the 16th Workshop on Hot Topics in Operating Systems. 30–36. https://doi.org/10.1145/3102980.3102986 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Benjamin C Pierce, Chris Casinghino, Marco Gaboardi, Michael Greenberg, Cătălin Hriţcu, Vilhelm Sjöberg, and Brent Yorgey. 2010. Software foundations. Webpage: http://www.cis.upenn.edu/bcpierce/sf/current/index.html. Google ScholarGoogle Scholar
  37. 2022. Quart. https://pgjones.gitlab.io/quart Google ScholarGoogle Scholar
  38. Ganesan Ramalingam and Kapil Vaswani. 2013. Fault tolerance via idempotence. In Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. 249–262. https://doi.org/10.1145/2429069.2429100 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. 2022. Ray Core: A faster, simpler approach to parallel Python. https://ray.io/ray-core Google ScholarGoogle Scholar
  40. Vikram Sreekanti, Chenggang Wu, Saurav Chhatrapati, Joseph E Gonzalez, Joseph M Hellerstein, and Jose M Faleiro. 2020. A fault-tolerance shim for serverless computing. In Proceedings of the Fifteenth European Conference on Computer Systems. 1–15. https://doi.org/10.1145/3342195.3387535 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Vikram Sreekanti, Chenggang Wu, Xiayue Charles Lin, Johann Schleier-Smith, Joseph E. Gonzalez, Joseph M. Hellerstein, and Alexey Tumanov. 2020. Cloudburst: Stateful Functions-as-a-Service. Proc. VLDB Endow., 13, 12 (2020), sep, 2438–2452. issn:2150-8097 https://doi.org/10.14778/3407790.3407836 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. 2022. Temporal. https://temporal.io/ Google ScholarGoogle Scholar
  43. Gil Tene. 2022. wrk2. https://github.com/giltene/wrk2 Google ScholarGoogle Scholar
  44. Jan Vitek, Suresh Jagannathan, Adam Welc, and Antony L Hosking. 2004. A semantic framework for designer transactions. In European Symposium on Programming. 249–263. https://doi.org/10.1007/978-3-540-24725-8_18 Google ScholarGoogle ScholarCross RefCross Ref
  45. Lucas Waye, Stephen Chong, and Christos Dimoulas. 2017. Whip: Higher-Order Contracts for Modern Services. Proc. ACM Program. Lang., 1, ICFP (2017), Article 36, aug, 28 pages. https://doi.org/10.1145/3110280 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Haoran Zhang, Adney Cardoza, Peter Baile Chen, Sebastian Angel, and Vincent Liu. 2020. Fault-tolerant and transactional stateful serverless workflows. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 1187–1204. https://dl.acm.org/doi/10.5555/3488766.3488833 Google ScholarGoogle Scholar
  47. Wen Zhang, Vivian Fang, Aurojit Panda, and Scott Shenker. 2020. Kappa: A Programming Framework for Serverless Computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC ’20). Association for Computing Machinery, New York, NY, USA. 328–343. isbn:9781450381376 https://doi.org/10.1145/3419111.3421277 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Executing Microservice Applications on Serverless, Correctly

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Article Metrics

              • Downloads (Last 12 months)423
              • Downloads (Last 6 weeks)78

              Other Metrics

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!