Abstract
Serverless, or Functions-as-a-Service (FaaS), is an increasingly popular paradigm for application development, as it provides implicit elastic scaling and load based billing. However, the weak execution guarantees and intrinsic compute-storage separation of FaaS create serious challenges when developing applications that require persistent state, reliable progress, or synchronization. This has motivated a new generation of serverless frameworks that provide stateful abstractions. For instance, Azure's Durable Functions (DF) programming model enhances FaaS with actors, workflows, and critical sections. As a programming model, DF is interesting because it combines task and actor parallelism, which makes it suitable for a wide range of serverless applications. We describe DF both informally, using examples, and formally, using an idealized high-level model based on the untyped lambda calculus. Next, we demystify how the DF runtime can (1) execute in a distributed unreliable serverless environment with compute-storage separation, yet still conform to the fault-free high-level model, and (2) persist execution progress without requiring checkpointing support by the language runtime. To this end we define two progressively more complex execution models, which contain the compute-storage separation and the record-replay, and prove that they are equivalent to the high-level model.
Supplemental Material
- 2020. AWS Step Functions. https://docs.aws.amazon.com/step-functions/latest/dg/welcome.htmlGoogle Scholar
- Joe Armstrong. 1997. The development of Erlang. In ICFP. 97, 196–203. https://doi.org/10.1145/258948.258967 Google Scholar
Digital Library
- Ioana Baldini, Perry Cheng, Stephen J. Fink, Nick Mitchell, Vinod Muthusamy, Rodric Rabbah, Philippe Suter, and Olivier Tardieu. 2017. The Serverless Trilemma: Function Composition for Serverless Computing. In Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2017). Association for Computing Machinery, New York, NY, USA. 89–103. isbn:9781450355308 https://doi.org/10.1145/3133850.3133855 Google Scholar
Digital Library
- Jonas Bonér. 2020. Towards Stateful Serverless. https://www.youtube.com/watch?v=DVTf5WQlgB8Google Scholar
- Sebastian Burckhardt, Chris Gillum, David Justo, Konstantinos Kallas, Connor McMahon, and Christopher S. Meiklejohn. 2021. Serverless Workflows with Durable Functions and Netherite. arxiv:2103.00033.Google Scholar
- Sergey Bykov, Alan Geller, Gabriel Kliot, James R Larus, Ravi Pandya, and Jorgen Thelin. 2011. Orleans: Cloud Computing for Everyone. In Proceedings of the 2nd ACM Symposium on Cloud Computing. 16. https://doi.org/10.1145/2038916.2038932 Google Scholar
Digital Library
- Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015. Apache flink: Stream and batch processing in a single engine. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 36, 4 (2015).Google Scholar
- 2021. Note on usage of yield for async/await.. https://twitter.com/cgillum/status/1378128701317992451 Accessed: 2021-04-13Google Scholar
- 2020. Using Durable Objects, Cloudflare Docs. https://developers.cloudflare.com/workers/learning/using-durable-objectsGoogle Scholar
- 2020. Workers Durable Objects Beta: A New Approach to Stateful Serverless. https://blog.cloudflare.com/introducing-workers-durable-objects/Google Scholar
- Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: Simplified Data Processing on Large Clusters. Commun. ACM, 51, 1 (2008), Jan., 107–113. issn:0001-0782 https://doi.org/10.1145/1327452.1327492 Google Scholar
Digital Library
- Cormac Flanagan and Matthias Felleisen. 1995. The Semantics of Future and Its Use in Program Optimization. In Proceedings of the 22nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’95). Association for Computing Machinery, New York, NY, USA. 209–220. isbn:0897916921 https://doi.org/10.1145/199448.199484 Google Scholar
Digital Library
- Sadjad Fouladi, Francisco Romero, Dan Iter, Qian Li, Shuvo Chatterjee, Christos Kozyrakis, Matei Zaharia, and Keith Winstein. [n. d.]. From laptop to lambda: Outsourcing everyday jobs to thousands of transient functional containers. In Proceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC ’19). 475–488. https://doi.org/10.1145/199448.199484 Google Scholar
Digital Library
- Sadjad Fouladi, Riad S Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. [n. d.]. Encoding, fast and slow: Low-latency video processing using thousands of tiny threads. In Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation (NSDI’17). 363–376. Google Scholar
Digital Library
- Pedro Garcia Lopez, Marc Sanchez-Artigas, Gerard Paris, Daniel Barcelona Pons, Alvaro Ruiz Ollobarren, and David Arroyo Pinto. 2018. Comparison of FaaS Orchestration Systems. 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), Dec, isbn:9781728103594 https://doi.org/10.1109/ucc-companion.2018.00049 Google Scholar
Cross Ref
- github. [n. d.]. Durable Task Framework. https://github.com/Azure/durabletaskGoogle Scholar
- github. 2021. DurableTask.MSSQL Backend. https://github.com/microsoft/durabletask-mssqlGoogle Scholar
- github. 2021. DurableTask.Netherite Backend. https://github.com/microsoft/durabletask-netheriteGoogle Scholar
- Jonathan Goldstein, Ahmed S. Abdelhamid, Mike Barnett, Sebastian Burckhardt, Badrish Chandramouli, Darren Gehring, Niel Lebeck, Christopher Meiklejohn, Umar Farooq Minhas, Ryan Newton, Rahee Peshawaria, Tal Zaccai, and Irene Zhang. 2020. A.M.B.R.O.S.I.A: Providing Performant Virtual Resiliency for Distributed Applications. Proc. VLDB Endow., 13, 5 (2020), 588–601. https://doi.org/10.14778/3377369.3377370 Google Scholar
Digital Library
- Philipp Haller. 2012. On the integration of the actor model in mainstream technologies: the Scala perspective. In Proceedings of the 2nd Edition on Programming Systems, Languages and Applications based on Actors, Agents, and Decentralized Control Abstractions. 1–6. https://doi.org/10.1145/2414639.2414641 Google Scholar
Digital Library
- Robert H. Halstead. 1985. MULTILISP: A Language for Concurrent Symbolic Computation. ACM Trans. Program. Lang. Syst., 7, 4 (1985), Oct., 501–538. issn:0164-0925 https://doi.org/10.1145/4472.4478 Google Scholar
Digital Library
- Abhinav Jangda, Donald Pinckney, Yuriy Brun, and Arjun Guha. 2019. Formal Foundations of Serverless Computing. Proceedings of the ACM on Programming Languages (PACMPL), 3, OOPSLA (2019), https://doi.org/10.1145/3360575 Google Scholar
Digital Library
- Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the cloud: Distributed computing for the 99%. In Proceedings of the 2017 Symposium on Cloud Computing. 445–451. https://doi.org/10.1145/3127479.3128601 Google Scholar
Digital Library
- Pedro García López, Aitor Arjona, Josep Sampé, Aleksander Slominski, and Lionel Villard. 2020. Triggerflow: trigger-based orchestration of serverless workflows. In Proceedings of the 14th ACM International Conference on Distributed and Event-Based Systems (DEBS ’20). 3–14. https://doi.org/10.1145/3401025.3401731 Google Scholar
Digital Library
- 2020. What are Durable Functions? - Microsoft Azure. https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overviewGoogle Scholar
- 2020. Azure Logic Apps Service. https://azure.microsoft.com/en-us/services/logic-apps/Google Scholar
- 2021. Azure SQL. https://azure.microsoft.com/en-us/services/azure-sql/ Accessed: 2021-04-13Google Scholar
- Heather Miller, Philipp Haller, and Martin Odersky. 2014. Spores: A Type-Based Foundation for Closures in the Age of Concurrency and Distribution. In Proceedings of the 28th European Conference on ECOOP 2014 — Object-Oriented Programming - Volume 8586. Springer-Verlag, Berlin, Heidelberg. 308–333. isbn:9783662442012 https://doi.org/10.1007/978-3-662-44202-9_13 Google Scholar
Digital Library
- Luc Moreau. 1996. The Semantics of Scheme with Future. In In In ACM SIGPLAN International Conference on Functional Programming (ICFP’96. 146–156. https://doi.org/10.1145/232627.232644 Google Scholar
Digital Library
- Ingo Müller, Renato Marroquín, and Gustavo Alonso. 2020. Lambada: Interactive Data Analytics on Cold Data Using Serverless Cloud Infrastructure. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 115–130. https://doi.org/10.1145/3318464.3389758 Google Scholar
Digital Library
- Matthew Perron, Raul Castro Fernandez, David DeWitt, and Samuel Madden. 2020. Starling: A scalable query engine on cloud functions. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 131–141. https://doi.org/10.1145/3318464.3380609 Google Scholar
Digital Library
- Benjamin C. Pierce. 2002. Types and programming languages. MIT Press. isbn:978-0-262-16209-8 Google Scholar
Digital Library
- Ganesan Ramalingam and Kapil Vaswani. 2013. Fault tolerance via idempotence. In Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. 249–262. https://doi.org/10.1145/2429069.2429100 Google Scholar
Digital Library
- German Shegalov, Michael Gillmann, and Gerhard Weikum. 2001. XML-enabled workflow management for e-services across heterogeneous platforms. The VLDB Journal, 10, 1 (2001), 91–103. Google Scholar
Digital Library
- Vikram Sreekanti, Chenggang Wu, Saurav Chhatrapati, Joseph E Gonzalez, Joseph M Hellerstein, and Jose M Faleiro. 2020. A fault-tolerance shim for serverless computing. In Proceedings of the Fifteenth European Conference on Computer Systems. 1–15. https://doi.org/10.1145/3342195.3387535 Google Scholar
Digital Library
- Vikram Sreekanti, Chenggang Wu, Xiayue Charles Lin, Johann Schleier-Smith, Jose M Faleiro, Joseph E Gonzalez, Joseph M Hellerstein, and Alexey Tumanov. 2020. Cloudburst: Stateful functions-as-a-service. arXiv preprint arXiv:2001.04592. Google Scholar
Digital Library
- Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauly, Michael J Franklin, Scott Shenker, and Ion Stoica. 2012. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12). 15–28. Google Scholar
Digital Library
- Haoran Zhang, Adney Cardoza, Peter Baile Chen, Sebastian Angel, and Vincent Liu. 2020. Fault-tolerant and transactional stateful serverless workflows. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI) 20). 1187–1204. Google Scholar
Digital Library
- Wen Zhang, Vivian Fang, Aurojit Panda, and Scott Shenker. 2020. Kappa: A Programming Framework for Serverless Computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC ’20). Association for Computing Machinery, New York, NY, USA. 328–343. isbn:9781450381376 https://doi.org/10.1145/3419111.3421277 Google Scholar
Digital Library
Index Terms
Durable functions: semantics for stateful serverless
Recommendations
Distributed transactions on serverless stateful functions
DEBS '21: Proceedings of the 15th ACM International Conference on Distributed and Event-based SystemsServerless computing is currently the fastest-growing cloud services segment. The most prominent serverless offering is Function-as-a-Service (FaaS), where users write functions and the cloud automates deployment, maintenance, and scalability. Although ...
Stateful Serverless Computing with Crucial
Serverless computing greatly simplifies the use of cloud resources. In particular, Function-as-a-Service (FaaS) platforms enable programmers to develop applications as individual functions that can run and scale independently. Unfortunately, applications ...
A Distributed Analysis and Benchmarking Framework for Apache OpenWhisk Serverless Platform
Middleware '18: Proceedings of the 19th International Middleware Conference (Posters)Serverless computing simplifies the life cycle of scalable web applications, through delegating most of the operational concerns to the cloud providers. One prominent serverless platform is Apache OpenWhisk which is employed by IBM Cloud. Despite the ...






Comments