research-article

Open Access

SPRIGHT: extracting the server from serverless computing! high-performance eBPF-based event-driven, shared-memory processing

Authors:
Shixiong Qi

University of California

University of California
View Profile

,
Leslie Monis

University of California

University of California
View Profile

,
Ziteng Zeng

University of California

University of California
View Profile

,
Ian-chin Wang

University of California

University of California
View Profile

,
K. K. Ramakrishnan

University of California

University of California
View Profile

SIGCOMM '22: Proceedings of the ACM SIGCOMM 2022 ConferenceAugust 2022Pages 780–794https://doi.org/10.1145/3544216.3544259

Published:22 August 2022Publication History

SIGCOMM '22: Proceedings of the ACM SIGCOMM 2022 Conference

Pages 780–794

ABSTRACT

Serverless computing promises an efficient, low-cost compute capability in cloud environments. However, existing solutions, epitomized by open-source platforms such as Knative, include heavyweight components that undermine this goal of serverless computing. Additionally, such serverless platforms lack dataplane optimizations to achieve efficient, high-performance function chains that facilitate the popular microservices development paradigm. Their use of unnecessarily complex and duplicate capabilities for building function chains severely degrades performance. 'Cold-start' latency is another deterrent.

We describe SPRIGHT, a lightweight, high-performance, responsive serverless framework. SPRIGHT exploits shared memory processing and dramatically improves the scalability of the dataplane by avoiding unnecessary protocol processing and serialization-deserialization overheads. SPRIGHT extensively leverages event-driven processing with the extended Berkeley Packet Filter (eBPF). We creatively use eBPF's socket message mechanism to support shared memory processing, with overheads being strictly load-proportional. Compared to constantly-running, polling-based DPDK, SPRIGHT achieves the same dataplane performance with 10× less CPU usage under realistic workloads. Additionally, eBPF benefits SPRIGHT, by replacing heavyweight serverless components, allowing us to keep functions 'warm' with negligible penalty.

Our preliminary experimental results show that SPRIGHT achieves an order of magnitude improvement in throughput and latency compared to Knative, while substantially reducing CPU usage, and obviates the need for 'cold-start'.

Supplemental Material

Available for Download

pdf

p780-qi-supp.pdf (74.3 KB)

Supplemental material.

References

2021. ab - Apache HTTP server benchmarking tool. https://httpd.apache.org/docs/2.4/programs/ab.html. [online].Google Scholar
2021. Kubernetes Components. https://kubernetes.io/docs/concepts/overview/components/. [online].Google Scholar
2021. wrk - a HTTP benchmarking tool. https://github.com/wg/wrk. [online].Google Scholar
2022. Apache OpenWhisk. https://openwhisk.apache.org/. [online].Google Scholar
2022. AWS Lambda. https://aws.amazon.com/lambda/. [online].Google Scholar
2022. AWS Serverless API. https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-api.html. [online].Google Scholar
2022. BPF-HELPERS - list of eBPF helper functions. https://man7.org/linux/man-pages/man7/bpf-helpers.7.html. [online].Google Scholar
2022. Chaining OpenFaaS functions. https://ericstoekl.github.io/faas/developer/chaining_functions/. [online].Google Scholar
2022. CloudEvents Spec. https://github.com/cloudevents/spec. [online].Google Scholar
2022. Dynamically Loaded (DL) Libraries. https://tldp.org/HOWTO/Program-Library-HOWTO/dl-libraries.html. [online].Google Scholar
2022. eBPF XDP: The Basics and a Quick Tutorial. https://www.tigera.io/learn/guides/ebpf/ebpf-xdp/. [online].Google Scholar
2022. Istio Architecture. https://istio.io/latest/docs/ops/deployment/architecture/. [online].Google Scholar
2022. Istio Traffic Management. https://istio.io/latest/docs/concepts/traffic-management/. [online].Google Scholar
2022. Knative. https://knative.dev. [online].Google Scholar
2022. Knative Eventing. https://knative.dev/docs/eventing/. [online].Google Scholar
2022. Knative Serving. https://knative.dev/docs/serving/. [online].Google Scholar
2022. Knative Serving Autoscaling System - Activator. https://github.com/knative/serving/blob/main/docs/scaling/SYSTEM.md#activator. [online].Google Scholar
2022. Locust: An open source load testing tool. https://locust.io/. [online].Google Scholar
2022. MQTT Version 5.0. https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html. [online].Google Scholar
2022. Multi-process Support of DPDK. https://doc.dpdk.org/guides/prog_guide/multi_proc_support.html. [online].Google Scholar
2022. NGINX. https://www.nginx.com/. [online].Google Scholar
2022. of-watchdog. https://github.com/openfaas/of-watchdog. [online].Google Scholar
2022. Online Boutique by Google. https://github.com/GoogleCloudPlatform/microservices-demo. [online].Google Scholar
2022. OpenFaaS. https://www.openfaas.com/. [online].Google Scholar
2022. OpenFaaS API Gateway / Portal. https://docs.openfaas.com/architecture/gateway/. [online].Google Scholar
2022. OpenFaaS Triggers. https://docs.openfaas.com/reference/triggers/. [online].Google Scholar
2022. OpenWhisk - Creating action sequences. https://github.com/apache/openwhisk/blob/master/docs/actions.md#creating-action-sequences. [online].Google Scholar
2022. OpenWhisk Composer. https://github.com/apache/openwhisk-composer. [online].Google Scholar
2022. Ring Library. https://doc.dpdk.org/guides/prog_guide/ring_lib.html. [online].Google Scholar
2022. SPRIGHT. https://github.com/ucr-serverless/spright.git. [online].Google Scholar
Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA, 419--434. https://www.usenix.org/conference/nsdi20/presentation/agacheGoogle Scholar
Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards High-Performance Serverless Computing. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 923--935. https://www.usenix.org/conference/atc18/presentation/akkusGoogle ScholarDigital Library
Giuseppe Amato, Fabio Carrara, Fabrizio Falchi, Claudio Gennaro, and Claudio Vairo. 2016. Car parking occupancy detection using smart camera networks and Deep Learning. In 2016 IEEE Symposium on Computers and Communication (ISCC). 1212--1217. Google ScholarCross Ref
Apache Software Foundation. 2022. APACHE KAFKA. https://kafka.apache.org/. [online].Google Scholar
Priscilla Benedetti, Mauro Femminella, Gianluca Reali, and Kris Steenhaut. 2021. Experimental Analysis of the Application of Serverless Computing to IoT Platforms. Sensors 21, 3 (2021). Google ScholarCross Ref
Matteo Bertrone, Sebastiano Miano, Jianwen Pi, Fulvio Risso, and Massimo Tumolo. 2018. Toward an eBPF-based clone of iptables. Netdev'18 (2018).Google Scholar
Vivek M. Bhasi, Jashwant Raj Gunasekaran, Prashanth Thinakaran, Cyan Subhra Mishra, Mahmut Taylan Kandemir, and Chita Das. 2021. Kraken: Adaptive Container Provisioning for Deploying Dynamic DAGs in Serverless Platforms. Association for Computing Machinery, New York, NY, USA, 153--167. Google ScholarDigital Library
Carsten Bormann, Angelo P Castellani, and Zach Shelby. 2012. Coap: An application protocol for billions of tiny internet nodes. IEEE Internet Computing 16, 2 (2012), 62--67.Google ScholarDigital Library
Qizhe Cai, Shubham Chaudhary, Midhul Vuppalapati, Jaehyun Hwang, and Rachit Agarwal. 2021. Understanding Host Network Stack Overheads. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference (Virtual Event, USA) (SIGCOMM '21). Association for Computing Machinery, New York, NY, USA, 65--77. Google ScholarDigital Library
Aditya Dhakal, Sameer G Kulkarni, and K. K. Ramakrishnan. 2020. ECML: Improving Efficiency of Machine Learning in Edge Clouds. In 2020 IEEE 9th International Conference on Cloud Networking (CloudNet). 1--6. Google ScholarCross Ref
Dmitry Duplyakin, Robert Ricci, Aleksander Maricq, Gary Wong, Jonathon Duerig, Eric Eide, Leigh Stoller, Mike Hibler, David Johnson, Kirk Webb, Aditya Akella, Kuangching Wang, Glenn Ricart, Larry Landweber, Chip Elliott, Michael Zink, Emmanuel Cecchet, Snigdhaswin Kar, and Prabodh Mishra. 2019. The Design and Operation of CloudLab. In Proceedings of the USENIX Annual Technical Conference (ATC). 1--14. https://www.flux.utah.edu/paper/duplyakin-atc19Google Scholar
Silvery Fu, Radhika Mittal, Lei Zhang, and Sylvia Ratnasamy. 2020. Fast and Efficient Container Startup at the Edge via Dependency Scheduling. In 3rd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 20). USENIX Association. https://www.usenix.org/conference/hotedge20/presentation/fuGoogle Scholar
Alexander Fuerst and Prateek Sharma. 2021. FaasCache: Keeping Serverless Computing Alive with Greedy-Dual Caching. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (Virtual, USA) (ASPLOS 2021). Association for Computing Machinery, New York, NY, USA, 386--400. Google ScholarDigital Library
Phani Kishore Gadepalli, Sean McBride, Gregor Peach, Ludmila Cherkasova, and Gabriel Parmer. 2020. Sledge: A Serverless-First, Light-Weight Wasm Runtime for the Edge. In Proceedings of the 21st International Middleware Conference (Delft, Netherlands) (Middleware '20). Association for Computing Machinery, New York, NY, USA, 265--279. Google ScholarDigital Library
Toke Høiland-Jørgensen, Jesper Dangaard Brouer, Daniel Borkmann, John Fastabend, Tom Herbert, David Ahern, and David Miller. 2018. The EXpress Data Path: Fast Programmable Packet Processing in the Operating System Kernel. In Proceedings of the 14th International Conference on Emerging Networking EXperiments and Technologies (Heraklion, Greece) (CoNEXT '18). Association for Computing Machinery, New York, NY, USA, 54--66. Google ScholarDigital Library
Jinho Hwang, K. K. Ramakrishnan, and Timothy Wood. 2014. NetVM: High Performance and Flexible Networking Using Virtualization on Commodity Platforms. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14). USENIX Association, Seattle, WA, 445--458. https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/hwangGoogle ScholarDigital Library
IBM. 2022. Creating serverless REST APIs. https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-apigateway. [online].Google Scholar
IBM. 2022. IBM Cloud Functions. https://cloud.ibm.com/functions/. [online].Google Scholar
Svilen Kanev, Juan Pablo Darago, Kim Hazelwood, Parthasarathy Ranganathan, Tipp Moseley, Gu-Yeon Wei, and David Brooks. 2015. Profiling a Warehouse-Scale Computer. SIGARCH Comput. Archit. News 43, 3S (jun 2015), 158--169. Google ScholarDigital Library
Jiaxin Lei, Manish Munikar, Kun Suo, Hui Lu, and Jia Rao. 2021. Parallelizing Packet Processing in Container Overlay Networks. In Proceedings of the Sixteenth European Conference on Computer Systems (Online Event, United Kingdom) (EuroSys '21). Association for Computing Machinery, New York, NY, USA, 261--276. Google ScholarDigital Library
Joshua Levin and Theophilus A. Benson. 2020. ViperProbe: Rethinking Microservice Observability with eBPF. In 2020 IEEE 9th International Conference on Cloud Networking (CloudNet). 1--8. Google ScholarCross Ref
Chuanpeng Li, Chen Ding, and Kai Shen. 2007. Quantifying the Cost of Context Switch. In Proceedings of the 2007 Workshop on Experimental Computer Science (San Diego, California) (ExpCS '07). Association for Computing Machinery, New York, NY, USA, 2--es. Google ScholarDigital Library
Junfeng Li, Sameer G. Kulkarni, K. K. Ramakrishnan, and Dan Li. 2019. Understanding Open Source Serverless Platforms: Design Considerations and Performance. In Proceedings of the 5th International Workshop on Serverless Computing (Davis, CA, USA) (WOSC '19). Association for Computing Machinery, New York, NY, USA, 37--42. Google ScholarDigital Library
Ping-Min Lin and Alex Glikson. 2019. Mitigating Cold Starts in Serverless Platforms: A Pool-Based Approach. arXiv:1903.12221 [cs.DC]Google Scholar
Sebastiano Miano, Matteo Bertrone, Fulvio Risso, Mauricio Vásquez Bernal, Yunsong Lu, and Jianwen Pi. 2019. Securing Linux with a Faster and Scalable Iptables. SIGCOMM Comput. Commun. Rev. 49, 3 (nov 2019), 2--17. Google ScholarDigital Library
Microsoft. 2022. Azure - Function chaining in Durable Functions. https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-sequence?tabs=csharp. [online].Google Scholar
Viyom Mittal, Shixiong Qi, Ratnadeep Bhattacharya, Xiaosu Lyu, Junfeng Li, Sameer G. Kulkarni, Dan Li, Jinho Hwang, K. K. Ramakrishnan, and Timothy Wood. 2021. Mu: An Efficient, Fair and Responsive Serverless Framework for Resource-Constrained Edge Clouds. Association for Computing Machinery, New York, NY, USA, 168--181. Google ScholarDigital Library
Jeffrey C Mogul and KK Ramakrishnan. 1997. Eliminating receive livelock in an interrupt-driven kernel. ACM Transactions on Computer Systems 15, 3 (1997), 217--252.Google ScholarDigital Library
Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-Optimized Containers. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 57--70. https://www.usenix.org/conference/atc18/presentation/oakesGoogle ScholarDigital Library
Jinwoo Park, Byungkwon Choi, Chunghan Lee, and Dongsu Han. 2021. GRAF: A Graph Neural Network Based Proactive Resource Allocation Framework for SLO-Oriented Microservices. In Proceedings of the 17th International Conference on Emerging Networking EXperiments and Technologies (Virtual Event, Germany) (CoNEXT '21). Association for Computing Machinery, New York, NY, USA, 154--167. Google ScholarDigital Library
Shixiong Qi, Sameer G. Kulkarni, and K. K. Ramakrishnan. 2021. Assessing Container Network Interface Plugins: Functionality, Performance, and Scalability. IEEE Transactions on Network and Service Management 18, 1 (2021), 656--671. Google ScholarDigital Library
Red Hat, Inc. 2022. Understanding the eBPF networking features in RHEL. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/assembly_understanding-the-ebpf-features-in-rhel_configuring-and-managing-networking. [online].Google Scholar
Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 205--218. https://www.usenix.org/conference/atc20/presentation/shahradGoogle Scholar
Arjun Singhvi, Arjun Balasubramanian, Kevin Houck, Mohammed Danish Shaikh, Shivaram Venkataraman, and Aditya Akella. 2021. Atoll: A Scalable Low-Latency Serverless Platform. Association for Computing Machinery, New York, NY, USA, 138--152. Google ScholarDigital Library
Ali Tariq, Austin Pahl, Sharat Nimmagadda, Eric Rozner, and Siddharth Lanka. 2020. Sequoia: Enabling Quality-of-Service in Serverless Computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (Virtual Event, USA) (SoCC '20). Association for Computing Machinery, New York, NY, USA, 311--327. Google ScholarDigital Library
The Linux Foundation. 2022. eBPF. https://ebpf.io/. [online].Google Scholar
Tigera, Inc. 2022. About eBPF. https://projectcalico.docs.tigera.io/about/about-ebpf. [online].Google Scholar
Tigera, Inc. 2022. Project Calico. https://www.tigera.io/project-calico/. [online].Google Scholar
William Tu, Yi-Hung Wei, Gianni Antichi, and Ben Pfaff. 2021. Revisiting the Open VSwitch Dataplane Ten Years Later. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference (Virtual Event, USA) (SIGCOMM '21). Association for Computing Machinery, New York, NY, USA, 245--257. Google ScholarDigital Library
Marcos A. M. Vieira, Matheus S. Castanho, Racyus D. G. Pacífico, Elerson R. S. Santos, Eduardo P. M. Câmara Júnior, and Luiz F. M. Vieira. 2020. Fast Packet Processing with EBPF and XDP: Concepts, Code, Challenges, and Applications. ACM Comput. Surv. 53, 1, Article 16 (feb 2020), 36 pages. Google ScholarDigital Library
Adam Wolnikowski, Stephen Ibanez, Jonathan Stone, Changhoon Kim, Rajit Manohar, and Robert Soulé. 2021. Zerializer: Towards Zero-Copy Serialization. In Proceedings of the Workshop on Hot Topics in Operating Systems (Ann Arbor, Michigan) (HotOS '21). Association for Computing Machinery, New York, NY, USA, 206--212. Google ScholarDigital Library
Christopher R. Wren, Yuri A. Ivanov, Darren Leigh, and Jonathan Westhues. 2007. The MERL Motion Detector Dataset. In Proceedings of the 2007 Workshop on Massive Datasets (Nagoya, Japan) (MD '07). Association for Computing Machinery, New York, NY, USA, 10--14. Google ScholarDigital Library
Muneer Bani Yassein, Mohammed Q Shatnawi, Shadi Aljwarneh, and Razan Al-Hatmi. 2017. Internet of Things: Survey and open issues of MQTT protocol. In 2017 international conference on engineering & MIS (ICEMIS). IEEE, 1--6.Google ScholarCross Ref
Wei Zhang, Guyue Liu, Wenhui Zhang, Neel Shah, Phillip Lopreiato, Gregoire Todeschi, K.K. Ramakrishnan, and Timothy Wood. 2016. OpenNetVM: A Platform for High Performance Network Service Chains. In Proceedings of the 2016 ACM SIGCOMM Workshop on Hot Topics in Middleboxes and Network Function Virtualization. ACM.Google ScholarDigital Library
Jianer Zhou, Xinyi Qiu, Zhenyu Li, Gareth Tyson, Qing Li, Jingpu Duan, and Yi Wang. 2021. Antelope: A Framework for Dynamic Selection of Congestion Control Algorithms. In 2021 IEEE 29th International Conference on Network Protocols (ICNP). 1--11.Google Scholar

Index Terms

SPRIGHT: extracting the server from serverless computing! high-performance eBPF-based event-driven, shared-memory processing
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing
2. Networks
  1. Network services
    1. Cloud computing

Recommendations

Architectural Implications of Function-as-a-Service Computing
MICRO '52: Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

Serverless computing is a rapidly growing cloud application model, popularized by Amazon's Lambda platform. Serverless cloud services provide fine-grained provisioning of resources, which scale automatically with user demand. Function-as-a-Service (FaaS)...
Read More
All but one: FaaS platform elasticity revisited

Serverless computing through Function-as-a-Service (FaaS) platforms is a popular new cloud computing model, offering high elasticity while being fully-managed. However, FaaS platforms do not quantify elasticity; thus, application developers are unaware ...
Read More
SeBS: a serverless benchmark suite for function-as-a-service computing
Middleware '21: Proceedings of the 22nd International Middleware Conference

Function-as-a-Service (FaaS) is one of the most promising directions for the future of cloud services, and serverless functions have immediately become a new middleware for building scalable and cost-efficient microservices and appli cations. However, ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Published in
SIGCOMM '22: Proceedings of the ACM SIGCOMM 2022 Conference
August 2022
858 pages
ISBN:9781450394208
DOI:10.1145/3544216
General Chairs:
Fernando Kuipers
Delft University of Technology
,
Ariel Orda
Technion Israel Institute of Technology
Copyright © 2022 Owner/Author
This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 22 August 2022
Check for updates
Badges
Author Tags
serverless
function chain
event-driven
eBPF
Qualifiers
- research-article
Conference

Acceptance Rates
Overall Acceptance Rate554of3,547submissions,16%
Funding Sources
Contributor Metrics
Expand All

Shixiong Qi University of California, Riverside

Publication Years2021 - 2022
Publication counts4
Available for Download3
Citation count34
Downloads (cumulative)7,967
Downloads (6 weeks)343
Downloads (12 months)4,611
Average Citation per Article9
Average Downloads per Article2,656
View Full Profile

Leslie Monis National Institute of Technology Karnataka

Publication Years2020 - 2022
Publication counts2
Available for Download2
Citation count15
Downloads (cumulative)3,302
Downloads (6 weeks)139
Downloads (12 months)1,955
Average Citation per Article8
Average Downloads per Article1,651
View Full Profile

Ziteng Zeng

Publication Years2022 - 2022
Publication counts1
Available for Download1
Citation count11
Downloads (cumulative)3,017
Downloads (6 weeks)138
Downloads (12 months)1,847
Average Citation per Article11
Average Downloads per Article3,017
View Full Profile

Ianchin Wang

Publication Years2022 - 2022
Publication counts1
Available for Download1
Citation count11
Downloads (cumulative)3,017
Downloads (6 weeks)138
Downloads (12 months)1,847
Average Citation per Article11
Average Downloads per Article3,017
View Full Profile

Kadangode k Ramakrishnan University of California, Riverside

Publication Years1983 - 2023
Publication counts159
Available for Download109
Citation count4,225
Downloads (cumulative)70,387
Downloads (6 weeks)1,162
Downloads (12 months)11,722
Average Citation per Article27
Average Downloads per Article646
View Full Profile
Other Metrics
View Article Metrics

Article Metrics
- 11
  Total Citations
  View Citations
- 3,017
  Total Downloads
- Downloads (Last 12 months)1,847
- Downloads (Last 6 weeks)138
Other Metrics
View Author Metrics
Cited By
View all

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

SPRIGHT: extracting the server from serverless computing! high-performance eBPF-based event-driven, shared-memory processing

Save to Binder

SIGCOMM '22: Proceedings of the ACM SIGCOMM 2022 Conference

ABSTRACT

Supplemental Material

Available for Download

References

Cited By

Index Terms

SPRIGHT: extracting the server from serverless computing! high-performance eBPF-based event-driven, shared-memory processing

Recommendations

Architectural Implications of Function-as-a-Service Computing

All but one: FaaS platform elasticity revisited

SeBS: a serverless benchmark suite for function-as-a-service computing