skip to main content
10.1145/3387514.3405865acmconferencesArticle/Chapter ViewAbstractPublication PagescommConference Proceedingsconference-collections
research-article
Public Access

BeauCoup: Answering Many Network Traffic Queries, One Memory Update at a Time

Published: 30 July 2020 Publication History

Abstract

Network administrators constantly monitor network traffic for congestion and attacks. They need to perform a large number of measurements on the traffic simultaneously, to detect different types of anomalies such as heavy hitters or super-spreaders. Existing techniques often focus on a single statistic (e.g., traffic volume) or traffic attribute (e.g., destination IP). However, performing numerous heterogeneous measurements within the constrained memory architecture of modern network devices poses significant challenges, due to the limited number of memory accesses allowed per packet. We propose BeauCoup, a system based on the coupon collector problem, that supports multiple distinct counting queries simultaneously while making only a small constant number of memory accesses per packet. We implement BeauCoup on PISA commodity programmable switches, satisfying the strict memory size and access constraints while using a moderate portion of other data-plane hardware resources. Evaluations show BeauCoup achieves the same accuracy as other sketch-based or sampling-based solutions using 4x fewer memory access.

Supplementary Material

MP4 File (3387514.3405865.mp4)
In this video, we present BeauCoup, a system for answering many different network traffic queries using only a constant number of memory updates per packet. BeauCoup formulates each query into a coupon collector problem, which allows it to approximately count distinct items while using much fewer than one memory update per packet. A query compiler finds the coupon-collector configuration for each query that minimizes the error, while a data-plane program maps each packet to at most one coupon and collects coupons into an in-memory table. BeauCoup further supports changing queries on-the-fly without the need to recompile the data-plane program. Evaluations show BeauCoup uses 4x-10x fewer memory accesses to achieve the same accuracy as state-of-the-art.

References

[1]
Erari Assaf, Ran Ben Basat, Gil Einziger, and Roy Friedman. 2018. Pay for a sliding bloom fi Iter and get counting, distinct elements, and entropy for free. In IEEE INFOCOM 2018. IEEE, 2204--2212.
[2]
Ziv Bar-Yossef, T. S. Jayram, Ravi Kumar, D Sivakumar, and Luca Trevisan. 2002. Counting distinct elements in a data stream. In International Workshop on Randomization and Approximation Techniques in Computer Science. Springer, 1--10.
[3]
Ran Ben Basat, Xiaoqi Chen, Gil Einziger, Shir Landau Feibish, Danny Raz, and Minlan Yu. 2020. Routing Oblivious Measurement Analytics. In IFIP Networking.
[4]
Ran Ben Basat, Xiaoqi Chen, Gil Einziger, and Ori Rottenstreich. 2020. Designing Heavy-Hitter Detection Algorithms for Programmable Switches. IEEE/ACM Transactions on Networking 28, 3 (2020), 1172--1185.
[5]
CAIDA. 2018. The CAIDA UCSD Anonymized Internet Traces 2018 - March 15th. (2018). https://www.caida.org/data/passive/passive_dataset.xml
[6]
Anne Chao. 1984. Nonparametric estimation of the number of classes in a population. Scandinavian Journal of Statistics (1984), 265--270.
[7]
Moses Charikar, Kevin C. Chen, and Martin Farach-Colton. 2004. Finding frequent items in data streams. Theoretical Computer Science 312, 1 (2004), 3--15.
[8]
Benoit Claise. 2004. Cisco Systems NetFlow Services Export Version 9. RFC 3954 (2004).
[9]
The P4 Language Consortium. 2018. P416 Language Specifications. (2018). https://p4.org/p4-spec/docs/P4-16-v1.0.0-spec.pdf
[10]
Graham Cormode. 2011. Sketch techniques for approximate query processing. Foundations and Trends in Databases. NOW publishers (2011).
[11]
Graham Cormode and S. Muthukrishnan. 2005. An improved data stream summary: The count-min sketch and its applications. Journal of Algorithms 55, 1 (2005), 58--75.
[12]
Marianne Durand and Philippe Flajolet. 2003. Loglog counting of large cardinalities. In European Symposium on Algorithms. Springer, 605--617.
[13]
Philippe Flajolet, Éric Fusy, Olivier Gandouet, and Frédéric Meunier. 2007. HyperLogLog: The analysis of a near-optimal cardinality estimation algorithm. In Analysis of Algorithms (AOFA).
[14]
Philippe Flajolet, Danièle Gardy, and Loÿs Thimonier. 1992. Birthday Paradox, Coupon Collectors, Caching Algorithms and Self-Organizing Search. Discrete Applied Mathematics 39, 3 (1992), 207--229.
[15]
Phillip B Gibbons. 2001. Distinct sampling for highly-accurate answers to distinct values queries and event reports. In VLDB, Vol. 1. 541--550.
[16]
Arpit Gupta, Rob Harrison, Marco Canini, Nick Feamster, Jennifer Rexford, and Walter Willinger. 2018. Sonata: Query-driven streaming network telemetry. In ACM SIGCOMM. 357--371.
[17]
Rob Harrison, Qizhe Cai, Arpit Gupta, and Jennifer Rexford. 2018. Network-Wide Heavy Hitter Detection with Commodity Switches. In ACM SIGCOMM Symposium on SDN Research. 8:1-8:7.
[18]
Daehyeok Kim, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, and Srinivasan Seshan. 2018. Generic External Memory for Switch Data Planes. In ACM Workshop on Hot Topics in Networks. 1--7.
[19]
Kasper Green Larsen, Jelani Nelson, and Huy L Nguyên. 2015. Time lower bounds for nonadaptive turnstile streaming algorithms. In ACM Symposium on Theory of Computing. ACM, 803--812.
[20]
Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. 2016. FlowRadar: A Better NetFlow for Data Centers. In USENIX NSDI. 311--324.
[21]
Zaoxing Liu, Ran Ben Basat, Gil Einziger, Yaron Kassner, Vladimir Braverman, Roy Friedman, and Vyas Sekar. 2019. NitroSketch: Robust and general sketch-based monitoring in software switches. In ACM SIGCOMM. 334--350.
[22]
Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One Sketch to Rule Them All: Rethinking Network Flow Monitoring with UnivMon. In ACM SIGCOMM. 101--114.
[23]
Zaoxing Liu, Samson Zhou, Ori Rottenstreich, Vladimir Braverman, and Jennifer Rexford. 2020. Memory-efficient performance monitoring on programmable switches with lean algorithms. In SIAM-ACM Symposium on Algorithmic Principles of Computer Systems. 31--44.
[24]
S. Muthukrishnan. 2005. Data Streams: Algorithms and Applications. Foundations and Trends in Theoretical Computer Science 1, 2 (2005).
[25]
Srinivas Narayana, Anirudh Sivaraman, Vikram Nathan, Prateesh Goyal, Venkat Arun, Mohammad Alizadeh, Vimalkumar Jeyakumar, and Changhoon Kim. 2017. Language-Directed Hardware Design for Network Performance Monitoring. In ACM SIGCOMM. 85--98.
[26]
Mihai Patrascu. 2008. Lower Bound Techniques for Data Structures. Ph.D. Dissertation. Massachusetts Institute of Technology, Cambridge, MA, USA.
[27]
Salvatore Pontarelli, Pedro Reviriego, and Michael Mitzenmacher. 2018. EMOMA: Exact Match in One Memory Access. IEEE Transactions on Knowledge and Data Engineering 30, 11 (2018), 2120--2133.
[28]
Daniel Rubio. 2017. Jinja templates in Django. In Beginning Django. Springer, 117--161.
[29]
Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, Shan Muthukrishnan, and Jennifer Rexford. 2017. Heavy-hitter detection entirely in the data plane. In ACM SIGCOMM Symposium on SDN Research. 164--176.
[30]
Bruce Spang and Nick McKeown. 2019. On estimating the number of flows. In Stanford Workshop on Buffer Sizing.
[31]
Shobha Venkataraman, Dawn Xiaodong Song, Phillip B. Gibbons, and Avrim Blum. 2005. New Streaming Algorithms for Fast Detection of Superspreaders. In Network and Distributed System Security Symposium.
[32]
Tong Yang, Jie Jiang, Peng Liu, Qun Huang, Junzhi Gong, Yang Zhou, Rui Miao, Xiaoming Li, and Steve Uhlig. 2018. Elastic Sketch: Adaptive and fast network-wide measurements. In ACM SIGCOMM. 561--575.
[33]
Andrew Chi-Chih Yao. 1978. Should Tables Be Sorted? (Extended Abstract). In Foundations of Computer Science. 22--27.

Cited By

View all
  • (2025)Altair: Resource-efficient optimization and deployment for data plane programsComputer Networks10.1016/j.comnet.2024.110917257(110917)Online publication date: Feb-2025
  • (2024)OctoSketchProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691914(1621-1639)Online publication date: 16-Apr-2024
  • (2024)SetD4: Sets With Deletions and Decay in the Data PlaneProceedings of the ACM on Networking10.1145/36963912:CoNEXT4(1-22)Online publication date: 25-Nov-2024
  • Show More Cited By

Index Terms

  1. BeauCoup: Answering Many Network Traffic Queries, One Memory Update at a Time

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGCOMM '20: Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication
      July 2020
      814 pages
      ISBN:9781450379557
      DOI:10.1145/3387514
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 30 July 2020

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Data Plane
      2. Distinct Counting
      3. Network Measurement
      4. Programmable Switch
      5. Sketching
      6. Streaming Algorithm

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      Conference

      SIGCOMM '20
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 462 of 3,389 submissions, 14%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)466
      • Downloads (Last 6 weeks)60
      Reflects downloads up to 11 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Altair: Resource-efficient optimization and deployment for data plane programsComputer Networks10.1016/j.comnet.2024.110917257(110917)Online publication date: Feb-2025
      • (2024)OctoSketchProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691914(1621-1639)Online publication date: 16-Apr-2024
      • (2024)SetD4: Sets With Deletions and Decay in the Data PlaneProceedings of the ACM on Networking10.1145/36963912:CoNEXT4(1-22)Online publication date: 25-Nov-2024
      • (2024)Incremental Specialization of Network ProgramsProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696870(264-272)Online publication date: 18-Nov-2024
      • (2024)P4-Ace: Resource-Efficient Optimization and Verification for Programmable SwitchesProceedings of the 2024 SIGCOMM Workshop on Formal Methods Aided Network Operation10.1145/3672199.3673884(8-13)Online publication date: 4-Aug-2024
      • (2024)Raising the Level of Abstraction for Sketch-Based Network Telemetry with SketchPlanProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3689016(651-658)Online publication date: 4-Nov-2024
      • (2024)P4BS: Leveraging Passive Measurements From P4 Switches to Dynamically Modify a Router’s Buffer SizeIEEE Transactions on Network and Service Management10.1109/TNSM.2023.330633521:1(1082-1099)Online publication date: Feb-2024
      • (2024)Resource-Efficient and Timely Packet Header Vector (PHV) Encoding on Programmable SwitchesIEEE/ACM Transactions on Networking10.1109/TNET.2024.341353032:5(4191-4206)Online publication date: Oct-2024
      • (2024)DynATOS+: A Network Telemetry System for Dynamic Traffic and Query WorkloadsIEEE/ACM Transactions on Networking10.1109/TNET.2024.336743232:4(2810-2825)Online publication date: Aug-2024
      • (2024)Distributed Network Telemetry With Resource Efficiency and Full AccuracyIEEE/ACM Transactions on Networking10.1109/TNET.2023.332734532:3(1857-1872)Online publication date: Jun-2024
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media