Huayi Duan
ABSTRACT
Running off-site software middleboxes at third-party service providers has been a popular practice. However, routing large volumes of raw traffic, which may carry sensitive information, to a remote site for processing raises severe security concerns. Prior solutions often abstract away important factors pertinent to real-world deployment. In particular, they overlook the significance of metadata protection and stateful processing. Unprotected traffic metadata like low-level headers, size and count, can be exploited to learn supposedly encrypted application contents. Meanwhile, tracking the states of 100,000s of flows concurrently is often indispensable in production-level middleboxes deployed at real networks.
We present LightBox, the first system that can drive off-site middleboxes at near-native speed with stateful processing and the most comprehensive protection to date. Built upon commodity trusted hardware, Intel SGX, LightBox is the product of our systematic investigation of how to overcome the inherent limitations of secure enclaves using domain knowledge and customization. First, we introduce an elegant virtual network interface that allows convenient access to fully protected packets at line rate without leaving the enclave, as if from the trusted source network. Second, we provide complete flow state management for efficient stateful processing, by tailoring a set of data structures and algorithms optimized for the highly constrained enclave space. Extensive evaluations demonstrate that LightBox, with all security benefits, can achieve 10Gbps packet I/O, and that with case studies on three stateful middleboxes, it can operate at near-native speed.
References
- Ittai Anati, Shay Gueron, Simon Johnson, and Vincent Scarlata. 2013. Innovative technology for CPU based attestation and sealing. In Proc. of ACM HASP.Google Scholar
- Sergei Arnautov, Bohdan Trach, Franz Gregor, Thomas Knauth, Andre Martin, Christian Priebe, Joshua Lind, Divya Muthukumaran, Dan OtextquoterightKeeffe, Mark L. Stillwell, David Goltzsche, Dave Eyers, Rüdiger Kapitza, Peter Pietzuch, and Christof Fetzer. 2016. SCONE: Secure Linux Containers with Intel SGX. In Proc. of USENIX OSDI.Google ScholarDigital Library
- Hassan Jameel Asghar, Luca Melis, Cyril Soldani, Emiliano De Cristofaro, Mohamed Ali Kaafar, and Laurent Mathy. 2016. SplitBox: Toward Efficient Private Network Function Virtualization. In Proc. of ACM HotMiddlebox.Google ScholarDigital Library
- Pierre-Louis Aublin, Florian Kelbert, Dan O'Keeffe, Divya Muthukumaran, Christian Priebe, Joshua Lind, Robert Krahn, Christof Fetzer, David Eyers, and Peter Pietzuch. 2017. TaLoS: Secure and transparent TLS termination inside SGX enclaves. Imperial College London, Tech. Rep 5 (2017).Google Scholar
- AWS. 2018. AWS Direct Connect. Online at: https://aws.amazon.com/directconnect/.Google Scholar
- Azure. 2018. Azure ExpressRoute. Online at: https://azure.microsoft.com/en-us/services/expressroute/.Google Scholar
- Andrew Baumann, Marcus Peinado, and Galen Hunt. 2014. Shielding applications from an untrusted cloud with Haven. In Proc. of USENIX OSDI.Google Scholar
- Theophilus Benson, Aditya Akella, Anees Shaikh, and Sambit Sahu. 2011. CloudNaaS: a cloud networking platform for enterprise applications. In Proc. of ACM SOCC.Google ScholarDigital Library
- CAIDA. 2016. The CAIDA UCSD Anonymized Internet Traces 2016 - April 6th. Online at: http://www.caida.org/data/passive/passive_2016_dataset.xml.Google Scholar
- Sébastien Canard, A"ida Diop, Nizar Kheir, Marie Paindavoine, and Mohamed Sabt. 2017. BlindIDS: Market-Compliant and Privacy-Friendly Intrusion Detection System over Encrypted Traffic. In Proc. of ACM AsiaCCS.Google ScholarDigital Library
- Martin Casado, Teemu Koponen, Daekyeong Moon, and Scott Shenker. 2008. Rethinking Packet Forwarding Hardware. In Proc. of ACM HotNets.Google Scholar
- Shanwei Cen and Bo Zhang. 2017. Trusted Time and Monotonic Counters with Intel Software Guard Extensions Platform Services. Online at: https://software.intel.com/sites/default/files/managed/1b/a2/Intel-SGX-Platform-Services.pdf.Google Scholar
- Sanchuan Chen, Xiaokuan Zhang, Michael K Reiter, and Yinqian Zhang. 2017. Detecting privileged side-channel attacks in shielded execution with Déjá Vu. In Proc. of ACM AsiaCCS.Google ScholarDigital Library
- Byungkwon Choi, Jongwook Chae, Muhammad Jamshed, Kyoungsoo Park, and Dongsu Han. 2016. DFC: Accelerating string pattern matching for network applications. In Proc. of USENIX NSDI.Google Scholar
- M. Conti, Q. Q. Li, A. Maragno, and R. Spolaor. 2018. The Dark Side(-Channel) of Mobile Devices: A Survey on Network Traffic Analysis. IEEE Communications Surveys Tutorials, Vol. 20, 4 (2018), 2658--2713.Google ScholarCross Ref
- Mauro Conti, Luigi Vincenzo Mancini, Riccardo Spolaor, and Nino Vincenzo Verde. 2016. Analyzing android encrypted network traffic to identify user actions. IEEE TIFS, Vol. 11, 1 (2016), 114--125.Google Scholar
- Victor Costan, Ilia Lebedev, and Srinivas Devadas. 2016. Sanctum: Minimal hardware extensions for strong software isolation. In Proc. of USENIX Security.Google Scholar
- Michael Coughlin, Eric Keller, and Eric Wustrow. 2017. Trusted Click: Overcoming Security Issues of NFV in the Cloud. In Proc. of ACM SDN-NFV Security.Google ScholarDigital Library
- Scott E. Coull and Kevin P. Dyer. 2014. Traffic Analysis of Encrypted Messaging Services: Apple iMessage and Beyond. ACM SIGCOMM CCR, Vol. 44, 5 (2014).Google Scholar
- Lorenzo De Carli, Robin Sommer, and Somesh Jha. 2014. Beyond pattern matching: A concurrency model for stateful deep packet inspection. In Proc. of ACM CCS.Google ScholarDigital Library
- Robert DeFrancesco. 2019. Securing The Cloud With Zscaler. Online at: https://www.forbes.com/sites/robertdefrancesco/2019/03/06/securing-the-cloud-with-zscaler/#29281a785d52.Google Scholar
- Mihai Dobrescu, Norbert Egi, Katerina Argyraki, Byung-Gon Chun, Kevin Fall, Gianluca Iannaccone, Allan Knies, Maziar Manesh, and Sylvia Ratnasamy. 2009. RouteBricks: Exploiting Parallelism to Scale Software Routers. In Proc. of ACM SOSP.Google ScholarDigital Library
- Kevin P Dyer, Scott E Coull, Thomas Ristenpart, and Thomas Shrimpton. 2012. Peek-a-boo, I still see you: Why efficient traffic analysis countermeasures fail. In Proc. of IEEE S&P.Google ScholarDigital Library
- Daniel E Eisenbud, Cheng Yi, Carlo Contavalli, Cody Smith, Roman Kononov, Eric Mann-Hielscher, Ardas Cilingiroglu, Bin Cheyney, Wentao Shang, and Jinnah Dylan Hosein. 2016. Maglev: A Fast and Reliable Software Network Load Balancer. In Proc. of USENIX NSDI.Google Scholar
- Ulfar Erlingsson, Mark Manasse, and Frank McSherry. 2006. A cool and practical alternative to traditional hash tables. In Proc. of 7th Workshop on Distributed Data and Structures (WDAS'06).Google Scholar
- Jingyuan Fan, Chaowen Guan, Kui Ren, Yong Cui, and Chunming Qiao. 2017. SPABox: Safeguarding Privacy During Deep Packet Inspection at a MiddleBox. IEEE/ACM Transactions on Networking, Vol. 25, 6 (2017), 3753--3766.Google ScholarDigital Library
- Edward Fjellskål. 2017. Passive Real-time Asset Detection System. Online at: https://github.com/gamelinux/prads.Google Scholar
- Chema García. 2018. libntoh. Online at: https://github.com/sch3m4/libntoh.Google Scholar
- Aaron Gember-Jacobson, Raajay Viswanathan, Chaithan Prakash, Robert Grandl, Junaid Khalid, Sourav Das, and Aditya Akella. 2014. OpenNF: Enabling Innovation in Network Function Control. In Proc. of ACM SIGCOMM.Google ScholarDigital Library
- Glen Gibb, Hongyi Zeng, and Nick McKeown. 2012. Outsourcing network functionality. In Proc. of ACM HotSDN.Google ScholarDigital Library
- Google. 2018. Google Dedicated Interconnect. Online at: https://cloud.google.com/interconnect/docs/concepts/dedicated-overview.Google Scholar
- Johannes Götzfried, Moritz Eckert, Sebastian Schinzel, and Tilo Müller. 2017. Cache attacks on Intel SGX. In Proc. of ACM EuroSec.Google ScholarDigital Library
- The Tcpdump Group. 2018. libpcap. Online at: https://www.tcpdump.org.Google Scholar
- Daniel Gruss, Julian Lettner, Felix Schuster, Olya Ohrimenko, Istvan Haller, and Manuel Costa. 2017. Strong and Efficient Cache Side-Channel Protection using Hardware Transactional Memory. In Proc. of USENIX Security.Google Scholar
- Juhyeng Han, Seongmin Kim, Jaehyeong Ha, and Dongsu Han. 2017. SGX-Box: Enabling Visibility on Encrypted Traffic Using a Secure Middlebox Module. In Proc. of the First Asia-Pacific Workshop on Networking.Google ScholarDigital Library
- Gregory L Heileman and Wenbin Luo. 2005. How Caching Affects Hashing. In Proc. of ALENEX/ANALCO.Google Scholar
- Tyler Hunt, Zhiting Zhu, Yuanzhong Xu, Simon Peter, and Emmett Witchel. 2016. Ryoan: a distributed sandbox for untrusted computation on secret data. In Proc. of USENIX OSDI.Google Scholar
- Mohsen Imani, Mohammad Saidur Rahman, and Matthew Wright. 2018. Adversarial Traces for Website Fingerprinting Defense. In Proc. of ACM CCS.Google ScholarDigital Library
- Zscaler Inc. 2019 a. Encrypted Traffic Dashboard. Online at: https://www.zscaler.com/threatlabz/encrypted-traffic-dashboard.Google Scholar
- Zscaler Inc. 2019 b. SSL Inspection. Online at: https://www.zscaler.com/products/ssl-inspection.Google Scholar
- Zscaler Inc. 2019 c. Zscaler Architecture: Cloud from the beginning. Online at: https://www.zscaler.com/products/cloud-architecture-security-as-a-service.Google Scholar
- Intel. 2017. Data plane development kit. Online at: http://www.dpdk.org.Google Scholar
- Prerit Jain, Soham Desai, Seongmin Kim, Ming-Wei Shih, J Lee, Changho Choi, Youjung Shin, Taesoo Kim, Brent Byunghoon Kang, and Dongsu Han. 2016. Opensgx: An open platform for sgx research. In Proc. of NDSS.Google ScholarCross Ref
- Muhammad Asim Jamshed, Jihyung Lee, Sangwoo Moon, Insu Yun, Deokjin Kim, Sungryoul Lee, Yung Yi, and KyoungSoo Park. 2012. Kargus: a highly-scalable software-based intrusion detection system. In Proc. of ACM CCS.Google ScholarDigital Library
- Muhammad Asim Jamshed, YoungGyoun Moon, Donghwi Kim, Dongsu Han, and KyoungSoo Park. 2014. mOS: A Reusable Networking Stack for Flow Monitoring Middleboxes. In Proc. of USENIX NSDI.Google Scholar
- Murad Kablan, Azzam Alsudais, Eric Keller, and Franck Le. 2017. Stateless Network Functions: Breaking the Tight Coupling of State and Processing. In Proc. of USENIX NSDI.Google Scholar
- Junaid Khalid, Aaron Gember-Jacobson, Roney Michael, Anubhavnidhi Abhashkumar, and Aditya Akella. 2016. Paving the way for NFV: Simplifying middlebox modifications using StateAlyzr. In Proc. of USENIX NSDI.Google Scholar
- Changhoon Kim, Matthew Caesar, Alexandre Gerber, and Jennifer Rexford. 2009. Revisiting route caching: The world should be flat. In Proc. of International Conference on Passive and Active Network Measurement.Google ScholarCross Ref
- Eddie Kohler, Robert Morris, Benjie Chen, John Jannotti, and M Frans Kaashoek. 2000. The Click modular router. ACM Transactions on Computer Systems (TOCS), Vol. 18, 3 (2000), 263--297.Google ScholarDigital Library
- Leslie Lamport. 1977. Proving the correctness of multiprocess programs. IEEE transactions on software engineering 2 (1977), 125--143.Google Scholar
- Chang Lan, Justine Sherry, Raluca Ada Popa, and Sylvia Ratnasamy. 2016. EMBArk: Securely Outsourcing Middleboxes to the Cloud. In Proc. of USENIX NSDI.Google Scholar
- Patrick PC Lee, Tian Bu, and Girish Chandranmenon. 2010. A lock-free, cache-efficient multi-core synchronization mechanism for line-rate network traffic monitoring. In Proc. of IEEE Parallel & Distributed Processing (IPDPS).Google ScholarCross Ref
- Paige Leskin. 2018. The 21 scariest data breaches of 2018. Online at: https://www.businessinsider.com/data-hacks-breaches-biggest-of-2018--2018--12.Google Scholar
- Hongda Li, Hongxin Hu, Guofei Gu, Gail-Joon Ahn, and Fuqiang Zhang. 2018. vNIDS: Towards Elastic Security with Safe and Efficient Virtualization of Network Intrusion Detection Systems. In Proc. of ACM CCS.Google ScholarDigital Library
- Joshua Lind, Christian Priebe, Divya Muthukumaran, Dan O'Keeffe, Pierre-Louis Aublin, Florian Kelbert, Tobias Reiher, David Goltzsche, David Eyers, Rüdiger Kapitza, et al. 2017. Glamdring: automatic application partitioning for intel SGX. In Proc. of USENIX ATC.Google Scholar
- Frank McKeen, Ilya Alexandrovich, Alex Berenzon, Carlos V Rozas, Hisham Shafi, Vedvyas Shanbhogue, and Uday R Savagaonkar. 2013. Innovative instructions and software model for isolated execution. In Proc. of ACM HASP.Google ScholarDigital Library
- Luca Melis, Hassan Jameel Asghar, Emiliano De Cristofaro, and Mohamed Ali Kaafar. 2016. Private Processing of Outsourced Network Functions: Feasibility and Constructions. In Proc. of ACM SDN-NFV Security.Google ScholarDigital Library
- David Mills. 1985. Network Time Protocol. Online at: https://tools.ietf.org/html/rfc958.Google Scholar
- P. Mishra, R. Poddar, J. Chen, A. Chiesa, and R. A. Popa. 2018. Oblix: An Efficient Oblivious Search Index. In Proc. of IEEE S&P.Google Scholar
- Ahmad Moghimi, Gorka Irazoqui, and Thomas Eisenbarth. 2017. Cachezoom: How SGX amplifies the power of cache attacks. In Proc. of CHES.Google ScholarCross Ref
- David Naylor, Richard Li, Christos Gkantsidis, Thomas Karagiannis, and Peter Steenkiste. 2017a. And Then There Were More: Secure Communication for More Than Two Parties. In Proc. of ACM CoNEXT.Google ScholarDigital Library
- David Naylor, Richard Li, Christos Gkantsidis, Thomas Karagiannis, and Peter Steenkiste. 2017b. And Then There Were More: Secure Communication for More Than Two Parties. In Proc. of ACM CoNEXT.Google ScholarDigital Library
- Meni Orenbach, Pavel Lifshits, Marina Minkin, and Mark Silberstein. 2017. Eleos: ExitLess OS Services for SGX Enclaves. In Proc. of ACM Eurosys.Google ScholarDigital Library
- Aurojit Panda, Sangjin Han, Keon Jang, Melvin Walls, Sylvia Ratnasamy, and Scott Shenker. 2016. NetBricks: Taking the V out of NFV. In Proc. of USENIX OSDI.Google Scholar
- Parveen Patel, Deepak Bansal, Lihua Yuan, Ashwin Murthy, Albert Greenberg, David A. Maltz, Randy Kern, Hemant Kumar, Marios Zikos, Hongyu Wu, Changhoon Kim, and Naveen Karri. 2013. Ananta: Cloud Scale Load Balancing. In Proc. of ACM SIGCOMM.Google ScholarDigital Library
- Rishabh Poddar, Chang Lan, Raluca Ada Popa, and Sylvia Ratnasamy. 2018. SafeBricks: Shielding Network Functions in the Cloud. In Proc. of USENIX NSDI.Google Scholar
- HAProxy Project. 2018. The reliable, high performance tcp/http load balancer. Online at: http://www.haproxy.org/.Google Scholar
- Andrew Reed and Michael Kranch. 2017. Identifying HTTPS-Protected Netflix Videos in Real-Time. In Proc. of ACM CODASPY.Google ScholarDigital Library
- Luigi Rizzo. 2012. netmap: A Novel Framework for Fast Packet I/O. In Proc. of USENIX ATC.Google Scholar
- Alan Rusbridger. 2013. The Snowden Leaks and the Public. The New York Review of Books (2013).Google Scholar
- Satori. 2017. Fast multi-core TCP and WebSockets load generator. Online at: https://github.com/machinezone/tcpkali.Google Scholar
- Felix Schuster, Manuel Costa, Cédric Fournet, Christos Gkantsidis, Marcus Peinado, Gloria Mainar-Ruiz, and Mark Russinovich. 2015. VC3: Trustworthy data analytics in the cloud using SGX. In Proc. of IEEE S&P.Google ScholarDigital Library
- Vyas Sekar, Norbert Egi, Sylvia Ratnasamy, Michael K Reiter, and Guangyu Shi. 2012. Design and implementation of a consolidated middlebox architecture. In Proc. of USENIX NSDI.Google Scholar
- Jaebaek Seo, Byounyoung Lee, Seongmin Kim, Ming-Wei Shih, Insik Shin, Dongsu Han, and Taesoo Kim. 2017. SGX-Shield: Enabling address space layout randomization for SGX programs. In Proc. of NDSS.Google ScholarCross Ref
- Fahad Shaon, Murat Kantarcioglu, Zhiqiang Lin, and Latifur Khan. 2017. SGX-BigMatrix: A Practical Encrypted Data Analytic Framework With Trusted Processors. In Proc. of ACM CCS.Google ScholarDigital Library
- Justine Sherry, Shaddi Hasan, Colin Scott, Arvind Krishnamurthy, Sylvia Ratnasamy, and Vyas Sekar. 2012. Making middleboxes someone else's problem: network processing as a cloud service. In Proc. of ACM SIGCOMM.Google ScholarDigital Library
- Justine Sherry, Chang Lan, Raluca Ada Popa, and Sylvia Ratnasamy. 2015. BlindBox: Deep Packet Inspection for Encrypted Traffic. In Proc. of ACM SIGCOMM.Google ScholarDigital Library
- Ming-Wei Shih, Mohan Kumar, Taesoo Kim, and Ada Gavrilovska. 2016. S-NFV: securing NFV states by using SGX. In Proc. of ACM SDN-NFV Security.Google ScholarDigital Library
- Shweta Shinde, Zheng Leong Chua, Viswesh Narayanan, and Prateek Saxena. 2016. Preventing page faults from telling your secrets. In Proc. of AsiaCCS. ACM.Google ScholarDigital Library
- Payap Sirinam, Mohsen Imani, Marc Juarez, and Matthew Wright. 2018. Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learning. In Proc. of ACM CCS.Google ScholarDigital Library
- Snort. 2018a. Network Intrusion Detection & Prevention System. Online at: https://www.snort.org/.Google Scholar
- Snort. 2018b. Rule Subscriptions. Online at: https://www.snort.org/products#rule_subscriptions.Google Scholar
- Chen Sun, Jun Bi, Zhilong Zheng, Heng Yu, and Hongxin Hu. 2017. NFP: Enabling Network Function Parallelism in NFV. In Proc. of ACM SIGCOMM.Google ScholarDigital Library
- Meysam Taassori, Ali Shafiee, and Rajeev Balasubramonian. 2018. VAULT: Reducing Paging Overheads in SGX with Efficient Integrity Verification Structures. In Proc. of ACM ASPLOS.Google ScholarDigital Library
- Hongliang Tian, Qiong Zhang, Shoumeng Yan, Alex Rudnitsky, Liron Shacham, Ron Yariv, and Noam Milshten. 2018. Switchless Calls Made Practical in Intel SGX. In Proc. of ACM SysTEX.Google ScholarDigital Library
- Bohdan Trach, Alfred Krohmer, Franz Gregor, Sergei Arnautov, Pramod Bhatotia, and Christof Fetzer. 2018. ShieldBox: Secure Middleboxes using Shielded Execution. In ACM Proc. of SOSR.Google ScholarDigital Library
- Chia-che Tsai, Donald E Porter, and Mona Vij. 2017. Graphene-SGX: A Practical Library OS for Unmodified Applications on SGX. In Proc. of USENIX ATC.Google Scholar
- Nirvan Tyagi, Yossi Gilad, Derek Leung, Matei Zaharia, and Nickolai Zeldovich. 2017. Stadium: A Distributed Metadata-Private Messaging System. In Proc. of ACM SOSP.Google ScholarDigital Library
- Jelle Van Den Hooff, David Lazar, Matei Zaharia, and Nickolai Zeldovich. 2015. Vuvuzela: Scalable private messaging resistant to traffic analysis. In Proc. of ACM SOSP.Google ScholarDigital Library
- Michael Walfish, Jeremy Stribling, Maxwell Krohn, Hari Balakrishnan, Robert Morris, and Scott Shenker. 2004. Middleboxes No Longer Considered Harmful. In Proc. of USENIX OSDI.Google Scholar
- Tao Wang, Xiang Cai, Rishab Nithyanand, Rob Johnson, and Ian Goldberg. 2014. Effective Attacks and Provable Defenses for Website Fingerprinting. In Proc. of USENIX Security.Google Scholar
- Tao Wang and Ian Goldberg. 2017. Walkie-Talkie: An Efficient Defense Against Passive Website Fingerprinting Attacks. In Proc. of USENIX Security.Google Scholar
- Wenhao Wang, Guoxing Chen, Xiaorui Pan, Yinqian Zhang, XiaoFeng Wang, Vincent Bindschaedler, Haixu Tang, and Carl A Gunter. 2017. Leaky cauldron on the dark land: Understanding memory side-channel hazards in sgx. In Proc. of ACM CCS.Google ScholarDigital Library
- Nico Weichbrodt, Anil Kurmus, Peter Pietzuch, and Rüdiger Kapitza. 2016. AsyncShock: Exploiting synchronisation bugs in Intel SGX enclaves. In Proc. of ESORICS.Google ScholarCross Ref
- Ofir Weisse, Valeria Bertacco, and Todd Austin. 2017. Regaining Lost Cycles with HotCalls: A Fast Interface for SGX Secure Enclaves. In Proc. of ACM ISCA.Google ScholarDigital Library
- Andrew M. White, Austin R. Matthews, Kevin Z. Snow, and Fabian Monrose. 2011. Phonotactic Reconstruction of Encrypted VoIP Conversations: Hookt on Fon-iks. In Proc. of IEEE S&P.Google ScholarDigital Library
- wolfSSL Inc. 2017. wolfSSL. Online at: https://www.wolfssl.com/wolfssl-with-intel-sgx/.Google Scholar
- Yuanzhong Xu, Weidong Cui, and Marcus Peinado. 2015. Controlled-channel attacks: Deterministic side channels for untrusted operating systems. In Proc. of IEEE S&P.Google ScholarDigital Library
- Xingliang Yuan, Xinyu Wang, Jianxiong Lin, and Cong Wang. 2016. Privacy-preserving Deep Packet Inspection in Outsourced Middleboxes. In Proc. of IEEE INFOCOM.Google ScholarCross Ref
- Fan Zhang. 2017. mbedtls-SGX. Online at: https://github.com/bl4ck5un/mbedtls-SGX.Google Scholar
- Fan Zhang, Ethan Cecchetti, Kyle Croman, Ari Juels, and Elaine Shi. 2016. Town crier: An authenticated data feed for smart contracts. In Proc. of ACM CCS.Google ScholarDigital Library
Supplemental Material
video
webm
115.4 MB
Index Terms
LightBox
Comments