research-article

Open access

Kua: a distributed object store over named data networking

Authors:

Hemil Desai, and

Lixia ZhangAuthors Info & Claims

ICN '22: Proceedings of the 9th ACM Conference on Information-Centric Networking

September 2022

Pages 56 - 66

https://doi.org/10.1145/3517212.3558083

Published: 06 September 2022 Publication History

Abstract

Applications such as machine learning training systems or log collection generate and consume large amounts of data. Object storage systems provide a simple abstraction to store and access such large datasets. These datasets are typically larger than the capacities of individual storage servers, and require fault tolerance through replication. In this paper, we present Kua, a distributed object storage system built over Named Data Networking (NDN). The data-centric nature of NDN helps Kua maintain a simple design while catering to requirements of storing large objects, providing fault tolerance, low latency and strong consistency guarantees, along with data-centric security. Our prototype Kua implementation provides easy-to-use primitives to let applications store and access data securely, and our initial evaluation suggests that Kua can leverage NDN's capabilities of multicast data delivery and in-network caching to achieve higher efficiency than existing object storage systems.

References

[1]

2022. Scaling with Redis Cluster. https://redis.io/docs/manual/scaling/. Accessed: 2022-05-18.

[2]

Alexander Afanasyev, Tamer Refaei, Lan Wang, and Lixia Zhang. 2018. A Brief Introduction to Named Data Networking. In Proc. of MILCOM.

Digital Library

[3]

Josiah Carlson. 2013. Redis in action. Simon and Schuster.

Digital Library

[4]

M. Factor, K. Meth, D. Naor, O. Rodeh, and J. Satran. 2005. Object storage: the future building block for storage systems. In 2005 IEEE International Symposium on Mass Storage Systems and Technology. 119--123.

Digital Library

[5]

Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. 2003. The Google file system. In Proceedings of the nineteenth ACM symposium on Operating systems principles. 29--43.

Digital Library

[6]

Garth A. Gibson, David F. Nagle, Khalil Amiri, Fay W. Chang, Eugene M. Feinberg, Howard Gobioff, Chen Lee, Berend Ozceri, Erik Riedel, David Rochberg, and Jim Zelenka. 1997. File Server Scaling with Network-Attached Secure Disks. SIGMETRICS Perform. Eval. Rev. 25, 1 (jun 1997), 272--284.

Digital Library

[7]

Developer Guide. 2008. Amazon Simple Storage Service. (2008).

[8]

John H. Howard. 1988. On Overview of the Andrew File System. In USENIX Winter.

[9]

Avinash Lakshman and Prashant Malik. 2010. Cassandra: A Decentralized Structured Storage System. SIGOPS Oper. Syst. Rev. 44, 2 (apr 2010), 35--40.

Digital Library

[10]

Youyou Lu, Jiwu Shu, Youmin Chen, and Tao Li. 2017. Octopus: an RDMA-enabled Distributed Persistent Memory File System. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 773--785. https://www.usenix.org/conference/atc17/technical-sessions/presentation/lu

Digital Library

[11]

Mini-NDN Authors. 2021. Mini-NDN: A Mininet-based NDN emulator. minindn. memphis.edu/ accessed: 2021-05-10.

[12]

Philipp Moll, Varun Patil, Lixia Zhang, and Davide Pesavento. 2021. Resilient Brokerless Publish-Subscribe Over NDN. In MILCOM 2021 - Special Session on Named Data Networking (MILCOM 2021 - NDN Session). San Diego, USA.

[13]

Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, et al. 2018. Ray: A distributed framework for emerging {AI} applications. In 13th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 18). 561--577.

Digital Library

[14]

NDN Project team. 2021. NDN Packet Format Specification version 0.3: Signed Interest. (2021). https://named-data.net/doc/NDN-packet-spec/current/signed-interest.html accessed: 2021-07-29.

[15]

Jon Postel. 1981. RFC793: Transmission Control Protocol. Technical Report.

[16]

Russel Sandberg. 2000. The Sun Network File System: Design, Implementation and Experience. (09 2000).

[17]

Junxiao Shi, Davide Pesavento, and Lotfi Benmohamed. 2020. NDN-DPDK: NDN Forwarding at 100 Gbps on Commodity Hardware. In Proceedings of the 7th ACM Conference on Information-Centric Networking (Virtual Event, Canada) (ICN '20). Association for Computing Machinery, New York, NY, USA, 30--40.

Digital Library

[18]

Yong Yoon Shin, Sae Hyong Park, Namseok Ko, and Arm Jeong. 2020. Chipmunk: Distributed Object Storage for NDN. In Proceedings of the 7th ACM Conference on Information-Centric Networking (Virtual Event, Canada) (ICN '20). Association for Computing Machinery, New York, NY, USA, 161--162.

Digital Library

[19]

Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. 2010. The hadoop distributed file system. In 2010 IEEE 26th symposium on mass storage systems and technologies (MSST). Ieee, 1--10.

Digital Library

[20]

NDN Project Team. 2014. repo-ng. https://github.com/named-data/repo-ng

[21]

NDN Project Team. 2018. Fast Repo. https://github.com/remap/fast-repo

[22]

The Kua Team. 2022. Kua: Distributed Object Storage over Named Data Networking. https://github.com/pulsejet/kua

[23]

Lan Wang, Vince Lehman, A. K. M. Mahmudul Hoque, Beichuan Zhang, Yingdi Yu, and Lixia Zhang. 2018. A Secure Link State Routing Protocol for NDN. IEEE Access 6 (2018), 10470--10482.

[24]

Sage A. Weil, Scott A. Brandt, Ethan L. Miller, Darrell D. E. Long, and Carlos Maltzahn. 2006. Ceph: A Scalable, High-Performance Distributed File System. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (Seattle, Washington) (OSDI '06). USENIX Association, USA, 307--320.

Digital Library

[25]

Huaxia Xia and Andrew A. Chien. 2007. RobuSTore: a distributed storage architecture with robust and high performance. In SC '07: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing. 1--11.

Digital Library

[26]

Zhiyi Zhang, Yingdi Yu, Sanjeev Kaushik Ramani, Alex Afanasyev, and Lixia Zhang. 2018. NAC: Automating Access Control via Named Data. In MILCOM 2018 - 2018 IEEE Military Communications Conference (MILCOM). 626--633.

Digital Library

Cited By

Kutscher DBurke JFioccola GMendes PCalvert KHjálmtýsson GCrowley PPapadopoulos C(2023)Statement: The Metaverse as an Information-Centric NetworkProceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623761(112-114)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3623565.3623761
Patil VYu TMa XZhang LCalvert KHjálmtýsson GCrowley PPapadopoulos C(2023)Decentralized Photo Sharing via Named Data NetworkingProceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623755(118-120)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3623565.3623755
Liu SZhang LCalvert KHjálmtýsson GCrowley PPapadopoulos C(2023)Mnemosyne: Distributed Application Logging over Named Data NetworkingProceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623708(26-35)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3623565.3623708

Index Terms

Kua: a distributed object store over named data networking
1. Information systems
  1. Information storage systems
    1. Storage architectures
      1. Distributed storage
    2. Storage replication
2. Networks
  1. Network services
    1. Cloud computing

Recommendations

WORMStore: A Specialized Object Store for Write-Once Read-Many Workloads
MASCOTS '14: Proceedings of the 2014 IEEE 22nd International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems

The recent increase in interest for batch analytics has resulted in extensive use of distributed frameworks such as Hadoop and Dryad. Batch analytics-as the name suggests, perform many computations on large volumes of data. That is, large quantities of ...
Read More
Using ceph's BlueStore as object storage in HPC storage framework
CHEOPS '21: Proceedings of the Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems

In times of ever-increasing data sizes, data management and insightful analysis are amidst the most severe challenges of high-performance computing. While high-level libraries such as NetCDF, HDF5, and ADIOS2, as well as the associated self-describing ...
Read More
Use case study of HDD-SSD hybrid storage, distributed storage and HDD storage on OpenStack
IDEAS '15: Proceedings of the 19th International Database Engineering & Applications Symposium

For typical IaaS cloud usage, frequent provisioning of virtual volumes is needed. However, performance of HDD storage is not sufficient and becomes bottleneck in IaaS cloud. In this paper, we studied a comparison of HDD-SSD hybrid storage, distributed ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICN '22: Proceedings of the 9th ACM Conference on Information-Centric Networking

September 2022

203 pages

ISBN:9781450392570

DOI:10.1145/3517212

Program Chairs:
Henning Schulzrinne
Columbia University
,
Matthias Wählisch
Freie Universität Berlin, Germany
,
Lixia Zhang
UCLA

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 September 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

ICN '22

Sponsor:

SIGCOMM

ICN '22: 9th ACM Conference on Information-Centric Networking

September 19 - 21, 2022

Osaka, Japan

Acceptance Rates

Overall Acceptance Rate 133 of 482 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
538
Total Downloads

Downloads (Last 12 months)249
Downloads (Last 6 weeks)23

Other Metrics

View Author Metrics

Citations

Cited By

Kutscher DBurke JFioccola GMendes PCalvert KHjálmtýsson GCrowley PPapadopoulos C(2023)Statement: The Metaverse as an Information-Centric NetworkProceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623761(112-114)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3623565.3623761
Patil VYu TMa XZhang LCalvert KHjálmtýsson GCrowley PPapadopoulos C(2023)Decentralized Photo Sharing via Named Data NetworkingProceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623755(118-120)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3623565.3623755
Liu SZhang LCalvert KHjálmtýsson GCrowley PPapadopoulos C(2023)Mnemosyne: Distributed Application Logging over Named Data NetworkingProceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623708(26-35)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3623565.3623708

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents