skip to main content
research-article

Depot: Cloud Storage with Minimal Trust

Published:01 December 2011Publication History
Skip Abstract Section

Abstract

This article describes the design, implementation, and evaluation of Depot, a cloud storage system that minimizes trust assumptions. Depot tolerates buggy or malicious behavior by any number of clients or servers, yet it provides safety and liveness guarantees to correct clients. Depot provides these guarantees using a two-layer architecture. First, Depot ensures that the updates observed by correct nodes are consistently ordered under Fork-Join-Causal consistency (FJC). FJC is a slight weakening of causal consistency that can be both safe and live despite faulty nodes. Second, Depot implements protocols that use this consistent ordering of updates to provide other desirable consistency, staleness, durability, and recovery properties. Our evaluation suggests that the costs of these guarantees are modest and that Depot can tolerate faults and maintain good availability, latency, overhead, and staleness even when significant faults occur.

References

  1. Abu-Libdeh, H., Princehouse, L., and Weatherspoon, H. 2010. RACS: A case for cloud storage diversity. In Proceedings of the ACM Symposium on Cloud Computing (SOCC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Ahamad, M., Neiger, G., Burns, J. E., Kohli, P., and Hutto, P. 1995. Causal memory: Definitions, implementation and programming. Distrib. Comput. 9, 1, 37--49.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Amazon. 2011. Summary of the Amazon EC2 and Amazon RDS service disruption in the US East region. http://aws.amazon.com/message/65648.Google ScholarGoogle Scholar
  4. Amazon S3 Team. 2008. Amazon S3 Team. Amazon S3 availability event: July 20, 2008. http://status.aws.amazon.com/s3-20080720.html.Google ScholarGoogle Scholar
  5. Beckmann, C. 2009. Google App Engine: Information regarding 2 July 2009 outage. http://groups.google.com/group/google-appengine/browse_thread/thread/e9237fc7b0aa7df5/ba95ded980c8c179.Google ScholarGoogle Scholar
  6. Belaramani, N., Dahlin, M., Gao, L., Nayate, A., Venkataramani, A., Yalagandula, P., and Zheng, J. 2006. PRACTI replication. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Birrell, A., Levin, R., Needham, R., and Schroeder, M. 1982. Grapevine: An exercise in distributed computing. Comm. ACM 25, 4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Cachin, C., Shelat, A., and Shraer, A. 2007. Efficient fork-linearizable access to untrusted shared memory. In Proceedings of the ACM Symposium on the Principles of Distributed Computing (PODC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Cachin, C., Keidar, I., and Shraer, A. 2009. Fail-aware untrusted storage. In Proceedings of the International Conference on Dependable Systems and Networks (DSN).Google ScholarGoogle Scholar
  10. Calore, M. 2009. Magnolia suffers major data loss, site taken offline. Wired Mag.Google ScholarGoogle Scholar
  11. Castro, M. and Liskov, B. 2002. Practical Byzantine fault tolerance and proactive recovery. ACM Trans. Comput. Syst. 20, 4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Chang, F., Dean, J., Ghemawat, S., Hsieh, W., Wallach, D., Burrows, M., Chandra, T., Fikes, A., and Gruber, R. 2006. Bigtable: A distributed storage system for structured data. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Chun, B.-G., Maniatis, P., Shenker, S., and Kubiatowicz, J. 2007. Attested append-only memory: Making adversaries stick to their word. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. CircleID. 2009. Survey: Cloud computing ‘no hype’, but fear of security and control slowing adoption. http://www.circleid.com/posts/20090226_cloud_computing_hype_security/.Google ScholarGoogle Scholar
  15. Clement, A., Kapritsos, M., Lee, S., Wang, Y., Alvisi, L., Dahlin, M., and Riché, T. 2009. UpRight cluster services. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. CNet News. 2011. Google probing lost Gmail messages, contacts. http://news.cnet.com/8301-1023 3-20037019-93.html.Google ScholarGoogle Scholar
  17. Cook, B. 2009. Seattle data center fire knocks out Bing Travel, other web sites. http://www.techflash.com/seattle/2009/07/Seattle_data_center_fire_knocks_out_Bing_Travel_other_Web_sites_49876777.html.Google ScholarGoogle Scholar
  18. Cooper, B., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H., Puz, N., Weaver, D., and Yerneni, R. 2008. PNUTS: Yahoo!’s hosted data serving platform. In Proceedings of the International Conference on Very Large Data Bases (VLDB).Google ScholarGoogle Scholar
  19. DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., and Vogels, W. 2007. Dynamo: Amazon’s highly available key-value store. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Demmer, M., Du, B., and Brewer, E. 2008. TierStore: A distributed filesystem for challenged networks in developing regions. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Feldman, A. J., Zeller, W. P., Freedman, M. J., and Felten, E. W. 2010. SPORC: Group collaboration using untrusted cloud resources. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Frigo, M. and Luchangco, V. 1998. Computation-centric memory models. In Proceedings of the ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Fu, K., Kaashoek, M. F., and Mazières, D. 2002. Fast and secure distributed read-only file system. ACM Trans. Comput. Syst. 20, 1, 1--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Gilbert, S. and Lynch, N. 2002. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33, 51--59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Goh, E.-J., Shacham, H., Modadugu, N., and Boneh, D. 2003. SiRiUS: Securing remote untrusted storage. In Proceedings of the Annual Network & Distributed System Security Symposium (NDSS).Google ScholarGoogle Scholar
  26. Gray, J. and Shenoy, P. 2000. Rules of thumb in data engineering. In Data Engineering. 3--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Guerraoui, R., Knezevic, N., Quema, V., and Vukolic, M. 2010. The next 700 BFT protocols. In Proceedings of the European Conference on Computer Systems (EuroSys). Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Haeberlen, A., Mislove, A., and Druschel, P. 2005. Glacier: Highly durable, decentralized storage despite massive correlated failures. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Haeberlen, A., Kouznetsov, P., and Druschel, P. 2007. PeerReview: Practical accountability for distributed systems. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Hendricks, J., Ganger, G. R., and Reiter, M. K. 2007. Low-overhead Byzantine fault-tolerant storage. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Herlihy, M. P. and Wing, J. M. 1990. Linearizability: A correctness condition for concurrent objects. ACM Trans. Program. Lang. Syst. 12, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kallahalla, M., Riedel, E., Swaminathan, R., Wang, Q., and Fu, K. 2003. Plutus: Scalable secure file sharing on untrusted storage. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Kang, B. 2004. S2D2: A framework for scalable and secure optimistic replication. Ph.D. thesis, University of California Berkeley. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Karger, D., Lehman, E., Leighton, T., Panigrahy, R., Levine, M., and Lewin, D. 1997. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the world wide web. In Proceedings of the ACM Symposium on Theory of Computing (STOC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Kistler, J. and Satyanarayanan, M. 1992. Disconnected operation in the coda file system. ACM Trans. Comput. Syst. 10, 1, 3--5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Kotla, R., Alvisi, L., and Dahlin, M. 2007. SafeStore: A durable and practical storage system. In Proceedings of the USENIX Annual Technical Conference (ATC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., and Zhao, B. 2000. OceanStore: An architecture for global-scale persistent storage. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Lamport, L. 1978. Time, clocks, and the ordering of events in a distributed system. Comm. ACM 21, 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Lamport, L. 1979. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Trans. Comput. 28, 9, 690--691. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Lamport, L., Shostak, R., and Pease, M. 1982. The Byzantine generals problem. ACM Trans. Program. Lang. Syst. 4, 3, 382--401. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Levin, D., Douceur, J. R., Lorch, J. R., and Moscibroda, T. 2009. TrInc: Small trusted hardware for large distributed systems. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Li, J. and Mazières, D. 2007. Beyond one-third faulty replicas in Byzantine fault tolerant systems. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Li, J., Krohn, M., Mazières, D., and Shasha, D. 2004. Secure untrusted data repository (SUNDR). In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Malkhi, D. and Reiter, M. 1998. Byzantine quorum systems. Distrib. Comput. 11, 4, 203--213. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Maniatis, P. 2003. Historic integrity in distributed systems. Ph.D. thesis, Stanford. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Mazières, D., Kaminsky, M., Kaashoek, M. F., and Witchel, E. 1999. Separating key management from file system security. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Miller, R. 2009. FBI siezes servers at Dallas data center. http://www.datacenterknowledge.com/archives/2009/04/03/fbi-seizes-servers-at-dallas-data-center/.Google ScholarGoogle Scholar
  48. Nath, S., Yu, H., Gibbons, P. B., and Seshan, S. 2006. Subtleties in tolerating correlated failures in wide-area storage systems. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. News, C. 2002. Victims of lost files out of luck. http://news.cnet.com/Victims-of-lost-files-out-of-luck/2100-1023_3-887849.html.Google ScholarGoogle Scholar
  50. Nightingale, E., Veeraraghavan, K., Chen, P., and Flinn, J. 2008. Rethink the sync. ACM Trans. Comput. Syst. 26, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Oppenheimer, D., Ganapathi, A., and Patterson, D. A. 2003. Why do Internet services fail, and what can be done about it? In Proceedings of the USENIX Symposium on Internet Technologies and Systems (USITS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Oprea, A. and Reiter, M. 2006. On consistency of encrypted files. In Proceedings of the International Symposium on Distributed Computing (DISC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Parker, D. S., Popek, G. J., Rudisin, G., Stoughton, A., Walker, B. J., Walton, E., Chow, J. M., Kiser, S., Edwards, D., and Kline, C. 1983. Detection of mutual inconsistency in distributed systems. IEEE Trans. Softw. Engin. 9, 3, 240--247. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Petersen, K., Spreitzer, M. J., Terry, D. B., Theimer, M. M., and Demers, A. J. 1997. Flexible update propagation for weakly consistent replication. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Pinheiro, E., Weber, W., and Barroso, L. 2007. Failure trends in a large disk drive population. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Popa, R. A., Lorch, J. R., Molnar, D., Wang, H. J., and Zhuang, L. 2011. Enabling security in cloud storage SLAs with CloudProof. In Proceedings of the USENIX Annual Technical Conference (ATC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Prabhakaran, V., Bairavasundaram, L., Agrawal, N., Gunawi, H., Arpaci-Dusseau, A., and Arpaci-Dusseau, R. 2005. IRON file systems. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Ramasubramanian, V., Rodeheffer, T., Terry, D. B., Walraed-Sullivan, M., Wobber, T., Marshall, C. C., and Vahdat, A. 2009. Cimbiosys: A platform for content-based partial replication. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Reiher, P., Heidemann, J., Ratner, D., Skinner, G., and Popek, G. 1994. Resolving file conflicts in the Ficus file system. In Proceedings of the USENIX Summer Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. S3 App Catalog. 2011. AWS forum: Customer app catalog. http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=66.Google ScholarGoogle Scholar
  61. Schiper, A., Birman, K., and Stephenson, P. 1991. Lightweight causal and atomic group multicast. ACM Trans. Comput. Syst. 9, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Shah, M., Baker, M., Mogul, J., and Swaminathan, R. 2007. Auditing to keep online storage services honest. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Shraer, A., Cachin, C., Cidon, A., Keidar, I., Michalevsky, Y., and Shaket, D. 2010. Venus: Verification for untrusted cloud storage. In Proceedings of the ACM Cloud Computing Security Workshop (CCSW). Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Singh, A., Fonseca, P., Kuznetsov, P., Rodrigues, R., and Maniatis, P. 2009. Zeno: Eventually consistent Byzantine fault tolerance. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Strunk, J., Goodson, G., Scheinholtz, M., Soules, C., and Ganger, G. 2000. Self-Securing storage: Protecting data in compromised systems. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Terry, D. B., Demers, A. J., Petersen, K., Spreitzer, M., Theimer, M., and Welch, B. W. 1994. Session guarantees for weakly consistent replicated data. In Proceedings of the International Conference on Parallel and Distributed Information Systems (PDIS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Terry, D. B., Theimer, M. M., Petersen, K., Demers, A. J., Spreitzer, M. J., and Hauser, C. H. 1995. Managing update conflicts in Bayou, a weakly connected replicated storage system. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. US. 2005. US Secret Service report on insider attacks. http://www.sei.cmu.edu/about/press/insider-2005.html.Google ScholarGoogle Scholar
  69. Vogels, W. 2006. Life is not a state-machine: The long road from research to production. In Proceedings of the ACM Symposium on the Principles of Distributed Computing (PODC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., and Joglekar, A. 2002. An integrated experimental environment for distributed systems and networks. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Wobber, T., Rodeheffer, T. L., and Terry, D. B. 2010. Policy-Based access control for weakly consistent replication. In Proceedings of the European Conference on Computer Systems (EuroSys). Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Yahoo. 2011. Yahoo’s offloading of delicious a reminder of cloud risks. http://www.infoworld.com/t/cloud-computing/yahoos-offloading-delicious-reminder-cloud-risks-735.Google ScholarGoogle Scholar
  73. Yang, J., Sar, C., and Engler, D. 2006. EXPLODE: A lightweight, general system for finding serious storage system errors. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Yip, A., Chen, B., and Morris, R. 2006. Pastwatch: A distributed version control system. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Yumerefendi, A. and Chase, J. 2007. Strong accountability for network storage. ACM Trans. Storage 3, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Depot: Cloud Storage with Minimal Trust

            Recommendations

            Reviews

            Veronica Lagrange

            Storage service providers (SSPs) are fault-prone black boxes operated by a third party. Prudent clients should avoid strong assumptions about the integrity of data stored remotely and implement some form of end-to-end checks. Based on these premises, this paper describes in detail a protocol, layered on top of most cloud storage services, that enforces fork-join-causal (FJC) consistency to guarantee data integrity. The protocol, dubbed Depot, increases reliability by replicating both data and metadata associated with update history. The paper is well organized and explains Depot's architecture, operation, protocol, consistency model, and implementation. It also briefly evaluates performance of a prototype and compares its cost with other simpler protocols. This exercise is geared toward evaluating the cost of trust. The authors conclude that, while Depot's read operations do not cost much, writes (updates) can be quite costly and add more than 50 percent to baseline central processing unit (CPU) consumption and more than 30 percent for remote storage space. There is also an interesting discussion on related work in which the authors classify existing systems in terms of high availability and fault tolerance. Online Computing Reviews Service

            Access critical reviews of Computing literature here

            Become a reviewer for Computing Reviews.

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!