skip to main content
research-article
Public Access

Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains

Authors Info & Claims
Published:28 February 2022Publication History
Skip Abstract Section

Abstract

In this paper, we study a canonical distributed resource allocation problem arising in blockchains. While distributed resource allocation is a well-studied problem in networking, the blockchain setting additionally requires the solution to be resilient to adversarial behavior from a fraction of nodes. Scaling blockchain performance is a basic research topic; a plethora of solutions (under the umbrella of sharding ) have been proposed in recent years. Although the various sharding solutions share a common thread (they cryptographically stitch together multiple parallel chains), architectural differences lead to differing resource allocation problems. In this paper we make three main contributions: (a) we categorize the different sharding proposals under a common architectural framework, allowing for the emergence of a new, uniformly improved, uni-consensus sharding architecture. (b) We formulate and exactly solve a core resource allocation problem in the uni-consensus sharding architecture -- our solution, Free2shard, is adversary-resistant and achieves optimal throughput. The key technical contribution is a mathematical connection to the classical work of Blackwell approachability in dynamic game theory. (c) We implement the sharding architecture atop a full-stack blockchain in 3000 lines of code in Rust -- we achieve a throughput of more than 250,000 transactions per second with 6 shards, a vast improvement over state-of-the-art.

References

  1. Jacob Abernethy, Peter L Bartlett, and Elad Hazan. 2011. Blackwell approachability and no-regret learning are equivalent. In Proceedings of the 24th Annual Conference on Learning Theory. JMLR Workshop and Conference Proceedings, 27--46.Google ScholarGoogle Scholar
  2. Ittai Abraham, TH Hubert Chan, Danny Dolev, Kartik Nayak, Rafael Pass, Ling Ren, and Elaine Shi. 2019. Communication complexity of byzantine agreement, revisited. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing . 317--326.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Mustafa Al-Bassam. 2019. LazyLedger: A Distributed Data Availability Ledger With Client-Side Smart Contracts. arxiv: 1905.09274 [cs.CR]Google ScholarGoogle Scholar
  4. Bowen Alpern and Fred B Schneider. 1987. Recognizing safety and liveness. Distributed computing , Vol. 2, 3 (1987), 117--126.Google ScholarGoogle Scholar
  5. Eitan Altman, Alexandre Reiffers, Daniel S Menasche, Mandar Datar, Swapnil Dhamal, and Corinne Touati. 2019. Mining competition in a multi-cryptocurrency ecosystem at the network edge: A congestion game approach. ACM SIGMETRICS Performance Evaluation Review , Vol. 46, 3 (2019), 114--117.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Sanjeev Arora, Elad Hazan, and Satyen Kale. 2012. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing , Vol. 8, 1 (2012), 121--164.Google ScholarGoogle ScholarCross RefCross Ref
  7. Georgia Avarikioti, Eleftherios Kokoris-Kogias, and Roger Wattenhofer. 2019. Divide and scale: Formalization of distributed ledger sharding protocols. arXiv preprint arXiv:1910.10434 (2019).Google ScholarGoogle Scholar
  8. Christian Badertscher, Juan Garay, Ueli Maurer, Daniel Tschudi, and Vassilis Zikas. 2018a. But why does it work? A rational protocol design treatment of bitcoin. In Annual international conference on the theory and applications of cryptographic techniques . Springer, 34--65.Google ScholarGoogle ScholarCross RefCross Ref
  9. Christian Badertscher, Peter Gazi, Aggelos Kiayias, Alexander Russell, and Vassilis Zikas. 2018b. Ouroboros Genesis: Composable Proof-of-Stake Blockchains with Dynamic Availability. Cryptology ePrint Archive, Report 2018/378. https://eprint.iacr.org/2018/378.Google ScholarGoogle Scholar
  10. Vivek Bagaria, Sreeram Kannan, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. sf Prism: Deconstructing the blockchain to approach physical limits. ACM Computer and Communications Security Conference (2019).Google ScholarGoogle Scholar
  11. Mahesh Balakrishnan, Dahlia Malkhi, Vijayan Prabhakaran, Ted Wobbler, Michael Wei, and John D Davis. 2012. CORFU: A Shared Log Design for Flash Clusters. In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12) . 1--14.Google ScholarGoogle Scholar
  12. Mahesh Balakrishnan, Dahlia Malkhi, Ted Wobber, Ming Wu, Vijayan Prabhakaran, Michael Wei, John D Davis, Sriram Rao, Tao Zou, and Aviad Zuck. 2013. Tango: Distributed data structures over a shared log. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. 325--340.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Eli Ben-Sasson, Iddo Bentov, Yinon Horesh, and Michael Riabzev. 2018. Scalable, transparent, and post-quantum secure computational integrity. IACR Cryptology ePrint Archive , Vol. 2018 (2018), 46.Google ScholarGoogle Scholar
  14. Eli Ben-Sasson, Alessandro Chiesa, Eran Tromer, and Madars Virza. 2014. Succinct non-interactive zero knowledge for a von Neumann architecture. In 23rd USENIX Security Symposium (USENIX Security 14). 781--796.Google ScholarGoogle Scholar
  15. David Blackwell et almbox. 1956. An analog of the minimax theorem for vector payoffs. Pacific J. Math. , Vol. 6, 1 (1956), 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  16. Miguel Castro, Barbara Liskov, et almbox. 1999. Practical Byzantine fault tolerance. In OSDI, Vol. 99. 173--186.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Danny Dolev and H. Raymond Strong. 1983. Authenticated algorithms for Byzantine agreement. SIAM J. Comput. , Vol. 12, 4 (1983), 656--666.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Cynthia Dwork, Nancy Lynch, and Larry Stockmeyer. 1988. Consensus in the presence of partial synchrony. Journal of the ACM (JACM) , Vol. 35, 2 (1988), 288--323.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Juan Garay, Aggelos Kiayias, and Nikos Leonardos. 2015. The bitcoin backbone protocol: Analysis and applications. In Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer.Google ScholarGoogle ScholarCross RefCross Ref
  20. Juan Garay, Aggelos Kiayias, and Nikos Leonardos. 2020. Full Analysis of Nakamoto Consensus in Bounded-Delay Networks. Cryptology ePrint Archive, Report 2020/277. https://eprint.iacr.org/2020/277.Google ScholarGoogle Scholar
  21. Juan A. Garay, Aggelos Kiayias, and Nikos Leonardos. 2016. The Bitcoin Backbone Protocol with Chains of Variable Difficulty. Cryptology ePrint Archive, Report 2016/1048. https://eprint.iacr.org/2016/1048.Google ScholarGoogle Scholar
  22. Adem Efe Gencer, Robbert van Renesse, and Emin Gün Sirer. 2017. Short paper: Service-oriented sharding for blockchains. In International Conference on Financial Cryptography and Data Security. Springer, 393--401.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Dileep Kalathil, Vivek S Borkar, and Rahul Jain. 2017. Approachability in Stackelberg stochastic games with vector costs. Dynamic games and applications , Vol. 7, 3 (2017), 422--442.Google ScholarGoogle Scholar
  24. Harry Kalodner, Steven Goldfeder, Xiaoqi Chen, S Matthew Weinberg, and Edward W Felten. 2018. Arbitrum: Scalable, private smart contracts. In 27th $$USENIX$$ Security Symposium ($$USENIX$$ Security 18). 1353--1370.Google ScholarGoogle Scholar
  25. Aggelos Kiayias, Alexander Russell, Bernardo David, and Roman Oliynykov. 2017. Ouroboros: A provably secure proof-of-stake blockchain protocol. In Annual International Cryptology Conference. Springer, 357--388.Google ScholarGoogle ScholarCross RefCross Ref
  26. Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. 2018. Omniledger: A secure, scale-out, decentralized ledger via sharding. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 583--598.Google ScholarGoogle ScholarCross RefCross Ref
  27. Leslie Lamport. 1977. Proving the correctness of multiprocess programs. IEEE transactions on software engineering 2 (1977), 125--143.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Andrew Lewis-Pye and Tim Roughgarden. 2020. Resource Pools and the CAP Theorem. arxiv: 2006.10698 [cs.DC]Google ScholarGoogle Scholar
  29. Nikolaos Liakopoulos, Apostolos Destounis, Georgios Paschos, Thrasyvoulos Spyropoulos, and Panayotis Mertikopoulos. 2019. Cautious regret minimization: Online optimization with long-term budget constraints. In International Conference on Machine Learning . PMLR, 3944--3952.Google ScholarGoogle Scholar
  30. Loi Luu, Viswesh Narayanan, Chaodong Zheng, Kunal Baweja, Seth Gilbert, and Prateek Saxena. 2016. A secure sharding protocol for open blockchains. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 17--30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Loi Luu and Nate Rush. 2017. PeaceRelay Merkle-Patricia Trie Proof Verification.Google ScholarGoogle Scholar
  32. Shie Mannor, John N Tsitsiklis, and Jia Yuan Yu. 2009. Online Learning with Sample Path Constraints. Journal of Machine Learning Research , Vol. 10, 3 (2009).Google ScholarGoogle Scholar
  33. Satoshi Nakamoto. 2008. Bitcoin: A peer-to-peer electronic cash system. (2008).Google ScholarGoogle Scholar
  34. Bryan Parno, Jon Howell, Craig Gentry, and Mariana Raykova. 2013. Pinocchio: Nearly practical verifiable computation. In 2013 IEEE Symposium on Security and Privacy. IEEE, 238--252.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. R. Pass and E. Shi. 2017. Fruitchains: A fair blockchain. In Proceedings of the ACM Symposium on Principles of Distributed Computing. ACM.Google ScholarGoogle Scholar
  36. Ethereum Research. 2020. Ethereum 2.0. https://github.com/ethereum/eth2.0-specs .Google ScholarGoogle Scholar
  37. Fred B Schneider. 1993. Replication management using the state-machine approach, distributed systems. (1993).Google ScholarGoogle Scholar
  38. Nahum Shimkin. 2016. An online convex optimization approach to Blackwell's approachability. The Journal of Machine Learning Research , Vol. 17, 1 (2016), 4434--4456.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Alex Skidanov and Illia Polosukhin. 2019. Nightshade: Near protocol sharding design. (2019).Google ScholarGoogle Scholar
  40. Alberto Sonnino, Shehar Bano, Mustafa Al-Bassam, and George Danezis. 2019. Replay Attacks and Defenses Against Cross-shard Consensus in Sharded Distributed Ledgers. arXiv preprint arXiv:1901.11218 (2019).Google ScholarGoogle Scholar
  41. Rayadurgam Srikant and Lei Ying. 2013. Communication networks: an optimization, control, and stochastic networks perspective .Cambridge University Press.Google ScholarGoogle Scholar
  42. Leandros Tassiulas and Anthony Ephremides. 1990. Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks. In 29th IEEE Conference on Decision and Control. IEEE, 2130--2132.Google ScholarGoogle ScholarCross RefCross Ref
  43. Coin Telegraph. 2019. 10,000 Nodes Are Running BTC Lightning Network in New All-Time High. https://cointelegraph.com/news/10-000-nodes-are-running-btc-lightning-network-in-new-all-time-high .Google ScholarGoogle Scholar
  44. Jason Teutsch and Christian Reitwießner. 2019. A scalable verification solution for blockchains. arXiv preprint arXiv:1908.04756 (2019).Google ScholarGoogle Scholar
  45. John Von Neumann. 1953. A certain zero-sum two-person game equivalent to the optimal assignment problem. Contributions to the Theory of Games , Vol. 2, 0 (1953), 5--12.Google ScholarGoogle Scholar
  46. Heinrich von Stackelberg. 2011. Market Structure and Equilibrium .Springer.Google ScholarGoogle Scholar
  47. Jiaping Wang and Hao Wang. 2019. Monoxide: Scale out blockchains with asynchronous consensus zones. In 16th $$USENIX$$ Symposium on Networked Systems Design and Implementation ($$NSDI$$ 19) . 95--112.Google ScholarGoogle Scholar
  48. Gavin Wood. 2016. Polkadot: Vision for a heterogeneous multi-chain framework. (2016).Google ScholarGoogle Scholar
  49. Lei Yang, Vivek Bagaria, Gerui Wang, Mohammad Alizadeh, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. Prism: Scaling Bitcoin by 10,000 x. arXiv preprint arXiv:1909.11261 (2019).Google ScholarGoogle Scholar
  50. Hao Yu and Michael J Neely. 2016. A Low Complexity Algorithm with $O(backslashsqrt T)$ Regret and $O(1)$ Constraint Violations for Online Convex Optimization with Long Term Constraints. arXiv preprint arXiv:1604.02218 (2016).Google ScholarGoogle Scholar
  51. Hao Yu, Michael J Neely, and Xiaohan Wei. 2017. Online Convex Optimization with Stochastic Constraints. In 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.Google ScholarGoogle Scholar
  52. Mingchao Yu, Saeid Sahraei, Songze Li, Salman Avestimehr, Sreeram Kannan, and Pramod Viswanath. 2019. Coded Merkle Tree: Solving Data Availability Attacks in Blockchains. Cryptology ePrint Archive, Report 2019/1139. https://eprint.iacr.org/2019/1139.Google ScholarGoogle Scholar
  53. Mahdi Zamani, Mahnush Movahedi, and Mariana Raykova. 2018. Rapidchain: Scaling blockchain via full sharding. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Zilliqa. 2017. The ZILLIQA Technical Whitepaper. https://docs.zilliqa.com/whitepaper.pdf .Google ScholarGoogle Scholar

Index Terms

  1. Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image Proceedings of the ACM on Measurement and Analysis of Computing Systems
          Proceedings of the ACM on Measurement and Analysis of Computing Systems  Volume 6, Issue 1
          POMACS
          March 2022
          695 pages
          EISSN:2476-1249
          DOI:10.1145/3522731
          Issue’s Table of Contents

          Copyright © 2022 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 28 February 2022
          Published in pomacs Volume 6, Issue 1

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!