Abstract
In this paper, we study a canonical distributed resource allocation problem arising in blockchains. While distributed resource allocation is a well-studied problem in networking, the blockchain setting additionally requires the solution to be resilient to adversarial behavior from a fraction of nodes. Scaling blockchain performance is a basic research topic; a plethora of solutions (under the umbrella of sharding ) have been proposed in recent years. Although the various sharding solutions share a common thread (they cryptographically stitch together multiple parallel chains), architectural differences lead to differing resource allocation problems. In this paper we make three main contributions: (a) we categorize the different sharding proposals under a common architectural framework, allowing for the emergence of a new, uniformly improved, uni-consensus sharding architecture. (b) We formulate and exactly solve a core resource allocation problem in the uni-consensus sharding architecture -- our solution, Free2shard, is adversary-resistant and achieves optimal throughput. The key technical contribution is a mathematical connection to the classical work of Blackwell approachability in dynamic game theory. (c) We implement the sharding architecture atop a full-stack blockchain in 3000 lines of code in Rust -- we achieve a throughput of more than 250,000 transactions per second with 6 shards, a vast improvement over state-of-the-art.
- Jacob Abernethy, Peter L Bartlett, and Elad Hazan. 2011. Blackwell approachability and no-regret learning are equivalent. In Proceedings of the 24th Annual Conference on Learning Theory. JMLR Workshop and Conference Proceedings, 27--46.Google Scholar
- Ittai Abraham, TH Hubert Chan, Danny Dolev, Kartik Nayak, Rafael Pass, Ling Ren, and Elaine Shi. 2019. Communication complexity of byzantine agreement, revisited. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing . 317--326.Google Scholar
Digital Library
- Mustafa Al-Bassam. 2019. LazyLedger: A Distributed Data Availability Ledger With Client-Side Smart Contracts. arxiv: 1905.09274 [cs.CR]Google Scholar
- Bowen Alpern and Fred B Schneider. 1987. Recognizing safety and liveness. Distributed computing , Vol. 2, 3 (1987), 117--126.Google Scholar
- Eitan Altman, Alexandre Reiffers, Daniel S Menasche, Mandar Datar, Swapnil Dhamal, and Corinne Touati. 2019. Mining competition in a multi-cryptocurrency ecosystem at the network edge: A congestion game approach. ACM SIGMETRICS Performance Evaluation Review , Vol. 46, 3 (2019), 114--117.Google Scholar
Digital Library
- Sanjeev Arora, Elad Hazan, and Satyen Kale. 2012. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing , Vol. 8, 1 (2012), 121--164.Google Scholar
Cross Ref
- Georgia Avarikioti, Eleftherios Kokoris-Kogias, and Roger Wattenhofer. 2019. Divide and scale: Formalization of distributed ledger sharding protocols. arXiv preprint arXiv:1910.10434 (2019).Google Scholar
- Christian Badertscher, Juan Garay, Ueli Maurer, Daniel Tschudi, and Vassilis Zikas. 2018a. But why does it work? A rational protocol design treatment of bitcoin. In Annual international conference on the theory and applications of cryptographic techniques . Springer, 34--65.Google Scholar
Cross Ref
- Christian Badertscher, Peter Gazi, Aggelos Kiayias, Alexander Russell, and Vassilis Zikas. 2018b. Ouroboros Genesis: Composable Proof-of-Stake Blockchains with Dynamic Availability. Cryptology ePrint Archive, Report 2018/378. https://eprint.iacr.org/2018/378.Google Scholar
- Vivek Bagaria, Sreeram Kannan, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. sf Prism: Deconstructing the blockchain to approach physical limits. ACM Computer and Communications Security Conference (2019).Google Scholar
- Mahesh Balakrishnan, Dahlia Malkhi, Vijayan Prabhakaran, Ted Wobbler, Michael Wei, and John D Davis. 2012. CORFU: A Shared Log Design for Flash Clusters. In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12) . 1--14.Google Scholar
- Mahesh Balakrishnan, Dahlia Malkhi, Ted Wobber, Ming Wu, Vijayan Prabhakaran, Michael Wei, John D Davis, Sriram Rao, Tao Zou, and Aviad Zuck. 2013. Tango: Distributed data structures over a shared log. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. 325--340.Google Scholar
Digital Library
- Eli Ben-Sasson, Iddo Bentov, Yinon Horesh, and Michael Riabzev. 2018. Scalable, transparent, and post-quantum secure computational integrity. IACR Cryptology ePrint Archive , Vol. 2018 (2018), 46.Google Scholar
- Eli Ben-Sasson, Alessandro Chiesa, Eran Tromer, and Madars Virza. 2014. Succinct non-interactive zero knowledge for a von Neumann architecture. In 23rd USENIX Security Symposium (USENIX Security 14). 781--796.Google Scholar
- David Blackwell et almbox. 1956. An analog of the minimax theorem for vector payoffs. Pacific J. Math. , Vol. 6, 1 (1956), 1--8.Google Scholar
Cross Ref
- Miguel Castro, Barbara Liskov, et almbox. 1999. Practical Byzantine fault tolerance. In OSDI, Vol. 99. 173--186.Google Scholar
Digital Library
- Danny Dolev and H. Raymond Strong. 1983. Authenticated algorithms for Byzantine agreement. SIAM J. Comput. , Vol. 12, 4 (1983), 656--666.Google Scholar
Digital Library
- Cynthia Dwork, Nancy Lynch, and Larry Stockmeyer. 1988. Consensus in the presence of partial synchrony. Journal of the ACM (JACM) , Vol. 35, 2 (1988), 288--323.Google Scholar
Digital Library
- Juan Garay, Aggelos Kiayias, and Nikos Leonardos. 2015. The bitcoin backbone protocol: Analysis and applications. In Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer.Google Scholar
Cross Ref
- Juan Garay, Aggelos Kiayias, and Nikos Leonardos. 2020. Full Analysis of Nakamoto Consensus in Bounded-Delay Networks. Cryptology ePrint Archive, Report 2020/277. https://eprint.iacr.org/2020/277.Google Scholar
- Juan A. Garay, Aggelos Kiayias, and Nikos Leonardos. 2016. The Bitcoin Backbone Protocol with Chains of Variable Difficulty. Cryptology ePrint Archive, Report 2016/1048. https://eprint.iacr.org/2016/1048.Google Scholar
- Adem Efe Gencer, Robbert van Renesse, and Emin Gün Sirer. 2017. Short paper: Service-oriented sharding for blockchains. In International Conference on Financial Cryptography and Data Security. Springer, 393--401.Google Scholar
Digital Library
- Dileep Kalathil, Vivek S Borkar, and Rahul Jain. 2017. Approachability in Stackelberg stochastic games with vector costs. Dynamic games and applications , Vol. 7, 3 (2017), 422--442.Google Scholar
- Harry Kalodner, Steven Goldfeder, Xiaoqi Chen, S Matthew Weinberg, and Edward W Felten. 2018. Arbitrum: Scalable, private smart contracts. In 27th $$USENIX$$ Security Symposium ($$USENIX$$ Security 18). 1353--1370.Google Scholar
- Aggelos Kiayias, Alexander Russell, Bernardo David, and Roman Oliynykov. 2017. Ouroboros: A provably secure proof-of-stake blockchain protocol. In Annual International Cryptology Conference. Springer, 357--388.Google Scholar
Cross Ref
- Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. 2018. Omniledger: A secure, scale-out, decentralized ledger via sharding. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 583--598.Google Scholar
Cross Ref
- Leslie Lamport. 1977. Proving the correctness of multiprocess programs. IEEE transactions on software engineering 2 (1977), 125--143.Google Scholar
Digital Library
- Andrew Lewis-Pye and Tim Roughgarden. 2020. Resource Pools and the CAP Theorem. arxiv: 2006.10698 [cs.DC]Google Scholar
- Nikolaos Liakopoulos, Apostolos Destounis, Georgios Paschos, Thrasyvoulos Spyropoulos, and Panayotis Mertikopoulos. 2019. Cautious regret minimization: Online optimization with long-term budget constraints. In International Conference on Machine Learning . PMLR, 3944--3952.Google Scholar
- Loi Luu, Viswesh Narayanan, Chaodong Zheng, Kunal Baweja, Seth Gilbert, and Prateek Saxena. 2016. A secure sharding protocol for open blockchains. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 17--30.Google Scholar
Digital Library
- Loi Luu and Nate Rush. 2017. PeaceRelay Merkle-Patricia Trie Proof Verification.Google Scholar
- Shie Mannor, John N Tsitsiklis, and Jia Yuan Yu. 2009. Online Learning with Sample Path Constraints. Journal of Machine Learning Research , Vol. 10, 3 (2009).Google Scholar
- Satoshi Nakamoto. 2008. Bitcoin: A peer-to-peer electronic cash system. (2008).Google Scholar
- Bryan Parno, Jon Howell, Craig Gentry, and Mariana Raykova. 2013. Pinocchio: Nearly practical verifiable computation. In 2013 IEEE Symposium on Security and Privacy. IEEE, 238--252.Google Scholar
Digital Library
- R. Pass and E. Shi. 2017. Fruitchains: A fair blockchain. In Proceedings of the ACM Symposium on Principles of Distributed Computing. ACM.Google Scholar
- Ethereum Research. 2020. Ethereum 2.0. https://github.com/ethereum/eth2.0-specs .Google Scholar
- Fred B Schneider. 1993. Replication management using the state-machine approach, distributed systems. (1993).Google Scholar
- Nahum Shimkin. 2016. An online convex optimization approach to Blackwell's approachability. The Journal of Machine Learning Research , Vol. 17, 1 (2016), 4434--4456.Google Scholar
Digital Library
- Alex Skidanov and Illia Polosukhin. 2019. Nightshade: Near protocol sharding design. (2019).Google Scholar
- Alberto Sonnino, Shehar Bano, Mustafa Al-Bassam, and George Danezis. 2019. Replay Attacks and Defenses Against Cross-shard Consensus in Sharded Distributed Ledgers. arXiv preprint arXiv:1901.11218 (2019).Google Scholar
- Rayadurgam Srikant and Lei Ying. 2013. Communication networks: an optimization, control, and stochastic networks perspective .Cambridge University Press.Google Scholar
- Leandros Tassiulas and Anthony Ephremides. 1990. Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks. In 29th IEEE Conference on Decision and Control. IEEE, 2130--2132.Google Scholar
Cross Ref
- Coin Telegraph. 2019. 10,000 Nodes Are Running BTC Lightning Network in New All-Time High. https://cointelegraph.com/news/10-000-nodes-are-running-btc-lightning-network-in-new-all-time-high .Google Scholar
- Jason Teutsch and Christian Reitwießner. 2019. A scalable verification solution for blockchains. arXiv preprint arXiv:1908.04756 (2019).Google Scholar
- John Von Neumann. 1953. A certain zero-sum two-person game equivalent to the optimal assignment problem. Contributions to the Theory of Games , Vol. 2, 0 (1953), 5--12.Google Scholar
- Heinrich von Stackelberg. 2011. Market Structure and Equilibrium .Springer.Google Scholar
- Jiaping Wang and Hao Wang. 2019. Monoxide: Scale out blockchains with asynchronous consensus zones. In 16th $$USENIX$$ Symposium on Networked Systems Design and Implementation ($$NSDI$$ 19) . 95--112.Google Scholar
- Gavin Wood. 2016. Polkadot: Vision for a heterogeneous multi-chain framework. (2016).Google Scholar
- Lei Yang, Vivek Bagaria, Gerui Wang, Mohammad Alizadeh, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. Prism: Scaling Bitcoin by 10,000 x. arXiv preprint arXiv:1909.11261 (2019).Google Scholar
- Hao Yu and Michael J Neely. 2016. A Low Complexity Algorithm with $O(backslashsqrt T)$ Regret and $O(1)$ Constraint Violations for Online Convex Optimization with Long Term Constraints. arXiv preprint arXiv:1604.02218 (2016).Google Scholar
- Hao Yu, Michael J Neely, and Xiaohan Wei. 2017. Online Convex Optimization with Stochastic Constraints. In 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.Google Scholar
- Mingchao Yu, Saeid Sahraei, Songze Li, Salman Avestimehr, Sreeram Kannan, and Pramod Viswanath. 2019. Coded Merkle Tree: Solving Data Availability Attacks in Blockchains. Cryptology ePrint Archive, Report 2019/1139. https://eprint.iacr.org/2019/1139.Google Scholar
- Mahdi Zamani, Mahnush Movahedi, and Mariana Raykova. 2018. Rapidchain: Scaling blockchain via full sharding. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM.Google Scholar
Digital Library
- Zilliqa. 2017. The ZILLIQA Technical Whitepaper. https://docs.zilliqa.com/whitepaper.pdf .Google Scholar
Index Terms
Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains
Recommendations
SoK: Sharding on Blockchain
AFT '19: Proceedings of the 1st ACM Conference on Advances in Financial TechnologiesBlockchain is a distributed and decentralized ledger for recording transactions. It is maintained and shared among the participating nodes by utilizing cryptographic primitives. A consensus protocol ensures that all nodes agree on a unique order in ...
Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains
SIGMETRICS '22In this paper, we formulate and study a new, but basic, distributed resource allocation problem arising in scaling blockchain performance. While distributed resource allocation is a well-studied problem in networking, the blockchain setting additionally ...
Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains
SIGMETRICS/PERFORMANCE '22: Abstract Proceedings of the 2022 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer SystemsIn this paper, we formulate and study a new, but basic, distributed resource allocation problem arising in scaling blockchain performance. While distributed resource allocation is a well-studied problem in networking, the blockchain setting additionally ...






Comments