Minimizing coordination in replicated systems

ABSTRACT

Replication has been widely adopted to build highly scalable services, but this goal is often compromised by the coordination required to ensure application-specific properties such as state convergence and invariant preservation. In this paper, we propose a principled mechanism to minimize coordination in replicated systems via the following components: a) a notion of restriction over pairs of operations, which captures the fact that the two operations must be ordered w.r.t. each other in any partial order; b) a generic consistency model which, given a set of restrictions, requires those restrictions to be met in all admissible partial orders; c) principles for identifying a minimal set of restrictions to ensure the above properties; and d) a coordination service that dynamically maps restrictions to the most efficient coordination protocols. Our preliminary experience with example applications shows that we are able to determine a minimal coordination strategy.

References

1. Amazon Web Services Webpage. http://aws.amazon.com/. {accessed 20-August-2014}.
2. Facebook Webpage. https://www.facebook.com/. {accessed 20-August-2014}.
3. Google Webpage. www.google.com. {accessed 20-August-2014}.
4. P. Alvaro, N. Conway, J. M. Hellerstein, and W. R. Marczak. Consistency Analysis in Bloom: a CALM and Collected Approach. In Proceedings of the 5th Biennial Conference on Innovative Data Systems Research (CIDR'11), 2011.
5. P. Alvaro, N. Conway, J. M. Hellerstein, and D. Maier. Blazes: coordination analysis for distributed programs. In Proceedings of the IEEE 30th International Conference on Data Engineering (ICDE'14), 2014.
6. P. Bailis, A. Fekete, M. J. Franklin, A. Ghodsi, J. M. Hellerstein, and I. Stoica. Coordination-avoiding database systems. Proc. VLDB Endow., 2015.
7. V. Balegas, S. Duarte, C. Ferreira, N. Preguiça, R. Rodrigues, M. Najafzadeh, and M. Shapiro. Putting consistency back into eventual consistency. In Proceedings of the 10th European Conference on Computer Systems (EuroSys'15), 2015.
8. P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. 1987.
9. S. Burckhardt, A. Gotsman, and H. Yang. Understanding eventual consistency. Technical report, Microsoft Research, 2013.
10. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: amazon's highly available key-value store. In Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles (SOSP'07), 2007.
11. M. P. Herlihy and J. M. Wing. Linearizability: a correctness condition for concurrent objects. ACM Transactions on Programming Languages and Systems, 1990.
12. P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. Zookeeper: Wait-free coordination for internet-scale systems. In Proceedings of the 2010 USENIX Annual Technical Conference (USENIX ATC'10), 2010.
13. R. Ladin, B. Liskov, L. Shrira, and S. Ghemawat. Providing high availability using lazy replication. ACM Transactions on Computer Systems (TOCS), 1992.
14. L. Lamport. The part-time parliament. ACM Transactions on Computer Systems (TOCS), 1998.
15. L. Lamport. Generalized consensus and paxos. Technical Report MSR-TR-2005-33, Microsoft Research, 2005.
16. C. Li, D. Porto, A. Clement, J. Gehrke, N. Preguiça, and R. Rodrigues. Making Geo-Replicated Systems Fast as Possible, Consistent when Necessary. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI'12), 2012.
17. C. Li, J. Leitão, A. Clement, N. Preguiça, R. Rodrigues, and V. Vafeiadis. Automating the choice of consistency levels in replicated systems. In Proceedings of the 2014 USENIX Annual Technical Conference (USENIX ATC'14), 2014.
18. W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Don't Settle for Eventual: Scalable Causal Consistency for Wide-Area Storage with COPS. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP'11), 2011.
19. F. Pedone and A. Schiper. Generic broadcast. In Proceedings of the 13th International Symposium on Distributed Computing (DISC'99), 1999.
20. M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. A comprehensive study of convergent and commutative replicated data types. Technical report, INRIA, 2011.
21. A. Singh, P. Fonseca, P. Kuznetsov, R. Rodrigues, and P. Maniatis. Zeno: Eventually Consistent Byzantine-Fault Tolerance. In Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation (NSDI'09), 2009.
22. Y. Sovran, R. Power, M. K. Aguilera, and J. Li. Transactional storage for geo-replicated systems. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP'11), 2011.

Index Terms

• Computer systems organization

  • Architectures

    • Distributed architectures

• Software and its engineering

  • Software organization and properties

    • Software system structures

      • Distributed systems organizing principles

• Theory of computation

  • Models of computation

    • Concurrency

      • Parallel computing models