Lasp: a language for distributed, eventually consistent computations with CRDTs

ABSTRACT

We propose Lasp, a novel programming model aimed to simplify correct, large-scale, distributed programming. Lasp leverages ideas from distributed dataflow programming extended with convergent data types. This provides support for computations where not all participants are online together at a given moment through Lasp's "convergent by design" applications. Lasp provides a familiar functional programming semantics, built on top of distributed systems infrastructure, targeted at the Erlang runtime system.

The initial Lasp design presented in this report supports synchronization free programming using convergent data types. It combines the expressiveness of these data types together with powerful primitives for composing them. This design lets us write long-lived fault-tolerant distributed applications with non-monotonic behavior. We show how to implement one nontrivial large-scale application, the ad counter scenario from the SyncFree project.

References

1. Lasp source code repository. https://github.com/cmeiklejohn/lasp. Accessed: 2015-02-14.
2. 263 Million Monthly Active Users In December. http://www.rovio.com/en/news/blog/261/263-million-monthly-active-users-in-december/. Accessed: 2015-02-13.
3. SyncFree: Large-scale computation without synchronisation. https://syncfree.lip6.fr. Accessed: 2015-02-13.
4. P. Alvaro, P. Bailis, N. Conway, and J. M. Hellerstein. Consistency without borders. In Proceedings of the 4th annual Symposium on Cloud Computing, page 23. ACM, 2013.
5. M. Bravo, Z. Li, P. Van Roy, and C. Meiklejohn. Derflow: distributed deterministic dataflow programming for Erlang. In Proceedings of the Thirteenth ACM SIGPLAN workshop on Erlang, pages 51--60. ACM, 2014.
6. R. Brown, S. Cribbs, C. Meiklejohn, and S. Elliott. Riak DT map: a composable, convergent replicated dictionary. In Proceedings of the First Workshop on Principles and Practice of Eventual Consistency, page 1. ACM, 2014.
7. T. D. Chandra, R. Griesemer, and J. Redstone. Paxos made live: an engineering perspective. In Proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing, pages 398--407. ACM, 2007.
8. N. Conway, W. R. Marczak, P. Alvaro, J. M. Hellerstein, and D. Maier. Logic and lattices for distributed programming. In Proceedings of the Third ACM Symposium on Cloud Computing, page 1. ACM, 2012.
9. R. Klophaus. Riak core: building distributed applications without shared state. In ACM SIGPLAN Commercial Users of Functional Programming, page 14. ACM, 2010.
10. C. Meiklejohn. On the composability of the Riak DT map: expanding from embedded to multi-key structures. In Proceedings of the First Workshop on Principles and Practice of Eventual Consistency, page 13. ACM, 2014.
11. C. Meiklejohn. Eventual Consistency and Deterministic Dataflow Programming. 8th Workshop on Large-Scale Distributed Systems and Middleware, 2014.
12. D. Miorandi, S. Sicari, F. De Pellegrini, and I. Chlamtac. Internet of things: Vision, applications and research challenges. Ad Hoc Networks, 10(7): 1497--1516, 2012.
13. M. Shapiro, N. Preguiça, C. Baquero, M. Zawirski, et al. A comprehensive study of convergent and commutative replicated data types. 2011.

Index Terms

• Computing methodologies

  • Concurrent computing methodologies

    • Concurrent programming languages

• Information systems

  • Information storage systems

    • Storage architectures

      • Distributed storage

• Software and its engineering

  • Software notations and tools

    • General programming languages

      • Language types

        • Concurrent programming languages