Scalaris: reliable transactional p2p key/value store

ABSTRACT

We present Scalaris, an Erlang implementation of a distributed key/value store. It uses, on top of a structured overlay network, replication for data availability and majority based distributed transactions for data consistency. In combination, this implements the ACID properties on a scalable structured overlay.

By directly mapping the keys to the overlay without hashing, arbitrary key-ranges can be assigned to nodes, thereby allowing a better load-balancing than would be possible with traditional DHTs. Consequently, Scalaris can be tuned for fast data access by taking, e.g. the nodes' geographic location or the regional popularity of certain keys into account. This improves Scalaris' lookup speed in datacenter or cloud computing environments.

Scalaris is implemented in Erlang. We describe the Erlang software architecture, including the transactional Java interface to access Scalaris.

Additionally, we present a generic design pattern to implement a responsive server in Erlang that serializes update operations on a common state, while concurrently performing fast asynchronous read requests on the same state.

As a proof-of-concept we implemented a simplified Wikipedia frontend and attached it to the Scalaris data store backend. Wikipedia is a challenging application. It requires - besides thousands of concurrent read requests per seconds - serialized, consistent write operations. For Wikipedia's category and backlink pages, keys must be consistently changed within transactions. We discuss how these features are implemented in Scalaris and show its performance.

References

1. J. Armstrong. Programming Erlang: Software for a Concurrent World. Pragmatic Programmers, ISBN: 978-1-9343560-0-5, July 2007
2. R. Baldoni, L. Querzoni, A. Virgillito, R. Jiménez-Peris, and M. Patiño-Martínez. Dynamic Quorums for DHT-based P2P Networks. NCA, pp. 91--100, 2005.
3. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon's Highly Available Key-Value Store Proceedings of the 21st ACM Symposium on Operating Systems Principles, Oct. 2007.
4. JJ Furman, J. S. Karlsson, J. Leon, A. Lloyd, S. Newman, and P. Zeyliger. Megastore: A Scalable Data System for User Facing Applications. SIGMOD 2008, Jun. 2008.
5. A. Ghodsi, L. O. Alima, and S. Haridi. Symmetric Replication for Structured Peer-to-Peer Systems. 3rd Intl. Workshop on Databases, Information Systems and P2P Computing, 2005.
6. R. Guerraoui and L. Rodrigues. Introduction to Reliable Distributed Programming. Springer-Verlag 2006.
7. C. Hewitt, P. Bishop, and R. Steiger. A Universal Modular ACTOR Formalism for Artificial Intelligence. IJCAI, 1973.
8. A. Lakshman, P. Malik, and K. Ranganathan. Cassandra: A Structured Storage System on a P2P Network. SIGMOD 2008, Jun. 2008.
9. L. Lamport. Fast Paxos. Distributed Computing 19(2):79--103, 2006.
10. M. M. Masud and I. Kiringa. Maintaining consistency in a failure-prone P2P database network during transaction processing. Proceedings of the 2008 International Workshop on Data management in peer-to-peer systems, pp. 27--34, 2008.
11. M. Moser and S. Haridi. Atomic Commitment in Transactional DHTs. 1st CoreGRID Symposium, Aug. 2007.
12. S. Plantikow, A. Reinefeld, and F. Schintke. Transactions for Distributed Wikis on Structured Overlays. 18th IFIP/IEEE Distributed Systems: Operations and Management (DSOM 2007), Oct. 2007.
13. T. Schütt, F. Schintke, and A. Reinefeld. Structured Overlay without Consistent Hashing: Empirical Results. GP2PC'06, May 2006.
14. T. Schütt, F. Schintke, and A. Reinefeld. A Structured Overlay for Multi-Dimensional Range Queries. Europar, Aug. 2007.
15. T. Schütt, F. Schintke, and A. Reinefeld. Scalable Wikipedia with Erlang. Google Scalability Conference, Jun. 2008.
16. T. M. Shafaat, M. Moser, A. Ghodsi, S. Haridi, T. Schütt, and A. Reinefeld. Key-Based Consistency and Availability in Structured Overlay Networks. Third Intl. ICST Conference on Scalable Information Systems, June 2008.
17. I. Stoica, R. Morris, M. F. Kaashoek D. Karger, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for Internet application. ACM SIGCOMM 2001, Aug. 2001.

Index Terms

• Computer systems organization

  • Architectures

    • Distributed architectures

• Information systems

  • Data management systems

    • Database management system engines

      • Parallel and distributed DBMSs

  • Information storage systems

    • Storage architectures

      • Distributed storage

• Software and its engineering

  • Software organization and properties

    • Software system structures

      • Distributed systems organizing principles

        • Organizing principles for web applications

      • Software architectures