Abstract
In distributed applications, the transmission of non-contiguous data structures is greatly slowed down by the need to serialize them into a buffer before sending. We describe Compact Normal Forms, an API that allows programmers to explicitly place immutable heap objects into regions, which can both be accessed like ordinary data as well as efficiently transmitted over the network. The process of placing objects into compact regions (essentially a copy) is faster than any serializer and can be amortized over a series of functional updates to the data structure in question. We implement this scheme in the Glasgow Haskell Compiler and show that even with the space expansion attendant with memory-oriented data structure representations, we achieve between x2 and x4 speedups on fast local networks with sufficiently large data structures.
- E. D. Berger, B. G. Zorn, and K. S. McKinley. Reconsidering custom memory allocation. ACM SIGPLAN Notices, 37:1, 2002. Google Scholar
Digital Library
- R. L. Bocchino Jr, V. S. Adve, D. Dig, S. V. Adve, S. Heumann, R. Komuravelli, J. Overbey, P. Simmons, H. Sung, and M. Vakilian. A type and effect system for deterministic parallel Java. ACM Sigplan Notices, 44(10):97–116, 2009. Google Scholar
Digital Library
- S. Breitinger, U. Klusik, and R. Loogen. From (sequential) Haskell to (parallel) Eden: An Implementation Point of View. Symposium on Programming Language Implementation and Logic Programming - PLILP, pages 318–334, 1998. Google Scholar
Digital Library
- L. Cardelli. The Functional Abstract Machine. Technical Report TR- 107, AT&T Bell Laboratories, 1983.Google Scholar
- J. Chase, F. Amador, E. Lazowska, H. Levy, and R. Littlefield. The Amber system: parallel programming on a network of multiprocessors. ACM SIGOPS Operating Systems Review, 23(December 1989):147–158, 1989. Google Scholar
Digital Library
- J. S. Chase, H. M. Levy, E. D. Lazowska, and M. Baker-Harvey. Lightweight shared objects in a 64-bit operating system. In ACM SIGPLAN Notices, volume 27, pages 397–413, 1992. Google Scholar
Digital Library
- S. Chaumette, P. Grange, B. Métrot, and P. Vignéras. Implementing a High Performance Object Transfer Mechanism over JikesRVM. 2004.Google Scholar
- T. M. Chilimbi and J. R. Larus. Using generational garbage collection to implement cache-conscious data placement. In ACM SIGPLAN Notices, volume 34, pages 37–48, 1999. Google Scholar
Digital Library
- L. Courtrai, Y. Maheo, and F. Raimbault. Expresso: a Library for Fast Java Objects Transfer. In Myrinet User Group Conference (MUG), 2000.Google Scholar
- J. Epstein, A. P. Black, and S. Peyton-Jones. Towards Haskell in the cloud. ACM SIGPLAN Notices, 46(Section 4):118, 2012. Google Scholar
Digital Library
- T. D. S. Gene Novark. Custom Object Layout for Garbage-Collected Languages. Techreport, 2006.Google Scholar
- D. Grossman, G. Morrisett, T. Jim, M. Hicks, Y. Wang, and J. Cheney. Region-based memory management in Cyclone. In PLDI, 2002. Google Scholar
Digital Library
- N. Hallenberg, M. Elsman, and M. Tofte. Combining region inference and garbage collection. In PLDI, volume 37, page 141, May 2002. Google Scholar
Digital Library
- G. C. Hunt and J. R. Larus. Singularity: rethinking the software stack. SIGOPS OSR, 2007. Google Scholar
Digital Library
- S. L. P. Jones. Implementing lazy functional languages on stock hardware: the Spineless Tagless G-machine. Journal of Functional Programming, 2:127–202, 1992.Google Scholar
Cross Ref
- A. Kemper and D. Kossmann. Adaptable pointer swizzling strategies in object bases: design, realization, and quantitative analysis. The VLDB Journal, 4:519–566, 1995. Google Scholar
Digital Library
- R. Lämmel and S. P. Jones. Scrap your boilerplate with class: Extensible generic functions. SIGPLAN Not., 40(9):204–215, Sept. 2005. Google Scholar
Digital Library
- J. Launchbury. A natural semantics for lazy evaluation. In Proceedings of the 20th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pages 144–154. ACM Press, 1993. Google Scholar
Digital Library
- R. Loogen, Y. Ortega-mallén, and R. Pe˜na mar´ı. Parallel functional programming in Eden. Journal of Functional Programming, 15(3):431–475, May 2005. Google Scholar
Digital Library
- P. Maier and P. Trinder. Implementing a high-level distributedmemory parallel haskell in haskell. In Implementation and Application of Functional Languages, pages 35–50. Springer, 2012. Google Scholar
Digital Library
- S. Marlow, T. Harris, R. P. James, and S. Peyton Jones. Parallel generational-copying garbage collection with a block-structured heap. In Proceedings of the 7th international symposium on Memory management, ISMM ’08, pages 11–20, New York, NY, USA, 2008. ACM. Google Scholar
Digital Library
- K. McKelvey and F. Menczer. Design and prototyping of a social media observatory. In Proceedings of the 22nd international conference on World Wide Web companion, WWW ’13 Companion, pages 1351– 1358, 2013. Google Scholar
Digital Library
- K. McKelvey and F. Menczer. Truthy: Enabling the Study of Online Social Networks. In Proc. 16th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (CSCW), 2013. Google Scholar
Digital Library
- Microsoft Corporation. Word (. doc) Binary File Format. https://msdn.microsoft.com/en-us/library/office/ cc313153(v=office.12).aspx, 2014.Google Scholar
- Microsoft Corporation. Bitmap Storage. https://msdn. microsoft.com/en-us/library/dd183391.aspx, 2015.Google Scholar
- H. Miller, P. Haller, E. Burmako, and M. Odersky. Instant pickles: Generating object-oriented pickler combinators for fast and extensible serialization. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA ’13, pages 183–202, New York, NY, USA, 2013. ACM. Google Scholar
Digital Library
- C. Nester, M. Philippsen, and B. Haumacher. A more efficient RMI for Java. In Proceedings of the ACM 1999 conference on Java Grande, pages 152–159. ACM, 1999. Google Scholar
Digital Library
- E. Shekita and M. Zwilling. Cricket: A Mapped, Persistent Object Store. Center for Parallel Optimization, Computer Sciences Department, University of Wisconsin, 1996.Google Scholar
- TIS Committee. Tool Interface Standard (TIS) Executable and Linking Format (ELF) Specification (Version 1.2). Technical Report May, 1995.Google Scholar
- M. Tofte, L. Birkedal, M. Elsman, and N. Hallenberg. A retrospective on region-based memory management. Higher-Order and Symbolic Computation, 17(3):245–265, 2004. Google Scholar
Digital Library
- K. Varda. Cap’n Proto. https://capnproto.org/, 2015.Google Scholar
- P. Wilson and S. Kakkad. Pointer swizzling at page fault time: efficiently and compatibly supporting huge address spaces on standard hardware. {1992} Proceedings of the Second International Workshop on Object Orientation in Operating Systems, 1992.Google Scholar
Index Terms
Efficient communication and collection with compact normal forms
Recommendations
Efficient communication and collection with compact normal forms
ICFP 2015: Proceedings of the 20th ACM SIGPLAN International Conference on Functional ProgrammingIn distributed applications, the transmission of non-contiguous data structures is greatly slowed down by the need to serialize them into a buffer before sending. We describe Compact Normal Forms, an API that allows programmers to explicitly place ...
Efficient Layering for High Speed Communication: Fast Messages 2.x
HPDC '98: Proceedings of the 7th IEEE International Symposium on High Performance Distributed ComputingWe describe our experience designing, implementing, and evaluating two generations of our high performance communication library, Fast Messages (FM) for Myrinet. In FM 1.x, we designed a simple interface and provided guarantees of reliable and in-order ...
Instant pickles: generating object-oriented pickler combinators for fast and extensible serialization
OOPSLA '13As more applications migrate to the cloud, and as "big data" edges into even more production environments, the performance and simplicity of exchanging data between compute nodes/devices is increasing in importance. An issue central to distributed ...






Comments