skip to main content
research-article

Efficient communication and collection with compact normal forms

Published:29 August 2015Publication History
Skip Abstract Section

Abstract

In distributed applications, the transmission of non-contiguous data structures is greatly slowed down by the need to serialize them into a buffer before sending. We describe Compact Normal Forms, an API that allows programmers to explicitly place immutable heap objects into regions, which can both be accessed like ordinary data as well as efficiently transmitted over the network. The process of placing objects into compact regions (essentially a copy) is faster than any serializer and can be amortized over a series of functional updates to the data structure in question. We implement this scheme in the Glasgow Haskell Compiler and show that even with the space expansion attendant with memory-oriented data structure representations, we achieve between x2 and x4 speedups on fast local networks with sufficiently large data structures.

References

  1. E. D. Berger, B. G. Zorn, and K. S. McKinley. Reconsidering custom memory allocation. ACM SIGPLAN Notices, 37:1, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. L. Bocchino Jr, V. S. Adve, D. Dig, S. V. Adve, S. Heumann, R. Komuravelli, J. Overbey, P. Simmons, H. Sung, and M. Vakilian. A type and effect system for deterministic parallel Java. ACM Sigplan Notices, 44(10):97–116, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. S. Breitinger, U. Klusik, and R. Loogen. From (sequential) Haskell to (parallel) Eden: An Implementation Point of View. Symposium on Programming Language Implementation and Logic Programming - PLILP, pages 318–334, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. L. Cardelli. The Functional Abstract Machine. Technical Report TR- 107, AT&T Bell Laboratories, 1983.Google ScholarGoogle Scholar
  5. J. Chase, F. Amador, E. Lazowska, H. Levy, and R. Littlefield. The Amber system: parallel programming on a network of multiprocessors. ACM SIGOPS Operating Systems Review, 23(December 1989):147–158, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. J. S. Chase, H. M. Levy, E. D. Lazowska, and M. Baker-Harvey. Lightweight shared objects in a 64-bit operating system. In ACM SIGPLAN Notices, volume 27, pages 397–413, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. S. Chaumette, P. Grange, B. Métrot, and P. Vignéras. Implementing a High Performance Object Transfer Mechanism over JikesRVM. 2004.Google ScholarGoogle Scholar
  8. T. M. Chilimbi and J. R. Larus. Using generational garbage collection to implement cache-conscious data placement. In ACM SIGPLAN Notices, volume 34, pages 37–48, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. L. Courtrai, Y. Maheo, and F. Raimbault. Expresso: a Library for Fast Java Objects Transfer. In Myrinet User Group Conference (MUG), 2000.Google ScholarGoogle Scholar
  10. J. Epstein, A. P. Black, and S. Peyton-Jones. Towards Haskell in the cloud. ACM SIGPLAN Notices, 46(Section 4):118, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. T. D. S. Gene Novark. Custom Object Layout for Garbage-Collected Languages. Techreport, 2006.Google ScholarGoogle Scholar
  12. D. Grossman, G. Morrisett, T. Jim, M. Hicks, Y. Wang, and J. Cheney. Region-based memory management in Cyclone. In PLDI, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. N. Hallenberg, M. Elsman, and M. Tofte. Combining region inference and garbage collection. In PLDI, volume 37, page 141, May 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. G. C. Hunt and J. R. Larus. Singularity: rethinking the software stack. SIGOPS OSR, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. S. L. P. Jones. Implementing lazy functional languages on stock hardware: the Spineless Tagless G-machine. Journal of Functional Programming, 2:127–202, 1992.Google ScholarGoogle ScholarCross RefCross Ref
  16. A. Kemper and D. Kossmann. Adaptable pointer swizzling strategies in object bases: design, realization, and quantitative analysis. The VLDB Journal, 4:519–566, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. R. Lämmel and S. P. Jones. Scrap your boilerplate with class: Extensible generic functions. SIGPLAN Not., 40(9):204–215, Sept. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. Launchbury. A natural semantics for lazy evaluation. In Proceedings of the 20th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pages 144–154. ACM Press, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. R. Loogen, Y. Ortega-mallén, and R. Pe˜na mar´ı. Parallel functional programming in Eden. Journal of Functional Programming, 15(3):431–475, May 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. P. Maier and P. Trinder. Implementing a high-level distributedmemory parallel haskell in haskell. In Implementation and Application of Functional Languages, pages 35–50. Springer, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. S. Marlow, T. Harris, R. P. James, and S. Peyton Jones. Parallel generational-copying garbage collection with a block-structured heap. In Proceedings of the 7th international symposium on Memory management, ISMM ’08, pages 11–20, New York, NY, USA, 2008. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. K. McKelvey and F. Menczer. Design and prototyping of a social media observatory. In Proceedings of the 22nd international conference on World Wide Web companion, WWW ’13 Companion, pages 1351– 1358, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. K. McKelvey and F. Menczer. Truthy: Enabling the Study of Online Social Networks. In Proc. 16th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (CSCW), 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Microsoft Corporation. Word (. doc) Binary File Format. https://msdn.microsoft.com/en-us/library/office/ cc313153(v=office.12).aspx, 2014.Google ScholarGoogle Scholar
  25. Microsoft Corporation. Bitmap Storage. https://msdn. microsoft.com/en-us/library/dd183391.aspx, 2015.Google ScholarGoogle Scholar
  26. H. Miller, P. Haller, E. Burmako, and M. Odersky. Instant pickles: Generating object-oriented pickler combinators for fast and extensible serialization. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA ’13, pages 183–202, New York, NY, USA, 2013. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. C. Nester, M. Philippsen, and B. Haumacher. A more efficient RMI for Java. In Proceedings of the ACM 1999 conference on Java Grande, pages 152–159. ACM, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. E. Shekita and M. Zwilling. Cricket: A Mapped, Persistent Object Store. Center for Parallel Optimization, Computer Sciences Department, University of Wisconsin, 1996.Google ScholarGoogle Scholar
  29. TIS Committee. Tool Interface Standard (TIS) Executable and Linking Format (ELF) Specification (Version 1.2). Technical Report May, 1995.Google ScholarGoogle Scholar
  30. M. Tofte, L. Birkedal, M. Elsman, and N. Hallenberg. A retrospective on region-based memory management. Higher-Order and Symbolic Computation, 17(3):245–265, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. K. Varda. Cap’n Proto. https://capnproto.org/, 2015.Google ScholarGoogle Scholar
  32. P. Wilson and S. Kakkad. Pointer swizzling at page fault time: efficiently and compatibly supporting huge address spaces on standard hardware. {1992} Proceedings of the Second International Workshop on Object Orientation in Operating Systems, 1992.Google ScholarGoogle Scholar

Index Terms

  1. Efficient communication and collection with compact normal forms

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 50, Issue 9
      ICFP '15
      September 2015
      436 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2858949
      • Editor:
      • Andy Gill
      Issue’s Table of Contents
      • cover image ACM Conferences
        ICFP 2015: Proceedings of the 20th ACM SIGPLAN International Conference on Functional Programming
        August 2015
        436 pages
        ISBN:9781450336697
        DOI:10.1145/2784731

      Copyright © 2015 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 29 August 2015

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!