skip to main content
research-article

Rate types for stream programs

Authors Info & Claims
Published:15 October 2014Publication History
Skip Abstract Section

Abstract

We introduce RATE TYPES, a novel type system to reason about and optimize data-intensive programs. Built around stream languages, RATE TYPES performs static quantitative reasoning about stream rates -- the frequency of data items in a stream being consumed, processed, and produced. Despite the fact that streams are fundamentally dynamic, we find two essential concepts of stream rate control -- throughput ratio and natural rate -- are intimately related to the program structure itself and can be effectively reasoned about by a type system. RATE TYPES is proven to correspond with a time-aware and parallelism-aware operational semantics. The strong correspondence result tolerates arbitrary schedules, and does not require any synchronization between stream filters.We further implement RATE TYPES, demonstrating its effectiveness in predicting stream data rates in real-world stream programs.

References

  1. Thies, W., Amarasinghe, S.: An empirical characterization of stream programs and its implications for language and compiler design. In: PACT '10. (2010) 365--376. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Nvidia: Compute unified device architecture programming guide. NVIDIA: Santa Clara, CA (2007).Google ScholarGoogle Scholar
  3. Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G., Stonebraker, M., Tatbul, N., Zdonik, S. B.: Monitoring streams - a new class of data management applications. In: VLDB. (2002) 215--226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Krishnaswami, N. R., Benton, N.: A semantic model for graphical user interfaces. In: ICFP '11. (2011) 45--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Czaplicki, E., Chong, S.: Asynchronous functional reactive programming for GUIs. In: PLDI'13. (June 2013). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Elliott, C., Hudak, P.: Functional reactive animation. In: ICFP '97. (1997) 263--273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Sorber, J., Kostadinov, A., Garber, M., Brennan, M., Corner, M. D., Berger, E. D.: Eon: a language and runtime system for perpetual systems. In: SenSys '07. (2007) 161--174. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Monsanto, C., Foster, N., Harrison, R.,Walker, D.: A compiler and run-time system for network programming languages. In: POPL '12. (2012) 217--230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Soulé, R., Hirzel, M., Grimm, R., Gedik, B., Andrade, H., Kumar, V., Wu, K. L.: A universal calculus for stream processing languages. In: ESOP'10. (2010) 507--528. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Botinčan, M., Babić, D.: Sigma*: symbolic learning of input-output specifications. In: POPL '13. (2013) 443--456. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Suenaga, K., Sekine, H., Hasuo, I.: Hyperstream processing systems: nonstandard modeling of continuous-time signals. In: POPL '13. (2013) 417--430. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Wilhelm, R., Engblom, J., Ermedahl, A., Holsti, N., Thesing, S., Whalley, D., Bernat, G., Ferdinand, C., Heckmann, R., Mitra, T., Mueller, F., Puaut, I., Puschner, P., Staschulat, J., Stenström, P.: The worst-case execution-time problem - overview of methods and survey of tools. ACM Trans. Embed. Comput. Syst. 7(3) (May 2008) 36:1--36:53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Blelloch, G. E., Greiner, J.: A provable time and space efficient implementation of nesl. In: ICFP '96. (1996) 213--225. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Igarashi, A., Kobayashi, N.: Resource usage analysis. In: POPL '02. (2002) 331--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Hoffmann, J., Aehlig, K., Hofmann, M.: Multivariate amortized resource analysis. In: POPL '11. (2011) 357--370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Lee, E. A., Messerschmitt, D. G.: Static scheduling of synchronous data flow programs for digital signal processing. IEEE Trans. Comput. 36(1) (January 1987) 24--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Cohen, A., Duranton, M., Eisenbeis, C., Pagetti, C., Plateau, F., Pouzet, M.: N-synchronous kahn networks: a relaxed model of Synchrony for real-time systems. In: POPL '06. (2006) 180--193. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Thies, W., Karczmarek, M., Amarasinghe, S. P.: StreamIt: A language for streaming applications. In: CC'02. (2002) 179--196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Kirkpatrick, S., Gelatt, C. D., Vecchi, M. P.: Optimization by simulated annealing. Science 220 (1983) 671--680.Google ScholarGoogle ScholarCross RefCross Ref
  20. Agha, G.: ACTORS: A model of Concurrent computations in Distributed Systems. MITP, Cambridge, Mass. (1990). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Lee, E., Messerschmitt, D.: Synchronous data flow. Proceedings of the IEEE 75(9) (Sept 1987) 1235--1245.Google ScholarGoogle ScholarCross RefCross Ref
  22. Spring, J. H., Privat, J., Guerraoui, R., Vitek, J.: StreamFlex: high-throughput stream programming in Java. In: OOPSLA '07. (2007) 211--228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Gordon, M. I., Thies,W., Amarasinghe, S.: Exploiting coarse-grained task, data, and pipeline parallelism in stream programs. In: ASPLOS'06. (2006). Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Furr, M., An, J. h. D., Foster, J. S.: Profile-guided static typing for dynamic scripting languages. In: OOPSLA '09. (2009) 283--300. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Chambers, C., Raniwala, A., Perry, F., Adams, S., Henry, R. R., Bradshaw, R., Weizenbaum, N.: FlumeJava: easy, efficient data-parallel pipelines. In: PLDI '10. (2010) 363--375. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Morton, K., Balazinska, M., Grossman, D.: ParaTimer: a progress indicator for MapReduce DAGs. In: SIGMOD '10. (2010) 507--518. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Bohannon, A., Foster, J. N., Pierce, B. C., Pilkiewicz, A., Schmitt, A.: Boomerang: resourceful lenses for string data. In: POPL '08. (2008) 407--419. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Bartenstein, T., Liu, Y. D.: Rate Types for Stream Programs - Technical Report. https://www.cs.binghamton.edu/?tbarten1/RateTypesForStreamPrograms_TechReport.pdf, (2014)Google ScholarGoogle Scholar
  29. Gordon, M. I.: Compiler Techniques for Scalable Performance of Stream Programs on Multicore Architectures. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA (May 2010). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Abadi, D. J., Ahmad, Y., Balazinska, M., Cetintemel, U., Cherniack, M., Hwang, J. H., Lindner, W., Maskey, A., Rasin, A., Ryvkina, E., et al.: The design of the borealis stream processing engine. In: CIDR. Volume 5. (2005) 277--289.Google ScholarGoogle Scholar
  31. Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G., Olston, C., Rosenstein, J., Varma, R.: Query processing, resource management, and approximation in a data stream management system, CIDR (2003).Google ScholarGoogle Scholar
  32. Meyerovich, L. A., Guha, A., Baskin, J., Cooper, G. H., Greenberg, M., Bromfield, A., Krishnamurthi, S.: Flapjax: a programming language for Ajax applications. In: OOPSLA '09. (2009) 1--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Olston, C., Reed, B., Srivastava, U., Kumar, R., Tomkins, A.: Pig latin: A not-so-foreign language for data processing. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. SIGMOD '08, New York, NY, USA, ACM (2008) 1099--1110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Zhou, J., Demsky, B.: Bamboo: a data-centric, object-oriented approach to many-core software. In: PLDI'10, ACM (2010) 388--399. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Ashcroft, E. A., Wadge, W. W.: Lucid, a non-procedural language with iteration. Commun. ACM 20(7) (July 1977). Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Caspi, P., Pilaud, D., Halbwachs, N., Plaice, J. A.: LUSTRE: a declarative language for real-time programming. In: POPL '87. (1987) 178--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Kahn, G.: The semantics of a simple language for parallel programming. In Rosenfeld, J. L., ed.: Information processing, Stockholm, Sweden, North Holland, Amsterdam (Aug 1974) 471--475Google ScholarGoogle Scholar
  38. Thiele, L., Stoimenov, N.: Modular Performance Analysis of Cyclic Dataflow Graphs. In: Proceedings of the Seventh ACM International Conference on Embedded Software. EMSOFT '09, New York, NY, USA, ACM (2009) 127--136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Sculthorpe, N., Nilsson, H.: Safe functional reactive programming through dependent types. In: ICFP '09. (2009) 23--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Krishnaswami, N. R., Benton, N., Hoffmann, J.: Higher-order functional reactive programming in bounded space. In: POPL '12. (2012) 45--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Cohen, M., Zhu, H. S., Emgin, S. E., Liu, Y. D.: Energy types. In: OOPSLA '12. (October 2012). Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Bartenstein, T., Liu, Y. D.: Green streams for data-intensive software. In: ICSE'13. (May 2013). Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Rate types for stream programs

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 49, Issue 10
      OOPSLA '14
      October 2014
      907 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2714064
      • Editor:
      • Andy Gill
      Issue’s Table of Contents
      • cover image ACM Conferences
        OOPSLA '14: Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications
        October 2014
        946 pages
        ISBN:9781450325851
        DOI:10.1145/2660193

      Copyright © 2014 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 October 2014

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!