Abstract
First-class functions dramatically increase expressiveness, at the expense of static guarantees. In ALGOL or PASCAL, functions could be passed as arguments but never escape their defining scope. Therefore, function arguments could serve as temporary access tokens or capabilities, enabling callees to perform some action, but only for the duration of the call. In modern languages, such programming patterns are no longer available.
The central thrust of this paper is to re-introduce second-class functions and other values alongside first-class entities in modern languages. We formalize second-class values with stack-bounded lifetimes as an extension to simply-typed λ calculus, and for richer type systems such as F<: and systems with path-dependent types. We generalize the binary first- vs second-class distinction to arbitrary privilege lattices, with the underlying type lattice as a special case. In this setting, abstract types naturally enable privilege parametricity. We prove type soundness and lifetime properties in Coq.
We implement our system as an extension of Scala, and present several case studies. First, we modify the Scala Collections library and add privilege annotations to all higher-order functions. Privilege parametricity is key to retain the high degree of code-reuse between sequential and parallel as well as lazy and eager collections. Second, we use scoped capabilities to introduce a model of checked exceptions in the Scala library, with only few changes to the code. Third, we employ second-class capabilities for memory safety in a region-based off-heap memory library.
Supplemental Material
Available for Download
This is the paper artifact packaged in a virtual machine image that contains: mechanized proofs in Coq, a Scala distribution that contains our Scala compiler plug-in, modified standard library discussed in the paper, and additional Scala code for case studies. This is the version accepted by the OOPSLA'16 Artifact Evaluation Committee, but you can easily update the code to the latest version (see the instructions). Please unpack the whole ZIP archive and go to directory: oopsla16-65 It contains a virtual machine image with instructions in README.pdf on how to: 1. reproduce the results from the paper 2. test and fiddle with our tools 3. reuse and possibly extend our work Hashes for integrity verification: MD5(oopsla16-65.vmdk)= 8e4d1cb296120ec8acbf6d2f92a7272f MD5(oopsla16-65-flat.vmdk)= 60651a50061cf81a27ccc28ecdcda2c4
- M. Abadi, A. Birrell, T. Harris, and M. Isard. Semantics of transactional memory and automatic mutual exclusion. In Proceedings of the 35th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, POPL ’08, pages 63–74, New York, NY, USA, 2008. ACM. Google Scholar
Digital Library
- N. Amin, A. Moors, and M. Odersky. Dependent object types. In FOOL, 2012.Google Scholar
- R. Atkey. Parameterised notions of computation. J. Funct. Program., 19(3-4):335–376, 2009. Google Scholar
Digital Library
- A. Banerjee and D. A. Schmidt. Stackability in the simplytyped call-by-value lambda calculus. Sci. Comput. Program., 31(1):47–73, 1998. Google Scholar
Digital Library
- E. Barendsen and S. Smetsers. Conventional and uniqueness typing in graph rewrite systems. In R. Shyamasundar, editor, Foundations of Software Technology and Theoretical Computer Science, volume 761 of Lecture Notes in Computer Science, pages 41–51. Springer Berlin Heidelberg, 1993. Google Scholar
Digital Library
- A. Bauer and M. Pretnar. Programming with algebraic effects and handlers. J. Log. Algebr. Meth. Program., 84(1):108–123, 2015.Google Scholar
Cross Ref
- J. Boyland. Checking interference with fractional permissions. In SAS, volume 2694 of Lecture Notes in Computer Science, pages 55–72. Springer, 2003. Google Scholar
Digital Library
- E. Brady. Programming and reasoning with algebraic effects and dependent types. To Appear in Proceedings of the ACM SIGPLAN international conference on Functional programming, 2013. Google Scholar
Digital Library
- K. J. Brown, A. K. Sujeeth, H. Lee, T. Rompf, H. Chafi, M. Odersky, and K. Olukotun. A heterogeneous parallel framework for domain-specific languages. In PACT, 2011. Google Scholar
Digital Library
- H. Chafi, Z. DeVito, A. Moors, T. Rompf, A. K. Sujeeth, P. Hanrahan, M. Odersky, and K. Olukotun. Language Virtualization for Heterogeneous Parallel Computing. Onward!, 2010. Google Scholar
Digital Library
- H. Chafi, A. K. Sujeeth, K. J. Brown, H. Lee, A. R. Atreya, and K. Olukotun. A domain-specific approach to heterogeneous parallelism. In Proceedings of the 16th ACM symposium on Principles and practice of parallel programming, PPoPP, 2011. Google Scholar
Digital Library
- D. Clarke, J. Östlund, I. Sergey, and T. Wrigstad. Ownership types: A survey. In Aliasing in Object-Oriented Programming, volume 7850 of Lecture Notes in Computer Science, pages 15–58. Springer, 2013. Google Scholar
Digital Library
- E. Ernst, K. Ostermann, and W. R. Cook. A virtual class calculus. In POPL, 2006. Google Scholar
Digital Library
- D. Gay. Memory management with explicit regions. PhD thesis, 1997. Google Scholar
Digital Library
- D. K. Gifford, P. Jouvelot, M. A. Sheldon, and J. W. O’Toole. Report on the FX programming language. Technical report, MIT/LCS/TR-531, 1992.Google Scholar
- D. K. Gifford and J. M. Lucassen. Integrating functional and imperative programming. In Proceedings of the 1986 ACM conference on LISP and functional programming, LFP ’86, pages 28–38, New York, NY, USA, 1986. ACM. Google Scholar
Digital Library
- J. Gosling, B. Joy, G. L. Steele, Jr., G. Bracha, and A. Buckley. The Java Language Specification, Java SE 7 Edition. Addison-Wesley Professional, 2013. Google Scholar
Digital Library
- J. Hannan. A type-based escape analysis for functional languages. J. Funct. Program., 8(3):239–273, 1998. Google Scholar
Digital Library
- J. Hogg. Islands: Aliasing protection in object-oriented languages. In OOPSLA, pages 271–285. ACM, 1991. Google Scholar
Digital Library
- T. Jim, J. G. Morrisett, D. Grossman, M. W. Hicks, J. Cheney, and Y. Wang. Cyclone: A safe dialect of C. In C. S. Ellis, editor, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, June 10-15, 2002, Monterey, California, USA, pages 275–288. USENIX, 2002. Google Scholar
Digital Library
- O. Kiselyov and C. Shan. Lightweight static capabilities. Electr. Notes Theor. Comput. Sci., 174(7):79–104, 2007. Google Scholar
Digital Library
- O. Kiselyov and C. Shan. Lightweight monadic regions. In Haskell, pages 1–12. ACM, 2008. Google Scholar
Digital Library
- H. Lee, K. J. Brown, A. K. Sujeeth, H. Chafi, T. Rompf, M. Odersky, and K. Olukotun. Implementing domain-specific languages for heterogeneous parallel computing. IEEE Micro, 31(5):42–53, 2011. Google Scholar
Digital Library
- D. Leijen. Koka: A language with effect inference. http: //research.microsoft.com/en-us/projects/koka/2012overviewkoka.pdf, April 2012.Google Scholar
- S. Liang, P. Hudak, and M. Jones. Monad transformers and modular interpreters. In Proceedings of the 22nd ACM SIGPLAN-SIGACT symposium on Principles of programming languages, POPL ’95, pages 333–343, New York, NY, USA, 1995. ACM. Google Scholar
Digital Library
- B. Lippmeier. Type Inference and Optimisation for an Impure World. PhD thesis, Australian National University, 2010.Google Scholar
- J. M. Lucassen and D. K. Gifford. Polymorphic effect systems. In Proceedings of the 15th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, POPL ’88, pages 47–57, New York, NY, USA, 1988. ACM. Google Scholar
Digital Library
- D. Marino and T. Millstein. A generic type-and-effect system. In Proceedings of the 4th international workshop on Types in language design and implementation, TLDI ’09, pages 39– 50, New York, NY, USA, 2009. ACM. Google Scholar
Digital Library
- N. D. Matsakis and F. S. Klock, II. The Rust language. Ada Lett., 34(3):103–104, Oct. 2014. Google Scholar
Digital Library
- M. S. Miller. The E language. http://erights.org/elang/ index.html, 1998.Google Scholar
- E. Moggi. Notions of computation and monads. Inf. Comput., 93(1):55–92, July 1991. Google Scholar
Digital Library
- J. Moses. The function of function in lisp or why the funarg problem should be called the environment problem. ACM Sigsam Bulletin, (15):13–27, 1970. Google Scholar
Digital Library
- K. Naden, R. Bocchino, J. Aldrich, and K. Bierhoff. A type system for borrowing permissions. In POPL, pages 557–570. ACM, 2012. Google Scholar
Digital Library
- F. Nielson and H. Nielson. Type and effect systems. In E.-R. Olderog and B. Steffen, editors, Correct System Design, volume 1710 of Lecture Notes in Computer Science, pages 114–136. Springer Berlin Heidelberg, 1999. Google Scholar
Digital Library
- J. Noble, J. Vitek, and J. Potter. Flexible alias protection. In ECOOP, volume 1445 of Lecture Notes in Computer Science, pages 158–185. Springer, 1998. Google Scholar
Digital Library
- M. Odersky. Scala - where it came from, where it is going. http://www.slideshare.net/Odersky/scala-dayssan-francisco-45917092, 2015.Google Scholar
- M. Odersky and T. Rompf. Unifying functional and objectoriented programming with scala. Commun. ACM, 57(4):76– 86, 2014. Google Scholar
Digital Library
- G. Ofenbeck, T. Rompf, A. Stojanov, M. Odersky, and M. Püschel. Spiral in scala: towards the systematic construction of generators for performance libraries. In J. Järvi and C. Kästner, editors, Generative Programming: Concepts and Experiences, GPCE’13, Indianapolis, IN, USA - October 27 - 28, 2013, pages 125–134. ACM, 2013. Google Scholar
Digital Library
- D. J. Pearce. JPure: A modular purity system for Java. In J. Knoop, editor, Compiler Construction, volume 6601 of Lecture Notes in Computer Science, pages 104–123. Springer Berlin Heidelberg, 2011. Google Scholar
Digital Library
- T. Petricek, D. A. Orchard, and A. Mycroft. Coeffects: Unified static analysis of context-dependence. In Automata, Languages, and Programming - 40th International Colloquium, ICALP 2013, Riga, Latvia, July 8-12, 2013, Proceedings, Part II, pages 385–397, 2013. Google Scholar
Digital Library
- T. Petricek, D. A. Orchard, and A. Mycroft. Coeffects: a calculus of context-dependent computation. In J. Jeuring and M. M. T. Chakravarty, editors, Proceedings of the 19th ACM SIGPLAN international conference on Functional programming, Gothenburg, Sweden, September 1-3, 2014, pages 123– 135. ACM, 2014. Google Scholar
Digital Library
- S. L. Peyton Jones and P. Wadler. Imperative functional programming. In Proceedings of the 20th ACM SIGPLANSIGACT symposium on Principles of programming languages, POPL ’93, pages 71–84, New York, NY, USA, 1993. ACM. Google Scholar
Digital Library
- G. D. Plotkin and M. Pretnar. Handling algebraic effects. Logical Methods in Computer Science, 9(4), 2013.Google Scholar
- A. Prokopec, P. Bagwell, and T. R. abd Martin Odersky. A generic parallel collection framework. Euro-Par, 2010. Google Scholar
Digital Library
- M. Püschel, J. Moura, J. Johnson, D. Padua, M. Veloso, B. Singer, J. Xiong, F. Franchetti, A. Gacic, Y. Voronenko, K. Chen, R. Johnson, and N. Rizzolo. Spiral: Code generation for dsp transforms. Proceedings of the IEEE, 93(2):232 –275, feb. 2005.Google Scholar
Cross Ref
- E. Reed. Patina : A Formalization of the Rust Programming Language. (February):1–37, 2015.Google Scholar
- T. Rompf. Lightweight Modular Staging and Embedded Compilers: Abstraction Without Regret for High-Level High-Performance Programming. PhD thesis, EPFL, 2012.Google Scholar
- T. Rompf and N. Amin. From F to DOT: Type soundness proofs with definitional interpreters. Technical report, Purdue University, July 2015. http://arxiv.org/abs/1510.05216.Google Scholar
- T. Rompf and N. Amin. Type soundness for dependent object types (dot). In OOPSLA, 2016. Google Scholar
Digital Library
- T. Rompf, K. J. Brown, H. Lee, A. K. Sujeeth, M. Jonnalagedda, N. Amin, G. Ofenbeck, A. Stojanov, Y. Klonatos, M. Dashti, C. Koch, M. Püschel, and K. Olukotun. Go meta! A case for generative programming and dsls in performance critical systems. In T. Ball, R. Bodík, S. Krishnamurthi, B. S. Lerner, and G. Morrisett, editors, 1st Summit on Advances in Programming Languages, SNAPL 2015, May 3-6, 2015, Asilomar, California, USA, volume 32 of LIPIcs, pages 238– 261. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2015.Google Scholar
- T. Rompf, I. Maier, and M. Odersky. Implementing first-class polymorphic delimited continuations by a type-directed selective cps-transform. In Proceedings of the 14th ACM SIGPLAN international conference on Functional programming, ICFP ’09, pages 317–328, New York, NY, USA, 2009. ACM. Google Scholar
Digital Library
- T. Rompf and M. Odersky. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled dsls. Commun. ACM, 55(6):121–130, 2012. Google Scholar
Digital Library
- L. Rytz, N. Amin, and M. Odersky. A flow-insensitive, modular effect system for purity. In W. Dietl, editor, Proceedings of the 15th Workshop on Formal Techniques for Java-like Programs, FTfJP 2013, Montpellier, France, July 1, 2013, pages 4:1–4:7. ACM, 2013. Google Scholar
Digital Library
- L. Rytz, M. Odersky, and P. Haller. Lightweight polymorphic effects. In J. Noble, editor, ECOOP 2012 - Object-Oriented Programming - 26th European Conference, Beijing, China, June 11-16, 2012. Proceedings, volume 7313 of Lecture Notes in Computer Science, pages 258–282. Springer, 2012. Google Scholar
Digital Library
- J. Siek. Type safety in three easy lemmas. http://siek.blogspot.ch/2013/05/type-safety-in-threeeasy-lemmas.html, 2013.Google Scholar
- J. G. Siek, M. M. Vitousek, and J. D. Turner. Effects for funargs. CoRR, abs/1201.0023, 2012.Google Scholar
- C. Strachey. Fundamental concepts in programming languages. Higher-Order and Symbolic Computation, 13(1/2):11– 49, 2000. Google Scholar
Digital Library
- A. K. Sujeeth, A. Gibbons, K. J. Brown, H. Lee, T. Rompf, M. Odersky, and K. Olukotun. Forge: Generating a high performance dsl implementation from a declarative specification. In Proceedings of the 12th International Conference on Generative Programming: Concepts &#38; Experiences, GPCE, 2013. Google Scholar
Digital Library
- A. K. Sujeeth, H. Lee, K. J. Brown, T. Rompf, M. Wu, A. R. Atreya, M. Odersky, and K. Olukotun. OptiML: an implicitly parallel domain-specific language for machine learning. In Proceedings of the 28th International Conference on Machine Learning, ICML, 2011.Google Scholar
- A. K. Sujeeth, T. Rompf, K. J. Brown, H. Lee, H. Chafi, V. Popic, M. Wu, A. Prokopec, V. Jovanovic, M. Odersky, and K. Olukotun. Composition and reuse with compiled domainspecific languages. In European Conference on Object Oriented Programming, ECOOP, 2013. Google Scholar
Digital Library
- K. N. Swadi, W. Taha, O. Kiselyov, and E. Pasalic. A monadic approach for avoiding code duplication when staging memoized functions. In J. Hatcliff and F. Tip, editors, Proceedings of the 2006 ACM SIGPLAN Workshop on Partial Evaluation and Semantics-based Program Manipulation, 2006, Charleston, South Carolina, USA, January 9-10, 2006, pages 160–169. ACM, 2006. Google Scholar
Digital Library
- W. Taha and M. F. Nielsen. Environment classifiers. In A. Aiken and G. Morrisett, editors, Conference Record of POPL 2003: The 30th SIGPLAN-SIGACT Symposium on Principles of Programming Languages, New Orleans, Louisisana, USA, January 15-17, 2003, pages 26–37. ACM, 2003. Google Scholar
Digital Library
- W. Taha and T. Sheard. Metaml and multi-stage programming with explicit annotations. Theor. Comput. Sci., 248(1-2):211– 242, 2000. Google Scholar
Digital Library
- J. Talpin and P. Jouvelot. The type and effect discipline. In Logic in Computer Science, 1992. LICS ’92., Proceedings of the Seventh Annual IEEE Symposium on, pages 162–173, 1992.Google Scholar
Cross Ref
- J.-P. Talpin and P. Jouvelot. Polymorphic type, region and effect inference. Journal of Functional Programming, 2:245– 271, 6 1992.Google Scholar
Cross Ref
- É. Tanter. Beyond static and dynamic scope. In J. Noble, editor, Proceedings of the 5th Symposium on Dynamic Languages, DLS 2009, October 26, 2010, Orlando, Florida, USA, pages 3–14. ACM, 2009. Google Scholar
Digital Library
- R. Tate. The sequential semantics of producer effect systems. In Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, POPL ’13, pages 15–26, New York, NY, USA, 2013. ACM. Google Scholar
Digital Library
- T. Terauchi and A. Aiken. Witnessing side-effects. In Proceedings of the tenth ACM SIGPLAN international conference on Functional programming, ICFP ’05, pages 105–115, New York, NY, USA, 2005. ACM. Google Scholar
Digital Library
- M. Tofte, L. Birkedal, M. Elsman, N. Hallenberg, T. Højfeld, and O. P. Sestoft. Programming with regions in the MLKit (revised for version 4.3.0). Technical report, IT University of Copenhagen, January 2006.Google Scholar
- M. Tofte and J.-P. Talpin. Implementation of the typed callby-value λ-calculus using a stack of regions. In Proceedings of the 21st ACM SIGPLAN-SIGACT symposium on Principles of programming languages, POPL ’94, pages 188–201, New York, NY, USA, 1994. ACM. Google Scholar
Digital Library
- P. Wadler. Linear types can change the world! In C. Jones, editor, Proceedings of the IFIP Working Group 2.2/2.3 Working Conference on Programming Concepts and Methods. North-Holland, 1990.Google Scholar
- P. Wadler. The essence of functional programming. In R. Sethi, editor, Conference Record of the Nineteenth Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Albuquerque, New Mexico, USA, January 19-22, 1992, pages 1–14. ACM Press, 1992. Google Scholar
Digital Library
- P. Wadler. The marriage of effects and monads. In Proceedings of the third ACM SIGPLAN international conference on Functional programming, ICFP ’98, pages 63–74, New York, NY, USA, 1998. ACM. Google Scholar
Digital Library
- J. Weizenbaum. The funarg problem explained. Technical report, MIT, Cambridge, Massachusetts, 1968.Google Scholar
- T. Zhao, J. Baker, J. Hunt, J. Noble, and J. Vitek. Implicit ownership types for memory management. Sci. Comput. Program., 71(3):213–241, 2008. Google Scholar
Digital Library
Index Terms
Gentrification gone too far? affordable 2nd-class values for fun and (co-)effect
Recommendations
Gentrification gone too far? affordable 2nd-class values for fun and (co-)effect
OOPSLA 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and ApplicationsFirst-class functions dramatically increase expressiveness, at the expense of static guarantees. In ALGOL or PASCAL, functions could be passed as arguments but never escape their defining scope. Therefore, function arguments could serve as temporary ...
Rust-like borrowing with 2nd-class values (short paper)
SCALA 2017: Proceedings of the 8th ACM SIGPLAN International Symposium on ScalaThe Rust programming language demonstrates that memory safety can be achieved in a practical systems language, based on a sophisticated type system that controls object lifetimes and aliasing through notions of ownership and borrowing. While Scala has ...
Recovering purity with comonads and capabilities
In this paper, we take a pervasively effectful (in the style of ML) typed lambda calculus, and show how to extend it to permit capturing pure expressions with types. Our key observation is that, just as the pure simply-typed lambda calculus can be ...







Comments