Pipelines and Beyond: Graph Types for ADTs with Futures

Parallel programs are frequently modeled as dependency or cost graphs, which can be used to detect various bugs, or simply to visualize the parallel structure of the code. However, such graphs reflect just one particular execution and are typically constructed in a post-hoc manner. Graph types, which were introduced recently to mitigate this problem, can be assigned statically to a program by a type system and compactly represent the family of all graphs that could result from the program. Unfortunately, prior work is restricted in its treatment of futures, an increasingly common and especially dynamic form of parallelism. In short, each instance of a future must be statically paired with a vertex name. Previously, this led to the restriction that futures could not be placed in collections or be used to construct data structures. Doing so is not a niche exercise: such structures form the basis of numerous algorithms that use forms of pipelining to achieve performance not attainable without futures. All but the most limited of these examples are out of reach of prior graph type systems. In this paper, we propose a graph type system that allows for almost arbitrary combinations of futures and recursive data types. We do so by indexing datatypes with a type-level vertex structure, a codata structure that supplies unique vertex names to the futures in a data structure. We prove the soundness of the system in a parallel core calculus annotated with vertex structures and associated operations. Although the calculus is annotated, this is merely for convenience in defining the type system. We prove that it is possible to annotate arbitrary recursive types with vertex structures, and show using a prototype inference engine that these annotations can be inferred from OCaml-like source code for several complex parallel algorithms.


INTRODUCTION
Decades of work on reasoning about parallel programs have focused on computation or cost graphs, directed graphs that represent the dependencies of threads.Computation graphs are a convenient target for analysis because they abstract away details of the program, language, and even the parallelism features that were used, while still capturing enough information about the relationships between threads to perform many useful analyses.For example, computation graphs have been used to study deadlock [Cogumbreiro et al. 2018], data races [Banerjee et al. 2006], priority inversions [Babaoğlu et al. 1993] and evaluation cost [Blelloch andGreiner 1995, 1996].
To analyze such properties, it is desirable to calculate the computation graph of a program statically, at compile time or analysis time.Doing so is often possible in languages and threading libraries for coarse-grained parallelism, such as pthreads, where thread creation and synchronization are expensive and rare.Much recent interest in parallel programming, however, has been in the area of fine-grained parallelism, in which threads are created cheaply and eagerly, often based on runtime conditions.For example, a program might fork at each level of a divide-and-conquer algorithm, or a web server might spawn a new thread to handle every incoming request asynchronously.Reasoning statically about the dependency structure of fine-grained parallel programs is difficult because of the highly dynamic nature of thread creation and synchronization in these programs.
This difficulty is compounded when programs use futures and related abstractions for fine-grained parallelism, which are becoming increasingly popular and have been made available in Python, Scala, Rust, and the most recent release of OCaml [Sivaramakrishnan et al. 2020], among other languages.Essentially, a future is a first-class handle to an asynchronous computation.The result of the computation can be demanded via a force or touch operation, which blocks if the result is not yet available.Because futures run in separate threads, we can model each future as its own vertex in the computation graph of a program.Edges leading into track the intermediate results used to compute the future, and when we touch the future, we add an edge from to the thread where the touch happened.Futures may be passed around a program arbitrarily and end up being touched in a very different part of the program from where it was spawned, leading to great power and flexibility but also complex computation graphs which are difficult to reason about.
To address the difficulty of predicting parallel dependences in fine-grained parallel programs, especially those with futures, Muller [2022] introduced the notion of graph types, which statically overapproximate the set of computation graphs that might result from running a program.A graph type system statically assigns graph types to programs, and its soundness theorem ensures that the actual computation graph resulting from any execution of a well-typed program is described by the program's graph type.
Much of the complexity of the graph type system centers around futures.Because futures can be touched in an entirely different part of the program from where they are created, each future type is annotated with a distinguished vertex name, so that the graph type system can refer to the correct vertex when tracking the dependencies of touch operations.(Explicit vertex names are not needed in simpler parallelism models such as fork-join, because it is clear what thread is being synchronized.)To avoid tracking spurious dependencies, the graph type system ensures that each vertex name is associated with at most one future during execution.More precisely, when spawning a new future, the graph type system annotates the type of the result with a fresh vertex name, which is tracked in a separate affine context to prevent reuse.
This treatment of futures leads to a significant limitation in prior work: it is difficult or impossible to build useful data structures containing futures.Even an expression as simple as [future e1; future e2] (a list containing two new futures) cannot be assigned a type.The reason is that the two elements of this list must have types future[ 1 ] and future[ 2 ], respectively, where 1 and 2 are distinct vertex names and future[ ] is the type of a future returning a value of type with the vertex named -these two elements can't be placed in a list because prior work supports only homogeneous lists.Although this example is simple and artificial, much of the power of futures, as opposed to more limited parallelism models such as fork-join, comes from the ability to program with data structures that contain an unbounded number of futures, such as lists and trees.As examples, Blelloch and Reid-Miller [1997]  data structures that use futures in complex ways to pipeline computations, resulting in asymptotic improvements over the best known fork-join implementations.These programming idioms exercise the full complexity of futures, motivating the need for techniques to reason statically about the computation graphs of these programs.
In this paper, we develop a graph type system, and accompanying inference algorithm, that can handle complex data structures using futures.As a motivating example, consider a function that produces a pipeline of increasingly precise approximations of .This could be, for example, the first stage in a graphics or simulation pipeline.We wish to compute the approximations asynchronously so that earlier approximations can be used while later ones are still being computed.Figure 1 shows two possible implementations of such a function.The implementation on the left produces a list of futures with the intermediate results.The function list_pi takes a number k and a future a, which computes the ( − 1)st approximation.Each iteration of list_pi spawns a new future to compute the kth term of the Gregory series multiplied by 4, adds it to the running total being computed by a, and adds the new future (which is completing the new running total) to a list, then calls list_pi recursively to compute the remaining terms.To illustrate a use of this structure, the main function takes the second approximation from the list.Note that list_pi, as written, doesn't terminate.
Because the function list_pi produces a list of futures, it cannot be given a graph type under prior work [Muller 2022].1This is a shame, because its computation graph would have revealed a subtle but fatal bug: despite the futures, there is no real asynchrony or pipelining because almost the entire list of approximations (which, in this example, is infinite) must be constructed before the program proceeds.This can be seen in the visualization on the left side of Figure 2, which is produced automatically by our implementation from the inferred graph type.In the figure, vertices in the graph, notated with either a text label or a small circle, represent pieces of computation.The vertices with labels like 1 • 1 are the final vertices of a future, and these labels are the vertex names assigned to the future.The reason for these particular labels will become clear later in the paper.Edges represent dependences: an edge out of a labeled vertex indicates a touch of the corresponding future, and other edges represent sequential dependences within a thread or the spawning of a future.A path of edges in the graph therefore represents a chain of sequential dependences and two vertices with no path between them indicate opportunities for parallelism.Long paths indicate a lack of parallelism.
The figure shows a visualization of the graph type of the program, with the recursion of list_pi unrolled a fixed number of times to make the recursive structure visually clear.A vertex labeled . . .indicates a recursive call that has been elided because of the cutoff on number of unrollings.The vertex representing the touch operation in main is circled in red: we can see that there is a long chain of dependences on the critical path to reach this operation, which means the operation will be significantly delayed when running the program.Indeed, the topmost . . .appears on the critical path, indicating a potentially (and, in this case, actually) infinite critical path.
The second implementation in Figure 1 instead uses a new data structure 'a pipe which resembles a lazy list: the head of the list is computed eagerly and may be used immediately but the tail of the list is computed asynchronously in a future.The function pipeline_pi takes the running total a (now as an actual float, rather than a future) and k.It adds the kth approximation to the running total, then returns the new running total as well as a future to call pipeline_pi recursively to compute the remainder of the pipeline.This is reminiscent of the "producer" example of Blelloch and Reid-Miller [1997].As we can see from the visualization on the right side of Figure 2, the graphs corresponding to pipeline_pi exhibit much more parallelism than the previous version.Here, the touch operation in main (again circled in red) occurs in parallel with the computation of the remainder of the list and there are only a small, finite number of operations on its critical path.In this paper, we present a graph type system that can statically compute graph types for the above pipelining examples (and many more), thus allowing us to detect and repair the parallelism bugs we discussed.We present the type system in , a core calculus containing both futures and recursive types.The key to our approach is parameterizing recursive data structures involving futures with a source of fresh vertex names called a vertex structure (or VS, for short).Conceptually, one can think of a VS as a separate structure of the same shape as the program's recursive data structure, containing unique vertex names.For example, both the float future list and the float pipe of Figure 1 would be parameterized by a stream of vertices.The two functions list_pi and pipeline_pi would take this vertex stream, let's call it , as an implicit parameter and, at each iteration, use the next vertex in the stream (fst ) to spawn the new future and pass the rest of the stream (snd ) to the recursive call.As a result, the returned list (resp., pipe) will be "zipped" together with the vertex stream in the sense that the first future in the list (resp., pipe) will use the first vertex of the stream, and so on.In this way, we need not "unroll" the vertex structure at compile time: the types will refer to projections of a VS parameter. 2Vertex structures are not limited to streams: in general, a VS can be an infinite (corecursive) tree with arbitrary branching patterns.We show that this allows us to construct a VS corresponding to arbitrary recursive data structures.
Recall that we require vertex names to be unique.The vertices contained by a vertex structure are all unique, but ensuring that each vertex is used at most once is non-trivial and requires numerous extensions to the graph type system.One source of complexity is that types can perform significant computation on vertex structures.As an example, as we discussed above, the same vertex cannot be used as a name for two futures.In the presence of computation on VSs, this is not a simple restriction to enforce syntactically: if is a vertex structure, then under a reasonable semantics for vertex structures, fst (fst , snd ) and fst refer to the same vertex name and therefore cannot both be used to spawn futures.
The calculus assumes that data structures are annotated with vertex structures and makes explicit many of the manipulations of VSs described above.However, should be seen as an intermediate representation-an inference algorithm can infer all necessary annotations from unannotated code in a high-level source language.As a proof of concept, we extend GML [Muller 2022], a graph type checker for a subset of OCaml (but which did not previously support lists containing futures, or any sort of user-defined algebraic data types) with support for user-defined algebraic data types containing futures.Our graph type checker is able to infer annotations and produce graph types from the examples in Figure 1, as well as all of the other examples contained in this paper, with no additional annotations or programmer burden, as well as to produce visualizations of their graph types.As shown in the example above, such visualizations can allow programmers to identify errors in the parallelization of their code, and can also be used to reason about parallel complexity and other features.Prior work [Muller 2022] also explains how graph types can be used to aid other analyses, such as deadlock detection.A formal presentation of the inference algorithm is out of the scope of this paper, but much of the challenge in our extension is constructing the vertex structure corresponding to an arbitrary user-defined algebraic data type.We describe this process formally and prove some metatheoretic results about it.
In sum, our contributions are: • , a parallel calculus with a graph type system supporting recursive data types (Section 3).• A soundness result for , guaranteeing that the graph type of a program correctly describes the computation graph that arises when running the program (Section 4).
• An algorithm for inferring the shape of a vertex structure that will provide the necessary vertex names for an arbitrary recursive data structure, and results showing (among other things) that such a VS exists for any valid recursive data type (Section 5).• A prototype implementation3 of graph type inference for an OCaml-like source language, including OCaml-style user-defined algebraic data types mixed with futures (Section 6).
Due to space limitations, we defer some of the technical details and many of the proofs to the full version of the paper [Rinaldi et al. 2023].We begin with an overview of graph types as well as a high-level description of our extensions.

OVERVIEW
We begin with an overview of graph types but refer the interested reader to the original paper [Muller 2022] for a more thorough presentation; we indicate using footnotes where we diverge from that paper's presentation.Our motivating example is a parallel implementation of Quicksort using futures (Figure 3).The code is supplemented with annotations in gray that are inserted during type inference and used in the formal presentation of , but are not written in actual code; these annotations will be explained later, in Section 3. The implementation returns immediately in the case of an empty list.On a non-empty list, the first element is selected as a pivot and used to partition the list using a sequential function partition, whose implementation we omit.A future is spawned to sort the first list recursively while the second list is sorted in the main thread.When the second list is sorted, we touch the future to retrieve the sorted first list, and append the lists.
The type of the function in is given below the code; the type indicates that qsort accepts and returns an 'a list.As is common in presentations of type-and-effect systems, we write an annotation over the arrow indicating the effects performed by running the function.In this case, the "effect" is the graph type of the function; that is, a graph type describing the family of computation graphs which might arise from executing qsort.The prefix .indicates that binds a recursive instance of itself as -this notation is taken from standard presentations of recursive types.The body of is a disjunction of two families of graphs, indicated by the ∨ symbol.This notation appears when the code executes a conditional or pattern match and indicates two possible families of graphs: 1 ∨ 2 indicates that the graph can take a form indicated by either 1 or 2 .In the example, the first graph type, •, indicates a sequential computation and corresponds to executing the base case.The second graph type corresponds to the recursive case, and indicates that a future is spawned.In order to refer to this future later in the graph type, futures are assigned unique names.By convention, these names are assumed to refer to a vertex "attached" to the computation graph of a future as a final vertex.We will refer to this vertex as the "sink" vertex of the future, borrowing a term from graph theory because, until the future is touched, it has no outgoing edges.The notation new : ♦., which appears in corresponding locations in the graph type and as an annotation in the code, indicates binding a new vertex variable which locally refers to a new, fresh vertex name; ♦ is the type of this variable and means that refers to a single vertex. 4When the annotated program is evaluated (we do this only to prove soundness; the vertex name annotations have no runtime meaning in the actual program) or the graph type is unrolled (e.g., to produce the visualizations of Figure 2), will be instantiated with a new, fresh vertex name.The sequential composition of two graph types is denoted 1 ⊕ 2 , indicating that the program performs a computation described by 1 followed by one described by 2 .In our example graph type, the graph type corresponding to the recursive case is the sequential composition of three operations.The graph type indicates that is the sink of a future whose graph is described by the graph type (which, recall, is a recursive instance of corresponding to a recursive call to qsort).In general, indicates a future whose computation graph can be described by and whose sink vertex is given the name .In , spawns using the future keyword are also annotated with the vertex that is used; this annotation is shown in gray in the code.The spawn in the graph type is then sequentially composed with another instance of for the other recursive call, and finally a touch of the future whose sink is (a touch of vertex is denoted ).
Pipelines and Beyond: Graph Types for ADTs with Futures 17:7 4. A function that iteratively approximates twice in a pipelined manner Note that the vertex in the Quicksort example exists only within the scope of the binding and so, in particular, cannot be allowed to escape the scope.If futures are, e.g., returned from a function, the vertices for those futures must be created outside and passed as parameters to the function.As an example, take the pipeline_pi2 function in Figure 4, which returns a future.This function is similar to the analogous function of Section 1, but limited to two approximations of .The vertex parameter is made explicit in the annotations, and also appears in the type of the function (shown on the right side of the figure) as a Π binding.This construct in a graph type binds two parameters.Both parameters stand for vertex structures (VSs), type-level (co)data structures containing vertices, and both are annotated with vertex structure types indicating their shapes.The first parameter, , will contain the vertices the function may use to spawn futures.In the case of pipeline_pi2, it is annotated with VS type ♦ × ♦, indicating a pair of vertices (recall that ♦ is a VS type representing a single vertex). 6The second parameter contains the vertices the function may touch; in the case of pipeline_pi2, it is empty as indicated by the unit VS type 1. 7The function pipeline_pi2 returns a future (spawned using the first component of the vertex structure ) that produces a pair of a float and another future, spawned with the second component.Note that the types of futures explicitly indicate the vertices with which the future was spawned.As in the Quicksort code, these vertices also appear as annotations on the future keyword which are inferred during type checking.Finally, the graph type of the function body shows that the function spawns a future using the vertex fst , which in turn spawns a future using the vertex snd , which finally does not spawn further threads.
The code in Figure 4 also shows a function that calls pipeline_pi2 and touches the two futures.The graph type of this function binds a new vertex structure for the two futures, which no longer need to escape the function.As in Quicksort, the new vertex structure is bound using a binding of the form new : U. , where the vertex structure type is now the product ♦ × ♦ instead of ♦.The call to pipeline_pi2 instantiating the bound vertex structure variable with the new VS ′ is indicated by substituting ′ for in .
Before proceeding, we make one additional note about the graph type system.We have referred to and similar as unique vertex names-each vertex name can be used to spawn a future at most once, otherwise the resulting graph will be ambiguous (if two futures have as a sink vertex, there is no way to know to which future a touch refers).The graph type system (both ours and that of prior work) enforce this using an affine type system that restricts the use of vertex names.Thus far, we have discussed examples that are within the capabilities of prior work.Now suppose we wish to generalize pipeline_pi2 to continue producing iterative approximations indefinitely.The code in Figure 5 does this, producing a value of type float pipe, also defined in the figure, which is a recursive type containing an approximation and a future to continue the pipeline.This is reminiscent of the "producer" example of Blelloch and Reid-Miller [1997], who use futures in this general pattern to construct a wide variety of pipelined data structures.As in the Introduction, each iteration computes the kth term in the approximation, adds it to a running total a, and returns the new running total as well as a future to call pipeline_pi recursively to compute the k + 1st term.
Useful instances of the recursive data type ′ a pipe cannot be typed with the existing graph type system, because doing so would require an infinite sequence of new vertex names and a way of associating each future in the pipeline with successive vertex names.One (incorrect but illustrative) approach would be to instantiate each future in the type with a fresh vertex name using, for example, an existential.The pipe type would then be annotated as follows: This is still not useful, however, because it doesn't allow any vertex name to escape the scope of the single future type, just as the vertex in the qsort function was confined to the qsort function.The type future[∃ : ♦. ] tells us that the future is spawned with some vertex, but gives no information about which, an untenable loss of precision when we try to touch this future and add an edge to its vertex.As an illustration of this loss of precision, consider the following program, and suppose we wish to use its graph type to check for deadlocks (simply put, a program may deadlock if its graph type can unroll to a cyclic graph): Each future in the list contains a function that touches the following future in the list.This is a fairly clear structure and a visualization or suitable analysis of the graph type produced by our system could show that the program is deadlock-free.However, if the type of the output list were given as (unit ], the most precise thing that could be said about this list is that it is a list of thunks under futures, each of which touches any future in the list (or, indeed, without further information, any future in the program), including itself.Thus, a sound deadlock detector would have to conclude that the program might deadlock.
As a more precise solution to the problem of generating unique vertex names for elements in a data structure, we introduce vertex structures, mentioned above, which we allow to be (co)recursive and thus serve as the source or collection of vertex names we need.In the code annotations and the type of pipeline_pi on the right side of the figure, the function takes a vertex structure parameter of vertex structure type vstream, which is defined in the lower right side of the figure to be a corecursive type of an infinite list or stream of vertices.The return type of the function is float pipe[ ], where the recursive pipe type is now parameterized by a vertex structure.This vertex structure is threaded through the recursive structure of the pipeline data type such that successive futures in the data type are associated with corresponding vertices from .The details of this are technical and so we defer them, as well as the formal presentation of recursive data types in , to the next section.As in pipeline_pi2, the first future uses the vertex fst , which appears as an annotation in the code and on the graph type ( contains fst , indicating a spawn of fst ).8However, now the function calls itself recursively to generate the rest of the pipeline.Because the function takes a vertex parameter, this recursive call must instantiate the vertex parameter with a vstream, and it does so with the tail of the stream, snd .This appears in the graph type as [snd ].
We complete this overview with a demonstration of how the pipeline can be consumed, which shows how vertex structures link individual futures to their touches.The nth function in Figure 5 consumes a pipeline recursively, returning the n ℎ value.It also takes a parameter of vertex structure type vstream, but this time as the second parameter, because the function uses these vertices to touch futures and does not spawn futures.The use of as the parameter to the pipe data type indicates that vertices for futures in the pipeline will be drawn from the stream , which is enough information to infer in the graph type ′ that the touch fst targets the first vertex of .The function then calls itself recursively with the tail of the vertex stream, which also appears in the recursive instantiation of in the graph type.Finally, main calls nth with the pipeline.As we have seen before, the vertex structure is bound here so that its scope covers its uses both for spawns (in pipeline_pi) and for touches (in nth).The calls to both functions instantiate the vertex structure parameter with the same vertex structure , linking the spawns and touches in the graph types.The graph type for main composes the graph types of the producer and the consumer and links the spawns and touches by instantiating both graph types with the same vertex structure.

GRAPH TYPES WITH VERTEX STRUCTURES
This section provides a formal presentation of , whose syntax is given in Figure 6.In the remainder of this section, we describe the features of the language in detail, focusing on the main novelties of compared to prior work: vertex structures (VSs) and recursive types.

Vertex Structures and Their Types
Vertex Structures ( ) contain vertices that represent futures in computation graphs.As shown in Figure 6, VSs appear in annotations within expressions, type constructors, and graph types; these annotations are not inserted into real code by programmers, but are filled in during type inference.Vertex structures are classified with VS types (U).The VS type ♦ represents a single vertex, and only VSs of type ♦ can be used to name futures.The product type U 1 1 × U 2 2 represents pairs of VSs in U 1 and U 2 .The availability annotations 1 and 2 indicate whether the corresponding component is available (■) or unavailable (□) for spawning new futures; their use is inspired by record types in the Cogent language [O'Connor et al. 2021].The need for availability will become clearer later, when we discuss the type system.Finally, we can also form corecursive VS types .U, which we will use to generate graph types that require a potentially unbounded number of vertices.
Figure 7 presents the rules for assigning VS types to VSs.The judgment Ω; Ψ ⊢ : U denotes that the VS has VS type U, where Ω and Ψ are contexts that map VS variables to their types.Vertices, and thus VSs, are treated in an affine manner to ensure that any vertex is used at most once to spawn a future-this affine treatment leads to the use of two contexts.The first, Ω, is an affine context storing vertices that may be used to spawn futures and the second, Ψ, is an unrestricted context for vertices that may be used to touch futures (we may touch a vertex any number of times).Because we wish to be able to touch any vertex we spawn, the set of variables in Ω will always be a subset of that in Ψ.A VS variable is well-typed if it is in either Ω or Ψ, and we assume that Ω does not contain multiple mappings for the same variable.
VSs can be variables ( ),9 pairs, and projections.As seen in the rules U:Fst and U:Snd, only available components can be projected.For example, if has VS type U ■ 1 × U □ 2 , then fst is safe to use, but snd is not.Rule U:Subtype is a subsumption rule for the subtyping relation on VS types, denoted U ′ ⊑ U and defined in Figure 8.We allow three forms of subtyping: first (UT:Corec1 and UT:Corec2), we can freely roll and unroll corecursive VS types.Second (UT:ProdLeft and UT:ProdRight), it is safe to take an available component and treat it as unavailable.Third, the types of unavailable components of VSs may be changed at will, which is safe since those sides can never be used.
. Vertex structure type system for .
(UT:ProdLeft) (US:Prod) (OM:Empty)   The VS typing rule U:Pair uses an auxiliary splitting relation which is described in Figure 10.This relation is responsible for enforcing the affine treatment of Ω contexts.The judgment Ω ⇝ Ω 1 ⊞ Ω 2 states that Ω splits into the disjoint contexts Ω 1 and Ω 2 .It is important that Ω 1 and Ω 2 be disjoint so that futures spawned under Ω 1 and under Ω 2 have distinct vertices.However, we allow a variable with a product VS type to appear in both contexts, as long as the availability of the products is in turn split between the two.This is allowed by OM:VarTypeSplit: Ω, : U may split to Ω 1 , : U 1 and Ω, : Intuitively, U, U 1 , and U 2 are the same types but with different availabilities: if a component of a product VS type is available in U, then that component is available in U 1 or U 2 or neither, but not both.For example, if : U ■ 1 × U ■ 2 appears in Ω, we may have : in Ω 2 , but we cannot have : U ■ 1 × U ■ 2 appear in either Ω 1 and Ω 2 .The VS type splitting judgment U ⇝ U 1 ⊞ U 2 is defined in Figure 9.The core mechanism of VS type splitting is US:Prod, which states a VS type Most of the other rules are "search" rules allowing applications of US:Prod in nested VS types.The other significant VS type splitting rule is US:Subtype, which allows for "weakening" the VS types resulting from a split (by turning available sides of product VS types to unavailable).

Graph Types and Type Constructors
There are two kinding judgments to characterize well-formed types: one for graph types, and another one for type constructors.For graph types, the judgment Δ; Ω; Ψ ⊢ : states that the graph type has graph kind (Figure 11).The context Δ maps graph type variables to their kinds, and is used to check that recursive graph types are well-formed (see DW:RecPi).We show selected graph type formation rules for space reasons; others are similar and are given in the full version of the paper [Rinaldi et al. 2023].
For type constructors, the judgment Δ; Ψ; Υ ⊢ :: states that the type constructor is wellformed and has kind (Figure 12).The context Υ maps type variables to their kinds.Type constructors can be ordinary types, which are given the kind Ty (and for which we sometimes use the metavariable ).Types may also be parameterized by vertex structures.This allows, for example, a type of lists of futures which is parameterized by the VS providing the vertices for the futures. 10The 10 We could use the same type parameter mechanism to allow types to be parameterized by other types, as in the ML type 'a list, but this is orthogonal and we do not consider it in the formalism to streamline the presentation.(UE:FstPair) kind U → Ty classifies type-level functions that take a VS of type U and return a type constructor of kind Ty.Rule K:Lambda describes how to assign the kind U → Ty to such functions.
As mentioned earlier, the motivation behind is to allow recursive types containing futures.We achieve this by parameterizing recursive types by VSs containing the vertices for these futures.The syntax for a parameterized recursive data type is ( ; : U. ; ), where : U. is a type level VS function (equivalent to : U. ), is a recursive binding of : U. , and is the argument applied to : U. when the recursive type is unrolled.The formation of parameterized recursive types is performed by K:Rec, in which has kind U → Ty within since it represents a type-level function that passes a VS argument to the VS argument of the recursive instance (seen in more detail below).By applying a sub-VS of to an instance of in , the type ( ; : U. ; ) is able to recur over .(Note that non-parameterized recursive types do not require special syntax because we can parameterize them in a trivial way by using a dummy VS as the parameter.)We can now implement the type constructor for a list of integer futures as Because VSs can occur in types, type checking programs may require performing some typelevel computation, notably when projecting vertices out of a VS.To address this, we introduce two judgments: an equivalence judgment on VSs, Ψ ⊢ ≡ ′ : U (see Figure 13), and an equivalence judgment on type constructors, Δ; Ψ; Υ ⊢ 1 ≡ 2 :: (see Figure 14).Regarding VSs, the most notable rules are UE:FstPair and UE:SndPair, which extract (available) components of a pair.For type constructors, equivalence has two purposes: performing a type-level VS function application, which is performed by CE:BetaEq; and changing VSs within types to equivalent VSs according to the VS equivalence rules (CE:Future is given as an example).Other rules are relatively straightforward and are deferred to the full version of the paper [Rinaldi et al. 2023] for space reasons.

Graph Type System
Figure 15 presents the type system for , which assigns graph types (and types) to expressions.The judgment Δ; Ω; Ψ; Γ ⊢ : | states that the expression has type and graph type .In addition to the graph type context Δ and VS contexts Ω and Ψ, this judgment uses the context Γ which, as usual, maps expression variables to their types.
We give a quick overview of the graph type system: • represents expressions that execute purely sequentially; 1 ⊕ 2 represents expressions that execute an expression with graph type 1 followed by an expression with graph type 2 ; 1 ∨ 2 represents expressions that execute an expression with graph type 1 or an expression with graph type 2 ; . is a recursive graph type; represents spawning a future at vertex that executes an expression with graph type in parallel; and represents touching the future at vertex .A parameterized graph type Π[ : U ; : U ]. accepts two VSs as arguments ( has VS type U and is added to Ω and Ψ while has VS type U and is only added to Ψ).Such graph type functions are applied with the syntax [ ; ].Finally, new : U. binds a new VS variable of VS type U and represents expressions that do the same.
Rule S:Fun types function expressions fun[ ; ] = where is the name of the function, is an expression parameter, and are two VS parameters, and is the body of the function.Excluding bindings of new VS variables within , the only VS variable that can be used for spawning futures in is , while future touches can use any VS variable within the context (including and ), hence the function having two VS parameters.The type of functions is where 1 is the type of the parameter , 2 is the type of the function body, is the graph type of the function body, and ( .Π[ : U ; : U ]. ) [ ; ] is the graph type representing graphs produced by applying the function.The function type is parameterized by the VS parameters and (the same ones from the function expression), which have VS types U and U respectively.The graph type ( .Π[ : U ; : U ]. ) [ ; ] contains a recursive binding to a graph type function whose body is (note that this function binds a new, separate and within ), and this recursive binding is applied to the and bound within the type.This allows to pass different VS arguments to recursive instances of itself (this is useful, for example, when recursive instances access deeper levels of a vertex stream).In addition to adding the function expression's parameters to the context when typing the function body, we add , the recursive binding of the function; and , the recursive binding of the graph type function whose body is .Rule S:Type-Eq ensures that typing respects type constructor equivalence.Rules S:Roll and S:Unroll roll and unroll parameterized recursive data types: ( ; : U. ; ) unrolls to [ / ] [ ′ : U. ( ; : U. ; ′ )/ ] by applying to : U. (hence [ / ]) and then substituting itself recursively.Instead of replacing the with an instance of the recursive type, we replace it with a type-level VS function that passes its argument to the recursive type.

SOUNDNESS
The goal of this section is to prove the soundness of the graph type system for ; that is, that the computation graph of a program is described by its graph type.In order to prove this theorem, we must first formalize 1) the notion of a computation graph being "described by" a graph type and 2) the operational semantics by which a program evaluates to produce a computation graph.The first notion is one of normalization [Muller 2022], a process for constructing the set of computation graphs corresponding to a given graph type.The second is a cost semantics, which we present as a big-step semantics that evaluates an expression to a value and a computation graph.
The rest of this section is structured as follows.In Section 4.1, we discuss how we represent the creation of new vertices (which occurs during both normalization, as new bindings are normalized, and during evaluation, as they are evaluated).We then formalize normalization (Section 4.2), and finally present the cost semantics and prove soundness (Section 4.3).

Generation of Runtime Vertex Names
Recall that the constructs new : U. in graph types and new : U. in expressions bind "fresh" vertex structure (VS) variables to be used in the graph type and the expression, respectively.Because VSs can be infinite and of arbitrary type, some care must be taken in how to represent them at "runtime", i.e., in normalization and the cost semantics.The key insight is that VSs are (possibly infinite) trees with unique vertices at each leaf.It is thus possible to uniquely identify a vertex by the VS it comes from, and the path taken to reach it from the root of the VS.Paths in a vertex structure are already represented in our syntax as sequences of projections, e.g., fst snd , so most of the new conceptual work is in representing the roots of the vertex structures.
We use the syntax ì and variants to represent a unique vertex name, called a generator, which serves as the root of a VS gen( ì ).Generators are included in the contexts Ω and Ψ like VS variables, but are meaningful runtime symbols representing unique VSs.We will use the notation Ω • and Ψ • to refer to contexts that contain only generators, and no variables.These are the only contexts that will exist at runtime for typing top-level terms and graph types, as such terms and graph types contain no free variables.We refer to these terms and graph types as closed even though they may contain free vertex names in the form of generators.The judgments for VS typing and Ω context splitting are extended with rules for generators that resemble the rules for variables.
Equipped with a way to represent the roots of new vertex structures, we turn our attention again to paths from the root to a vertex.Currently, the same path can be represented in multiple ways, for example Ψ ⊢ fst (fst , snd ) ≡ fst : U.It will be useful to have a normal form for paths, so we introduce a normalization operation on VSs with the judgment ⇓ , defined in Figure 16 Proc.ACM Program.Lang., Vol. 8, No. POPL, Article 17. Publication date: January 2024.(UR:Seq1) . ↩→ [ ./ ] Fig. 19.Selected rules for graph type unrolling.(rules symmetric to these are omitted).Intuitively, the operation -reduces any projections of pairs (but leaves alone projections of vertex structures that are not syntactically pairs, e.g.fst gen( ì )).
We use the term vertex paths, and the notation VP, to refer to closed, normal VSs. Figure 17 extends the syntax for VSs with generators and gives the syntax for vertex paths.We now have a way of producing references to unique vertices: generators give rise to unique, non-intersecting VSs, and unique vertex paths in a given VS refer to unique vertices.When evaluating a new binding, we will simply create a fresh generator.The remainder of the vertices in the VS are then created implicitly, and will be accessed by the program as it traverses the VS.

Normalization
Figures 19-21 present the machinery for normalization.Because a recursive graph type represents an infinite set of graphs (as it can be unrolled any number of times), we stage the construction of these sets so that every set constructed is finite.Constructing a set of graphs consists of three operations, each of which performs some of the required tasks.First, recursive graph types are unrolled a desired number of times, yielding another graph type that is equivalent up to unrollings of recursive bindings.Next, the graph type is reduced to "New-normal form" (NBNF), which "evaluates" any exposed "new" bindings by generating and substituting fresh vertex structures.This process also performs any applicable reductions on exposed applications.At this point, we are left with a valid graph type, but one with no exposed "new" bindings or applications.Finally, the resulting graph type is expanded into the set of graphs; because there are no exposed "new" bindings, this process does not involve generating any new vertex names or structures.We will now discuss each of these steps in more detail.
Figure 19 gives a small-step semantics for unrolling recursive bindings in graph types.The bulk of the work is done by rule UR:Rec, which steps a binding .to [ ./ ].The remaining rules "search" the graph type for recursive bindings, so we defer most of these rules to the full version of the paper [Rinaldi et al. 2023].Note that, unlike in a standard left-to-right (or right-to-left) operational semantics, the rules UR:Seq1 and UR:Seq2 allow any instance of recursion in the type to be unrolled at any time in a nondeterministic fashion.For example, both steps below are valid: The rules for reducing to NBNF, given in Figure 20, eliminate "new" bindings by substituting fresh vertex structure generators, and perform any exposed applications.Evaluation proceeds recursively through sequential compositions and alternatives, but not under binders.As a result, closed sub-graph-types of NBNF graph types are themselves NBNF.
Finally, Figure 21 gives the rules for expanding a graph type into the set of graphs it represents.We represent a graph formally as a 4-tuple ( , , , ) containing the sets of vertices and edges , as well as a designated "start" vertex and "end" vertex .We use shorthands for combining graphs sequentially and in parallel; these shorthands use many of the same operators as graph type composition, but should not be confused.Figure 22 gives formal definitions for these shorthands; for more description of them, the reader is referred to prior work [Muller 2022].In brief, sequential composition ⊕ joins the end vertex of the first graph to the start vertex of the second graph.The "left composition" operator [Spoonhower 2009], written , adds a subgraph corresponding to a future to the graph, with an edge representing the spawn.It also adds the vertex as the sink of the future's graph.The "touch" operator adds an edge from .Sequential compositions are expanded by expanding both subgraphs, and then sequentially composing the resulting graphs.Alternation simply takes the union of the two sets of graphs.Expansion does not perform any additional unrolling, so the expansion of a recursive graph type is the empty set of graphs.Expansion of the future is performed by left-composing all of the resulting graphs with the vertex , and touches simply expand to the singleton graph Fig. 22. Shorthands for combining graphs.consisting of the touch.Note that because NBNF has already expanded all "new" bindings, there is no rule for expanding these, and expansion does not generate new vertex names.The latter is a key property in guaranteeing that expansion results in well-formed graphs.These three operations combine to form a normalization process that is correct: any set of graphs that results from unrolling, normalizing, and expanding a well-formed graph type is well-formed.This result is formalized by Theorem 1, which is proven in the full version of the paper alongside several necessary technical lemmas.

Cost Semantics and Soundness
We equip with a cost semantics, a big-step operational semantics that evaluates an expression and also produces the computation graph that represents the execution.The judgment is ⇓ | , meaning that expression evaluates to value , producing the cost graph .The rules for this judgment are in Figure 23, and the syntax for values are in Figure 17.In C:Future, the body of the future is evaluated (in a real execution, the body of the future will be evaluated in parallel, but the big-step cost semantics deliberately abstracts away evaluation order) and the future evaluates to a handle, a new syntactic form which records the result of the future.In addition, we evaluate the vertex structure used to spawn the future to a vertex path VP, which is recorded by the handle.The C:Touch rule extracts both the vertex path and future result from the handle.In C:New, a new generator ì is created and used to generate a vertex structure which instantiates the variable .
The soundness theorem for the graph type system of is that if a program has a graph type and evaluates to produce a graph , then is described by (that is, should be in the set of graphs obtained by normalizing using the machinery in the previous subsection).This is stated formally as Theorem 2. The formal statement of the theorem also includes a context Ψ • containing generators created during execution which may be captured in the result value .The proof of the theorem, as well as statements and proofs of several necessary technical lemmas, appears in the full version of the paper [Rinaldi et al. 2023].These lemmas include: • A number of substitution results for expressions, types, vertex structures, etc.

ELABORATION OF RECURSIVE TYPES WITH VERTEX STRUCTURES
Thus far, we have presented the annotated language containing recursive data types ( ; : U. ; VP), annotated with a vertex path VP of type U that provides vertex names for data structures of the recursive type.We have motivated that VP should have a structure that in some sense "maps on" to the recursive structure of the list so that any futures in the structure have a corresponding vertex name.As examples, a list data type corresponds to an infinite stream of vertices, and a binary tree data type corresponds to an infinite binary tree of vertices.As discussed, the annotated language is provided merely as a core calculus for expressing the ideas of the graph type system; the annotations can be inferred from unannotated code by our implementation.
Other than the addition of vertex structures, the general structure of the algorithm for inferring these annotations is similar to that of GML [Muller 2022], and the details of the algorithm are largely outside the scope of this paper.However, one important and non-obvious fact for inferring annotations for is that it is indeed possible to annotate any recursive data structure with a corresponding vertex path.Showing this fact is the goal of this section.We do so by defining a set of rules for annotating unannotated types and values with vertex structure annotations.For simplicity, the system we present in this section is declarative and not algorithmic, so it still abstracts away many of the complexities of our inference algorithm, but we show that the rules are complete and thus that any recursive type may be so annotated.
We first define a syntax for unannotated types and unannotated values .
Unannotated types consist of the unit type, functions, products, and sums, as well as an unannotated future type and an unannotated recursive type.Note that the annotation of functions is orthogonal to the annotation of recursive data types; we assume that function types and function values have already been annotated and include annotated function types and annotated function values as unannotated types and unannotated values, respectively.The unannotated future type future is similar to the annotated future type future[ ] but is not annotated with a VS.Similarly, the unannotated recursive type .binds a type variable but does not bind a VS variable and does not take a VS as an argument.Because unannotated types do not interact with vertex structures, there is no type-level lambda and all unannotated types have kind Ty (and so we do not distinguish between "unannotated types" and "unannotated type constructors").Unannotated values differ from values only in that future handles are not annotated with vertex paths.
Figure 24 defines the judgment Υ ⊢ ; U.This indicates that may be annotated to the type constructor (which will, by construction, have kind Ty).It also returns a vertex structure type U that "corresponds" to the type .For recursive types ., the VS type U is the type of the VS annotation for the recursive type (that is, .will be annotated to be ( ; : U. ; ) for some and some ).As an example, if is the type of int future lists, then U will be (equivalent to) the type of vertex streams, .♦■ × ■ .The judgment takes a type variable context Υ mapping type variables to kinds (these will be annotated types and so their kinds will not be Ty).It is also parameterized by a vertex structure to use for annotations.When annotating a closed unannotated type , this parameter will simply be instantiated with a fresh vertex path gen( ì ) to derive • ⊢ gen( ì ) ; U. The returned type would be annotated with projections of gen( ì ).The returned VS type U would be the type that ì should be assigned in order for to be well-kinded.
Rule F:TyVar looks up the type variable in the context.By construction, its kind will be of the form U → Ty, indicating that to properly annotate the use of the variable , it must be applied to a VS of type U. We use the VS for the annotation and return the type U as the required type of .The unit and function types do not require additional annotations, and so are simply returned.11Rule F:Prod takes a VS and annotates the first component 1 with the left projection of and the second component 2 with the right projection.The type required for is thus the product of the two returned types.Rule F:Sum, somewhat counterintuitively, also returns a product of the two VS types.This is because if a data structure can take one of two forms, the corresponding VS must offer either possibility.12Rule F:Fut takes to be a product whose second component is a single vertex, which it uses to annotate the future; the first component is used to annotate the future's return type.Finally, rule F:Rec annotates a recursive type . .It begins by adding to the context with kind .U → Ty (this is the only truly non-algorithmic feature of these rules; we do not discuss how to construct U).With this context, it annotates .The resulting VS type is rolled back into the corecursive type .U, which is the type required for .
Example.We can represent the type of a list of integer futures as an unannotated type : Using the rules of Figure 24, we can infer the following annotation for : The VS corresponding to is a stream of vertices (note that because we treat VS types equicorecursively, the VS type above is equivalent to .♦■ × ■ but unrolled slightly).In the body of the recursive annotated type, which is unit + (int future[fst snd ] × (snd snd )), the first vertex of the stream is discarded (this is an effect of mapping the type unit to the VS type ♦ even though it does not need a vertex), the second vertex of the stream (the first vertex of the tail) is used for the future and the remainder (the tail of the tail) is passed to the recursive instance of the type.
The judgment described above declaratively shows a correspondence between unannotated types and the vertex structure types required to annotate them.Later in this section, we show that this relation is complete with respect to well-kinded unannotated types, and thus that any type has a corresponding VS type.We next wish to show that a VS of the returned VS type actually does suffice to provide all necessary vertices for a data structure of the given type.We do this using  another judgment that annotates unannotated values.This judgment is defined in Figure 25 and takes the form ⊢ VP , where VP is a vertex path to use for annotation (similar to the type annotation judgment above), is an unannotated value, and is the annotated value.We restrict annotations of values to vertex paths since the value handle[VP] may only use vertex paths VP as the handle.Otherwise, annotation of values proceeds in much the same way as annotation of types.Rule FE:Pair uses the two components of VP to annotate the components of the pair.Rules FE:InL and FE:InR use the first and second components, respectively, of VP to annotate left and right injections (recall that, for a sum type, VP is given a product type so that the two components of VP may be used for the two injections).Finally, just as F:Fut uses the first component of to annotate the type of the future's payload and the second component as the vertex for the future, rule FE:Handle uses fst VP to annotate the payload and snd VP to annotate the handle itself.
Example.Consider the list of integer futures from above.We claimed that the correct VS type for this type is U = .♦■ × ■ .The rules of Figure 25    As described above, the futures take consecutive odd vertices from the stream VP.
The main result of this section has three components.First, any well-kinded unannotated type may be matched with an annotated type by the rules of Figure 24.Second, if a well-kinded unannotated type is annotated with a VS of the VS type returned by the annotation judgment, then the annotated type is also well-kinded.Third, if an unannotated type is annotated with a vertex path (that is, if • ⊢ VP ; U), then any well-typed unannotated value of type may be annotated with VP by the rules of Figure 25, and the annotated value is well-typed when VP has type U.Moreover, to show that VP has "enough" vertices to fully annotate the value with unique vertices, we show that the annotated value is well-typed under a new typing judgment that uses only an affine context for vertices.Usually, values would be typed with the unrestricted context Ψ, because a data structure is allowed to contain multiple handles to the same future, but in this case, we wish to show that we can restrict data structures to use new vertices for each handle.We write the new judgment Ω ⊢ : .The rules are similar to the standard typing rules, but use the affine context Ω for typing handles.This rule for typing handle values is given in Figure 26.Values always have the graph type •, so we omit the graph type from the judgment.The remaining rules are straightforward and are deferred to the full version of the paper [Rinaldi et al. 2023].
Theorem 3 formalizes the main result of this section, that is, that 1) type annotation is complete with respect to well-kinded unannotated types, 2) type annotation annotates well-kinded unannotated types into well-kinded types, and 3) annotating values with vertex structures of the returned VS type results in well-typed values.In order to show this, we introduce a kinding judgment for unannotated types, Υ ⊢ :: Ty, and a typing judgment for unannotated values, ⊢ : .The rules for these judgments are similar to those for annotated types and expressions, so we defer them to the full version of the paper [Rinaldi et al. 2023].Since the kinds of type variables bound by unannotated and annotated recursive types are different (Ty and U → Ty respectively), we need some way to change the kinds that these type variables are bound to.We address this with the functions Unann and Ann.Unann(Υ) takes a context Υ suitable for annotating types and kinding annotated types (where type variables can, and will always, have kind U → Ty), and return an unannotated context, one suitable for kinding unannotated types (where every type variable has kind Ty).Ann(Υ) performs this process in reverse, where the VS type expected by every type variable in Υ is a fresh VS type variable unique to that type variable (each which can be substituted with the desired VS type).
The proof of Theorem 3, as well as statements and proofs of several necessary technical lemmas, appears in the full version of the paper [Rinaldi et al. 2023].

IMPLEMENTATION AND EXAMPLES
We have implemented a prototype graph inference algorithm for on top of GML [Muller 2022], an existing graph type inference algorithm.The goal of the implementation, which we call GML , is to infer vertex structure annotations and graph types from ordinary, unannotated OCaml programs.GML extends OCaml syntax with the keywords future for spawning expressions into a future, touch for joining a future handle's value to the current thread, and a type 'a future.Additionally, GML supports OCaml's user-definable recursive datatypes, which were not previously supported by GML (there are some limitations, which we discuss at the end of this section).For example, we can define the 'a pipe type from Sections 1 and 2 using standard OCaml syntax: type 'a pipe = Pipe of 'a * 'a pipe future ;; Our extension of GML successfully infers the corresponding vertex structure annotations, for the type itself and for all of its uses in the code in Figure 5.
In addition, we implemented (by extending facilities existing in GML) a visualizer that uses several heuristics to generate a visualization of a representative graph corresponding to each inferred graph type. 13This allows developers to see at a glance how their program will parallelize.We have used GML to infer graph types for all example programs in this paper.
The details of the implementation are out of the scope of the paper.However, the main challenge in extending GML with support for algebraic data types is generating the VS type corresponding to an ADT.Our algorithm for this closely follows the presentation of Section 5. 14 When processing a type declaration, GML generates the associated VS type, and also generates a constructor and deconstructor function for each constructor.Constructor applications are desugared to ordinary applications of the constructor function and the deconstructor function is used during pattern matching.Another major challenge is implementing unification on vertex structures.At the moment, our implementation uses a set of heuristics that are not guaranteed to be complete (i.e., unification may fail for VSs that could be unified, resulting in a spurious type error) but work well in practice on the large examples tested.
In addition to extending the subset of OCaml supported by GML, we have also substantially re-architected the code.In GML , the graph type checker is completely separate from the type checker.This simplifies the implementation and has a number of other benefits.First, all futures in a program are known by the time graph checking begins.This allows the implementation to infer graph types in several instances where type annotations would previously have been required (one such instance is noted in prior work [Muller 2022] as a limitation of GML, which is not a limitation of GML ).Additionally, this architecture would simplify the process of integrating graph checking as an extension of the OCaml compiler, as an additional pass on type-checked ASTs.

Examples
To show the utility of GML , we discuss several example programs for which it can infer and visualize graph types.
Produce-consume.The producer-consumer example of Blelloch and Reid-Miller [1997], shown in Figure 28, is similar to the pi_pipeline function of Figure 5, but allows the pipelined list to be finite (ending with FNil).As in the pipeline example, the FCons constructor allows the tail of the flist to continue being computed in a future.We compose produce, which (for the sake of a simple example) outputs a list of the numbers 1-n, with the consume function which calculates the sum of the list.In the graph of the composed functions (right side of the figure), the touches of consume happen in parallel with the production of the list.
Tree Sum.In Figure 29, we present operations on a pipelined tree data structure [Blelloch and Reid-Miller 1997].As with flist, the two subtrees of an ftree are futures, so they may be computed asynchronously while the value at the node is used.The function bst generates a tree of numbers 0 to 10, then tree_sum calculates the sum of elements in the generated tree.While the particular application of summing a binary tree is fairly simple, one can imagine using the same structure for more complicated use-cases.Because of the design of the data structure, bst immediately returns a future and then tree_sum can perform its calculation as later recursive steps of bst are still executing.
Tree Reverse.The function in Figure 30 reverses a pipelined tree of the type defined in Figure 29.Here, the interesting feature of the output was not the visualization of the function's graph type, which shows a similar structure to tree_sum and bst, but the function type, which is reverse : Π[ : vtree; : vtree].ree − → ree where vtree = .× ♦ × × ♦.We omit the graph type for clarity.The function takes two VS parameters and a tree indexed by and returns a tree indexed by .At first glance, this may seem imprecise because one might expect the VSs parameterizing the input and output tree to be related (after all, the output tree is the reverse of the input).However, this is not correct: reverse  touches (in a pipelined way) all of the futures of the input tree and constructs a new tree with the reversed values but new futures from the VS .Here, not just the graph type but the return type parameterized by its VS can correct a misunderstanding about the parallel behavior of a program.

Limitations
Though our inference algorithm checks most useful programs, there are some limitations.First, polymorphic types cannot be instantiated with types that include futures.For example, a list of futures would have to be explicitly defined as a new type 'a futlist rather than by instantiating built-in lists to form the type 'a future list.This represents a design trade-off; graph types are not currently expressive enough to represent, say, a reverse function on lists of futures.If the standard list type could be instantiated with a future, the polymorphic reverse function would need to be assigned a general enough type to cover all instantiations of 'a, which wouldn't be possible.A limitation we inherit from GML is that functional arguments of higher order functions cannot spawn futures.This is possible in , but cannot be inferred without annotations.

RELATED WORK
Graph Types and Related Analyses.The use of graphs to represent the parallel programs dates back to at least the late 1960s [Karp and Miller 1966;Rodriguez Bezos 1969].Our notation is most directly inspired by the work of Blelloch andGreiner [1995, 1996] and Spoonhower [2009], who extended these graphs with notations for futures.In this work, graphs were produced dynamically from programs using a cost semantics, which abstractly evaluates the program to form the graph (or a family of graphs if execution is nondeterministically).The first work we are aware of on statically approximating such graphs for fine-grained parallel programs was the prior work of one of us [Muller 2022], which developed a calculus and corresponding graph type system for inferring graph types of parallel programs with futures.Our work builds on , including the use of an affine type system to ensure that vertex names are unique and therefore do not appear twice in a graph, which would result in an invalid graph.However, the main thrust of this paper is overcoming the significant limitation in that affine treatment of vertex names prevents building collections of futures.
Dependency graphs are frequently used to represent control dependencies in coarse-grained parallel programs and these have been the target of several static analyses (e.g., [Chen et al. 2002;Cheng 1993;Kasahara et al. 1995]) but such tools do not contend with the substantial dynamicity inherent in fine-grained parallel programs, especially those with futures.Dependency graphs are also used to represent other dependencies in a program, including data dependencies; analyzing the structure of such dependencies is a form of program slicing (e.g., [Korel 1987;Weiser 1984]).
As observed in prior work [Muller 2022], graph type systems draw on ideas from region type systems [Tofte and Talpin 1997], where assigning a vertex to a future corresponds to allocating an object within a region of memory, in order to aid in memory management and/or ensure safety (e.g.[Fluet et al. 2006]), including in the presence of concurrency and complex, dynamic data structures [Milano et al. 2022].It is not possible to list all of the related work on regions and related systems, so we refer the interested reader to the chapter by Henglein et al. [2005].Two major differences with region systems are that vertex assignments must be unique (whereas typically many objects are allocated within a single region) and that, to generate useful graphs, we wish for vertex assignments to be visible at a global scope (see the example from the Introduction of why locally allocated vertices are not suitable for graph types of data structures).
Heterogeneous and Indexed Data Structures.Indexed types [Xi and Pfenning 1999;Zenger 1997], a limited form of dependent types in which a type is indexed by a value from a specified domain, have long been used to add expressiveness to types-a classic example is a type of vectors indexed with a natural number giving the vector's length.We index recursive data types with a vertex structure to assign unique vertices to futures in recursive data structures.Vertex structures have a non-trivial semantics of their own but are not first-class objects at the expression level, so computation on VSs may be seen as an instance of type-level computation.A similar indexing idea and type-level computation have been previously combined to achieve heterogeneity in HList [Kiselyov et al. 2004], a Haskell library that expresses heterogeneous lists by indexing the list type constructor with a type-level list.Their work does not appear to generalize beyond lists or to infinite indices.
Richer forms of type-level computation have been explored, and could be used to further generalize the theory of vertex structures.Yorgey et al. [2012] extend Haskell's kind system with features for expressing a variety of type-level data structures.As another example, we have considered extending vertex structures with sums (so a vertex structure could, e.g., represent a finite list) and using type-level matching [Blanvillain et al. 2022] to constrain the lengths of lists by the length of the vertex structure parameter.While this would expand the expressiveness of vertex structures, extending VSs beyond tree-like corecursive structures causes a problem for inference and so it seems likely that such an extension would require programmer annotations in some cases.
We note that this paper enters a rich design space of combining data and codata (e.g.[Thibodeau et al. 2016]).We have shown in Section 5 that any data type can be "overapproximated by" a codata type in the sense that there is a straightforward, local mapping from nodes in the data type's AST to those of the codata type (the vertex structure in our case).Whether this has a deeper meaning in the theory of data and codata is left to future work.
Affine Type Systems.We use an affine type system to handle vertex structures and ensure that vertex names in output graphs are unique.Affine type systems have been used in a number of languages to ensure safe usage of resources (broadly construed), notably including Cyclone [Jim et al. 2002] and Rust [Rus [n. d.]].Our notation for splitting and availability is inspired by Cogent [O'Connor et al. 2021], which uses these ideas for an affine treatment of record types.
Encoding in Rust.As a memory safety focused language, Rust's type system would likely benefit from the features GML.Though Rust is able to encode other type systems such as session types [Jespersen et al. 2015;Lagaillardie et al. 2020], we do not believe GML could be usefully or at least conveniently encoded in Rust as is.The main limiting factor we foresee is that we believe each vertex name would need its own lifetime.This is a problem because Rust requires all lifetimes to be declared statically, and each piece of code can only refer to finitely many lifetimes.However, in GML, vertex names are generated dynamically, therefore a function might manipulate infinitely many vertices.

CONCLUSION
We have presented a type system for annotating parallel programs with futures with graph types, which compactly represent the parallel structure of the program.Unlike prior work, we support complex data structures containing futures.As evidenced by our prototype implementation, it is possible to infer these graph types automatically for examples that use this feature for efficient pipelined algorithms.Our implementation is also able to generate visualizations of the resulting graph types, which can aid in understanding the structure of parallel code and finding bugs.
In the future, we hope to expand on the types of program analysis and bug-finding that can be done with these graph types.For example, we could build on the prototype analyses of Muller [2022] to study deadlock and asymptotic complexity in the complex, pipelined graphs that arise from programs with data structures of futures.We also plan to scale the implementation up to support a larger subset of OCaml, with the goal of integrating the analysis into the OCaml compiler.Finally, because the graph type system itself does not depend on a particular source language, we plan to explore implementing the graph type system in front-ends for a variety of languages, so that more programmers can benefit from graph types as an analysis tool and reasoning aid.
Fig. 1.Two implementations of a function that iteratively computes with futures.

Fig. 2 .
Fig. 2. Visualizations for list_pi (le ) and pipeline_pi (right) showing the differences in parallelization strategies.In both figures, the node corresponding to the touch operation in main is circled in red for emphasis. 5

Fig. 3 .
Fig. 3. Code and types for parallel-recursive icksort using futures.Code annotations in gray are shown for convenience; these are not wri en by the programmer.

Fig. 5 .
Fig. 5. Code and types for a function that iteratively approximates indefinitely in a pipelined manner.
Fig. 17.Extended syntax for generators, vertex paths, and values
provide a "recipe" for constructing a future list using a vertex path of VS type U.As an example, consider the unannotated value roll inr (handle 1, roll inr (handle 2, roll inl ⟨⟩)) Proc.ACM Program.Lang., Vol. 8, No. POPL, Article 17. Publication date: January 2024.