An Exceptional Actor System (Functional Pearl)

The Glasgow Haskell Compiler is known for its feature-laden runtime system (RTS), which includes lightweight threads, asynchronous exceptions, and a slew of other features. Their combination is powerful enough that a programmer may complete the same task in many different ways -- some more advisable than others. We present a user-accessible actor framework hidden in plain sight within the RTS and demonstrate it on a classic example from the distributed systems literature. We then extend both the framework and example to the realm of dynamic types. Finally, we raise questions about how RTS features intersect and possibly subsume one another, and suggest that GHC can guide good practice by constraining the use of some features.


Introduction
Together with its runtime system (RTS), the Glasgow Haskell Compiler (GHC) is the most commonly used implementation of Haskell [Fausak 2022].The RTS is featureful and boasts support for lightweight threads, two kinds of profiling, transactional memory, asynchronous exceptions, and more.Combined with the base package, a programmer can get a lot done without ever reaching into the extensive set of community packages on Hackage.
In that spirit, we noticed that there is nothing really stopping one from abusing the tools throwTo and catch to pass data between threads.Any user-defined datatype can be made into an asynchronous exception.Why not implement message-passing algorithms on that substrate?
We pursued this line of thought, and in this paper we present an actor framework hidden just under the surface of the RTS.The paper is organized as follows: -Section 2 provides a concise summary of asynchronous exceptions in GHC and the actor model of programming.-Section 3 details the implementation of our actor framework.We first show how actors receive messages of a single type, and then extend the framework to support dynamically typed actors, which receive messages of more than one type.-Section 4 shows an implementation of a classic protocol for leader election using our actor framework.We then extend the actors with an additional message type and behavior without changing the original implementation.-We reflect on whether this was a good idea in Section 5, by considering the practicality and performance of our framework, and conclude in Section 6 that asynchronous exceptions might be more constrained.
This paper is a literate Haskell program.1

Brief background
In this section, we briefly review the status of asynchronous exceptions in GHC (Section 2.1) and the actor model of programming (Section 2.2); readers already familiar with these topics may wish to skip this section.Readers unfamiliar with the behavior of throwTo, catch, or mask from the Control.Exception module may wish to first scan the documentation of throwTo in GHC Contributors [2021].

Asynchronous exceptions in GHC
The Glasgow Haskell Compiler (GHC) is unusual in its support for asynchronous exceptions.Unlike synchronous exceptions, which are thrown as a result of executing code in the current thread, asynchronous exceptions are thrown by threads distinct from the current one, or by the RTS itself.They are used to communicate conditions that may require the current thread to terminate: thread cancellation, user interrupts, or memory limits.
Asynchronous exceptions allow syntactically-distant parts of a program to interact in unexpected ways, much like mutable references.A thread needs only the ThreadId of another to throw a ThreadKilled exception to it.The standard library function killThread is even implemented as (\x -> throwTo x ThreadKilled).2There is no permission or capability required to access this powerful feature.
Asynchronous exceptions are peculiar because they aren't constrained to their stated purpose of "signaling (or killing) one thread by another" [Marlow et al. 2001].A thread may throw any exception to any thread for any reason.This absence of restrictions means that standard exceptions may be reused for any purpose, such as to extend greetings: (\x -> throwTo x $ AssertionFailed "hello").Even user-defined datatypes may be thrown as asynchronous exceptions by declaring an empty instance of Exception [Marlow 2006].For example, with the declarations in Figure 1, it is possible to greet in vernacular: (\x -> throwTo x Hi).
Asynchronous exceptions may be caught by the receiving thread for either cleanup or, surprisingly, recovery.An example of recovery includes "inform[ing] the program when memory is running out [so] it can take remedial action" [Marlow et al. 2001].The ability to recover from a termination signal seems innocuous, but it leaves asynchronous exceptions open to being repurposed.

The actor model
The actor model is a computational paradigm characterized by message passing.Hewitt et al. [1973] write that "an actor can be thought of as a kind of virtual processor that is never 'busy' [in the sense that it cannot be sent a message]."In our setting, we interpret an actor to be a green thread3 with some state and an inbox.When a message is received by an actor, it is handled by that actor's intent function.An intent function may perform some actions: send a message, update state, create a new actor, destroy an actor, or terminate itself.Unless terminated, the actor then waits to process the next message in its inbox.We will approximate this model with Haskell's asynchronous exceptions as the mechanism for message passing.
More concretely, we think of an actor framework as having the characteristics of a concurrency-oriented programming language (COPL), a notion due to Armstrong [2003].After describing our framework, we will make the case (in Section 5.1) that it has many of the characteristics of a COPL.To summarize Armstrong [2003], a COPL (1) has processes, (2) which are strongly isolated, (3) with a unique hidden data Greet = Hi | Hello deriving Show instance Exception Greet Figure 1.Show and Exception instances are all that is required to become an asynchronous exception.identifier, (4) without shared state, (5) that communicate via unreliable message passing, and (6) can detect when another process halts.Additionally, (5a) message passing is asynchronous so that no stuck recipient may cause a sender to become stuck, (5b) receiving a response is the only way to know that a prior message was sent, and (5c) messages between two processes obey FIFO ordering.While an actor system within an instance of the RTS cannot satisfy all of these requirements (e.g., termination of the main thread is not strongly isolated from the child threads), we will show that our framework satisfies many requirements of being a COPL with relatively little effort.

Actor framework implementation
In our framework, an actor is a Haskell thread running a provided main loop function.The main loop function mediates message receipt and makes calls to a user-defined intent function.Here we describe the minimal abstractions around such threads that realize the actor model.These abstractions are so minimal as to seem unnecessary; we have sought to keep them minimal to underscore our point.

Sending (throwing) messages
To send a message, we will throw an exception to the recipient's thread identifier.So that the recipient may respond, we define a self-addressed envelope data type in Figure 2 and declare the required instances.
Figure 3 defines a send function, sendStatic, which reads the current thread identifier, constructs a self-addressed envelope, and throws it to the specified recipient.For the purpose of explication in this paper, it also prints an execution trace.

Receiving (catching) messages
An actor is defined by how it behaves in response to messages.A user-defined intent function, with the type Intent shown in Figure 2, encodes behavior as a state transition that takes a self-addressed envelope argument.
Every actor thread will run a provided main loop function to manage message receipt and processing.The main loop function installs an exception handler to accumulate messages in an inbox and calls a user-defined intent function on each.Figure 3 defines a main loop, runStatic, that takes an Intent function and its initial state and does not return.It masks asynchronous exceptions so they will only be raised at well-defined points within the loop: during threadDelay or possibly during the Intent function.
The loop in Figure 3 has two pieces of state: that of the intent function, and an inbox of messages to be processed.The loop body is divided roughly into three cases by an exception handler and a case-split on the inbox list: (1) If the inbox is empty, sleep for an arbitrary length of time and then recurse on the unchanged actor state and the empty inbox.
(2) If the inbox has a message, call the intent function and recurse on the updated actor state and the remainder of the inbox.(3) If, during cases (1) or (2), an Envelope exception is received, recurse on the unchanged actor state and an inbox with the new envelope appended to the end.
In the normal course of things, an actor will start with an empty inbox and go to sleep.If a message is received during sleep, the actor will wake (because threadDelay is defined to be interruptible), add the message to its inbox, and recurse.
On the next loop iteration, the actor will process that message and once again have an empty inbox.Exceptions are masked (using mask_4 ) outside of interruptible actions so that the bookkeeping of recursing with updated state through the loop is not disrupted.
Unsafety.Before moving forward, let us acknowledge that this is not safe.An exception may arrive while executing the intent function.Despite our use of mask_, if the intent function executes an interruptible action, then it will be preempted.In this case the intent function's work will be unfinished.Without removing the message currently being processed, the loop will continue on an inbox extended with the new message.The next iteration will begin by processing the same message that the preempted iteration was, effecting a double-send.
To avoid the possibility of a double-send, a careful implementor of an actor program might follow the documented recommendations for code in the presence of asynchronous exceptions: use software transactional memory (STM), avoid interruptible actions, or apply uninterruptibleMask.However, recall that message sends are implemented with throwTo, which is "always interruptible, even if it does not actually block" [GHC Contributors 2021].A solution is obtained "by forking a new thread" [Marlow et al. 2001] each time we run an intent function, but this sacrifices serializable executions -an actor must be safe to run concurrently with itself.We opt for the simple presentation in Figure 3 > >= loop

Dynamic types
The actor main loop in Figure 3 constrains an actor thread to handle messages of a single type.An envelope containing the wrong message type will not be caught by the exception handler, causing the receiving actor to crash.We think the recipient should not crash when another actor sends an incorrect message. 5n this section, we correct this issue by extending the framework to support actors that may receive messages of different types.With this extension, our framework could be thought of as dynamically typed in the sense that a single actor can process multiple message types.This is similar to the dynamic types support in the Data.Dynamic module.
Furthermore, any actor may be extended by wrapping it ("has-a" style) with an actor that uses a distinct message type and branches on the type of a received message, delegating to the wrapped actor where desired. 6It may seem natural to encapsulate such actor-wrapping in combinators that generalize the patterns by which an actor is given additional behavior.However, here our goal is not to lean into the utility of a dynamically typed actor framework, but to point out how little scaffolding is required to obtain one from the RTS.

Sending dynamic messages. Instead of sending an
Envelope of some application-specific message type we convert messages to the "any type" in Haskell's exception hierarchy, SomeException [Marlow 2006].Figure 4 defines a new send function that converts messages, so that all inflight messages will have the type Envelope SomeException.

Receiving dynamic messages.
On the receiving side, messages must now be downcast to the Intent function's message type.This is an opportunity to treat messages of the wrong type specially.In Figure 4 we define a new main loop, runDyn, that lifts any intent function to one that can receive envelopes containing SomeException.If the message downcast fails, instead of the recipient crashing, it performs a "return to sender." Specifically, it throws an exception (not an envelope) using the built-in TypeError exception. 7hese changes do not directly empower actor intent functions to deal with messages of different types.We have only removed application-specific type parameters from envelopes.Actors intending to receive messages of different types will do so by downcasting from SomeException themselves.Such actors will use an intent function handling messages of type SomeException.We will see an example of this usage pattern in Section 4.2.

Safe initialization
When creating an actor thread, it is important that no exception arrive before the actor main loop (runStatic in Figure 3) installs its exception handler.If this happened, the exception would cause the newly created thread to die.To avoid this, the fork prior to entering the main loop must be masked (in addition to the mask within the main loop).
Figure 5 defines the main loop wrapper we will use for examples in Section 4. It performs a best-effort check and issues a helpful reminder to mask the creation of actor threads.

Example: Ring leader election
The problem of ring leader election is to designate one node among a network of communicating nodes organized in a ring topology.Each node has a unique identity, and identities are totally ordered.Nodes know their immediate successor, or "next" node, but do not know the number or identities of the other nodes in the ring.A correct solution will result in exactly one node being designated the leader.This classic problem from the distributed systems literature serves to illustrate our actor framework, despite leader election being unnecessary in the context of threads in a process.Chang and Roberts [1979] describe a solution to the ring leader election problem that begins with every node sending a message to its successor to nominate itself as the leader (Figure 6).Upon receiving a nomination, a node forwards the nomination to its successor if the identity of the nominee is greater than its own identity.Otherwise, the nomination is ignored.We implement and extend that solution below.

Implementing a leader election
Each node begins uninitialized, and later becomes a member of the ring when it learns the identity of its successor.To represent this we define two constructors in Figure 7 for node state type, Node. Figure 6.In-progress ring leader election with seven nodes (Chang and Roberts' 1979 solution).The node identities are unique and randomly distributed.Two nomination chains are shown: Node 5 nominated itself and was accepted by nodes 3, 1, and 4; next node 4 will nominate 5 to node 6 (who will reject it).Concurrently, node 6 nominated itself and was accepted by node 2 but rejected by node 7.For this election to result in a leader, node 7 must nominate itself.Three messages (also defined in Figure 7 as type, Msg) will be used to run the election: -Init: After creating nodes, the main thread initializes the ring by informing each node of its successor.-Start: The main thread rapidly instructs every node to start the leader election.-Nominate: The nodes carry out the election by sending and receiving nominations.
4.1.1Election termination.The node with the greatest identity that nominates itself will eventually receive its own nomination after it has circulated the entire ring.That same node will ignore every other nomination.Therefore the algorithm will terminate because node identities are unique and only one nomination can circumnavigate the ring.9 4.1.2Node-actor behavior.The intent function for a node actor will have state of type Node and receive messages of type Msg, as defined in Figure 7.We show its implementation and describe each case below.
node :: Intent Node Msg When an uninitialized node receives an Init message, it becomes a member of the ring and remembers its successor.8.It takes the size of the ring and an unevaluated IO action representing node behavior, and then performs the following steps to start an election: 10 (1) Create actors (with asynchronous exceptions masked).
(2) Randomize the order of actor ThreadIds.11 (3) Inform each actor of the ThreadId that follows it in the random order (its successor) with an Init message.(4) Send each actor the Start message to kick things off.To call the election initialization function, we construct an IO action by passing the node intent function and the initial node state to the actor main loop from Figure 5: An election execution trace appears in Figure 9.

Extending the leader election
The solution we have shown solves the ring leader election problem insofar as a single node concludes that it has won.However, it is also desirable for the other nodes to learn the outcome of the election.Since it is sometimes necessary to extend a system without modifying the original, we will  show how to extend the original ring leader election to add a winner-declaration round.
Since there is no message constructor to inform nodes of the election outcome, we will define a new message type whose constructor indicates a declaration of who is the winner.We will extend the existing node intent function by wrapping it with a new intent function that processes messages of either the old or the new message types, with distinct behavior for each, leveraging the dynamic types support described in Section 3.3.The new behaviors are: -Each node remembers the greatest nominee it has seen.
-When the winner self-identifies, they will start an extra round declaring themselves winner.-Upon receiving a winner declaration, a node compares the greatest nominee it has seen with the declared-winner.
If they are the same, then the node forwards the declaration to its successor.
Extended nodes will store the original node state (Figure 7) paired with the identity of the greatest nominee they have seen.This new extended node state is shown in Figure 10 type 4.2.1 Declaration-round termination.When an extended node receives a declaration of the winner that matches their greatest nominee seen, they have "learned" that that node is indeed the winner.When the winner receives their own declaration, everyone has learned they are the winner, and the algorithm terminates.

4.2.2
Exnode-actor behavior.The intent function for the new actor will have state Exnode and receive messages of type SomeException.This will allow it to receive either Msg or Winner values and branch on which is received.

exnode :: Intent Exnode SomeException
Recall the implementation of the actor main loop function, runDyn from Figure 4.When we apply exnode to runDyn, the call to fromException in runDyn is inferred to return Maybe SomeException, which succeeds unconditionally.The exnode intent function must then perform its own downcasts, and we enable ViewPatterns to ease our presentation.There are two main cases, corresponding to the two message types the actor will handle, which we explain below.
The first case of exnode, shown in Figure 11, applies when an extended node downcasts the envelope contents to Msg.In each of its branches, node state is updated by delegating part of message handling to the held node.We annotate the rest of Figure 11 as follows: (1) Delegate to the held node by putting the revealed Msg back into its envelope and passing it through the intent function, node, from Section 4.1.2.(2) If the message is a nomination of the current node, start the winner round, because the election is over.(3) Otherwise, the election is ongoing, so keep track of the greatest nominee seen.
The second case of exnode applies when a node downcasts the envelope contents to a winner declaration.Its implementation is shown in Figure 12.If the current node is declared winner, the algorithm terminates successfully.If the greatest nominee the current node has seen is declared winner, the node forwards the declaration to its successor.State is unchanged in each of these branches.

Extended election initialization.
The extended ring leader election reuses the initialization scaffolding from before (Figure 8).The only change is that the IO action passed to ringElection initializes the greatest nominee seen to itself, prior to calling run.It is called like this: A trace of an extended election appears in Appendix A.8.
5 What have we wrought?
Figure 3 shows that we have, in only a few lines of code, discovered an actor framework within GHC's RTS that makes no explicit use of channels, references, or locks and imports just a few names from default modules.The support for dynamic types, shown in Figure 4 as separate definitions, can be folded into Figure 3 for only a few additional lines. 12e find it intriguing that this is possible and shocking that it is so easy.

Almost a COPL
In Section 2.2 we described an actor framework as having the characteristics of a concurrency-oriented programming language (COPL) [Armstrong 2003].Which of the COPL requirements does our framework satisfy?Here we review the criteria listed in Section 2.2: (1) ✓ Threads behave as independent processes.
(2) ✗/✓ Threads are not strongly isolated because termination of the main thread terminates all others.However, if the main thread is excluded as a special case, then the set of other threads are strongly isolated.(3) ✓ ThreadID is unique, hidden, and unforgeable.(4) ✗ Threads may have shared state.
(5) ✗ Asynchronous exceptions do not behave as unreliable message passing.( 6) ✓ An actor can reliably inform others when it halts using forkFinally.
The message-passing semantics of our actor framework is nuanced.Documentation for the interfaces we use indicates that the framework provides reliable synchronous message passing with FIFO order.We call it synchronous because "throwTo does not return until the exception is received by the target thread" [GHC Contributors 2021]. 13This means that a sender may block if the recipient is not well-behaved (e.g., its intent function enters an infinite loop in pure computation).We distinguish well-behaved intent functions, which eventually terminate or reach an interruptible point, from poorly-behaved intent functions, which do not.Assuming intent functions are well-behaved, the framework will tend to exhibit the behavior of reliable asynchronous message passing with FIFO order and occasional double-sends, because senders will not observe the blocking behavior of throwTo.By wrapping calls to the send function with forkIO [Marlow  et al. 2001], we can achieve reliable asynchronous message passing without FIFO order even in the presence of poorlybehaved intent functions.14FIFO can then be recovered by message sequence numbers or by (albeit, jumping the shark) use of an outbox thread per actor.With those caveats in mind, our framework mostly satisfies Armstrong [2003]'s criteria for message-passing semantics: (5a) ✗/✓ A stuck recipient may cause a sender to become stuck, unless senders use forkIO or we assume the recipient is well-behaved.
(5b) ✗/✓ Actors know that a message is received (stored in the recipient inbox) as soon as send returns.However, they do not know that a message is delivered (processed by the recipient) until receiving a response.(5c) ✓/✗ Messages between two actors obey FIFO ordering, unless forkIO is used when sending.
Our choice to wrap a user-defined message type in a known envelope type has the benefit of allowing the actor main loop to distinguish between messages and exceptions, allowing the latter to terminate the thread as intended.At the same time, though, this choice runs afoul of the name distribution problem [Armstrong 2003] by indiscriminately informing all recipients of the sender process identifier.One strategy to hide to an actor's name and restore the lost security isolation is to wrap calls to the send function with forkIO.Another strategy would be to define two constructors for envelope, and elide the "sender" field from one.
We claim that our actor framework is almost a COPL.It also meets our informal requirements that actors can send and receive messages, update state, and spawn or kill other actors (though we have not shown examples of all of these).However, we do not mean to imply that our actor framework is practical; we merely mean to point out that it is, indeed, an actor framework.

Summary of performance evaluation
We have described a novel approach to inter-thread communication.We believe it is prudent to compare the performance this unintended communication mechanism against the performance of an intended communication mechanism to restore a sense that the ship is indeed upright.To that end, we re-implemented the extended ring leader election from Section 4 using channels -a standard FIFO communication primitive.We also implemented a "control" 15 to establish a lower bound on the expected running time of the actor-based and channel-based implementations.
We compared the running time of these implementations at ring sizes up to 65536 nodes on machines with 8, 32, and 192 capabilities.We also compared their total allocations over the program run at various ring sizes.
Our running time results (Figure 13a) show that the actorbased implementation is significantly slower than the channelbased implementation for ring sizes less than 8192 nodes, but surprisingly, it is marginally faster for more than 32768 nodes.The total-allocations result (Figure 13b) shows that allocations made by the channel-based implementation catch up to that of the actor-based implementation at large ring sizes, and we hypothesize that this convergence explains why the running time results swap places.Additionally, our results show that the running time of the extended ring 15 The "control" forks some number of threads that do nothing and immediately kills them.leader election algorithm is invariant to the number of capabilities used by the RTS, making it a poor choice for a general evaluation of our actor framework, but sufficient for our purpose of confirming that channels are faster.
Appendices A.2 to A.5 give the source code for these benchmarks.Appendix A.6 details our experimental setup, and Appendix A.7 discusses more of the results.

Conclusion
Can we implement an actor framework with Haskell's threads and asynchronous exceptions?Our implementation and results show that we can, and this fact hints that perhaps asynchronous exceptions are at least as general as actors.
However, the actor framework we present is not an advancement: It is easy to use, but easy to use wrongly.It has acceptable throughput, but is slower than accepted tools.It requires no appreciable dependencies, no explicitly mutable data structures or references, no effort to achieve synchronization, and very little code only because those things already exist, abstracted within the RTS.
Should it have been possible to implement the actor framework we present here?Like many people, we choose Haskell because it is a tool that typically prevents "whole classes of errors, " and also because it is a joy to use.But in this paper we achieve dynamically typed "spooky action at a distance" with frighteningly little effort.Perhaps the user-accessible interface to the asynchronous exception system should be constrained.
More broadly, with the 9.6.1 release of GHC, a user of the RTS enjoys software transactional memory, asynchronous exceptions, delimited continuations (and extensible algebraic effects), and more, together in the same tub.The water is warm -jump in!Will all the members of this new extended "awkward squad" [Peyton Jones 2001] bob gently together, or will they knock elbows?Which of them can be implemented in terms of the others, and should their full power be exposed so that we can do so?We hope the reader will draw their own conclusions.The reason we aren't using message passing to notify about termination is because it is difficult to communicate between the "actor world" and the "functional world."The functional world expects IO actions to terminate with return values, but we didn't bother to implement clean termination in our actor framework.Lacking that, we could try spawning an actor and then setting up an exception handler to receive messages from it, but we choose not to do this because of the potential for race conditions.
We benchmark time to termination using the criterion package.For this, we need an IO action that executes the algorithm, cleans up its resources, and then returns.The function benchActors does this: it runs an election with benchmark-nodes, waits for termination, kills the nodes, and asserts a correct result.

A.3 Control benchmark implementation
The experimental control, benchControl, only forks threads and then kills them.It is useful to establish whether or not, for example, laziness has caused our non-control implementations to perform no work.The other implementations should take longer than the control because they do more work.It is unnecessary to split the channel-based implementation into a simple node and an extended node, but we split them anyway to ease comparison to the actor-based implementation.This structural similarity hopefully has the added benefit of focusing benchmark differences onto the communication mechanisms instead of anecdotal differences.
In chanNode we implement the main loop.The only state maintained is the greatest nominee seen.It leaves off with definitions of communication functions in its where-clause.

Figure 2 .
Figure 2. Message values are contained in a self-addressed envelope.Actor behavior is encoded as a transition system.

Figure 3 .
Figure 3. Message sends are implemented by throwing an exception.Actor threads run a main loop to receive messages. 8

data
Figure 7. Election nodes can be in one of two states, and they accept three different messages.
Figure10.Extended nodes store node state alongside the greatest nominee seen.They accept one message in addition to those in Figure7.

Figure 11 .Figure 12 .
Figure11.When exnode receives a Msg, it delegates to node.It may also update the greatest nominee seen or trigger the winner-declaration round.
The channel-based implementation is faster than the actorbased implementation, except at very large numbers of threads.We reproduced this result on machines with 8, 32, and 192 capabilities.size (MacBookPro11,5; +RTS -N8) (b) The growth of allocations by the channel-based implementation eventually catches up to that of the actor-based implementation.

Figure 13 .
Figure 13.Representative selection of experimental results.