When an Actor stops, its children stop in an undefined order. Child termination is asynchronous and thus non-deterministic.

If an Actor has children that have order dependencies, then you might need to ensure a particular shutdown order of those children so that their postStop() methods get called in the right order. This pattern approaches that problem by introducing an Actor type that we’ll call a Terminator.

The Goal

The goal is to provide control over how the “children” die. In order to achieve this goal, we are going to change the surface of the problem so that we no longer have these children, in the normal sense.

Fig2: This is what we’re going to build. We’ve removed the Workers from their Master and put them under a Terminator. The Terminator now uses the Master as a hook to trigger its business logic; the business of killing things.

We’re going to remove the responsibility of terminating the Workers from the Master and delegate that to the Terminator. The Master still has whatever role it used to have, but that isn’t relevant to this pattern. What is relevant is the relationship between the Terminator and the Master. Termination of the Workers happens as a result of the Master’s death event.

The tricky part is in getting the Workers instantiated and providing references to them, but it’s really not that big of a deal.

The Master

The Master is initialized with an ActorRef to the Terminator, which has already been instantiated. On its preStart() it will ask for the children of the Terminator, after which it can go about its business.

The key point here is to make sure you understand the issues surrounding supervision and potential restart. If the Master restarts, then its Mailbox becomes interesting.

Fig3: The usual rules apply to a restart when the Actor is stateful. The messages that are queued in the Mailbox need to be dealt with in some manner.

If the Master restarts then there may be a number of client requests sitting in its Mailbox that are going to be processed once it is finished restarting. However, the Children response coming from the Terminator won’t get there until all of the pre-existing requests have been flushed through the Master. Not good.

If your Master is subject to a restart strategy then you need to deal with this somehow. The standard rules apply:

• Don’t care.
• Use the stash to stash these away until you’re ready to deal with them.
• Tell the client it should come back later.

Or, there’s another option, which may or may not be decent depending on your problem:

• Block in preStart(). Instead of saying surrogate ! GetChildren(self), say Await.result(surrogate ? GetChildren(self), 5.seconds) and store the children in a var instead of become()ing a new state with the data passed to it.

It’s up to you. If your situation warrants a (seemingly) short thread-block and it honestly is the best solution to the problem, then go for it.

One thing that should be mentioned here is that the notion of using a PriorityMailbox to solve the problem might be a bad idea. You could conceivably use a PriorityMailbox that ensures the Children message gets hanlded first, but there’s a race condition here.

If you have 500 messages in the Mailbox, and then send the GetChildren request to the Terminator, you may end up processing 47 messages from the Mailbox before you get the Children response. A PriorityMailbox can only prioritize what it sees. PriorityMailbox can be a bit of a tempting solution in these situations but, in the world of non-determinism, it’s not usually the silver bullet you might think it is.

The Worker

For completeness, we’ll include the worker, but it’s really silly.

The Terminator

The Terminator is an abstract class that implements a simple protocol between itself and the Master. It’s only real job is to shut down the children in a deterministic order, specified by the implementation.

The key here is the killKids() method. We use the gracefulStop pattern, which stops an Actor such that the Future it returns completes only after the Actor’s postStop() method has been called. We can thus chain these together using flatMap and then signal the Terminator when the terminations are complete. This ensures that any side-effects processed by the postStop() occur in the intended order. And that order is presented to the Terminator by the derivation’s implemenation of the order() method.

Testing the Terminator

Things are easiest to illustrate with testing. We’ll create a TestWorker, which is given an ActorRef to the TestKit’s testActor so we can verify death order. We’ll instantiate the Terminator, give it a preStart() method that creates 20 TestWorker instances, and an order() method that provides the order in which we want to kill them. We then use a TestProbe to sit in for the Master, and put it through its paces. When we kill the probe, we verify the resulting deaths by waiting for messages from the kids.

Extensions

The interesting issue here concerns life-cycle. If you want the Master (assuming you actually need one) to start up before the Workers then you would need to alter the implementation of the Terminator to something like this:

The relationship between the Terminator and the Master remains the same (i.e. the Terminator starts first and is given to the Master) but the life-cycle between the Master and the Workers, with respect to start, is inverted.

Conclusion

This pattern illustrates an interesting use of the Actor model to enhance the Actor model. Often times we might worry that a particular desire simply isn’t possible because it needs to be provided by Akka itself. However, the Actor model is so flexible that we can generally provide what we need by the simple application of another level of indirection. You just have to decompose your problem, as you would with any paradigm in software development.

This pattern also achieves its goal in a non-blocking manner, with respect to killing the kids. We could have chosen to Await.result(...) instead of using gracefulStop and built a synchronous algorithm as a result, but there’s no need to. Akka delivers on asynchrony; it’s our job to wield it correctly.

About the Author

Derek Wyatt is a Software Developer and Architect at Primal, helping to create Interest Networks using graphs and other cool stuff. He is also the author of an upcoming book on Akka to be published by Artima Publishing. You should go get a copy when it comes out… really. It’ll have pictures! Nothing dirty, though…