A few weeks ago, Jorge Ortiz gave a good talk at BASE about actors. I want to summarize a bit of that and explain why I wrote naggati, and why I think mina-plus-actors is a big deal.
The history
In the 90s, we server coders wrote daemons using the fork/exec process model. Apache still does it this way, though you can switch to threaded models in the current version.
Having a bunch of processes makes coordination difficult (you have to use shared memory, pipes, or something similar) but works well for a web server where there isn’t really any shared state except configuration, and you can just respawn new processes if the configuration changes.
For a chat server, you’d like a lot of shared state and communication between the various sessions. So in the late 90s (early 2000s on Linux), we started using threads. Each incoming connection spawns a new thread.
Each session is exactly one thread. Because threads all share the same memory space, communication between sessions is easy. You just have to make sure to use locks and condition variables, and to get the locking logic right – which can be tricky.
When you start to have a lot of sessions, the number of active threads begins to get a little unmanagable. At a past job, we had a daemon that would handle between 3000-4000 sessions this way. We couldn’t push it past that, because the OS wasn’t able to allocate stack space for more than about 4000 threads, and the task switching overhead was getting pretty ridiculous anyway.
Most sessions are blocked waiting for new data most of the time, so in the mid-2000s, server coders migrated to a thread-pool model.
Java even included a good thread pool library in java 5. In this model, you
can have thousands of open sessions or socket connections, but you farm out
work across a smaller pool of threads. Usually you have one or two threads in
a socket poll() call (or Selector in java) across all the open sockets,
and when incoming data arrives, it posts a work item into a queue, which gets
handled by the next available thread from the pool.
Here’s where mina comes in
Mina takes this thread-pool pattern and pulls it
out into a reusable library. You can register a socket with mina and get an
IoSession object back. There’s a Selector running in its own thread, and
when an I/O event happens on one of your registered sockets, mina calls an
event method on your IoHandler, passing in the IoSession. These methods
represent events like “new data has arrived” or “some data you sent has been
successfully transmitted”, and they are called from within a thread pool, or
other executor of your choosing.
The great thing mina adds to this mix is the idea of a “protocol decoder”.
Instead of handling I/O events in your session and doing stateful buffering
and protocol decoding there, mina lets you write a ProtocolFilter that
receives incoming data as a byte buffer. This filter can do the buffering and
decoding, and pass decoded objects on to the session. For example, a web
server could have a decoder that takes byte buffers and generates
HttpRequests.
The protocol decoding happens in a special set of I/O processor threads
maintained by the mina library, so your IoHandler only gets events for fully
decoded objects, and the handler code can be small and simple. In the web
server example, an IoHandler would receive HttpRequest objects as events
that run in a worker thread from a thread pool. The repsonse can be created
and queued for writing, and then the event is over. Since the queued work
items are small, discrete tasks that finish quickly, many active sessions can
be handled across a much smaller number of threads.
There are only two big speedbumps with this system:
You’re using threads, so you need to be very careful about locking and all the other threading issues.
The protocol decoder is isolated into its own filter class, but it still has to be stateful and buffered. So it will still be ugly and gnarly.
Enter actors…
Actors solve problem 1. There are several good articles about actors out there, so I won’t belabor the point. With actors, you can write code in a way similar to writing threaded code, but all communication happens through message passing. The actor library promises that any actor will only be running in one thread at a time, so you can avoid using locks entirely if you want. Shared state can be passed around as “messages” that are immutable objects.
There’s a pretty clear one-to-one mapping between the events that mina would
like to notify you about, and messages passed to an actor. It’s
straightforward enough that I put it into naggati as a helper class called
IoHandlerActorAdapter.
It creates an IoHandler that handles every event by sending a corresponding
message to the actor handling that session. (For example, the
messageReceived method causes MinaMessage.MessageReceived to be sent.)
This line in kestrel is all that’s needed to tell mina to create a new actor for each incoming connection:
acceptor.setHandler(new IoHandlerActorAdapter(session => new KestrelHandler(session, config)))
…and naggati
Mina’s written in java, so they can’t do much about the way protocol filters
need to be written, beyond giving you some tools to store state between calls
and some base classes you can use to get implementations of some common bits
(like reading a line). But scala’s syntax supports attaching code blocks to
method calls as Function objects, so we can write code that looks sequential
but is actually a series of continuations.
Naggati is a library that makes it easy to build protocol filters for mina 2.0 using this sequential style. A DSL allows you to write the decoder in sequence, but each decoding step is actually a state in a state machine, and when one step is finished decoding, it calls the code block for the next step.
As an example, let’s say there’s a simple protocol that sends packets. Each packet starts with an (ascii) count of the number of bytes in the packet, followed by a linefeed, and then the actual bytes. (This is how HTTP chunked encoding works.) We could write the decoder like this:
case class DataBlob(data: Array[Byte])
val decode = readLine { line =>
readByteBuffer(line.toInt) { data =>
state.out.write(DataBlob(data))
End
}
}
val decoder = new Decoder(decode)
The first line just defines a case class for the decoded objects we’re going
to be throwing to an actor inside a MinaMessage.MessageReceived event.
The decode step is where it gets interesting. It just calls readLine with
a block of code that will process that line when it’s read. readLine returns
a Step object that contains that code block and some logic for reading bytes
from mina’s buffers until a linefeed is found. Once a complete line has been
read, readLine passes it to the saved code block.
In this example, that code block just contains another call (readByteBuffer)
which returns a Step that reads a certain number of bytes and then passes
them on to the attached code block. The nested code block takes that byte
array and sends it back to mina as a decoded object, to be handed off to a
session actor. End is a special state object that means decoding is
complete, and the decoder should reset and start over at the beginning again.
We’ve written a tiny state machine here, which buffers data as it arrives and progresses through each state as the previous state is completed. But we didn’t have to write it that way. By passing code blocks around, we got to write a very simple, sequential description of the protocol without worrying about callbacks or buffers.
There are a couple of more in-depth examples in the naggati source code, including an ASN.1 tag decoder, and a simple HTTP request decoder. They’re too long to quote here, but they demonstrate things like using a value from one decoded state to help decode a later state. Kestrel is another good example, because it uses nagatti to decode memcache requests in a pretty small chunk of code.
With actors, mina, and naggati, I think the promise of a simple programming model for servers with thousands of connections is here. You can spend your time thinking about and coding up the actual logic of a server, and not worry so much about how to juggle all those sockets and threads.
Hopefully in this tiny space I’ve convinced you too.