Erlang & Server-Sent Events: made for each other

In the HTTP world, having real-time applications implies server-push technologies, and for long time doing it properly was painful because not well understood by our browsers.

But with the rise of modern technologies like Websockets and Server-Sent Events, we can build serious HTTP based (soft) real-time applications.

Today, we will see how to use Erlang super powers for the good of the real-time Web. For this, we will write a Server-Sent Event API for Cowboy.

About Server-Sent Events

In a previous blog post, François told you a bit about Server-Sent Events. But for those of you who did not follow so far, here is a little recap.

Server-Sent Events (SSE) is a simple but not well known specification which comes with HTML5. Unlike WebSockets, implementing Server-Sent Events is trivial because you don't need to UPGRADE your request.

The only thing your server have to know is how to stream data. And guess what? Every Erlang HTTP server is capable of that!

Here is an example of what a SSE response looks like:

HTTP1/1 200 OK
Content-Type: text/event-stream

data: an event

data: a new one

As you see, it's a every simple protocol:

Respond to the request with a text/event-stream Content-Type
Prefix your streamed events with a data: string

On the client side the javascript API is very simple too:

var source = new EventSource("http://example.com/streaming")

source.onmessage = function(message) {
    console.log("got a new message:", message.data)
}

Which should ouputs in your console:

got a new message: an event
got a new message: a new one

The Cowboy example

So let's start with an Erlang example. For this we will write a Cowboy handler:

-module(eventsource_emitter).
-behaviour(cowboy_http_handler).
-export([init/3, handle/2, terminate/2]).

init({_Any, http}, Req, []) ->
    {ok, Req, undefined}.

handle(Req, State) ->
    Headers = [{'Content-Type', <<"text/event-stream">>}],
    {ok, Req2} = cowboy_http_req:chunked_reply(200, Headers, Req),
    handle_loop(Req2, State).

handle_loop(Req, State) ->
    {ok, Req, State}.

terminate(_Req, _State) ->
    ok.

For the moment the handler does nothing. It's a basic skeleton which responds by a chunked reply and the right Content-Type. You certainly have noticed the handle_loop/2 function which does not belong to the behaviour. This is where the event streaming will occur.

In Cowboy, each request has its own lightweight process. We want this process to wait for Erlang events and send them to the browser. Moreover, we want the process to stop when it receive a shutdown message. So our handle_loop/2 method now looks like the following code:

handle_loop(Req, State) ->
    receive
        shutdown ->
            {ok, Req, State};
        Message ->
            Event = ["data: ", Message, "\n\n"],
            ok = cowboy_http_req:chunk(Event, Req),
            handle_loop(Req, State)
    end.

Ok, so we now have a proper handler which is able to listen for external string events and send them to the browser.

But who will send these Erlang messages to our process? As we are writing an example here, we'll keep it simple and set 2 timers when initializing our handler:

The first is a recurrent timer which sends our messages
The second is a simpler timer which sends a shutdown message to stop the streaming.

So we modify the init/3 function according to our specs:

init({_Any, http}, Req, []) ->
    timer:send_interval(1000, "Tick"),
    timer:send_after(10000, shutdown),
    {ok, Req, undefined}.

The handler is now complete. But what our example does exactly?

First, it sends a Tick message every second to the browser.
After 10 messages (i.e. 10 seconds) it closes the connection as the process terminates.
Then on the client side, the browser waits for a few seconds (depending on your browser) and retries a connection. This re-open an event stream and everything happens again.

Here the benefits of Erlang are obvious, because each connection has its own process and is able to accept messages from other processes (another client connection for instance). Think of this as a start for higher applications like pubsubs or queues.

See it in action

This example is part of the cowboy_examples repository, so feel free to run it an see it in action:

$ git clone https://github.com/extend/cowboy_examples
$ cd cowboy_examples
$ make
$ ./start

Then open a browser to http://localhost:8080/eventsource and inspect the javascript source.

Going further

The point of this article was to show you how it's damn simple to implement Server-Sent Events with Erlang.

But don't forget that it's a richer API. Among other:

Ids: attach ids to your events and detect disconnections
Type: attach types to your events and listen for these specific ones from the Javascript API
Retry: modify the interval between reconnections

To dig a little deeper, you can take a look at those links: