Using Thrift with HTTP Transfer-Encoding As3 and Java

A few months ago I started to look for something that could take a place of BlazeDS for my flash game. BlazeDS is nice but I had a few concerns:

1. BlazeDS by default uses threaded IO. Each connection uses one thread. If you want to use Java NIO and hence better performance you should use the commercial offering.
2. BlazeDS is open source, but I encourage everyone to look at the code, debug a call stack and make their own judgement. At minimum a faulty config file should not cause a Nullpointer exception.
3. Finally BlazeDS, AMF is very much As3->Java only. While AS3 and Java is what I use today, for multi platform game development having different language bindings is a good things to have.

So I started to look around and came across Google Protocol Buffer. Cool product, I wish I could be google cool using it but sadly the AS3 port really has very little traction. More importantly it was a little too complicated to construct my own As3 port. Luckily there is a Google Protocol Buffer Diet version, called Thrift. After reading the history of Thrift, it is fair to say I was not the only one who found Google Protocol Buffer a little too feature rich and more complex than needed.

Incidentally there was a pretty functional Thrift As3 port. It was a patch much like with Google but it was more complete patch, and more importantly a straight port of Java which made it easy to follow and debug.

Using Thrift AS3 101.

By default the AS3 port only uses HTTP as transport with a class called HTTPClient. Which is fine because Flash and raw socket connections are troublesome to manage anyway with Flash 10 security sandbox. Also most game platform has some method of "web access" which is pretty hard to do without HTTP.

HttpClient uses a POST request to send a service request in the body, and the service response in the body of the HTTP response. Like this:

My problem was that with small messages this is a fairly wasteful transport. Since Thrift can condense the message into a nice little payload, often times the HTTP headers end up bigger than the payload. Plus unless you use Keep-Alive, the connection will be created and destroyed every time.

Next step I started to wonder. HTTP is just text over a socket, a few headers in request, a few headers in response but after that is exchanged, you get input and output on both side.. Can I just tunnel a socket connection over HTTP?

Not exactly.

1. The URLConnection will not let you write to a connection after you read.
2. The As3 URLConnection/Loader really does not have semantics for such operation.

So is that means I can not tunnel over HTTP? No, I just have to follow the HTTP spec.

Thank to a really good explanation for Rex Young I found HTTP/1.1 Chunked Encoding .

The good news is you can tunnel over HTTP (I know ADOBE did it). Jetty as a Servlet container handles the Chunk encoded request by letting you write and read the streams in a single invocation of service(). The bad news it default Java/Flash http clients do not support it AFAIK. So I had to take Rex's example make my own Transport handler to get a Java client (just to test). For Flash I had to create and process and HTTP request over Socket.

The result:

I know some of your reading might say, HTTP header, socket opening not that expensive, to worth the hassle. I would agree, but it depends on the use case. For my int:getTime() benchmark it was 40ms for chunked request, 100ms for all new post request each time. Therefore it depends on the frequency,size and acceptable latency of the messages.

In my case for games with lot of small messages, with low latency HTTP chunk encoding is a significant performance improvement.