Alternative Erlang bindings for Thrift

At Amie Street we use Thrift, an open source project from Facebook, to offload various services from our PHP frontend to backends written in various other languages. Last week we released our first backend service written in Erlang, using the facebook-provided bindings.

At the beginning of this week we began to work on our backlog of improvements for this service. In working on these improvements, I realized that the Thrift bindings for Erlang were written as a fairly close translation of the bindings in object oriented languages, to the point of an OOP emulation layer (thrift_oop_server). This resulted in a lot of code that was both confusing for developers to read through and inefficient for the Erlang VM to execute.

Since Erlang is quick to write, I spent the last couple of days rewriting the bindings, and have the new version online in a git repository. Here's the list of notable new features:

• Improved idiomatic style - the whole thing is more "Erlangy" -- there are less gratuitous processes, and things are named thrift_binary_protocol rather than tBinaryProtocol


• Improved performance - as a result of ditching the OOP layer, the new bindings are at least 5x faster than the old.


• More robust code generation - the old version generated broken code from the ThriftTest.thrift file in the repository. The new one passes the tests.


• Bug fixes - for example, the old version did not properly serialize/unserialize doubles


• No unexpected crashing processes - the old thrift bindings crashed at the end of every connection when the client disconnected. The new version only crashes when something goes wrong
  

Hopefully we'll be able to get these new bindings merged upstream in the next couple of weeks. But if you use Erlang and Thrift, please check it out and try it on your project. And if you don't use Erlang and Thrift, it may be worth writing a quick server to see how useful it really is!

Reloading all code from the Erlang shell (take two)

So, it turns out that the snippet from earlier doesn't quite work right.

Thanks to #erlang on freenode, I found out that you can add commands to the shell by putting them in the user_default module. So here's the improved version:

In ~/erl/user_default.erl

-module(user_default).

-export([la/0]).

la() ->
    Modules = [M || {M, P} <- code:all_loaded(), is_list(P) andalso string:str(P, "amiest") > 0],
    [shell_default:l(M) || M <- Modules].

Then compile that module using c(user_default) from within ~/erl.

In your ~/.erlang:

code:load_abs("/home/amiest/erl/user_default").

Reloading all code from the Erlang shell

One of the reasons Erlang is pretty cool is that it supports hot code reloading. With OTP (Open Telecom Platform) applications, this can be done through a complicated mechanism involving release files, code_change callbacks to update state, and special upgrade/downgrade scripts.

In development, however, that's a bit of a nuisance. So, how do you simply reload all of the loaded modules after you've recompiled them? Here's a quick solution to paste into your ~/.erlang:

Reload = fun(M) ->
                 code:purge(M),
                 code:soft_purge(M),
                 {module, M} = code:load_file(M),
                 {ok, M}
         end.
 
ReloadAll =
 fun() ->
         Modules = [M || {M, P} <- code:all_loaded(), is_list(P) andalso string:str(P, "/home/amiest") > 0],
         [Reload(M) || M <- Modules]
 end.

Obviously, replace /home/amiest with a directory within which your modules reside. Then, simply run ReloadAll(). from the shell to load the newest versions of your modules.