RPC framework evaluation

Network protocol design is hard to get exactly right, and networking/serialization code has high standards of correctness – all the other code in a project depends on the right data getting sent across the wire. If you're trying to communicate between programs written in multiple languages, that code has to be written (and tested, and maintained) separately in each language.

RPC frameworks offer a nice solution to this problem: write the interface you want and your message definition in some Interface Definition Language (IDL), and then compile it to working (and hopefully idiomatic) code in a multitude of languages.

However, there are drawbacks:

• We don't control the implementation, which means it is hard to optimize for performance, and harder to add new features later.
• many of the RPC frameworks don't support push notifications from the server to client (implemented by some as "streaming").
• Generally, IDLs require static type definition. This means that we would have to define remote procedure calls as only taking one type. The only really workable approach would be to take a byte array serialized with some other data serializer and pass that through the RPC. This reduces the usefulness of an RPC framework somewhat, as it can't handle everything. However, having function stubs and having the transport taken care of is still quite useful, especially as the code works in multiple languages.

"Server-Client push" in the table below refers to the ability of the server to push messages back over the same communication channel the client uses to connect to it. BERT, for example, can take an address to deliver a message to when some remote call is done, but it has to be able to connect to that server. Many Geode users use firewalls, which would cause issues connecting to the client. Also, it's more adapted to the actor model than to a client-server model, as we use.

Testing Setup

All perf tests were run on a  4x 2.4Ghz (4-Core), 48Gb RAM.

Two sets of performance tests were run:

• 1x server with 1 client doing 10mil blocking put messages
• 1x server with 2 clients doing 10mil blocking put messages each

Each test was run with the same message structure and key/value size.

Key Size: 64bytes

Value Size: 512bytes

The put message definition for each framework are linked in the table:

The results can be found in Results.zip. This archive file contains all the results for both testing scenarios, including GFS files for more detailed stats.

Results

1 Server, 1 Client (1 Thread)

1 Server, 2 Client (1 Thread)