...
RPC frameworks offer a nice solution to this problem: write the interface you want and your messaging to take message definition in some Interface Definition Language (IDL), and then compile it to working (and hopefully idiomatic) code in a multitude of languages.
...
- We don't control the implementation, which means it is hard to optimize for performance, and harder to add new features later.
- many of the RPC frameworks don't support push notifications from the server to client (implemented by some as "streaming").
- Generally, IDLs require static type definition. This means that we would have to define remote procedure calls as only taking one type. The only really workable approach would be to take a byte array serialized with some other data serializer , and pass that through the RPC. This reduces the usefulness of an RPC framework somewhat, as it can't handle everything. However, having function stubs and having the transport taken care of is still quite useful, especially as the code works in multiple languages.
...
RPC Framework | Supported Languages | Serialization | Transport Layer | Server-Client push? | Dynamic type definition? | Throughput (msg/s) | Comments |
---|---|---|---|---|---|---|---|
Avro | at least Java, Python, Ruby, C#, C++ | Binary or JSON | transport-agnostic | No | Yes | ||
BERT | scroll to the bottom on the website to see. NO native Java support (though there is Scala). | Erlang | transport-agnostic or BERP (custom) | No | No | ||
gRPC | 10 (incl. C++,Java,C#,Go) | Protobuf | Http2, Java uses Netty | Yes | No | ||
Apache Thrift | quite a few (incl. C++,Java,C#, (unofficial) Go) | Custom | Pluggable: TTransport interface. | No | No | ||
Raw Socket with Serialization Framework | See supported languages of the serialization framework | Protobuf, Avro, Cap n Proto | SocketChannel | No | No |
Testing Setup
All perf tests were run on a 4x 2.4Ghz (4-Core), 48Gb RAM.
Two sets of performance tests were run:
- 1x server with 1 client doing 10mil blocking put messages
- 1x server with 2 clients doing 10mil blocking put messages each
Each test was run with the same message structure and key/value size.
Key Size: 64bytes
Value Size: 512bytes
The put message definition for each framework are linked in the table:
Framework | IDL Message definition files |
---|---|
Avro (using raw socket) | client.avdl |
Cap n Proto (using raw socket) | protocol.capnp |
Protobuf (using raw socket) | clientProtocol.proto region_API.proto |
GRPC + Protobuf | grpcService.proto clientProtocol.proto region_API.proto |
The results can be found in Results.zip. This archive file contains all the results for both testing scenarios, including GFS files for more detailed stats.
Results
1 Server, 1 Client (1 Thread)
Protobuf | Avro | Cap n Proto | GRPC Streaming | GRPC Blocking | |
---|---|---|---|---|---|
Median (msg/s) | 37593.98496 | 21012.81782 | 23245.00232 | 47370.91426 | 6931.447979 |
Average (msg/s) | 37040.22695 | 20541.22109 | 23182.0977 | 45986.4068 | 6868.469895 |
1 Server, 2 Client (1 Thread)
Protobuf | Avro | Cap n Proto | GRPC Streaming | GRPC Blocking | |
---|---|---|---|---|---|
Median (msg/s) | 65307.3915 | 40355.25818 | 43851.95917 | 88949.4447 | 11974.32611 |
Average (msg/s) | 65471.2589 | 40375.60386 | 43770.10225 | 83513.36694 | 11933.20999 |