Page History

Introduction

Apache Geode is a data management platform that provides real-time, consistent access to data-intensive applications throughout widely distributed cloud architectures. While it currently has high-speed client interfaces for Java, C++ and .NET there is both a need to create lighter-weight clients and a demand to access Geode from other programming languages. Unfortunately the the existing client-server protocol is undocumented.  It evolved over time and is overly complex to meet either of these needs.

This document describes a new, lighter-weight client/server protocol that may be implemented in phases, moving from a "basic client" to a more advanced "smart client".  It endeavors to provide a protocol that is also more amenable to more modern APIs such as those using asynchronous or reactive patterns.

You will notice that this document describes the protocol down to the level of byte-ordering and "bytes on the wire".  This is a description of the native serialization of the protocol that will be used for the initial implementation.  It is our intent to make this pluggable so that alternative serialization technologies may be used instead, such as protobuf or thrift.  How that is done will be left as a goal for the architecture of the server component, with an additional goal of providing an IDL description of the protocol.

At the same time we realize that serialization of application keys, values, callback arguments, function parameters and so forth are a separate matter and are not necessarily tied to the serialization protocol used for client/server messaging.  The initial protocol will support primitive types such as scalars, strings and byte-arrays.  It will also support JSON documents as values and convert between these and Geode PDX-serialized objects in the servers.

Goals

Here are the high level goals.

...

Connect 

The new protocol will be integrated with In order to fit into the existing Geode client/server infrastructure we will be leveraging the current Geode "cache server. The new client driver " component.  It accepts an initial byte that tells it what type of client is connecting to the server and how the client should be handled. A client using the new protocol can connect with the Geode server by sending a protocol byte. Initially, the Geode we will support the following two protocols.:

• byte - 110: Message will contain the whole request or response. 
• byte - 111: If the message is large then client or server can divide the message into the set of small messages. Then they need to collect all the small message and parse the whole request or response.

We may add additional protocol definitions here to indicate use of an alternative serialization mechanism.  For instance bytes 112 & 113 might indicate the but require the server to use protobuf for serialization of client/server messages.  How we handle other serialization libraries will be addressed when the pluggable-serialization architecture is roughed out.

API Id

The ApiId recognizes the API, a client wants to invoke on the server. The request format will contain the 2-byte(int16) for Api id. It will be marked as ApiId in the request format.

...

We will support all the object types which Geode understands. This would include all the primitive java types, an array of primitive types, collections, java serialization, data serializable, pdx serialization and custom user data serializers. The client needs to serialize objects as described here.  This might be extended as part of making the Geode storage format pluggable.

ObjectKeyType

Geode supports only few object types as the region Key. The region key will be marked as Key in the request format. the client needs to serialize key as described here.

...

ProtocolType

The protocol descriptions use the following types.

...

These may be mapped to a pluggable serialization description language but their native serialization is described here.  The native serialization uses network byte-order ("big-endian").

...

ProtocolType

The format contains following fixed and variable types. They need to serialize in Big Endian byte order as showed for the example.

...

PutRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
	    hasMetaData, 1 byte, boolean
 	}
 	PutRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
 		key, variable, bytes
		CallbaclArg, variable, bytes
 		Value {
 			ValueHeader {
				Size, 4 byte, int32
				hasPartialBytes, 1 byte, boolean
 			}
			value {
 				bytes, series of bytes
			}
 		}
 	}
}

PutResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
    ResponseHeader {
 		ResponseTypeId, 2 byte, int16
 		hasMetaData, 1 byte, boolean
 	}
  	Success, 1 byte, boolean
}

GetRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
	    hasMetaData, 1 byte, boolean
 	}
 	GetRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
 		key, variable, bytes
		CallbaclArg, variable, bytes
 	}
}

GetResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
    ResponseHeader {
 		ResponseTypeId, 2 byte, int16
 		hasMetaData, 1 byte, boolean
 	}
  	Result, variable, bytes
}

PutAllRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
	    hasMetaData, 1 byte, boolean
 	}
 	PutRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
		NumberOfKeyValuePair, 4 byte, int32 
		KeyValuePair  { 
	 		key, variable, bytes		
 			Value {
 				ValueHeader {
					Size, 4 byte, int32
					hasPartialBytes, 1 byte, boolean
 				}
				value {
 					bytes, series of bytes
				}
			}
 		}
		CallbaclArg, variable, bytes
 	}
}

PutAllResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
	ResponseHeader {
 		ResponseTypeId, 2 byte, int16
 		hasMetaData, 1 byte, boolean
 	}
  	NumberOfKeysFailed , 4 byte, int32
}

GetAllRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
	    hasMetaData, 1 byte, boolean
 	}
 	GetRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
		NumberOfKeys, 4 byte, int32 
		keys {
 			key, variable, bytes
		}
		CallbaclArg, variable, bytes
 	}
}

GetAllResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
    ResponseHeader {
 		ResponseTypeId, 2 byte, int16
 		hasMetaData, 1 byte, boolean
 	}
  	NumberOfKeyValuePair, 4 byte, int32 
	KeyValuePair  { 
		key, variable, bytes		
 		value, variable, bytes
	}
}

RPC Frameworks

The RPC frameworks such as Thrift or Apache Avro provide tools to generate client -server library based on a message schema. We plan to are talking about how we may support various RPC framework to create the GEODE client frameworks to facilitate quick creation of GEODE clients in various languages quicklysupported by popular RPC frameworks.