New Client Server Protocol

Introduction

Geode is a reliable distributed data management tool. There is a demand to access Geode from various programming languages. But the existing client-server protocol is too complex to understand, undocumented. This establishes the need for a new client-server protocol.

Goals

Here are the high level goals.

Connect 

The new protocol will be integrated with the current Geode server. The new client driver can connect with the Geode server by sending a protocol byte. Initially, the Geode will support the following two protocols.

• byte - 110: Message will contain the whole request or response. 
• byte - 111: If the message is large then client or server can divide the message into the set of small messages. Then they need to collect all the small message and parse the whole request or response.

API Id

The ApiId recognizes the API, a client wants to invoke on the server. The request format will contain the 2-byte(int16) for Api id. It will be marked as ApiId in the request format.

API Version

API version will be associated with the request api. The request format will contain a 1-byte(int8) for the version. It will be marked as ApiVersion in the request format. Its current value will be 1.

Correlation Id

The purpose of correlation id to match the request and its corresponding response. The request format will contain the 4-bytes(int32) for correlation Id. It will be marked as CorrelationId in the request and response format. The client needs to send correlation id with the request and server will send the same id with the response.

Object Type

We will support all the object types which Geode understands. This would include all the primitive java types, an array of primitive types, collections, java serialization, data serializable, pdx serialization and custom user data serializers. The client needs to serialize objects as described here.

ObjectKeyType

Geode supports only few object types as the region Key. The region key will be marked as Key in the request format. the client needs to serialize key as described here.

ResponseType

ReponseType will indicate that whether the response is partial or complete. A client can process a partial response.  Response with the FullResponse type id will indicate the completion of that request. 

The response format will contain the 2-bytes(int16) for response type. It will be marked as ResponseTypeId in the response format.

Error Codes

Error codes indicate the issue with the invocation of API at the server. We have following error code for various issues at the server. The response format will contain the 2-bytes(int16) for error codes. It will be marked as ErrorCode in the response format.

Format grammar

The format is described using Extended Backus–Naur form grammar.

ProtocolType

The format contains following fixed and variable types. They need to serialize in Big Endian byte order as showed for the example.

Message(Framing) 

A message is series of bytes which contains the request or response. If the message is large, then we will have provision to divide the message into small messages. The message will be sent in following way.  A client can send the multiple messages on the connection and server will respond to those messages in same order.

MessageHeader

The Message header is a fixed size header which contains the size of a message, boolean flag to indicates whether a message is partial, and correlation id for that request message. The correlation id is used for the dual purpose here.

• If a message is sent in multiple sub-messages, then it will be used for combining the whole message. 
• It will be used for correlating the request to its response.

Request Format

The request would contain the fixed size request header, optional metadata and request api parameters. 

RequestHeader

The request header contains the ApiId, ApiVersion, and hasMetaData flag to indicate whether the request contains some metadata.

Response Format

The response would contain the fixed size response header, optional metadata and return values.

ResponseHeader

The response header will have resposneType, which indicates its partial response, full response or error. A hasMetaData flag indicates whether the response contains some metadata.

Value

The Value is serialized bytes for the Geode region value. It contains value header and series of bytes. Using value header, we can send a large number of bytes in more than one chunk. 

ValueHeader

The value header contains the value bytes size, and a flag indicates whether it contains all the value bytes. 

APIS

PutRequest

PutResponse

GetRequest

GetResponse

GetAllRequest

GetAllResponse

PutAllRequest

PutAllResponse

Metadata

The purpose of metadata to pass defined key value pair with request and response. That will be optional for client. If there is any metadata associated with request or response, then need to set "hasMetadata" flag to "true" in request or response header. After that send metadata in following format.

We would have following pre-defined key and value for metadata. Note this list will grow over time.

ServerConfigRequest

The purpose of ServerConfigRequest request to get server config from the server. The client needs to send this request after connecting to the server. If the client knows server properties, then there is no need to send this request.

ClientConfigRequest

The Purpose of ClientConfigRequest request to send client config to a server. The client needs to send this request after connecting to the server. 

AuthRequest

The purpose of AuthRequest to authenticate the client connection. A client can send the auth request in the following format. Before sending the auth request a client can verify the server config by sending the ServerConfigRequest. A client can send key-value pair of credentials to authenticate itself.

Those pair needs to serialized in following way. If Diffie-Hellman encryption is enabled on the server side then need to encrypt those serialized bytes. And then send those encrypted bytes to the server. The server will decrypt those bytes and create the key-value pair.

Examples

PutRequest

string regionName = "ExampleRegion"

Key = 101

Value = "New Geode Client Server Protocol"

CallbackArg = Null

PutResponse

Messages

PutRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
 	}
 	PutRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
 		key, variable, bytes
		CallbaclArg, variable, bytes
 		Value {
 			ValueHeader {
				Size, 4 byte, int32
				hasPartialBytes, 1 byte, boolean
 			}
			value {
 				bytes, series of bytes
			}
 		}
 	}
}

PutResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
  	Success, 1 byte, boolean
}

GetRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
 	}
 	GetRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
 		key, variable, bytes
		CallbaclArg, variable, bytes
 	}
}

GetResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
  	Result, variable, bytes
}

PutAllRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
 	}
 	PutRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
		NumberOfKeyValuePair, 4 byte, int32 
		KeyValuePair  { 
	 		key, variable, bytes		
 			Value {
 				ValueHeader {
					Size, 4 byte, int32
					hasPartialBytes, 1 byte, boolean
 				}
				value {
 					bytes, series of bytes
				}
			}
 		}
		CallbaclArg, variable, bytes
 	}
}

PutAllResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
  	Success, 1 byte, boolean
}

GetAllRequestMessage {
	MessageHeader {
  		Size, 4 byte, int32
 		isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
 	}
 
 	RequestHeader {
 		ApiId, 2 byte, int16
 		ApiVersion, 1 byte, int8
 	}
 	GetRequest {
 		regionName, varaible , String {
			len, 2 byte, int16
			variable, utf encoding		
		}
		NumberOfKeys, 4 byte, int32 
		keys {
 			key, variable, bytes
		}
		CallbaclArg, variable, bytes
 	}
}

GetAllResponseMessage {
	MessageHeader {
    	Size, 4 byte, int32
        isPartialMessage, 1 byte, boolean
		CorrelationId, 4 byte, int32
    }
  	NumberOfKeyValuePair, 4 byte, int32 
	KeyValuePair  { 
		key, variable, bytes		
 		Value {
 			ValueHeader {
				Size, 4 byte, int32
				hasPartialBytes, 1 byte, boolean
 			}
			value {
 				bytes, series of bytes
			}
		}
 	}
}