THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Table of Contents
Protocol negotiation
| Note | ||
|---|---|---|
| ||
In order to use the existing Geode client/server infrastructure, we will be leveraging the current Geode "cache server" component. In the current server implementation, the server expects an initial "handshake" byte, which tells the server the client protocol and how it should be handled. A client, using the new protocol, can connect with the Geode server by sending a protocol byte. Initially, we will support the following two protocols:
We may add additional protocol definitions here to indicate the use of an alternative serialization mechanism. For instance bytes 112 & 113 might indicate the but require the server to use Protobuf for serialization of client/server messages. How we handle other serialization libraries will be addressed when the pluggable-serialization architecture is roughed out. |
Protocol Field Definitions
Understanding the message definitions it is advised that one references the Protocol Field Definitions page and Supported Data Type Definitions.
Message Definition
A message is a series of bytes which contains the request or response. If the message is large, then we will have provision to divide the message into small messages. In that case, client/server needs to collect all messages to parse the request/response.
...
Message definition grammar
In order to consistently define messages the Extended Backus–Naur form grammar will be used.
Usage | Notation |
|---|---|
| definition | = |
| alterationalternation | | |
| optional | [ ... ] |
| repetition | { ... } |
| grouping | ( ... ) |
Generic Message definition
Every message will message (except for connection initializing handshakes, which are necessarily insulated from future changes to the message structure) will adhere to the following generic message definition. A Message will comprise of a MessageHeader and either a Request or Response component.
| Message = MessageHeader (Request Request | Response) | Description | |
|---|---|---|
| MessageHeader | variable size, type = MessageHeader | The MessageHeader corresponding to this Message. |
| Request | variable size, type = Request | This field will contain the Request component. |
| Response | variable size, type = Response
| This field will contain the Response component. |
MessageHeader
The Message header is a fixed size header which contains the size of a message, boolean flag to indicates whether a message is partial, and correlation id for that request message. The correlation id is used for the dual purpose here.
- If a message is sent in multiple sub-messages, then it will be used for combining the whole message.
- It will be used for correlating the request to its response.
| MessageHeader = Size CorrelationId isPartialMessage hasMetaData | Description | |
|---|---|---|
| Size | fixedSize = 4 bytes, type = int32 | Size of request or response |
| CorrelationId | fixedSize = 4 bytes, type = int32 | The correlationID used to track a request/response |
| isPartialMessage | fixedSize = 1 byte, type = boolean | Is this a partial message |
| hasMetaData | fixedSize = 1 byte, type = boolean | Does the message have meta data associated with it |
| Anchor | ||||
|---|---|---|---|---|
|
The request would contain the fixed size request header, optional metadata and request API parameterswill contain an operation specific request structure.
Notes | Description | |
|---|---|---|
| RequestHeader | variable size, type = RequestHeader | The RequestHeader corresponding to this Request |
| MetaData | optional, variable size, type = MetaData | This field is optional and will be populated if the MessageHeader.hasMeta is true |
RequestAPI | variable size, type = variable | The Api specific Request message. Here are some examples of Request messages. PutRequest | GetRequest | PutAllRequest | GetAllRequest |ServerConfigRequest | ClientConfigRequest | AuthRequest |
RequestHeader
The request header contains the ApiId, ApiVersion, and hasMetaData flag to indicate whether the request contains some metadata.
| RequestHeader = ApiId apiVersion | Description | |
|---|---|---|
| ApiId | fixedSize = 2 bytes, type = int16 | The Api ID for the Request. Supported ID's are defined on the API's page |
| apiVersion | fixedSize = 1 byte, type = int8 | The version of the API being used. |
| Anchor | ||||
|---|---|---|---|---|
|
The response would contain contains the fixed size response header, optional metadata and return values. response object corresponding to the request, or an error response object.
Response = ResponseHeader [MetaData]APIResponse | Description | |
|---|---|---|
| ResponseHeader | variable size, type = ResponseHeader | The ResponseHeader corresponding to this Response | MetaData | optional, variable size, type = MetaData | This field is optional and will be populated if the MessageHeader.hasMeta is true
| APIResponse | variable size, type = variable | The Api specific Reponse message. Here are some examples of Response messages PutResponse | GetResponse | PutAllResponse | GetAllResponse | ServerConfigResposneServerConfigResponse | ClientConfigResponse | AuthResponse | ErrorResponse |
...
| Anchor | ||||
|---|---|---|---|---|
|
The response header will have resposneType, which indicates its partial response, full response or error. A hasMetaData flag indicates whether the response contains some metadata.
| ResponseHeader = ResponseTypeId | ErrorCode | Description | |
|---|---|---|
| ResponseTypeId | fixedSize = 2 bytes, type = int16 | The ResponeTypeId corresponding to either a Full or Partial message |
| ErrorCode | fixedSize = 2 bytes, type = int16 | The error code for the error that occurred. A list of error codes can be found on the page Error Codes |
...
server will raise an error when it failed to execute API request from the client. The error response wraps an error object detailing the failure.
| ErrorResponse = Error | Description | |
|---|---|---|
| error | type = Error | The error message |
| Anchor | ||||
|---|---|---|---|---|
|
The server will raise the error when it failed to execute API request from the client. The error code and message should help the client to diagnose the issue.
| ErrorResponse = errorMessageerrorCode message | Description | |||
|---|---|---|---|---|
| errorCode | type = int | Numeric code referencing the type of failure | ||
| message | type = String | errorMessage | variable size, type = String | The error message |
| Anchor | ||
|---|---|---|
|
The Value is serialized bytes for the Geode region value. It contains value header and series of bytes. Using value header, we can send a big serialized object in more than one chunk.
| Value = {ValueHeader value} | Description | |
|---|---|---|
| ValueHeader | variable size, type = ValueHeader | The ValueHeader corresponding to this Value entry |
| value | variable size, type = bytes | Serialized Value Object which Geode can understand |
ValueHeader
The value header contains the value bytes size, and a flag indicates whether it contains all the value bytes.
| ValueHeader = Size hasPartialBytes | Description | |
|---|---|---|
| Size | fixedSize = 4 bytes, type = int32 | Number of serialized bytes |
| hasPartialBytes | fixedSize = 1 byte, type = boolean | Whether this contains partial bytes of value |
MetaData
The purpose of a metadata to pass defined key value pair with request and response. That will be optional for a client. If there is any metadata associated with request or response, then need to set "hasMetadata" flag to "true" in request or response header. After that send metadata in the following format.
| MetaData = NumberOfMetadata { MetadataKeyId MetadataKeyValue} | Description | |
|---|---|---|
| NumberOfMetadata | fixedSize = 2 bytes, type = int16 | The number of MetaDataKeyId : MetaDataKeyValue combinations |
| MetadataKeyId | fixedSize = 2 bytes, type = int16 | Supported MetaDataKeyId's can be found in the table "Supported MetaData - MetaData KeyId" |
| MetadataKeyValue | variable size, type = variable | The MetaData value for the KeyId. The supported dataType can be found in the table "Supported MetaData - MetaData Value Type" |
Supported MetaData
We would have following pre-defined key and value for a metadata. Note this list will grow over time.
MetaData for Requests
Request MetaData Key | MetaData KeyId | MetaData Value example | MetaData Value Type | Description |
|---|---|---|---|---|
| JSON_KEY | 1 | true | fixedSize = 1 byte, type = boolean | Geode will expect key as JSON string(or bytes) and it will convert that string into PDX key. If the response requires a key, then it will convert the PDX key to JSON string(or bytes) back. |
| JSON_VALUE | 2
| true | fixedSize = 1 byte, type = boolean | Geode will expect Value as JSON string(or bytes) and it will convert that string into PDX value. If the response requires a value, then it will convert PDX value to JSON string(or bytes) back. |
| EVENT_ID | 3
| EventId { uniqueId: type = String ThreadId:type=int64 SequenceId: type=int64 } | variable size, type = bytes | The eventid is used to identify same region event in Geode. Geode keeps map of "uniqueId + threadId" Vs "SequenceId" to know whether region event has been already seen or not.
|
MetaData for Responses
...
Response MetaData Key
...
MetaData KeyId
...
MetaData Value example
...
MetaData Value Type
...
Description
|
Some data being sent between the client and server, for the most part these rely on protobuf for proper serialization, except for CustomEncodedValues
| EncodedValue = (one of the below) | Description |
|---|---|
| intResult | 32-bit signed integer |
| longResult | 64-bit signed integer |
| shortResult | 16-bit signed integer |
| byteResult | 8-bit signed integer |
| booleanResult | True or False. |
| doubleResult | 64-bit floating point number. |
| floatResult | 32-bit floating point number. |
| binaryResult | A binary blob to be stored in the region. |
| stringResult | Character string |
| customEncodedValue | Data which doesn't use protobuf serialization |
| Anchor | ||||
|---|---|---|---|---|
|
EncodedValue represents a serialized value in a format that Geode can understand. It is used for both keys and values in database requests.
| Value = ValueHeader value | Description | |
|---|---|---|
| encodingType | type = EncodingType | The encoding type of the following bytes. |
| value | type = bytes | Serialized Value Object which Geode can understand |
| Anchor | ||||
|---|---|---|---|---|
|
Currently supported types are:
| Name | Description |
|---|---|
| JSON | UTF-8 encoded string, containing a JSON document that will be deserialized in the same way as the REST API. |
| Anchor | ||||
|---|---|---|---|---|
|
This structure represents a pair of data corresponding to a key and associated value.
| Entry = EncodedValue Error | Description | |
|---|---|---|
| key | type = EncodedValue | Data referencing the corresponding value |
| value | type = EncodedValue | Data associated to the corresponding key |
| Anchor | ||||
|---|---|---|---|---|
|
Some responses may contain multiple errors keyed to different inputs. This object enables this behavior.
| KeyedError = EncodedValue Error | Description | |
|---|---|---|
| key | type = EncodedValue | Key corresponding to this error |
| error | type = Error | Details about the failure |
Request and Response Definitions
The request and response messages are further broken down by category and can be found on the following pages.
Connection API
Initial operations for correctly setting up a connection can be found here
Region API
Operations for getting, creating, and modifying data regions can be found here
Locator API
Connections to locators behave a bit differently from normal cache server connections. For more details and information about locator operations, visit this page.
...
1
...
fixedSize = 1 byte, type = boolean
[optional]The server accepted and forwarded the request to the appropriate
node holding the affected cache entry. A smart client should refresh its partitioned
...