...
Ignite Type | Size | MsgPack Type | Notes |
---|
Bitmask(n) | ⌈n/8⌉ bytes | bin8 / bin16 / bin32 | Extension 8 |
IntX, UintX | 1-8 bytes | fixint / intX / uintX | Integer types are interchangeable when possible. fixint (1 byte) can be passed as a value for uint64 column. |
Float | 4 bytes | float32 |
|
Double | 8 bytes | float64 |
|
Number([n]) | Variable | ext16 (up to 2^8 - 1 bytes) | Extension 1 |
Decimal | Variable | ext16 (up to 2^8 - 1 bytes) | Extension 2 |
UUID | 16 bytes | fixext16 | Extension 3 |
String | Variable | str |
|
Date | 3 bytes | fixext4 | Extension 4 |
Time | 5 bytes | ext8 | Extension 5 |
Datetime | 7 bytes | fixext8 | Extension 6 |
Timestamp | 10 bytes | ext8 | Extension 7 |
IgniteUuid | 24 bytes | ext8 | Extension 9 |
NoValue | 1 byte | fixext1 | Extension 10 |
Binary | Variable | bin32 (up to 2^32 - 1 bytes) |
|
...
Key tuples are tuples with key columns only. Key columns always come first in the schema. So if there are 2 key columns, first two values of the tuple is the key.
Dynamic schema expansion (Live-schema)
IEP-54: Schema-first Approach includes "dynamic schema expansion" feature, which means that users can insert arbitrary tuples and Ignite will automatically update the schema with extra fields, if any.
To support this feature, we provide schemaless alternatives for all operations that can involve automatic schema updates (upsert, insert, etc).
Null vs NoValue
Ignite Table API handles "column set to null" (1) and "column not set" (2) differently.
- Non-nullable column does not allow (1), but allows (2) as long as there is a default value.
- Nullable column will be set to null in (1) and to default value in (2).
NoValue custom protocol type reflects this distinction.
Handshake
Request |
---|
4 bytes | Magic number 49 47 4E 49, or "IGNI". Helps to identify misconfigured SSL or other apps probing the port. |
int | Payload length |
int | Version major |
int | Version minor |
int | Version patch |
int | Client code (1 for JDBC, 2 for general-purpose client) |
bin | Features (bitmask) |
map (string → any) | Extensions (auth, attributes, etc). Server can skip unknown extension data (when client is newer). |
...
Response |
---|
4 bytes | Magic number 49 47 4E 49, or "IGNI". Helps to identify misconfigured SSL or different server on the port. |
int | Payload length |
int | Server version major |
int | Server version minor |
int | Server version patch |
int | Error code (0 for success) |
string | Error message (when error code is not 0) |
int | When error code is 0: Server idle timeout |
string | When error code is 0: Server node id |
string | When error code is 0: Server node name (consistent id) |
bin | When error code is 0: Server features (bitmask) |
map (string → any) | When error code is 0: Extensions (auth, attributes, etc). Client can skip unknown extension data (when server is newer). |
...
Request |
int | Operation code |
int | Request id, generated by client and returned as-is in response |
... | Operation-specific data |
Response |
int | Type = 0 |
int | Request id |
int | Error code (0 for successFlags (1 = partition assignment changed) |
stringlong | Error message (when error code is not 0) |
... | Operation-specific data |
Observable timestamp (causality token) |
uuid or null | Trace id (null for success) |
int | Error code (when trace id is not null) |
string | Error message (when trace id is not null) |
string or null | Error stack trace (when trace id is not null) |
map or null | Error details (when trace id is not null) |
... | Operation-specific data (when trace id is null) |
Request or response without operation-Request or response without operation-specific data is called basic request or basic response below.
...
Operation codes and request ids are omitted everywhere below for brevity.
...
TABLES_
...
GET =
...
3
...
Basic request.
Response |
---|
map | configuration (name, partitions, replicas, columns, indices, ...) - according to TableConfigurationSchema |
Response |
---|
UUID | table ID |
TABLE_DROP = 2
...
Basic response.
TABLES_GET = 3
Basic request.
...
(UUID -> string) | map of table ids and names |
Note: tables can only be created/deleted with SQL, there are no TABLE_CREATE or TABLE_DROP operations.
...
TABLE_GET = 4
...
Response |
---|
map (int → array (array)) | Map from schema ID to columns. Column is represented by an array of values for: name, type, isKey, isNullable. The array can contain extra data in future for additional properties. |
TUPLE_UPSERT = 10
...
Basic response.
Client side is supposed to match provided columns against the latest known schema.
- If columns don't match, request the latest schema and try again.
- If the latest schema still does not match, and live schema is enabled, use TUPLE_UPSERT_SCHEMALESS
TUPLE_UPSERT_SCHEMALESS = 11
...
Basic response.
TUPLE_GET = 12
...
TUPLE_UPSERT
...
=
...
10
Request |
---|
UUID | table ID |
int or nil | schema transaction ID |
introw | countschema ID |
values | rows with values for all columns in given schema (nil when value is missing for a column) |
Basic response.
Basic response.
Client side is supposed to match provided columns against the latest known schema.
- If columns don't match, request the latest schema and try again.
- If the latest schema still does not match, and live schema is enabled, use TUPLE_UPSERT_
...
TUPLE_
...
GET =
...
12
Request |
---|
UUID | table ID |
arr of map | array of tuples |
Basic response.
TUPLE_GET_ALL = 15
int or nil | transaction |
Request |
---|
UUID | table ID |
int | schema ID, or nil when result set is empty | int | row count |
values | array of rows with values for key columns (in schema order) |
Response |
---|
int or nil | schema ID (for all tuples in response) | int | row countid for the current tuple, or nil when there is no matching record |
values | rows with values in schema order |
...
values for value columns in schema order, when schema id is not nil |
Clients should retrieve schemas with SCHEMAS_GET and cache them per table.
TUPLE_UPSERT_ALL = 13
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
int | row count |
values | rows with values for all columns in given schema (nil when value is missing for a column)) |
Basic response.
TUPLE_GET_ALL = 15
Request |
---|
UUID | table ID | Response |
int or nil | transaction ID |
int | schema id for the current tupleID, or nil when there is no matching recordresult set is empty |
int | row count |
values | tuple values in schema order |
TUPLE_GET_AND_UPSERT_SCHEMALESS = 17
...
array of rows with values for key columns (in schema order) |
Response |
---|
int | schema ID (for all tuples in response) |
int | row count |
values | rows with values in schema order |
TUPLE_GET_AND_UPSERT = 16
...
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
values | values for all columns in given schema (nil when value is missing for a column) |
Response |
---|
bool | Insert result |
TUPLE_INSERT_SCHEMALESS = 19
...
int or nil | schema id for the current tuple, or nil when there is no matching record |
values | values for value columns in schema order |
TUPLE_INSERT = 18
Request |
---|
UUID | table ID |
int or nil | schema transaction ID |
introw | countschema ID |
values | rows with values for all columns in given schema (nil when value is missing for a column) |
Response |
---|
bool | Insert result |
TUPLE_INSERT_ALL = 20
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema |
id, or nil when no rows were skippedskipped skipped (with values for all columns in |
schema order)TUPLE_INSERT_ALL_SCHEMALESS = 21
given schema (nil when value is missing for a column) |
Request |
---|
UUID | table ID |
arr of map | array of tuples |
Response |
---|
arr of int | indices (in the request array) of skipped rows, or nil when none were skippedno rows were skipped |
int | skipped row count |
values | skipped rows (values in schema order) |
TUPLE_REPLACE = 22
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
values | values for all columns in given schema (nil when value is missing for a column) |
Response |
---|
bool | Replace result |
TUPLE_REPLACE_SCHEMALESS = 23
Request |
---|
UUID | table ID |
map | tuple as mapmissing for a column) |
Response |
---|
bool | Replace result |
TUPLE_REPLACE_EXACT = 24
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
values | oldRec: values for all columns in given schema (nil when value is missing for a column) |
values | newRec: values for all columns in given schema (nil when value is missing for a column) |
Response |
---|
bool | Replace result |
TUPLE_REPLACE_EXACT_SCHEMALESS = 25
Request |
---|
UUID | table ID |
map | oldRec: tuple as map |
map | newRec: tuple as map |
Response |
---|
bool | Replace result |
TUPLE_GET_AND_REPLACE = 26
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
values | values for all columns in given schema (nil when value is missing for a column) |
...
Response |
---|
int or nil | schema id for the current tuple, or nil when there is no matching recordmatching record |
values | values for value columns in schema order |
TUPLE_DELETE = 28
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
valuestuple | values for key columns in schema ordergiven schema |
Response |
---|
bool | Delete result |
TUPLE_
...
DELETE_ALL = 29
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
int | row count |
map | tuple as mapvalues | rows with values for key columns in a given schema |
or nil for the current tuple there is no matching recordno rows were skipped |
int | skipped row count |
values |
tuple values skipped rows (values for key columns in schema order) |
TUPLE_DELETE_EXACT =
...
30
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
values | values for key all columns in given schema (nil when value is missing for a column) |
Response |
---|
bool | Delete result |
TUPLE_DELETE_ALL_EXACT =
...
31
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
int | row count |
values | rows with values for key all columns in given schema (nil when value is missing for a column) |
arr of int | Indices (in the request) of rows that were not deleted |
...
|
---|
int | schema id, or nil when no rows were skipped |
int | skipped row count |
values | skipped rows (values for key columns in schema order) |
TUPLE_GET_AND_DELETE = 32
Request |
---|
UUID | table ID |
int or nil | transaction ID |
int | schema ID |
values | values for all columns in given schema (nil when value is missing for a column) |
Response |
---|
bool | Delete result |
...
|
---|
int or nil | schema id for the current tuple, or nil when there is no matching record |
values | values for value columns in schema order, when schema id is not null |
TUPLE_CONTAINS_KEY = 33
Request |
---|
UUID | table ID |
int or nil | schema transaction ID |
introw | countschema ID |
valuesrows | with values for all columns in given schema (nil when value is missing for a column)key columns |
Response |
---|
arr of int | Indices (in the request) of rows that were not deleted |
TUPLE_GET_AND_DELETE = 32
...
bool | whether a tuple with the given key exists |
TX_BEGIN = 43
Request |
---|
bool | Read-only tx flag |
long | Observable timestamp (value from server - see standard response header) |
Response |
---|
int | Transaction ID |
TX_COMMIT = 44
Basic response
TX_ROLLBACK = 45
Basic response
...
Risks and Assumptions
- This IEP covers handshake and Tables API, which is the only public API available at the moment.
- Invoke API is out of scope: code deployment and processor serialization should be designed separately.
...