Ignite Type	Size	MsgPack Type	Notes
Bitmask(n)	⌈n/8⌉ bytes	bin8 / bin16 / bin32	Extension 8
IntX, UintX	1-8 bytes	fixint / intX / uintX	Integer types are interchangeable when possible. fixint (1 byte) can be passed as a value for uint64 column.
Float	4 bytes	float32
Double	8 bytes	float64
Number([n])	Variable	ext16 (up to 2^8 - 1 bytes)	Extension 1
Decimal	Variable	ext16 (up to 2^8 - 1 bytes)	Extension 2
UUID	16 bytes	fixext16	Extension 3
String	Variable	str
Date	3 bytes	fixext4	Extension 4
Time	5 bytes	ext8	Extension 5
Datetime	7 bytes	fixext8	Extension 6
Timestamp	10 bytes	ext8	Extension 7
IgniteUuid	24 bytes	ext8	Extension 9
NoValue	1 byte	fixext1	Extension 10
Binary	Variable	bin32 (up to 2^32 - 1 bytes)

Key tuples are tuples with key columns only. Key columns always come first in the schema. So if there are 2 key columns, first two values of the tuple is the key.

Dynamic schema expansion (Live-schema)

IEP-54: Schema-first Approach includes "dynamic schema expansion" feature, which means that users can insert arbitrary tuples and Ignite will automatically update the schema with extra fields, if any.

Null vs NoValue

Ignite Table API handles "column set to null" (1) and "column not set" (2) differently.

Non-nullable column does not allow (1), but allows (2) as long as there is a default value.
Nullable column will be set to null in (1) and to default value in (2).

NoValue custom protocol type reflects this distinctionTo support this feature, we provide schemaless alternatives for all operations that can involve automatic schema updates (upsert, insert, etc).

Handshake

Request
4 bytes	Magic number 49 47 4E 49, or "IGNI". Helps to identify misconfigured SSL or other apps probing the port.
int	Payload length
int	Version major
int	Version minor
int	Version patch
int	Client code (1 for JDBC, 2 for general-purpose client)
bin	Features (bitmask)
map (string → any)	Extensions (auth, attributes, etc). Server can skip unknown extension data (when client is newer).

...

Response
4 bytes	Magic number 49 47 4E 49, or "IGNI". Helps to identify misconfigured SSL or different server on the port.
int	Payload length
int	Server version major
int	Server version minor
int	Server version patch
int	Error code (0 for success)
string	Error message (when error code is not 0)
binint	When error code is 0: Server features (bitmask)idle timeout
string	When error code is 0: Server node id
string	When error code is 0: Server node name (consistent id)
bin	When error code is 0: Server features (bitmask)
map (string map (string → any)	When error code is 0: Extensions (auth, attributes, etc). Client can skip unknown extension data (when server is newer).

...

Request
int	Operation code
int	Request id, generated by client and returned as-is in response
...	Operation-specific data

Error code (0 for success

Response
int	Type = 0
int	Request id
int	Flags (1 = partition assignment changed)
string	Error message (when error code is not 0)
...	Operation-specific data
long	Observable timestamp (causality token)
uuid or null	Trace id (null for success)
int	Error code (when trace id is not null)
string	Error message (when trace id is not null)
string or null	Error stack trace (when trace id is not null)
map or null	Error details (when trace id is not null)
...	Operation-specific data (when trace id is null)

Request or response without operation-Request or response without operation-specific data is called basic request or basic response below.

...

Clients should expect notification messages at any moment after the successful handshake.

...

Operation codes and request ids are omitted everywhere below for brevity.

TABLE_CREATE = 1

...

TABLE_DROP = 2

...

Basic response.

TABLES_GET = 3

Basic request.

...

TABLE_GET = 4

...

SCHEMAS_GET = 5

...

handshake.

Notification
int	Type = 1
int	Notification code
...	Notification-specific data

Operation codes and request ids are omitted everywhere below for brevity.

TABLES_GET = 3

Basic request.

Response
map (UUID -> string)	map of table ids and names

Note: tables can only be created/deleted with SQL, there are no TABLE_CREATE or TABLE_DROP operations.

TABLE_GET = 4

Request
string	table name

Response
UUID or nil	table ID or null when a table with the given name does not exist

SCHEMAS_GET = 5

TUPLE_UPSERT = 10

...

Basic response.

Client side is supposed to match provided columns against the latest known schema.

If columns don't match, request the latest schema and try again.
If the latest schema still does not match, and live schema is enabled, use TUPLE_UPSERT_SCHEMALESS

TUPLE_UPSERT_SCHEMALESS = 11

...

Basic response.

...

int id for the current tuple, or nil when there is no matching record
Request
UUID table ID
int schema ID
values values for key columns (in schema order)
Response
arr or nil schema
values values for value columns in schema order, when schema id is not nil
...
IDs, or null to get latest

Response
map (int → array (array)) Map from schema ID to columns. Column is represented by an array of values for: name, type, isKey, isNullable. The array can contain extra data in future for additional properties.

TUPLE_UPSERT

...

=

...

10

Request
UUID	table ID
int or nil	schema transaction ID
introw	countschema ID
values	rows with values for all columns in given schema (nil when value is missing for a column)

Basic response.response.

Client side is supposed to match provided columns against the latest known schema.

If columns don't match, request the latest schema and try again.
If the latest schema still does not match, and live schema is enabled, use TUPLE_UPSERT

...

_SCHEMALESS

...

Basic response.

TUPLE_GET

...

=

...

12

Request
UUID	table ID
int or nilschema ID, or nil when result set is	emptytransaction ID
introw	countschema ID
values	array of rows with values for key columns (in schema order)

Response
int or nil	schema ID (for all tuples in response)
int	row count
values	rows with values in schema order

...

id for the current tuple, or nil when there is no matching record
values	values for value columns in schema order, when schema id is not nil

Clients should retrieve schemas with SCHEMAS_GET and cache them per table.

TUPLE_UPSERT_ALL = 13

values for value columns in schema order

Request
UUID	table ID
int or nil	schema transaction ID
values	values for all columns in given schema (nil when value is missing for a column)
Response
int or nil	schema id for the current tuple, or nil when there is no matching record
values
int	schema ID
int	row count
values	rows with values for all columns in given schema (nil when value is missing for a column)

Basic response.

TUPLE_GET_

...

ALL = 15

Request
UUID	table ID		table ID
int or nil	transaction ID
int	schema ID, or nil when result set is empty
int	row count
values	array of rows with values for key columns (in schema order)	map	tuple as map

Response
int or nil	schema id for the current tuple, or nil when there is no matching recordID (for all tuples in response)
int	row count
valuestuple	rows with values in schema order when schema id in not null

TUPLE_GET_AND_

...

UPSERT =

...

16

Request
UUID	table ID
int or nil	transaction ID
int	schema ID
values	values for all columns in given schema (nil when value is missing for a column)

Response
bool	Insert result

TUPLE_INSERT_SCHEMALESS = 19

Insert result

Request
int or nil	schema id for the current tuple, or nil when there is no matching record
values	values for value columns in schema order
UUID	table ID
map	tuple as map
Response
bool

TUPLE_INSERT

...

=

...

18

Request
UUID	table ID
int or nil	schema transaction ID
introw	countschema ID
values	rows with values for all columns in given schema (nil when value is missing for a column)

Response

intschema id, or nil when no rows were skippedintskipped row countvaluesskipped rows (values in schema order)

bool	Insert result

TUPLE_INSERT_ALL

...

=

...

20

Request
UUID	table ID
arr of map	array of tuples

Responsearr of intindices (in the request array) of skipped rows, or nil when none were skipped


int or nil	transaction ID
int	schema ID
int	row count
values	rows with values for all columns in given schema (nil when value is missing for a column)

Response
int	schema id, or nil when no rows were skipped
int	skipped row count
values	skipped rows (values in schema order)

TUPLE_REPLACE = 22

Request
UUID	table ID
int or nil	transaction ID
int	schema ID
values	values for all columns in given schema (nil when value is missing for a column)
Response
bool	Replace result

TUPLE_REPLACE_SCHEMALESS = 23

missing for a column)

Request
UUID	table ID
map	tuple as map
Response
bool	Replace result

TUPLE_REPLACE_EXACT = 24

Request
UUID	table ID
int or nil	transaction ID
int	schema ID
values	oldRec: values for all columns in given schema (nil when value is missing for a column)
values	newRec: values for all columns in given schema (nil when value is missing for a column)
Response
bool	Replace result

TUPLE_REPLACE_EXACT_SCHEMALESS = 25

Request
UUID	table ID
map	oldRec: tuple as map
map	newRec: tuple as mapa column)

Response
bool	Replace result

TUPLE_GET_AND_REPLACE = 26

Request
UUID	table ID
int or nil	transaction ID
int	schema ID
values	values for all columns in given schema (nil when value is missing for a column)

Response
int or nil	schema id for the current tuple, or nil when there is no matching record
values	values for value columns in schema order

TUPLE_DELETE = 28

Response

Request
UUID	table ID
int or nil	transaction ID
int	schema IDschema id for the current tuple, or nil when there is no matching record
values	values for value key columns in schema ordergiven schema

Response
bool	Delete result

TUPLE_

...

DELETE_ALL = 29

Request
UUID	table ID
int or nil	transaction ID
int	schema ID
int	row count
values	rows with values for key columns in a given schema

...

RequestUUIDtable IDmaptuple as map

Response
int

or nil

schema id

for the current tuple

, or nil when

there is no matching recordvaluestuple values in schema order

TUPLE_DELETE = 28

no rows were skipped
int	skipped row count
values	skipped rows (

RequestUUIDtable IDintschema IDvalues

values for key columns in

given

schema

ResponseboolDelete result

order)

TUPLE_DELETE_

...

EXACT =

...

30

Request
UUID	table ID
int or nil	schema transaction ID
introw	countschema ID
valuesrows	with values for key all columns in given schema (nil when value is missing for a column)

Indices (in the request) of rows that were not deleted

Response		arr of int
bool	Delete result

TUPLE_DELETE_ALL_EXACT =

...

31

Request
UUID	table ID
int or nil	transaction ID
int	schema ID
int	row count
values	rows with values for all columns in given schema (nil when value is missing for a column)
Response
bool	Delete result

...

for a column)

Response
int	schema id, or nil when no rows were skipped
int	skipped row count
values	skipped rows (values for key columns in schema order)

TUPLE_GET_AND_DELETE = 32

Request
UUID	table ID
int or nil	schema transaction ID
introw	countschema ID
values	rows with values for all columns in given schema (nil when value is missing for a column)

Response
arr of int	Indices (in the request) of rows that were not deleted

...

int or nil	schema id for the current tuple, or nil when there is no matching record
values	values for value columns in schema order, when schema id is not null

TUPLE_CONTAINS_KEY = 33

Request
UUID	table ID
int or nil	transaction ID
int	schema ID
values	values for all columns in given schema (nil when value is missing for a column)
Response
int or nil	schema id for the current tuple, or nil when there is no matching record
values	values for value columns in schema order, when schema id is not null

TUPLE_CONTAINS_KEY = 33

...

key columns

Response
bool	whether a tuple with the given key exists

TX_BEGIN = 43

Request
bool	Read-only tx flag
long	Observable timestamp (value from server - see standard response header)

Response
int	Transaction ID

TX_COMMIT = 44

Request
int	Transaction ID

Basic response

TX_ROLLBACK = 45

Request
int	Transaction ID

Basic response

...

Risks and Assumptions

This IEP covers handshake and Tables API, which is the only public API available at the moment.
Invoke API is out of scope: code deployment and processor serialization should be designed separately.

...

Page tree

Page History

Versions Compared

Old Version 65

New Version Current

Key

Dynamic schema expansion (Live-schema)

Null vs NoValue

Handshake

TABLE_CREATE = 1

TABLE_DROP = 2

TABLES_GET = 3

TABLE_GET = 4

SCHEMAS_GET = 5

TABLES_GET = 3

TABLE_GET = 4

SCHEMAS_GET = 5

TUPLE_UPSERT = 10

TUPLE_UPSERT_SCHEMALESS = 11

TUPLE_UPSERT

=

10

TUPLE_GET

=

12

TUPLE_UPSERT_ALL = 13

TUPLE_GET_

ALL = 15

TUPLE_GET_AND_

UPSERT =

16

TUPLE_INSERT_SCHEMALESS = 19

TUPLE_INSERT

=

18

TUPLE_INSERT_ALL

=

20

TUPLE_REPLACE = 22

TUPLE_REPLACE_SCHEMALESS = 23

TUPLE_REPLACE_EXACT = 24

TUPLE_REPLACE_EXACT_SCHEMALESS = 25

TUPLE_GET_AND_REPLACE = 26

TUPLE_DELETE = 28

TUPLE_

DELETE_ALL = 29

TUPLE_DELETE = 28

TUPLE_DELETE_

EXACT =

30

TUPLE_DELETE_ALL_EXACT =

31

TUPLE_GET_AND_DELETE = 32

TUPLE_CONTAINS_KEY = 33

TUPLE_CONTAINS_KEY = 33

TX_BEGIN = 43

TX_COMMIT = 44

TX_ROLLBACK = 45

Risks and Assumptions