Page History

...

• Flag 0: no value. If the flag is set, the value chunk is omitted, e.g. the row represents a tombstone
• Flag 1: skip key nullmap. If the flag is set, all column values in the key chunk are non-null, so that the null map for the key chunk is omitted
• Flag 2: skip value nullmap. If the flag is set, all column values in the value chunk are non-null, so that the null map for the value chunk is omitted
• Flag 3: skip key varlen table. If flag is set, all column values in the key chunk either of fix-sized type or null, so that the varlen table for key chunk is omitted.
• Flag 4: skip value varlen table. If flag is set, all column values in the value chunk either of fix-sized type or null, so that the varlen table for value chunk is omitted.
• Flags 5-15: Reserved for future use

Hash calculation and key comparison

Row hash can be calculated from affinity field values while marshalling to byte array. Because od field order is defined by scheme, hash can be calculated consistently regarding the column order.

Key can be compared as byte[] for compatible schemas (that has same key column set), otherwise oldest row should be upgrade first.
It is possible to compare keys column-by-column regarding the schema if same key can be serialized in more than one way. E.g. kind of compression will be supported and compressed rows could be marked with a flag.

Schema evolution

Unlike Ignite 2.x approach, where binary object schema ID is defined by a set of fields that are present in a binary object, for the schema-first approach we assign a monotonically growing identifier to each version of the cache schema. The ordering guarantees should be provided by the underlying metadata storage layer (for example, the current distributed metastorage implementation or consensus-based metadata storage). The schema identifier should be stored together with the data rows (but not necessarily with each row individually: we can store schema ID along with a page or larger chunks of data). The history of schema versions must be stored for a long enough period of time to allow upgrade all existing data stored in a given cache.

...