Page History

...

The following changes are proposed:

• Field lengths are replaced with relative offsets;
• FIeld IDs and lengths offsets are moved to object footer.;
• Relative footer offset is added to object header.;
• All field IDs are hashed in order they are written. Resulting value is saved to object header. We refer to is as schema ID.

...

• Schema ID which is a hash of all field IDs. E.g. ID =  HASH("C" + HASH("B" + HASH("A")));
• Total fields count;
• (string, fieldId) pairsOrdered collection of field IDs.

Known schemas are stored in read-only structure. If new schema is detected during read or write, it is updated atomically. Normally object will have only 1 schema, 2-3 schemas in rare cases, >3 schemas in very rare cases. For this reason we can store them in volatile array or so.

...

• We save object field IDs and offsets to array during write. When all fields are written, we simply copy this array to object footer in a single System.arrayCopy() or Unsafe.copyMemory() operation. To minimize GC garbage we can use thread-local array instead of "new byte[]";
• If new schema is detected at the end of object write, it is added to the list of known object schemas.
• If collision is detected on (schema ID, fields count) pair, we throw an excpetion in the same way as if we had field ID conflict. This should be very unlikely event. N.B.: we can support collisions with some additional computational overhead.

No or almost no additional overhead is expected comparing to Ignite 1.4 after warmup.

Object read

• First of all we check Check if object's schema is known. If . Normally this will require only 1-2 int comparisons. If no, we scan the whole object, create the schema and save it.
• Until object read schema matches object write schema, we just read the object sequentially. Schema matching is performed using trivial int field ID comparisons.
• If read/write schemas mismatch is detected, we fallback to random field read. Normally mismatch will only occur if different object versions co-exist in runtime. 

No or almost no additional overhead is expected comparing to Ignite 1.4 after warmup.

Random object field read

• Check if object's schema is known. Normally this will require only 1-2 int comparisons;. If no, we scan the whole object, create the schema and save it.
• Field name is converted to field ID as usual;
• Schema is queried for footer offset for this field ID order. We need to evaluate possible techniques for fast int lookup: normal HashMap, open-addressing like in ThreadLocal's, specialized int-int maps. Anyways, even HashMap should usually sustain 0(1) complexity;
• Go to footer and get field offset: FieldOffset = valueOf(FooterOffset + SchemaFieldIdOffsetINT_VALUE_AT(ObjectStart + FooterOffset + FieldIdOrder * 8 + 4);
• Use FIeldOffset to get the field.

...