Motivation

Customers may want to 

• minimize (compress)
• protect (encrypt)

user's data at the network and memory layer.

Ignite supports Disk Compression and Transparent Data Encryption, but they are able to transform the data at the persistent layer only.

Description

To cover both layers (network and memory) and make the feature compatible with the existing data, it is proposed to transform/restore CacheObject's bytes on the fly.

A possible solution is to transform the byte arrays they provided during the marshaling/unmarshalling phase. This will cover both layers, messaging (network) and storage (in-memory + persist).

Transformation

All we need is to cover all CacheObjects.

CacheObjects

Most of them has the following structure:

 protected Object val; // Unmarshalled value.
 protected byte[] valBytes; // Marshalled value bytes.

and all we need - is to add transformation during the marshaling/unmarshalling phase:

protected byte[] valueBytesFromValue(CacheObjectValueContext ctx) throws IgniteCheckedException {
    byte[] bytes = ctx.kernalContext().cacheObjects().marshal(ctx, val);

    return CacheObjectsTransformer.transformIfNecessary(bytes, ctx);
}


protected Object valueFromValueBytes(CacheObjectValueContext ctx, ClassLoader ldr) throws IgniteCheckedException {
    byte[] bytes = CacheObjectsTransformer.restoreIfNecessary(valBytes, ctx);

    return ctx.kernalContext().cacheObjects().unmarshal(ctx, bytes, ldr);
}

...

if (valBytes == null)
	valBytes = valueBytesFromValue(ctx);

...

if (val == null) 
    val = valueFromValueBytes(ctx, ldr);

BinaryObjects

BinaryObject(Impl)s have the different structure:

private byte[] arr;
private Object obj;

Risks and Assumptions

Transformation requires additional memory allocation and subsequent GC work.

Transformation requires additional CPU utilization.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

// Links to various reference documents, if applicable.

Tickets