Page History

...

A configurable Kafka serializer that:

• Check if the estimated size of the data (bytes) after applying provided compression (if there is one) it needs to serialize is larger than the configured threshold.
  • If it is large than the provided threshold (`large.message.threshold.bytes`):
    • Use the provided PayloadStore implementation to publish the large message into payload store, generating an id which is the reference to access this later.
    • Encapsulate that ID into a simple Kafka event using a structured format.
    • Pass the new Kafka Event down
    • Add alarge-message: trueheader
  • If it’s not large then provided threshold (large.message.threshold.bytes):
    • Do nothing, pass the data as it is.

...