...
- Kafka Exactly Once - Solving the problem of spurious OutOfOrderSequence errors
- Kafka Exactly Once - Dealing with older message formats when idempotence is enabled
Public Interfaces
RecordMetadata (in 1.0.0)
With the changes in Kafka Exactly Once - Solving the problem of spurious OutOfOrderSequence errors, the broker may return a new DUPLICATE_SEQUENCE
error code in some cases where a duplicate is detected but the metadata for the existing batch isn't cached in memory. When the producer receives this error, it is considered a success, but will not have the offset and timestamp information for the appended records. To help identify this state, we add hasOffset
and add hasTimestamp
methods to the RecordMetadata
.
...
Code Block |
---|
// TopicMetadataV3 TopicMetadata => TopicErrorCode Topic IsInternal MessageFormatVersion MaxMessageBytes [PartitionMetadata] TopicErrorCode => int16 Topic => String IsInternal => Boolean MessageFormatVersion => int8 (NEW) MaxMessageBytes => int32 (NEW) PartitionMetadata => PartitionMetadataV2 |
ProduceResponse (in 1.0.0)
We add a logStartOffset
field to the produce response to help the producer identify when producer state has been lost due to retention time elapsing. See Kafka Exactly Once - Solving the problem of spurious OutOfOrderSequence errors for a precise description of how this will be used.
...