Page History

...

Normal EndTxn (not epoch overflow)

Prepare

• Keep producer ID but increment epoch

• Return new epoch and producer ID in EndTxnResponse

• If we retry and see epoch - 1 + ID in last seen fields and are issuing the same command (ie commit not abort) we can return (with the new epoch)

Complete

• Write the previous ID in the tagged field. (Epoch - 1 is assumed)

• If we retry and see epoch - 1 + ID in last seen field and are issuing the same command we can return (with new epoch)

Epoch Overflow EndTxn

Prepare

• Write the producer ID and max epoch in the producer ID and epoch fields. Get a new producer ID and write it in the nextProducerId field. The previous producer ID tagged field will remain empty.

• Return epoch 0 and new producer ID in EndTxnResponse

• If we retry and see epoch - 1 + ID in last seen fields and are issuing the same command we can return (with the new producer id and epoch)

Complete

• Set the producer ID to nextProducerID and epoch to 0 and nextProduceId can go back to empty. Set the previous producer ID field.

• If we retry and see epoch max - 1 + ID in last seen fields and are issuing the same command we can return (with new producer id and epoch)

Once we move past the Prepare and Complete states, we don’t need to worry about lastSeen fields and clear them, just handle state transitions as normal.

In KIP-890 part 2, when writing to a partition we will check the epoch when we add the partition, and use the existing fencing logic at the log layer (and we do not worry about the verification mechanism described in part 3).

...

Instead of  INVALID_TXN_STATE and INVALID_PID_MAPPING are we considered using UNKNOWN_MEMBER_ID which is abortable. However, this is not a clear message and is not guaranteed to be abortable on non-Java clients. Since we can't specify a message in the response, we thought it would be better to just send the actual (but fatal) errors.

...