Versions Compared

Old Version 5

changes.mady.by.user Apurva Mehta

Saved on

compared with

New Version Current

changes.mady.by.user Apurva Mehta

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Update the level of effort section to match the new design.

...

To solve this problem, we propose introducing a new idempotence mode in 1.0.0, ie. enable.idempotence=requested. The other modes would be enable.idempotence=off and enable.idempotence=required. In the safe requested mode, idempotence would be best effort, ie. you get it if the broker supports it, otherwise no error is raised. In required mode, requests would fail if the broker did not support idempotence. In off mode, there is no idempotence (similar to the false value today).

The new enable.idempotence=requested mode would be introduced in the 1.0.0 release. In addition, the 1.0.0 broker will automatically downgrade messages sent in 'requested' mode from the 0.11.0 message format if the topic being written to is configured to use an older message format versionclient will disable idempotence if the message format of the destination topic doesn't support it.. In required mode, the broker would client will raise an error if it detects an incompatible message format.

This means we would need to upgrade the ProduceRequest version the TopicMetadata  in the MetadataResponse version to include the idempotent producer mode.. One potential downside here is the extra work to convert messages when the format of the client and broker is mismatchedversion of the message format for the topic in question.

Additionally, if a 1.0.0 client with enable.idempotence=requested is writing to an 0.11.0 broker, idempotence will be disabled.

Finally, a 1.0.0 producer would interpret enable.idempotence=true as enable.idempotence=required and enable.idempotence=false as enable.idempotence=off.

Level of Effort

  1. Implement broker code to downgrade Update MetadataResponse to include the message format from 0.11.0 to an older version when receiving a 'safe' mode request - 1 day.Upgrade the ProduceRequest to
  2. Use the message format version to determine whether a partition supports idempotence - 5 include the idempotent producer mode, with KIP - 2 days.
  3. Client side changes to send the idempotent producer mode, interpret old configurations of enable.idempotence– 0.5 days.

Total: 36.5 days.