Page History

...

This proposal will introduce an optional set of ids that will provide a unique identifier for messages a producer sends to avoid duplicates.

Some Nuances

 Opt-in

Producer implementations that don't care about idempotency should not need to do anything special.

...

Now I will make the proposal a bit more explicit.

We will add a new API lease_pid which will be used to allocate a unique producer id. The API will have the following format:

Request:

lease_pid_request => topic partition pid expire_ms

...

The intended usage is as follows. The producer would issue a lease_pid request for each partition it wants to send messages to every time it establishes a new connection. When the producer first initializes it would send the pid=-1 which would cause the server to issue it a new pid in the response.

...