Status

Current state: Under Discussion

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

This KIP aims to enforce offsets.topic.replication.factor upon __consumer_offsets topic creation.

...

Rationale for the prior behavior can be found in 

Public Interfaces

Set the offsets.topic.replication.factor to 1 in config/server.properties to maintain existing single-broker quickstart behavior.

Proposed Changes

Internal topic creation can happen in five paths:

...

This is a bug fix KIP impacting the setup of new clusters. Users setting up a cluster should keep in mind that the __consumer_offsets topic will not be created until their cluster satisfies the offsets.topic.replication.factor.

Rejected Alternatives

One rejected alternative was to push __consumer_offsets topic creation logic out of the brokers and into the KafkaController. Since the KafkaController can detect broker membership changes through zookeeper, it can create the __consumer_offets topic as soon as the offsets.topic.replication.factor is met. While doable, it is more complicated as it would add even more logic to keep track of in the already complicated KafkaController and would additionally require KafkaApis to instead lookup cluster readiness based on its MetadataCache when responding to GroupCoordinatorRequests.