You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Apache Kafka version 3.7.0 is shipped with an early access of the new generation of the consumer rebalance protocol (KIP-848). The next generation of the consumer rebalance protocol has three major changes: (1) the new consumer rebalance protocol; (2) a brand new group coordinator; and (3) a brand new threading model for the consumer. It is not recommended for production environments. Instead, we advice users to test it in clusters created for testing the feature.

Limitations:

  • We don't plan to support upgrading clusters using the early access feature. We are still actively developing the feature and we want to keep the flexibility for doing non-backward compatible changes.
  • The kafka-consumer-groups.sh command line tool is not able to describe consumer groups using the new protocol.
  • The transaction verification (KIP-890) is not supported and must be disabled.  

How to test it?

The simplest way is to create a new KRaft cluster following the instruction here but with the additional configurations added to config/kraft/server.properties:

group.coordinator.rebalance.protocols=classic,consumer
transaction.partition.verification.enable=false

Then, it is possible to configure the kafka-console-consumer.sh command line tool to use it as follow:

bin/kafka-console-consumer.sh --topic <TOPIC> --from-beginning --bootstrap-server localhost:9092 --consumer-property group.protocol=consumer

It is also possible to configure the KafkaConsumer by setting the group.protocol property to consumer.

final Map<String, Object> configs = new HashMap<>();
configs.put(ConsumerConfig.GROUP_PROTOCOL_CONFIG, "consumer");
final ConsumerConfig consumerConfig = new ConsumerConfig(configs);

How to report issues?

Feel free to file a bug in the Apache Kafka project for any discovered issue. We appreciate your help to test it. Happy testing!

  • No labels