Status

Current state: "Vote in progressUnder Discussion"

Discussion thread: here

Vote thread: here

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

TopicBasedRemoteLogMetadataManager(TBRLMM) is the key built-in implementation for tiered-storage feature. It uses the Kafka topic to maintain the metadata for remote storage.

Thus. when we begin to support tiered-storage with it. We found that the TBRLMM's initialization failed in some Kafka clusters sometimes.

The root cause is the initialization(TopicBasedRemoteLogMetadataManager#initializeResources) will use the __remote_log_metadata  but the server is not ready for handle the request.

We can check with broker's startup sequence refer to follow snapshot:

Image Removed

So the initialization will have to relay on the retry to get success until the broker is ready. The retry many happen with multiple times:

FYI:  according to one of kamalcph 's test result: 
Image Removed

Thus. the critical bad case is that the default retry time (DEFAULT_REMOTE_LOG_METADATA_INITIALIZATION_RETRY_MAX_TIMEOUT_MS: 2 Minutes) is not enough for some Kafka cluster which take > 2 minutes to complete the startup. 

Base on this. the fail will happen. As one result. The fail will cause the feature broken and local disk never get deleted and some other issues for different cases.

Image Removed

As one workaround solution. we can check every kafka cluster's  startup consume time and set a very big value for the DEFAULT_REMOTE_LOG_METADATA_INITIALIZATION_RETRY_MAX_TIMEOUT_MS due to the startup time may increased in future.

But It is not reasonable for one configure need to change again and again or set to a very large value . What's more, as we mentioned above you will find there are lots of warn log which hint the connection not available. This can be avoided by this change.

...

[2025-07-19 18:00:17,923] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (10.20.4.98:9559) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)

Apache Kafka supports rack-aware partition assignment to improve fault tolerance and reduce cross-rack network traffic. However, the current Admin API does not expose rack information for all type of consumer group members, despite this information being available at the protocol level.

Current State

The ConsumerGroupDescribeResponse (API Key 69) protocol includes a rackId field for each group member, as defined in the protocol specification:

{ "name": "RackId", "type": "string", "versions": "0+", 
  "nullableVersions": "0+", "default": "null",
  "about": "The member rack ID." }

However, when users call AdminClient.describeConsumerGroups(), the returned MemberDescription objects do not include this rack information. The rack ID is available in the wire protocol but is discarded during response processing in DescribeConsumerGroupsHandler.

The Similar case is for AdminClient.describeShareGroups():


Problem Statement

This limitation creates several issues:

1. Monitoring and Observability: Operators cannot determine the rack distribution of consumer group members through the Admin API, making it difficult to verify that rack-aware assignment is working correctly.

2. Inconsistency: Other group types expose rack information:

   • StreamsGroupMemberDescription includes rackId() method
   • The underlying protocol supports it for consumer groups
   • Only the public Admin API omits this information
3. Diagnostics: When troubleshooting rack-aware assignment issues or network problems, operators need to use lower-level tools or custom code to access rack information that should be readily available.
4. Third-party Tools: Monitoring and management tools built on the Admin API cannot display rack information, limiting their usefulness for rack-aware deployments

...

1. .

Public Interfaces

add dedicated public interface for this case:

...