...

PR: https://github.com/apache/kafka/pull/20811

Motivation

The __remote_log_metadata topic is a critical component of Currently, Kafka's Tiered Storage feature (KIP-405) as the default implemetation, storing essential metadata including:

• RemoteLogSegmentMetadata: Remote log segment lifecycle state transitions along with segment identifiers, offset ranges 
      (startOffset, endOffset), leader epoch mappings, timestamps, segment sizes, 
      and custom RSM-specific metadata (e.g., remote storage paths)
• RemoteLogSegmentMetadataUpdate: State transition events for existing remote log segments
• RemotePartitionDeleteMetadata: Partition-level deletion state tracking

Currently, this topic lacks an explicit min.insync.replicas (min.isr) configuration and relies on the broker-level default, which is typically 1. 
With the default configuration of replication.factor=3 and min.isr=1, the system is vulnerable to metadata loss in the following scenario:
Image Removed

This data loss is particularly severe because remote data can becomes unreachable even though it still exists in remote storage.

As we knonw, .the 3 factor + 2 min sync in the stardard best practice for kafka topic. Besides this. it is inconsistency with other critical internal metadata topics: 

 __transaction_state: Explicitly sets min.isr via the  transaction.state.log.min.isr broker configuration (default: 2) 

tiered storage implementation uploads all non-active local log segments to remote storage immediately, even when they are still within the local retention period. 
This results in redundant storage of the same data in both local and remote tiers.

 Whenthere is no requirement for real-time analytics or immediate consumption from remote storage. It has the following drawbacks:

1. Wastes storage capacity and costs: The same data is stored twice during the local retention window
2. Provides no immediate benefit: During the local retention period, reads prioritize local data, making the remote copy unnecessary

Example Scenario：

Consider a topic with:
- Local retention: 1 day (24 hours)
- Remote retention: 7 days (168 hours)

Currently, data is stored in both tiers for the first 24 hours, resulting in 24 hours of 
redundant storage. With typical data volumes, this can lead to significant cost increases.

However, some users/topics rely on remote storage for real-time analytics and need the 
latest data to be available as soon as possible. Therefore, this optimization should be 
offered as an **optional configuration** rather than the default behaviorGiven that __remote_log_metadata stores equally critical metadata, it should have equivalent durability guarantees.

Public Interfaces

This KIP introduces one new broker configuration property:  remote.log.metadata.topic.min.isr

...