...

However, some users/topics rely on remote storage for real-time analytics and need the latest data to be available as soon as possible (In fact, it only tries to stay as up-to-date as possible, but it still can’t include the latest data because the active segment hasn’t been uploaded yet.). Therefore, this optimization is offered as a topic's optional configuration rather than the default behavior.

...

This KIP introduces one new broker topic configuration property:  remote remote.log.metadata.topic.min.isrkeep.latest

public static final String REMOTE_LOG_KEEP_LATEST_CONFIG = "remote.log.

Type: short
Default: 2
Valid Values: atLeast(1)
Importance: LOW
Documentation: The minimum number of replicas that must acknowledge a write to remote log metadata topic.

keep.latest";
public static final String REMOTE_LOG_KEEP_LATEST_DOC = "Determines whether to upload all segments to remote storage including the latest ones within local retention. " +
            "When set to true (default), all committed segments will be uploaded without checking local retention constraints. " +
            "When set to false, only segments beyond local retention period will be uploaded to remote storage.";

The default value is true so that the whole remote stroage module keeps the orginial behavior when topic don't set it to falseWhen the __remote_log_metadata topic is created, the topic-level configuration (min.insync.replicas) will be set to the value of the configure.

Proposed Changes

You can refer to https://github.com/apache/kafka/pull/20811 for the detailed changes.

...