...

1. Wastes storage capacity and costs: The same data is stored twice during the local retention window
2. Provides no immediate benefit: During the local retention period, reads prioritize local data, making the remote copy unnecessary

Example Scenari Scenario and The goal:

Consider a topic with remote stroage storage enable:
- Local retention: 1 day (24 hours)
- Remote retention: 3 days (72 hours)

...

So we can reduce the tiered storage redundancy for cost saving. In this case, it will save about 25% cost pay for total size of disk.

Image AddedImage Removed

However, some users/topics rely on remote storage for real-time analytics and need the latest data to be available as soon as possible (In fact, it only tries to stay as up-to-date as possible, but it still can’t include the latest data because the active segment hasn’t been uploaded yet.). Therefore, this optimization is offered as a topic's optional configuration rather than the default behavior.

...

You can refer to https://github.com/apache/kafka/pull/20913 for the detailed changes.

We change the candidateLogSegments for remote upload as followed: Image RemovedRemoteLogManager.RLMCopyTask#candidateLogSegments's logic for decide one segment if need to upload to remote: 

Image Added

You can see the uploading will be delayed if the configure remoteLogKeepLatest is false. And After the change, the remote tiered storage redundancy will be reduced with delayed upload.
You can refer to the test case and result: https://github.com/apache/kafka/pull/20913#issuecomment-3547156286

Compatibility, Deprecation, and Migration Plan 

• Backward Compatibility
  This change is fully backward compatible:
  Backward compatible: Default value (true) maintains current behavior
  Forward compatible: Older clients unaware of this config will ignore it
  
  

• Deprecation
  N/A
• Migration for Existing Deployments
  N/A. The feature is optional with topic level.

...