DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
This KIP introduces one new topic configuration propertyitem: remote.log.keep.latest
...
You can see the uploading will be delayed if the configure remoteLogKeepLatest is false. And After the change, the remote tiered storage redundancy will be reduced with delayed upload.
You can refer to the test case and result: https://github.com/apache/kafka/pull/20913#issuecomment-3547156286
BTW: Here are some additional thoughts/considerations.
- Local files won’t be deleted until they’ve been uploaded to the remote storage, so this change is very safe
You don’t need to worry about files being cleaned up before they be upload to the remote. - Considering the latency of remote storage, the local retention period won’t be set too short.
For example, in our production environment, we keep one day of local data alongside 3-7 days in remote storage, so there’s still one day of redundancy.
Compatibility, Deprecation, and Migration Plan
...
Reason for rejection: Some users require real-time remote analytics and need data uploaded as soon as possible. Breaking their use case would be unacceptable. after all it is the default behavior before this change.
Alternative 2: Global broker-level configuration only
...