DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Public Interfaces
This KIP introduces:
(1) one new topic configuration item: remote.logcopy.latestlazy.enable
BTW: topic's remote storage feature already had some others configure items such as remote.log.delete.on.disable, remote.log.copy.disable, etc.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
public static final String REMOTE_LOGCOPY_LATESTLAZY_ENABLE_CONFIG = "remote.logcopy.latestlazy.enable"; public static final String REMOTE_LOGCOPY_LATESTLAZY_ENABLE_DOC = "Determines whether to upload all non-activedelay uploading segments to remote storage, including those still within local retention. " + "When set to truefalse (default), all non-active segments will be uploaded immediately without checking local retention constraints. " + "When set to falsetrue, only segments beyondwithin local retention periodwill willnot be uploaded to remote storage.", the upload is delayed until they expire."; |
(2) one new server configuration item: log.remote.copy.lazy.enable
| Code Block | ||||
|---|---|---|---|---|
| ||||
public static final String LOG_REMOTE_COPY_LAZY_ENABLE_CONFIG = LOG_PREFIX + TopicConfig.REMOTE_COPY_LAZY_ENABLE_CONFIG;
public static final String LOG_REMOTE_COPY_LAZY_ENABLE_DOC = TopicConfig.REMOTE_COPY_LAZY_ENABLE_DOC; |
The default value is true so that the whole remote storage module keeps the original behavior when topic don't set it to false.
...
You can see the uploading will be delayed if the configure remoteLogKeepLatestremote.copy.lazy.enable is falsetrue. And After the change, the remote tiered storage redundancy will be reduced with delayed upload.
You can refer to the test case and result: https://github.com/apache/kafka/pull/20913#issuecomment-3547156286
...