Page History

remote.log.storage.enable - Whether to enable remote log storage or not. Valid values are `true` or `false` and the default value is false. This property gives backward compatibility.

remote.log.storage.manager.class.name - This is mandatory if the remote.log.storage.enable is set as true.

remote.log.metadata.manager.class.name(optional) - This is an optional property. If this is not configured, Kafka uses an inbuilt metadata manager backed by an internal topic.

(These configs are dependent on remote storage manager implementation)

remote.log.storage.*

(These configs are dependent on remote log metadata manager implementation)

remote.log.metadata.*

remote.log.manager.thread.pool.size
Remote log thread pool size, which is used in scheduling tasks to copy segments, fetch remote log indexes and clean up remote log segments.

remote.log.manager.task.interval.ms
The interval at which remote log manager runs the scheduled tasks like copy segments, fetch remote log indexes and clean up remote log segments.

remote.log.reader.threads
Remote log reader thread pool size

remote.log.reader.max.pending.tasks
Maximum remote log reader thread pool task queue size. If the task queue is full, broker will stop reading remote log segments.

Below retention configs are similar to the local log retention. This configuration is used to determine how long the remote log segments are to be retained in the remote local storage.  remote.Existing log.retention.* are retention configs for the topic partition which includes both local and remote storage. 

localms
The number of milli seconds to keep the remote log segment before it gets deleted. If not set, the value in `remote.log.retention.minutes` is used. If set to -1, no time limit is applied.remote.log.retention.minutesms
The number of minutes milli seconds to keep the remote local log segment before it gets deleted. If not set, the value in `remote.log.retention.hours` minutes` is used. If set to -1, no time limit is applied.

remote.log.retention.hours
The number of hours to keep the remote log segment before it gets deleted. If not set, default value of 336 (2 weeks) is used. If set to -1, no time limit is applied.

remote.local.log.retention.bytes
The maximum size of remote local log segments that can grow for a partition before it deletes the old segments. There is no default value, but the above time based retention always applies.