Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This change will impact external monitoring systems that rely on the old metric name; corresponding updates will be required.Due to 

Since `REMOTE_LOG_READER_TASK_QUEUE_SIZE_METRIC` and `REMOTE_LOG_READER_AVG_IDLE_PERCENT_METRIC` are part of the public API, we introduce corresponding deprecated fields as a temporary bridge.

These fields are marked as deprecated because they serve as a transitional solution. They will be removed in Kafka 5.0.

Code Block
languagejava
    @Deprecated(since = "4.2")
    public static final MetricName BRIDGE_REMOTE_LOG_READER_TASK_QUEUE_SIZE_METRIC = getMetricName(
            "kafka.log.remote", "RemoteStorageThreadPool", REMOTE_LOG_READER_TASK_QUEUE_SIZE);
    @Deprecated(since = "4.2")
    public static final MetricName BRIDGE_REMOTE_LOG_READER_AVG_IDLE_PERCENT_METRIC = getMetricName(
            "kafka.log.remote", "RemoteStorageThreadPool", REMOTE_LOG_READER_AVG_IDLE_PERCENT);

Compatibility, Deprecation, and Migration Plan

  1. We will add a deprecated annotation on following constant
    1. `RemoteStorageMetrics#REMOTE_LOG_READER_TASK_QUEUE_SIZE_METRIC`
    2. `RemoteStorageMetrics#REMOTE_LOG_READER_AVG_IDLE_PERCENT_METRIC`
    3. `AssignmentsManager#QUEUED_REPLICA_TO_DIR_ASSIGNMENTS_METRIC`
  2. Add two new bridge fields to `RemoteStorageMetrics` and mark them as deprecated:
    1. `BRIDGE_REMOTE_LOG_READER_TASK_QUEUE_SIZE_METRIC`
    2. `BRIDGE_REMOTE_LOG_READER_AVG_IDLE_PERCENT_METRIC`
  3. Update the Kafka documentation to note MBean will be replaced
    1. `org.apache.kafka.storage.internals.log:type=RemoteStorageThreadPool.RemoteLogReaderTaskQueueSize`
    2. `org.apache.kafka.storage.internals.log:type=RemoteStorageThreadPool.RemoteLogReaderAvgIdlePercent`
    3. `org.apache.kafka.server:type=AssignmentsManager.QueuedReplicaToDirAssignments`
  4. Register folowing new metrics
    1. `kafka.log.remote:type=RemoteStorageThreadPool.RemoteLogReaderTaskQueueSize`
    2. `kafka.log.remote:type=RemoteStorageThreadPool.RemoteLogReaderAvgIdlePercent`
    3. `kafka.server:type=AssignmentsManager.QueuedReplicaToDirAssignments`
  5. Delete following all usage in code base at the Kafka 5.0
    1. `RemoteStorageMetrics#REMOTE_LOG_READER_TASK_QUEUE_SIZE_METRIC`
    2. `RemoteStorageMetrics#REMOTE_LOG_READER_AVG_IDLE_PERCENT_METRIC`
    3. `AssignmentsManager#QUEUED_REPLICA_TO_DIR_ASSIGNMENTS_METRIC`
    4. `BRIDGE_REMOTE_LOG_READER_TASK_QUEUE_SIZE_METRIC`
    5. `BRIDGE_REMOTE_LOG_READER_AVG_IDLE_PERCENT_METRIC`
  6. Update the group name to `kafka.log.remote` for the following fields in `RemoteStorageMetrics`:
    1. REMOTE_LOG_READER_TASK_QUEUE_SIZE_METRIC
    2. REMOTE_LOG_READER_AVG_IDLE_PERCENT_METRIC

Test Plan

  • Add new tests to verify the new metric is correctly registered and behaves as expected.
  • Ensure backward compatibility by verifying deprecated metric behavior until removal.

...