Current state: Accepted
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
As per the Tiered Storage feature introduced in KIP-405, we added several metrics related to reads(from) and writes(to) for remote storage. The naming convention that was followed is confusing to the users.
For eg. in regular Kafka, BytesIn means bytes written to the log, and BytesOut means bytes read from the log. But with tiered storage, the concepts are reversed.
We should rename the tiered storage related metrics to remove any ambiguity.
The following metrics will be renamed:
Original Name | Description | New Name |
kafka.server:type=BrokerTopicMetrics, name=RemoteBytesInPerSec, topic=([-.w]+) | The number of bytes read from remote storage per second for a topic | kafka.server:type=BrokerTopicMetrics, name=RemoteFetchBytesPerSec, topic=([-.w]+) |
kafka.server:type=BrokerTopicMetrics, name=RemoteReadRequestsPerSec, topic=([-.w]+) | The number of remote storage read requests per second for a topic | kafka.server:type=BrokerTopicMetrics, name=RemoteFetchRequestsPerSec, topic=([-.w]+) |
kafka.server:type=BrokerTopicMetrics, name=RemoteReadErrorPerSec, topic=([-.w]+) | The number of remote storage read errors per second for a topic | kafka.server:type=BrokerTopicMetrics, name=RemoteFetchErrorsPerSec, topic=([-.w]+) |
kafka.server:type=BrokerTopicMetrics, name=RemoteBytesOutPerSec, topic=([-.w]+) | The number of bytes copied to remote storage per second for a topic | kafka.server:type=BrokerTopicMetrics, name=RemoteCopyBytesPerSec, topic=([-.w]+) |
kafka.server:type=BrokerTopicMetrics, name=RemoteWriteRequestsPerSec, topic=([-.w]+) | The number of remote storage write requests per second for a topic | kafka.server:type=BrokerTopicMetrics, name=RemoteCopyRequestsPerSec, topic=([-.w]+) |
kafka.server:type=BrokerTopicMetrics, name=RemoteWriteErrorPerSec, topic=([-.w]+) | The number of remote storage write errors per second for a topic | kafka.server:type=BrokerTopicMetrics, name=RemoteCopyErrorsPerSec, topic=([-.w]+) |
kafka.server:type=BrokerTopicMetrics, name=RemoteBytesInPerSec | Total number of bytes read from remote storage per second on a broker | kafka.server:type=BrokerTopicMetrics, name=RemoteFetchBytesPerSec |
kafka.server:type=BrokerTopicMetrics, name=RemoteReadRequestsPerSec | Total number of remote storage read requests per second on a broker | kafka.server:type=BrokerTopicMetrics, name=RemoteFetchRequestsPerSec |
kafka.server:type=BrokerTopicMetrics, name=RemoteReadErrorPerSec | Total number of remote storage read errors per second on a broker | kafka.server:type=BrokerTopicMetrics, name=RemoteFetchErrorsPerSec |
kafka.server:type=BrokerTopicMetrics, name=RemoteBytesOutPerSec | Total number of bytes copied to remote storage per second on a broker | kafka.server:type=BrokerTopicMetrics, name=RemoteCopyBytesPerSec |
kafka.server:type=BrokerTopicMetrics, name=RemoteWriteRequestsPerSec | Total number of remote storage write requests per second on a broker | kafka.server:type=BrokerTopicMetrics, name=RemoteCopyRequestsPerSec |
kafka.server:type=BrokerTopicMetrics, name=RemoteWriteErrorPerSec | Total number of remote storage write errors per second on a broker | kafka.server:type=BrokerTopicMetrics, name=RemoteCopyErrorsPerSec |
The metrics that are being renamed were only added as part of the same release. Hence, we do not need to maintain backward compatibility for the renamed metrics.
Existing Unit Tests for the metrics being renamed.
None