Status
Current state: Accepted
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
When running kafka-metadata-quorum script to get the quorum replication status, the LastFetchTimestamp and LastCaughtUpTimestamp output is not human-readable.
I will be convenient to add an optional flag (-hr, --human-readable) to enable a human-readable format showing the delay in ms (i.e. 366 ms ago).
This dealy is computed as (now - timestamp), where they are both represented as Unix time (UTC based).
Public Interfaces
A new optional flag called --human-readable with the shortcut -hr will be added to the kafka-metadata-quorum tool.
$ bin/kafka-metadata-quorum.sh describe -h usage: kafka-metadata-quorum --bootstrap-server BOOTSTRAP_SERVER describe [-h] [--status] [--replication] [-hr] optional arguments: -h, --help show this help message and exit Status: --status A short summary of the quorum status and the other provides detailed information about the status of replication. Replication: --replication Detailed information about the status of replication -hr, --human-readable Print human-readable timestamps
Proposed Changes
The current default --replication output is as follows and will remain unchanged.
$ bin/kafka-metadata-quorum.sh --bootstrap-server :9092 describe --replication NodeId LogEndOffset Lag LastFetchTimestamp LastCaughtUpTimestamp Status 2 551 0 1683732558012 1683732558012 Leader 3 551 0 1683732557573 1683732557573 Follower 4 551 0 1683732557573 1683732557573 Follower
Instead, when passing the new flag, we will have the following output.
$ bin/kafka-metadata-quorum.sh --bootstrap-server :9092 describe --replication -hr NodeId LogEndOffset Lag LastFetchTimestamp LastCaughtUpTimestamp Status 2 8609 0 4 ms ago 4 ms ago Leader 3 8609 0 155 ms ago 155 ms ago Follower 4 8609 0 155 ms ago 155 ms ago Follower $ bin/kafka-metadata-quorum.sh --bootstrap-server :9092 describe --replication --human-readable NodeId LogEndOffset Lag LastFetchTimestamp LastCaughtUpTimestamp Status 2 8689 0 4 ms ago 4 ms ago Leader 3 8689 0 148 ms ago 148 ms ago Follower 4 8689 0 148 ms ago 148 ms ago Follower
Compatibility, Deprecation, and Migration Plan
The default behavior will remain unchanged to avoid breaking any tool relying on the current timestamp output.
Test Plan
A new test method will be added to MetadataQuorumCommandTest to verify default and new behaviors with regard to the timestamp output.
Rejected Alternatives
Using a date/time formatting (i.e. May 10, 15:54:55 UTC) won't add much information in most cases and you will have to deal with locale.