Table of Contents |
---|
Status
Current state: Under DiscussionApproved
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: KAFKA-13883
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
With KRaft, Kafka added a new controller quorum to the cluster. This quorum These controllers needs to be available able to commit records for Kafka to be available. We are going One way to measure availability, is by periodically causing the high-watermark and the last committed offset to increase. Monitoring service services can compare that these last committed offset against every controller and brokeroffsets are advancing. They can also use these metrics to check that all of the brokers and controllers are relatively within each other's offset.
Public Interfaces
Cluster Metadata Records
...
NoOpRecord
Add a new record to periodically advancement of the LEO and high-watermark. Controller or broker state will not change when applying this record. This record will not be included in the cluster metadata snapshot.
Code Block |
---|
{ “apiKey”: TBD, “type”: “metadata”, “name”: “NoopRecord”“NoOpRecord”, “validVersions”: “0”, “flexibleVersions”: “0+”, “fields”: [] } |
...
kafka.controller:type=KafkaController,name=MetadataLastCommittedOffsetMetadataLastAppliedRecordOffset
Reports the offset of the last committed offset record consumed by the controller. For the active controller this may include uncommitted records. For the inactive controller this always includes committed records.kafka.controller:type=KafkaController,name=MetadataWriteOffsesMetadataLastCommittedRecordOffset
The active controller will report the offset of the latest writethat last committed offset it consumed. Inactive controllers will always report 0the same value in MetadataLastAppliedRecordOffset.kafka.controller:type=KafkaController,name=MetadataLastCommittedTimestampMetadataLastAppliedRecordTimestamp
Reports the append time of the last committed applied record batch.kafka.controller:type=KafkaController,name=MetadataLastCommittedLagMsMetadataLastAppliedRecordLagMs
Reports the difference between the local time and the append time of the last committed applied record batch.
Broker
kafka.server:type=broker-metadata-metrics,name=last-applied-record-offset
Reports the offset of the last record consumed by the broker.kafka.server.metadata:type=BrokerMetadataPublisherbroker-metadata-metrics,name=MetadataLastCommittedOffsetlast-applied-record-timestamp
kafka.server:type=broker-metadata-metrics,name=last-applied-record-lag-ms
kafka.server:type=broker-metadata-metrics,name=pending-record-processing-time-us-avg
Reports the average amount of time it took for the broker to process all pending committed records when there are pending records in the cluster metadata partition. The time unit for this metric is microseconds.kafka.server:type=BrokerMetadataPublisher,name=MetadataLastCommittedTimestamp
kafka.server.metadata:type=BrokerMetadataPublisher,name=MetadataLastCommittedLagMs
Configuration
broker-metadata-metrics,name=pending-record-processing-time-us-max
Reports the maximum amount of time it took for the broker to process all pending committed records when there are pending records in the cluster metadata partition. The time unit for this metric is microseconds.kafka.server:type=broker-metadata-metrics,name=record-batch-size-byte-avg
Reports the average byte size of the record batches in the cluster metadata partition.kafka.server:type=broker-metadata-metrics,name=record-batch-size-byte-max
Reports the maximum byte size of the record batches in the cluster metadata partition.
Configuration
- metadata.max.idle.interval.ms - The interval between writing NoOpRecord confluent.metadata.monitor.write.ms - Frequency for writing NoopRecord to the cluster metadata log. The default for this value is 500 ms.
Proposed Changes
Active Controller
The active controller will increase the LEO and high-watermark by periodically writing a no-op record (NoOpRecord) to the metadata log. The active controller will write this new record only if the IBP and metadata.version supports this feature. See the backward compatibility section for details.
The implementation will only append the NoOpRecord, if the LEO wasn’t already advanced in the defined period.
Controllers and Brokers
In both the controller and broker, the metadata replaying code will be extended to ignore NoOpRecordDescribe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.
Compatibility, Deprecation, and Migration Plan
The IBP and metadata.version will be bumped. This feature and record will only be produced if the active controller is at the expected version or greater.
Users must use the same software version of DumpLogSegment
associated with the server node when reading the metadata cluster log segments.
Rejected Alternatives
Control Records
Instead of using the NoopRecord NoOpRecord metadata record. We could have added a control record in the KRaft layer. This solution has two problems.
...