THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- New PushTelemetryRequest protocol request type.
- Updated metrics receiver interface in the broker to accustom for the new semantics and format.
- A new CLIENT_METRICS ConfigEntry resource type for configuring metrics subscriptions.
- kafka-client-metrics.sh CLI script to configure and list client metrics subscriptions.
Default behaviour
The metric collection is opt-in on the broker and opt-out on the client.
For metrics to be collected a MetricsPlugin (see below) must be configured on the brokers, and at least one metrics subscription must be configured through the Admin API. Only then will metrics subscriptions be propagated to clients, and only then will clients push metrics to the broker. It is thus up to the cluster operator to explicitly enable client metrics collection.
Protocol
| Code Block |
|---|
GetTelemetrySubscriptionsRequestV0 {
ClientInstanceId uuid // UUID4 unique for this client instance.
// Must be set to Null on the first request, and to the
// returned ClientInstanceId from the first response
// for all subsequent requests to any broker.
}
GetTelemetrySubscriptionsResponseV0 {
ThrottleTime int32 // Standard throttling
ErrorCode int16 // Error code
SubscriptionId int32 // Unique identifier for the current subscription set for this client instance.
AcceptedCompressionTypes Array[int8] // The compression types the broker accepts for PushTelemetryRequest.CompressionType.
// Calculated as a bitmask of (1 << MessageHeaderV2.Attributes.CompressionType).
PushIntervalMs int32 // Configured push interval, which is the lowest configured interval in the current subscription set.
RequestedMetrics Array[string] // Requested Metrics prefix string match.
// Empty array: No metrics subscribed.
// Array[0] empty string: All metrics subscribed.
// Array[..]: prefix string match
}
PushTelemetryRequestV0 {
ClientInstanceId uuid // UUID4 unique for this client instance, as retrieved in the first GetTelemetrySubscriptionsRequest.
SubscriptionId int32 // SubscriptionId from the GetTelemetrySubscriptionsResponse for the collected metrics.
Terminating bool // Client is terminating.
CompressionType int8 // Compression codec used for .Metrics (ZSTD, LZ4, Snappy, GZIP, None).
// Same values as that of the current MessageHeaderV2.Attributes.
Metrics binary // Format specified by ContentType, possibly compressed.
}
PushTelemetryResponseV0 {
ThrottleTime int32 // Standard and metric-specific throttling
ErrorCode int16 // Error code
}
|
...
enable.metrics.push=true|false (default: true) -Allows disabling the functionality outlined in this proposal which should be enabled by default. (Or enable.telemetry?)
New error codes
UnknownSubscriptionId - See Error handling chapter.
...