THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Code Block |
|---|
GetTelemetrySubscriptionsRequestV0 {
ClientInstanceId uuid // UUID4 unique for this client instance.
// Must be set to Null on the first request, and to the returned ClientInstanceId
// from the first response for all subsequent requests to any broker.
}
GetTelemetrySubscriptionsResponseV0 {
ThrottleTimeMs int32 // The duration in milliseconds for which the request was throttled due to a quota violation,
// or zero if the request did not violate any quota.
ErrorCode int16 // The error code, or 0 if there was no error.
ClientInstanceId uuid // Assigned client instance id if ClientInstanceId was Null in the request, else Null.
SubscriptionId int32 // Unique identifier for the current subscription set for this client instance.
AcceptedCompressionTypes Array[int8] // The compression types the broker accepts for PushTelemetryRequest.CompressionType
// as listed in MessageHeaderV2.Attributes.CompressionType. The array will be sorted in
// preference order from higher to lower. The CompressionType of NONE will not be
// present in the response from the broker, though the broker does support uncompressed
// client telemetry if none of the accepted compression codecs are supported by the client.
PushIntervalMs int32 // Configured push interval, which is the lowest configured interval in the current subscription set.
TelemetryMaxBytes int32 // The maximum bytes of binary data the broker accepts in PushTelemetryRequest.
DeltaTemporality bool // If True; monotonic/counter metrics are to be emitted as deltas to the previous sample.
// If False; monotonic/counter metrics are to be emitted as cumulative absolute values.
RequestedMetrics Array[string] // Requested telemetry metrics prefix string match.
// Empty array: No metrics subscribed.
// Array[0] empty string"*": All metrics subscribed.
// Array[..]: prefix string match.
}
PushTelemetryRequestV0 {
ClientInstanceId uuid // UUID4 unique for this client instance, as retrieved in the first GetTelemetrySubscriptionsRequest.
SubscriptionId int32 // SubscriptionId from the GetTelemetrySubscriptionsResponse for the collected metrics.
Terminating bool // Client is terminating.
CompressionType int8 // Compression codec used for .Metrics (ZSTD, LZ4, Snappy, GZIP, None).
// Same values as that of the current MessageHeaderV2.Attributes.
Metrics binary // Metrics encoded in OpenTelemetry MetricsData v1 protobuf format.
}
PushTelemetryResponseV0 {
ThrottleTimeMs int32 // The duration in milliseconds for which the request was throttled due to a quota violation,
// or zero if the request did not violate any quota.
ErrorCode int16 // The error code, or 0 if there was no error.
}
|
...
| Configuration | Description | Values |
|---|---|---|
metrics | A list of telemetry metric name prefixes which specify the metrics of interest. | An empty list means no metrics subscribed. A list containing just an empty string Otherwise, the list entries are prefix-matched against the metric names. |
interval.ms | The client metrics push interval in milliseconds. | Default: 300000 (5 minutes), minimum: 100, maximum: 3600000 (1 hour) |
match | The match criteria for selecting which clients the subscription matches. If a client matches all of these criteria, the client matches the subscription. | A list of key-value pairs. The valid keys are:
The values are anchored regular expressions. |
...
API Request | Resource | ACL operation |
|---|---|---|
DescribeConfigs | CLUSTER | DESCRIBE_CONFIGS |
AlterConfigs | CLUSTER | ALTER_CONFIGS |
| GetTelemetrySubscriptions | N/A | N/A |
PushTelemetry | N/A | N/A |
...
The kafka-configs.sh tool can be used to administer client metrics subscriptionsconfiguration. The kafka-client-metrics.sh tool is preferred because it is more useable.
...
A new kafka-client-metrics.sh tool is added which provides a simpler interface for administering client metrics subscriptions configuration resources than using kafka-configs.sh directly.
...
| Code Block |
|---|
This tool helps to manipulate and describe client metrics subscriptionsconfigurations. Option Description ------ ----------- --alter Alter the configuration of the client metrics subscriptionresource. --block Block metrics collection. --bootstrap-server <String: server to REQUIRED: The Kafka server to connect to. This connect to> --command-config <String: command Property file containing configs to be config property file> is required for describing and passed to Admin Client. --delete altering broker configs. --command-config <String: commandDelete the configuration of the client Property file containing configs to be config property file> passed to Admin Client. This is used metrics resource. --describe List configurations for client metrics resources. only with --bootstrapgenerate-servername option Generate a UUID to for describing and altering broker configs. --delete Delete the configuration of the client metrics subscription. --describeuse as the name. --help List configs for the clientPrint metricsusage subscriptions. --generate-name information. Generate a UUID to use as the name. --interval The metrics push interval in milliseconds. --match Matching selector ‘k1=v1,k2=v2’. The following is a list of valid selector names: client_instance_id client_id client_software_name client_software_version client_source_address client_source_port --metrics Telemetry metric name prefixes ‘m1,m2’. --name <String> Name of client metrics subscription. --help Print usage information. configuration resource. --version Display Kafka version. |
Here are some examples.
List all client metrics
...
configuration resources
| Code Block | ||
|---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --describe $ kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type client-metrics |
Describe a client metrics
...
configuration resources
| Code Block | ||
|---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --describe --name METRICSUBMYMETRICS $ kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type client-metrics --entity-name METRICSUBMYMETRICS |
Create a client metrics
...
configuration resource, generating a unique name
In this example, --generate-name causes the tool to create a type-4 UUID to use as the client metrics subscription configuration resource name. There is no equivalent in kafka-configs.sh .
| Code Block | ||
|---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --generate-name \ --metrics org.apache.kafka.producer.node.request.latency.,org.apache.kafka.consumer.node.request.latency. \ --interval 60000 |
Create a client metrics
...
configuration resource for all Python v1.2.* clients
| Code Block | ||
|---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --name METRICSUBMYMETRIC \ --metrics org.apache.kafka.consumer. \ --interval 60000 \ --match "client_software_name=kafka_python,client_software_version=1\.2\..*" $ kafka-configs.sh --bootstrap-server $BROKERS --alter --entity-type client-metrics --entity-name METRICSUBMYMETRICS \ --add-config "metrics=org.apache.kafka.consumer.,interval.ms=60000,match=[client_software_name=kafka.python,client_software_version=1\.2\..*]" |
Block an existing client metrics
...
configuration resource from pushing metrics
Blocking pushing metrics for a client metrics subscription configuration resource is achieved by setting the push interval to 0ms.
| Code Block | ||
|---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --name METRICSUBMYMETRICS --block $ kafka-configs.sh --bootstrap-server $BROKERS --alter --entity-type client-metrics --entity-name METRICSUBMYMETRICS \ --add-config "interval.ms=0" |
Delete a client metrics
...
configuration resource
Deletion of a client metrics subscription configuration resource with kafka-configs.sh requires listing the names of the configs to delete.
| Code Block | ||
|---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --delete --name METRICSUBMYMETRICS $ kafka-configs.sh --bootstrap-server $BROKERS --alter --entity-type client-metrics --entity-name METRICSUBMYMETRICS \ --delete-config metrics,interval.ms,match |
...