...
Code Block |
---|
GetTelemetrySubscriptionsRequestV0 { ClientInstanceId uuid // UUID4 unique for this client instance. // Must be set to Null on the first request, and to the returned ClientInstanceId // from the first response for all subsequent requests to any broker. } GetTelemetrySubscriptionsResponseV0 { ThrottleTimeMs int32 // The duration in milliseconds for which the request was throttled due to a quota violation, // or zero if the request did not violate any quota. ErrorCode int16 // The error code, or 0 if there was no error. ClientInstanceId uuid // Assigned client instance id if ClientInstanceId was Null in the request, else Null. SubscriptionId int32 // Unique identifier for the current subscription set for this client instance. AcceptedCompressionTypes Array[int8] // The compression types the broker accepts for PushTelemetryRequest.CompressionType // as listed in MessageHeaderV2.Attributes.CompressionType. The array will be sorted in // preference order from higher to lower. The CompressionType of NONE will not be // present in the response from the broker, though the broker does support uncompressed // client telemetry if none of the accepted compression codecs are supported by the client. PushIntervalMs int32 // Configured push interval, which is the lowest configured interval in the current subscription set. TelemetryMaxBytes int32 // The maximum bytes of binary data the broker accepts in PushTelemetryRequest. DeltaTemporality bool // If True; monotonic/counter metrics are to be emitted as deltas to the previous sample. // If False; monotonic/counter metrics are to be emitted as cumulative absolute values. RequestedMetrics Array[string] // Requested telemetry metrics prefix string match. // Empty array: No metrics subscribed. // Array[0] empty string"*": All metrics subscribed. // Array[..]: prefix string match. } PushTelemetryRequestV0 { ClientInstanceId uuid // UUID4 unique for this client instance, as retrieved in the first GetTelemetrySubscriptionsRequest. SubscriptionId int32 // SubscriptionId from the GetTelemetrySubscriptionsResponse for the collected metrics. Terminating bool // Client is terminating. CompressionType int8 // Compression codec used for .Metrics (ZSTD, LZ4, Snappy, GZIP, None). // Same values as that of the current MessageHeaderV2.Attributes. Metrics binary // Metrics encoded in OpenTelemetry MetricsData v1 protobuf format. } PushTelemetryResponseV0 { ThrottleTimeMs int32 // The duration in milliseconds for which the request was throttled due to a quota violation, // or zero if the request did not violate any quota. ErrorCode int16 // The error code, or 0 if there was no error. } |
...
Configuration | Description | Values |
---|---|---|
metrics | A list of telemetry metric name prefixes which specify the metrics of interest. | An empty list means no metrics subscribed. A list containing just an empty string Otherwise, the list entries are prefix-matched against the metric names. |
interval.ms | The client metrics push interval in milliseconds. | Default: 300000 (5 minutes), minimum: 100, maximum: 3600000 (1 hour) |
match | The match criteria for selecting which clients the subscription matches. If a client matches all of these criteria, the client matches the subscription. | A list of key-value pairs. The valid keys are:
The values are anchored regular expressions. |
...
API Request | Resource | ACL operation |
---|---|---|
DescribeConfigs | CLUSTER | DESCRIBE_CONFIGS |
AlterConfigs | CLUSTER | ALTER_CONFIGS |
GetTelemetrySubscriptions | N/A | N/A |
PushTelemetry | N/A | N/A |
...
The kafka-configs.sh
tool can be used to administer client metrics subscriptionsconfiguration. The kafka-client-metrics.sh
tool is preferred because it is more useable.
...
A new kafka-client-metrics.sh
tool is added which provides a simpler interface for administering client metrics subscriptions configuration resources than using kafka-configs.sh
directly.
...
Code Block |
---|
This tool helps to manipulate and describe client metrics subscriptionsconfigurations. Option Description ------ ----------- --alter Alter the configuration of the client metrics subscriptionresource. --block Block metrics collection. --bootstrap-server <String: server to REQUIRED: The Kafka server to connect to. This connect to> --command-config <String: command Property file containing configs to be config property file> is required for describing and passed to Admin Client. --delete altering broker configs. --command-config <String: commandDelete the configuration of the client Property file containing configs to be config property file> passed to Admin Client. This is used metrics resource. --describe List configurations for client metrics resources. only with --bootstrapgenerate-servername option Generate a UUID to for describing and altering broker configs. --delete Delete the configuration of the client metrics subscription. --describeuse as the name. --help List configs for the clientPrint metricsusage subscriptions. --generate-name information. Generate a UUID to use as the name. --interval The metrics push interval in milliseconds. --match Matching selector ‘k1=v1,k2=v2’. The following is a list of valid selector names: client_instance_id client_id client_software_name client_software_version client_source_address client_source_port --metrics Telemetry metric name prefixes ‘m1,m2’. --name <String> Name of client metrics subscription. --help Print usage information. configuration resource. --version Display Kafka version. |
Here are some examples.
List all client metrics
...
configuration resources
Code Block | ||
---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --describe $ kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type client-metrics |
Describe a client metrics
...
configuration resources
Code Block | ||
---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --describe --name METRICSUBMYMETRICS $ kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type client-metrics --entity-name METRICSUBMYMETRICS |
Create a client metrics
...
configuration resource, generating a unique name
In this example, --generate-name
causes the tool to create a type-4 UUID to use as the client metrics subscription configuration resource name. There is no equivalent in kafka-configs.sh
.
Code Block | ||
---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --generate-name \ --metrics org.apache.kafka.producer.node.request.latency.,org.apache.kafka.consumer.node.request.latency. \ --interval 60000 |
Create a client metrics
...
configuration resource for all Python v1.2.* clients
Code Block | ||
---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --name METRICSUBMYMETRIC \ --metrics org.apache.kafka.consumer. \ --interval 60000 \ --match "client_software_name=kafka_python,client_software_version=1\.2\..*" $ kafka-configs.sh --bootstrap-server $BROKERS --alter --entity-type client-metrics --entity-name METRICSUBMYMETRICS \ --add-config "metrics=org.apache.kafka.consumer.,interval.ms=60000,match=[client_software_name=kafka.python,client_software_version=1\.2\..*]" |
Block an existing client metrics
...
configuration resource from pushing metrics
Blocking pushing metrics for a client metrics subscription configuration resource is achieved by setting the push interval to 0ms.
Code Block | ||
---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --name METRICSUBMYMETRICS --block $ kafka-configs.sh --bootstrap-server $BROKERS --alter --entity-type client-metrics --entity-name METRICSUBMYMETRICS \ --add-config "interval.ms=0" |
Delete a client metrics
...
configuration resource
Deletion of a client metrics subscription configuration resource with kafka-configs.sh
requires listing the names of the configs to delete.
Code Block | ||
---|---|---|
| ||
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --delete --name METRICSUBMYMETRICS $ kafka-configs.sh --bootstrap-server $BROKERS --alter --entity-type client-metrics --entity-name METRICSUBMYMETRICS \ --delete-config metrics,interval.ms,match |
...