THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
The existing Kafka client metric names consist of two parts: a group name and a metric name. The telemetry metric name is derived by concatenation of:
- The prefix
"messagingorg.apache.kafka." - The group name with
"-metrics"removed, and every '-' replaced with '.' - The metric name with every '-' replaced with '.'
...
| Kafka metric name | Telemetry metric name |
|---|---|
| "connection-creation-rate", group="producer-metrics" |
|
| "rebalance-latency-max", group="consumer-coordinator-metrics" |
|
Other vendor or implementation-specific metrics can be added according to the following examples, using "contrib" followed by an implementation-specific reverse domain name as the namespace to avoid clashes:
| Implementation-specific metric name | Telemetry metric name |
|---|---|
Client "io.confluent.librdkafka" Metric name "client.produce.xmitq.latency" |
|
Python client "com.example.client.python" Metric name "object.count" |
|
...
| Kafka metric name | Telemetry metric name |
|---|---|
| "request-latency-avg", group="producer-node-metrics", client-id={client-id}, node-id={node-id} |
Attribute keys: |
...
All standard telemetry metric names begin with the prefix "messagingorg.apache.kafka.". This is omitted from the table for brevity. The required metrics are bold.
...
All standard telemetry metric names begin with the prefix "messagingorg.apache.kafka.". This is omitted from the table for brevity. The required metrics are bold.
...
metrics- a comma-separated list of telemetry metric name prefixes, e.g.,"messagingorg.apache.kafka.producer.node.request.latency., messagingorg.apache.kafka.consumer.coordinator.rebalance.latency.max". Whitespace is ignored.interval.ms- metrics push interval in milliseconds. Defaults to 5 minutes if not specified.match- Client matching selector that is evaluated as a list of an anchored regular expressions (i.e., "something.*" is treated as "^something.*$"). Any client that matches all of the selectors will be eligible for this metrics subscription. Initially supported selectors are:client_instance_id- CLIENT_INSTANCE_ID UUID string representation.client_id- client's reported client.id in the GetTelemetrySubscriptionsRequest.client_software_name- client software implementation name.client_software_version- client software implementation version.client_source_address- client connection's source address from the broker's point of view.client_source_port- client connection's source port from the broker's point of view.
...
| Code Block | ||
|---|---|---|
| ||
$ kafka-configs.sh --bootstrap-server $BROKERS \ --entity-type client-metrics \ --entity-name "basic_producer_metrics" \ --alter \ --add-config "metrics=[messagingorg.apache.kafka.producer., messagingorg.apache.kafka.consumer.coordinator.rebalance.latency.max],interval.ms=15000,match=[client_instance_id=b69cc35a-7a54-4790-aa69-cc2bd4ee4538]" |
...
| Code Block |
|---|
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --generate-name \ --metrics messagingorg.apache.kafka.producer.node.request.latency.,messagingorg.apache.kafka.consumer.node.request.latency. \ --interval 60000 |
...
| Code Block |
|---|
$ kafka-client-metrics.sh --bootstrap-server $BROKERS --alter --name METRICSUB \ --metrics messagingorg.apache.kafka.consumer. \ --interval 60000 \ --match "client_software_name=kafka_python,client_software_version=1\.2\..*" $ kafka-configs.sh --bootstrap-server $BROKERS --alter --entity-type client-metrics --entity-name METRICSUB \ --add-config "metrics=messagingorg.apache.kafka.consumer.,interval.ms=60000,match=[client_software_name=kafka.python,client_software_version=1\.2\..*]" |
...
The Kafka operator adds a metrics subscription for metrics matching prefix "messagingorg.apache.kafka.consumer." and with the client_source_address and client_id as matching selector. Since this is a live troubleshooting case, the metrics push interval is set to a low 10 seconds.
...