Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
{
  "apiKey": 2,
  "type": "coordinator-key",
  "name": "MirrorPartitionStateKey",
  "validVersions": "0",
  "flexibleVersions": "none",
  "fields": [
    { "name": "MirrorName", "type": "string", "versions": "0",
      "about": "The cluster mirror name."}
  ]
}

{
  "apiKey": 2,
  "type": "coordinator-value",
  "name": "MirrorPartitionStateValue",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "TopicName", "type": "string", "versions": "0",
      "about": "The topic name."},
    { "name": "Partition", "type": "int32", "versions": "0",
      "about": "The partition index."},
    { "name": "State", "type": "int8", "versions": "0+",
      "about": "The mirror partition state." }
  ]
}

Errors

List of protocol-level errors returned in API responses:

...

Configuration

A new configuration resource type is added for cluster mirrors, which is stored in the cluster metadata internal log. Both DescribeConfigs and IncrementalAlterConfigs will be bumped to new versions so that clients can check ApiVersionsResponse to determine whether the broker supports mirror configuration management, rather than having to send a request and interpret the error.

...

Name

Type

Group

Tags

Description

JMX Bean

MaxLag

MirrorFetcherManager

kafka.server.mirror

clientId=MirrorReplica

Max lag in messages between destination leader and source leader replicas.

kafka.server.mirror:type=MirrorFetcherManager,name=MaxLag,clientId=MirrorReplica

MinFetchRate

MirrorFetcherManager

kafka.server.mirror

clientId=MirrorReplica

The min fetch rate between destination leader and source leader replicas.

kafka.server.mirror:type=MirrorFetcherManager,name=MirrorReplica

ConsumerLag

FetcherLagMetrics

kafka.server

clientId=MirrorFetcherThread-{sourceBroker.id}-{fetcherId}-{mirrorName},topic=([-.\w]+),partition=([0-9]+)

Lag in messages per remote leader replica.

kafka.serverr:type=FetcherLagMetrics,name=ConsumerLag,clientId=MirrorFetcherThread-{sourceBroker.id}-{fetcherId}-{mirrorName},topic=([-.\w]+),partition=([0-9]+)

DeadThreadCount

MirrorFetcherManager

kafka.server.mirror

clientId=MirrorReplica

Number of dead mirror fetcher threads.

kafka.server,mirror:type=MirrorFetcherManager,name=DeadThreadCount,clientId=MirrorReplica

FailedPartitionsCount

MirrorFetcherManager

kafka.server.mirror

clientId=MirrorReplica

Total count for failed partitions for any reason like auth, authorization, failed network with source.

kafka.serve.mirror:type=MirrorFetcherManager,name=FailedPartitionsCount,clientId=MirrorReplica

BytesPerSec

FetcherStats

kafka.server

clientId=MirrorFetcherThread-{sourceBroker.id}-{fetcherId}-{mirrorName},brokerHost={host},brokerPort={port}

Extend kafka.server.FetcherStats to report mirror fetcher threads.

kafka.server:type=FetcherStats,name=BytesPerSec,clientId=MirrorFetcherThread-{sourceBroker.id}-{fetcherId}-{mirrorName},brokerHost={host},brokerPort={port},mirror-name={mirrorName}

RequestsPerSec

FetcherStats

kafka.server

clientId=MirrorFetcherThread-{sourceBroker.id}-{fetcherId}-{mirrorName},brokerHost={host},brokerPort={port}

Extend kafka.server.FetcherStats to report mirror fetcher threads.

kafka.server:type=FetcherStats,name=RequestsPerSec,cclientId=MirrorFetcherThread-{sourceBroker.id}-{fetcherId}-{mirrorName}, brokerHost={host},brokerPort={port},mirror-name={mirrorName}



LocalTimeMs,

MessageConversionsTimeMs,

RemoteTimeMs,

RequestBytes,

RequestQueueTimeMs,

ResponseQueueTimeMs,

ResponseSendTimeMs,

TemporaryMemoryBytes,

TotalTimeMs

RequestMetrics

kafka.network

request=[mirror_requests]

Extend kafka.network:type=RequestMetrics to list cluster mirror requests.

kafka.network:type=RequestMetrics,name=*, request=*

ErrorsPerSec

RequestMetrics

kafka.network

request=[mirror_requests],error=*

Extend kafka.network:type=RequestMetrics to list cluster mirror requests.

kafka.network:type=RequestMetrics,name=ErrorsPerSec, request=*, error=*

RequestsPerSec

RequestMetrics

kafka.network

request=[mirror_requests],version=*

Extend kafka.network:type=RequestMetrics to list cluster mirror requests.

kafka.network:type=RequestMetrics,name=RequestsPerSec, request=*, version=*

connection-close-rate,

connection-close-total,

connection-count,

connection-creation-rate,

connection-creation-total,

failed-authentication-rate,

failed-authentication-total,

failed-reauthentication-rate,

failed-reauthentication-total,

incoming-byte-rate,

incoming-byte-total,

network-io-rate,

network-io-total,

outgoing-byte-rate,

outgoing-byte-total,

reauthentication-latency-avg,

reauthentication-latency-max,

request-rate,

request-size-avg,

request-size-max,

request-total,

response-rate,

response-total,

select-rate,

select-total,

successful-authentication-no-

reauth-total,

successful-authentication-rate,

successful-authentication-total,

successful-reauthentication-rate,

successful-reauthentication-total

mirror-broker-{DestinationBroker.id}-fetcher-{fetcherId}-mirror-{mirrorName}-metrics

kafka.server

broker-id={sourceBroker.id},fetcher-id={fetcherId}

Fetcher requests in the cluster mirror metrics.

kafka.server:type=mirror-broker-{sourceBroker.id}-fetcher-{fetcherId}-mirror-{mirrorName}-metrics,broker-id={sourceBroker.id},fetcher-id={fetcherId}

MetadataRefreshError

MirrorMetadataManager

kafka.server.mirror


Number of topic metadata refresh sync errors.

kafka.server.mirror:type=MirrorMetadataManager,name=aclSyncError

TopicConfigMetadataSyncError

MirrorMetadataManager

kafka.server.mirror


Number of topic configuration sync errors.


ConsumerGroupOffsetSyncError

MirrorMetadataManager

kafka.server.mirror


Number of CGs sync errors.


AclSyncError

MirrorMetadataManager

kafka.server.mirror


Number of ACLs sync errors.

kafka.server.mirror:type=MirrorMetadataManager,name=aclSyncError

ByteRate

MirrorReplication

kafka.server


Bandwidth quota metrics. Indicates the throttled data mirror replication rate of the broker in bytes/sec.

kafka.server:type=MirrorReplication

FailedPartitionState

MirrorMetadataManager

kafka.server.mirror


Number of partitions in failed state.

kafka.server.mirror:type=MirrorMetadataManager,name=FailedPartitionState

StoppedPartitionState

MirrorMetadataManager

kafka.server.mirror


Number of partitions in a stopped state.

kafka.server.mirror:type=MirrorMetadataManager,name=StoppedPartitionState

StoppingPartitionState

MirrorMetadataManager

kafka.server.mirror


Number of partitions in stopping state.

kafka.server.mirror:type=MirrorMetadataManager,name=StoppingPartitionState

MirroringPartitionState

MirrorMetadataManager

kafka.server.mirror


Number of partitions in mirroring state.

kafka.server.mirror:type=MirrorMetadataManager,name=MirroringPartitionState

PreparingPartitionState

MirrorMetadataManager

kafka.server.mirror


Number of partitions in preparing state.

kafka.server.mirror:type=MirrorMetadataManager,name=PreparingPartitionState

Errors

List of protocol-level errors returned in API responses:

Error CodeNameUsed ByContext
3UNKNOWN_TOPIC_OR_PARTITIONMirrorMetadataManagerSource topic metadata has errors during sync
15COORDINATOR_NOT_AVAILABLEMirrorCoordinatorMirror coordinator is not active
31CLUSTER_AUTHORIZATION_FAILEDKafkaApis, ControllerApisMissing authorization for mirror operations
35UNSUPPORTED_VERSIONKafkaApis, ControllerApisCluster mirroring disabled (mirror.version=0)
38INVALID_REPLICATION_FACTORReplicationControlManagerInsufficient brokers for mirror topic replication
39INVALID_REPLICA_ASSIGNMENTReplicationControlManagerBrokers fenced or in controlled shutdown
42INVALID_REQUESTKafkaApis, ConfigurationControlManagerValidation failures (direct mirror.name modification, mirror disabled, invalid config)

Compatibility, Deprecation, and Migration Plan

...