Versions Compared

Old Version 176

changes.mady.by.user Federico Valeri

Saved on

compared with

New Version 177

changes.mady.by.user Federico Valeri

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This section describes all protocol level changes and new RPCs.

EntityType

A new entity type is added for the message generator to provide schema-level type validation for mirror name fields:

Code Block
languagejava
public enum EntityType {
    // ... existing types ...
    @JsonProperty("mirrorName")
    MIRROR_NAME(FieldType.StringFieldType.INSTANCE);
}

ResourceType

A new resource type is added to the ResourceType enum to enable per-mirror authorization:

...

languagejava

...

.

...

CoordinatorType

The FindCoordinatorRequest object is extended to support a new coordinator type:

Code Block
languagejava
public enum CoordinatorType { 
    // ... existing types ...
    MIRROR((byte) 3);
}

CreateTopic

The CreateTopic API is extended to add information required for mirror topic creation.

...

Code Block
{
  "apiKey": TBD,
  "type": "response",
  "name": "BumpLeaderEpochsResponse",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "Topics", "type": "[]TopicPartitions", "versions": "0+",
      "about": "Each topic in the mirror.", "fields": [
      { "name": "TopicName", "type": "string", "versions": "0+",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionDetail", "versions": "0+",
        "about": "Each partition state.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0+",
          "about": "The partition index." },
        { "name": "ErrorCode", "type": "int16", "versions": "0",
          "about": "The error code, or 0 if there was no error." }
      ]}
    ]}
  ]
}

Cluster Metadata Records

PartitionChangeRecord

New version adds MinLeaderEpoch to support leader epoch bumps triggered by the mirror state machine during failover.

Code Block
{
  "apiKey": 5,                                                                                                                                                                                                                                                                           
  "type": "metadata",                                                                                                                                                                                                                                                                    
  "name": "PartitionChangeRecord",
  "validVersions": "0-3",                                                                                                                                                                                                                                                                
  "flexibleVersions": "0+",
  "fields": [
    // ... existing fields unchanged...
    {"name": "MinLeaderEpoch", "type": "int32", "versions": "3+", "default": -1,
      "about": "The minimum leader epoch requested."}
  ]
}

Mirror Metadata Records

LastMirrorEpochs

The greatest leader epoch of a given partition that a destination cluster recognizes from the source cluster.

Code Block
{
  "apiKey": 1,
  "type": "coordinator-key",
  "name": "LastMirrorEpochsKey",
  "validVersions": "0",
  "flexibleVersions": "none",
  "fields": [
    { "name": "MirrorName", "type": "string", "versions": "0",
      "about": "The cluster mirror name."}
  ]
}

{
  "apiKey": 1,
  "type": "coordinator-value",
  "name": "LastMirrorEpochsValue",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "Topics", "type": "[]Topic", "versions": "0+",
      "about": "The mirror topics for which we want to store the last mirrored epochs.",  "fields": [
      { "name": "Name", "type": "string", "versions": "0",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]Partition", "versions": "0+",
        "about": "Each partition to record the last mirrored epochs.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0+",
          "about": "The partition index." },
        { "name": "", "type": "int32", "versions": "0+",
          "about": "The last mirror leader epoch for this partition." }
      ]}
    ]}
  ]
}

MirrorPartitionState

MirrorPartitionState record represents the lifecycle states of a mirrored partition.

Code Block
{
  "apiKey": 2,
  "type": "coordinator-key",
  "name": "MirrorPartitionStateKey",
  "validVersions": "0",
  "flexibleVersions": "MirrorPartitionStateKey"none",
  "fields": [
    { "name": "MirrorName", "type": "string", "versions": "0",
      "about": "The cluster mirror name."}
  ]
}

{
  "apiKey": 2,
  "type": "coordinator-value",
  "name": "MirrorPartitionStateValue",
  "validVersions": "0",
  "flexibleVersions": "none0+",
  "fields": [
    { "name": "MirrorNameTopicName", "type": "string", "versions": "0",
      "about": "The cluster mirrortopic name."}
  ]
}

{
  "apiKey": 2,
  "type,
    { "name": "coordinator-valuePartition",
   "nametype": "MirrorPartitionStateValueint32",
   "validVersionsversions": "0",
        "flexibleVersionsabout": "0+",
  "fields": [
The partition index."},
    { "name": "TopicNameState", "type": "stringint8", "versions": "0+",
      "about": "The mirror topicpartition namestate." },
    { "name": "Partition", "type": "int32", "versions": "0",
      "about": "The partition index."},
    { "name": "State", "type": "int8", "versions": "0+",
      "about": "The mirror partition state." }
  ]
}

...


  ]
}

Type Enumerations

EntityType

A new entity type is added for the message generator to provide schema-level type validation for mirror name fields:

Code Block
languagejava
public enum EntityType {
    // existing types unchanged
    @JsonProperty("mirrorName")
    MIRROR_NAME(FieldType.StringFieldType.INSTANCE);
}

ResourceType

A new resource type is added to the ResourceType enum to enable per-mirror authorization:

Code Block
languagejava
public enum ResourceType {
    // existing types unchanged
    CLUSTER_MIRROR((byte) 8);

CoordinatorType

The FindCoordinatorRequest object is extended to support a new coordinator type:

Code Block
languagejava
public enum CoordinatorType { 
    // existing types unchanged
    MIRROR((byte) 3);
}

ConfigurationType

A new configuration resource type is added for cluster mirrors, which is stored in the cluster metadata internal log. Both DescribeConfigs and IncrementalAlterConfigs will be bumped to new versions so that clients can check ApiVersionsResponse to determine whether the broker supports mirror configuration management, rather than having to send a request and interpret the error.

Code Block
languagejava
public enum Type {
    // ... existing types ... 
    MIRROR unchanged 
    MIRROR((byte) 64, "mirror"); // New type
}

Configuration

Mirror Configuration

Set via CreateMirror or IncrementalAlterConfigs. Stored in cluster metadata records.

...