Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Role

ZooKeeper Path

Type

Data Description

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="3c3ea8c70c186830-e7690df8-4a5f4230-9e009361-c74bacec71ab2e2edd150db4"><ac:plain-text-body><![CDATA[

ID Registry

/brokers/ids/[0..N]

Ephemeral

String in the format of "creator:host:port" of the broker.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="216942039db9fb0d-005071c1-42404816-94cbbad9-4a7ebd1376bd5befb69b7443"><ac:plain-text-body><![CDATA[

Topic Registry

/brokers/topics/[topic]/[0..N]

Ephemeral

Number of partitions that topic has on that Broker.

]]></ac:plain-text-body></ac:structured-macro>

...

  • All Consumers in a ConsuerGroup will come to a consensus as to who is consuming what.
  • Each Broker+Topic+Partition combination is consumed by one and only one Consumer, even if it means that some Consumers don't get anything at all.
  • Wiki Markup
    A Consumer should try to have as many partitions on the same Broker as possible, so sort the list by \[Broker ID\]-\[Partition\] (0-0, 0-1, 0-2, etc.), and assign them in chunks.
  • Consumers are sorted by their Consumer IDs. If there are three Consumers, two Brokers, and two three partitions in each, the split might look like:
    • Wiki Markup
      Consumer A: \[0-0, 0-1\]
    • Wiki Markup
      Consumer B: \[0-2, 1-0\]
    • Wiki Markup
      Consumer C: \[1-1, 1-2\]
  • If the distribution can't be even and some Consumers must have more partitions than others, the extra partitions always go to the earlier consumers on the list. So you could have a distribution like 4-4-4-4 or 5-5-4-4, but never 4-4-4-5 or 4-5-4-4.