Page History

...

1. Data in diskless topics is durably stored in object storage at all times. 
   1. Local segments on broker disks serve as caches and not sources of truth
   2. Remote storage may have higher latency than local disks, increasing the latency of Kafka requests and end-to-end data latency.
2. Kafka delegates replication of diskless topics to object storage, and does not perform replication itself.
   1. Replicas placement is still used to control client traffic and cache placement
   2. Any broker may build a replica of any set of diskless partitions by contacting the diskless coordinator, lowering load on other brokers and eliminating unclean leader elections.
   3. All operators can use efficient types of storage backends, such as ones with erasure coding.
   4. Hyperscaler operators can avoid most inter-zone data replication costs.
3. All brokers are capable of interacting with all diskless topics, and produce requests do not need to be handled by the partition leader.
   1. Produce requests are preferentially served by replicas of the partition, and do not need to be directed to the partition leader.
   2. Partition leaders are still elected to upload to manage the ISR state, upload to tiered storage, and handle share fetches.
   3. Clusters are able to perform fine-grained client balancing across the cluster independently of topic/partition hot spots.
   4. Hyperscaler operators can avoid most inter-zone data ingress/egress costs.

...