...

With Diskless Topics, Apache Kafka will become a streaming engine that supports a wide spectrum of latencies, balancing cost and performance for an extremely diverse set of workloads.

Disk usage and lack thereof

It's important to clarify what exactly "diskless" means. "Diskless" primarily refers to not using broker disk for storing user data. There are no index files stored on broker disk for diskless topics as well. However, diskless topics still require some broker disk usage, particularly:

• normal topic KRaft metadata;
• batch metadata may be stored on broker disk depending on the batch coordinator implementation (e.g. in a Kafka topic);
• brokers may require some limited amount of disk space to perform certain operations like object compaction;
• caching in the read path may optionally use broker disk instead of memory.
  

Proposed Changes

This KIP will not require any changes to the codebase or documentation upon acceptance. By accepting this KIP, we will come to a consensus on the need for this feature, and its end-user requirements, but not any specific implementation details.

...