Page History

...

This proposal suggests adding compression level and compression buffer size options to the producer and , topic, and broker config.

Basically, CPU (running time) and I/O (compressed size) are trade-offs in compression. Since the best is use case dependent, lots of compression algorithms provide a way to control the compression level with reasonable default level, which results in a good performance in general. Add to this, the compression ratio is also affected by the buffer size - some users may want to trade compression for less/more memory.

...

This feature introduces two new options, 'compression.level' and 'compression.buffer.size,' to the producer and , topic, and broker configuration.

• If specified option is not supported for the codec ('compression.type'), it is ignored (e.g., compression level for snappy or buffer size for zstd.)
• If specified value is not available (or invalid), it raises an error.
• If there is no specified value, it falls back to the default one.

...