...

Public Interfaces

This feature introduces a 3 new optionoptions, 'compression.gzip.level','compression.lz4.level' and 'compression.snappy.level' to the producer, topic, and broker configuration. Snappy is excluded since it does not support any compression level. The type of this option these options is an integer, with a default value of null.

For example

compression.type=gzip
compression.gzip.level=4				# NEW: Compression level to be used.

The table below shows the valid range of compression. level per compression.type. (note: snappy is excluded since it does not support any compression level. ) The valid range and default value of the compression level are entirely up to the compression library, so they may be changed in the future.

Proposed Changes

...

• If 'compression.type' is none of or snappy, 'compression.<codec>.level' is ignored.
• If 'compression.<codec>.level' is not in the valid range, it raises an error.
• If If 'compression.<codec>.level' is in the valid range, the producer compresses the records with the given level.
• If If 'compression.<codec>.level' is not set, it falls back to the default level.

...

With 1.3.2, Zstandard introduced compression/decompression with long window size. This option can improve the compression/decompression speed for some levels.

Rejected Alternatives

...

• Introduce a single configuration: compression.level. This was the original proposal that was voted. However while reviewing the PR we noticed that it was hard to use as each codec has its own range of compression levels. So for example setting compression.level to 20 would be valid with zstd but an error with gzip. Also a follow up KIP, KIP-780, aims at introducing per codec configurations using the compression.<codec>.<option> syntax. For these reason, it seems preferable to use configurations with the 'compression.<codec>.level' format for consistency.