...

In many use cases it is required to drop a few specific Kafka headers but a set of headers whose names can also dynamically change (e.g. when used with some end-to-end-encryption libraries, tracing etc.). To prevent those headers following a special pattern (which for example may or may not not comply with downstream system format) to be further forwarded/processed downstream (e.g.header forwarding in Http HTTP Sinks), this KIP proposes to add a wildcard/regexp matching feature to the DropHeaders Transform.

...

The SMT org.apache.kafka.connect.transforms.DropHeaders method apply is proposed to be extended by the new functionality. The existing interface will not be changed.

Proposed Changes

It is proposed to use regexp-matchers inside the apply method instead of or in addition to a set-based contains (existing implementation). Link to the relevant code lines:
https://github.com/apache/kafka/blob/7b5d640cc656443a078bda096d01910b3edfdb37/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/DropHeaders.java#L54 

...

If the functionality is added with regexp-matching in addition to the existing implementation (Option B), no breaking will be done, no migration is required as the additional feature comes with a standalone configuration option

changes need to be done. In case the contains-implementation is completely replaced (Option A), there might be are possible breaking changes in rare cases for the case of headers that contain characters which are which can be interpreted as Java regular expressions like e.g. "headers.*". In that case of Option B, no migration needs to be done as the additional feature comes with a standalone configuration option. In case of Option A only changes in configuration are required in case the configured header-names contain characters which can be interpreted as Java regular expressions, the configurations need to be adapted before upgrading such that special characters are escaped and not interpreted as a regular expression.

Rejected Alternatives

None.