Status

Current state: "Under Discussion"

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently when a user sends a null key to a log compacted topic, the broker returns CorruptRecordException, which is a retriable exception. As such, the producer keeps retrying until retries are exhausted or request.timeout.ms expires and eventually throws a TimeoutException. This is confusing and not user-friendly. 

The broker should throw a non-retriable exception with a message explaining the error.

New or Changed Public Interfaces

This KIP introduces a new error type INVALID_COMPACTION_KEY.

It introduces a new non-retriable exception InvalidCompactionKeyException. Going forward user request will fail without doing retries, if they try to produce to a log compacted topic with a null key.

Proposed Changes

Add a new error code and exception as follows :

...

public class InvalidCompactionKeyException extends ApiException {
    
private static final long serialVersionUID = 1L;

    public InvalidCompactionKeyException() {
        super();
    }

    public InvalidCompactionKeyException(String message, Throwable cause) {
        super(message, cause);
    }

    public InvalidCompactionKeyException(String message) {
        super(message);
    }

    public InvalidMessageKeyException(Throwable cause) {
        super(cause);
    }
}

Compatibility, Deprecation, and Migration Plan

• We should upgrade the clients before the broker is upgraded, so that the clients would be able to understand the new exception. During this phase the broker will still return a CorruptRecordException in case it receives a null key message for a log compacted topic and the clients would treat it as a retriable exception.
• Once the clients are upgraded, the broker should be upgraded to handle the error scenario more efficiently and throw the non-retriable InvalidCompactionKeyException. At this point the clients would recognize that its a non-retriable exception and would fail the request without doing any retries.

Rejected Alternatives