Status

Current state: "Under Discussion"

Discussion thread: here

JIRA:  Unable to render Jira issues macro, execution error.

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently when a user sends a null key to a log compacted topic, the broker returns CorruptRecordException, which is a retriable exception. As such, the producer keeps retrying until retries are exhausted or request.timeout.ms expires and eventually throws a TimeoutException. This is confusing and not user-friendly. 

The broker should throw a non-retriable exception with a message explaining the error.

New or Changed Public Interfaces

This KIP introduces a new error type INVALID_KEY.

It introduces a new non-retriable exception InvalidKeyException. Going forward user request will fail without doing retries, if they try to produce to a log compacted topic with a null key.

Proposed Changes

Add a new error code and exception as follows :

Error Code :

INVALID_KEY(45, new InvalidKeyException("Message key is invalid."))

Exception :

public class InvalidKeyException extends ApiException {
    
private static final long serialVersionUID = 1L;

    public InvalidKeyException() {
        super();
    }

    public InvalidKeyException(String message, Throwable cause) {
        super(message, cause);
    }

    public InvalidKeyException(String message) {
        super(message);
    }

    public InvalidKeyException(Throwable cause) {
        super(cause);
    }
}

Compatibility, Deprecation, and Migration Plan

• It is recommended that we upgrade the clients before the broker is upgraded, so that the clients would be able to understand the new exception. During this phase the broker will still return a CorruptRecordException in case it receives a null key message for a log compacted topic and the clients would treat it as a retriable exception.
• Once the clients are upgraded, the broker should be upgraded to handle the error scenario more efficiently and throw the non-retriable KeyException. At this point the clients would recognize that its a non-retriable exception and would fail the request without doing any retries.

Rejected Alternatives