Status

Current state: "Under Discussion"

Discussion thread: here 

JIRA: KAFKA-5925 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

The KIP-107 provides a way to delete messages starting from a specified offset inside a topic partition which we don’t want to take anymore so without relying on time-based and size-based log retention policies. The already implemented protocol request and response messages (DeleteRecords API, key 21) are used only by the “legacy” Admin Client in Scala and aren’t provided by the new Admin Client API in Java. This KIP is about adding this feature to the new Admin Client API.

Note that this KIP is related to KIP-107.

Public Interfaces

The AdminClient API will have new methods added (plus overloads for options):

deleteRecords(Map<TopicPartition, Long> partitionsAndOffsets)

Proposed Changes

AdminClient : deleteRecords()

public DeleteRecordsResult deleteRecords(Map<TopicPartition, Long> partitionsAndOffsets)
public DeleteRecordsResult deleteRecords(Map<TopicPartition, Long> partitionsAndOffsets, DeleteRecordsOptions options)

Where :

TopicPartition comes from org.apache.kafka.common package

DeleteRecordsResult is defined as follow. 

public class DeleteRecordsResult {
    // package access constructor
    Map<TopicPartition, KafkaFuture<Long>> values() { ... }
    KafkaFuture<Long> all() { ... }
}

In the partitionsAndOffsets parameter map, the Long value specifies the offset as a starting point for deleting subsequent messages in the corresponding topic partition.

In the DeleteRecordsResult, the Long value accessed by values() and all() method specifies the low watermark as described in the KIP-107.

Compatibility, Deprecation, and Migration Plan

This is a new API and won't directly affect existing users.

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.