Versions Compared

Old Version 7

changes.mady.by.user Jorge Otoya

Saved on

compared with

New Version 8

changes.mady.by.user Jorge Otoya

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Currently `kafka-streams-application-reset` only supports returning to the earliest offset on input topics. But `kafka-consumer-groups` `reset-offsets` option tool support more options detailed in KIP-122.

This KIP is considering include current `reset-offsets` options scenarios on `kafka-streams-application-reset` to have more options over input-topics offsets.Also, this KIP is considering removing Zookeeper dependency on `kafka-streams-application-reset` and use the new AdminClient tool instead.

 

Public Interfaces

"kafka-streams-application-reset" supports the current features[2]:

...

1.Execute(no execution arguments)

This execution option will run the reset offset process based on scenario and scope.

Prints and execute resetting all topics and partitions to earliest:

--application-id app1 --input-topics foo,bar --to-earliest

2.Dry-Run--dry-run

This execution option will only print out the result of the scenario by scope. (i.e. dry-run)

The output will look like this:

INPUT
TOPIC                 PARTITION NEW-OFFSET NEW-LAG LOG-END-OFFSET 
input-topic           0         90         10      100            
INTERMEDIATE
TOPIC                 PARTITION NEW-OFFSET NEW-LAG LOG-END-OFFSET 
intermediate-topic    0         0          100      100            
INTERNAL
TOPIC                 PARTITION NEW-OFFSET NEW-LAG LOG-END-OFFSET 
internal-topic           0         0         0      0

Prints result of resetting all topics and partitions to earliest:

--application-id app1 --input-topics foo,bar --to-earliest --dry-run

 

Proposed Changes

1. Add Implement reset-offset Scenarios option described above to on `kafka-streams-application-reset` using `KafkaConsumer`.

2. Use `kafka-consumer-groups` `reset-offsets` tool in the background to reset offset of format to define input topics (it will support the same format to specify partitions, e.g. `topic2:0,1,2`) on `kafka-streams-application-reset` execution.

3. Change `kafka-streams-application-reset` `dry-run` option for Execution Options described above. This is proposed to align execution options between both tools.4. Keep execution parameters uniform between both tools: It will execute by default, and have a `dry-run` parameter just show the results. This will involve change current `ConsumerGroupCommand` to change execution options.

Compatibility, Deprecation, and Migration Plan

  • Code Defaults should make code that used the class `StreamResetter.java` will have to be adapted to the new interface. Basically all parameters could be keeped as current version but `–zookeeper` parameter will not be supported.work as it used to. 

Rejected Alternatives

Current `kafka-streams-application-reset` as mentioned in the Motivation supports this functionality partially, but it will be deprecated.

We will keep both tools `kafka-streams-application-reset` and `reset-offsets`, reusing `reset-offsets` on the background when `kafka-streams-application-reset` is executed.

This KIP was considering to remove Zookeeper dependency will be , but this has been handled by KIP-198.


[1] KIP-122: Add Reset Consumer Group Offsets tooling

...