You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

Status

Current state: "Under Discussion"

Discussion thread: here

JIRA: KAFKA-6435 – Application Reset Tool might delete incorrect internal topics

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Users may want to specify which internal topics should be deleted instead of deleting all internal topics that start with "<app.id>-". This option will allow them to do so.

E.g., if we have two versions of the same application named "app" and "app-v2", we want to be able to delete the internal topics of "app" without deleting the internal topics of "app-v2", but both of their internal topics start with "app-".

Public Interfaces

Add an option to the streams reset tool.

--internal-topics <String: list>    Comma-separated list of internal topics. 
                                     If specified, these are the only topics 
                                     that the tool will attempt to delete.

Proposed Changes

Add an internal-topics option to streams reset tool. If specified, these will be the only topics considered for deletion. 

Usage

Usage of the "internal-topic" options will involve:

  1. Run `kafka-streams-application-reset.sh` with `--dry-run` to check which internal topics will be deleted.
  2. If everything looks okay,  run the script again without `--dry-run`. Otherwise, run it with `--internal-topics` to specify the internal topics to be deleted.

Compatibility, Deprecation, and Migration Plan

If the new option is not specified, the stream resetter's behaviour will remain unchanged.

Rejected Alternatives

Disallow "dash" in the application.id

This fixes the example given above in the motivation, but does not allow users to specify the internal topics. Also, it will raise a lot of backward compatibility issues.


Have a "negative prefix" option

This may be more convenient than having to list out relevant internal topics which may be tedious. However, the internal-topics option is a lot more explicit and it's usage will be more familiar because of it's similarity to other options (e.g. input-topics and intermediate-topics).  Having displayed the internal-topics marked for deletion in the dry-run, it is more straightforward to simply specify the subset to be deleted instead of finding a prefix (or prefixes) to omit everything by.


  • No labels