THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Table of Contents |
|---|
Status
Current state: "Under Discussion"Accepted
Vote thread: here
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: KAFKA-63456435 – Application Reset Tool might delete incorrect internal topics
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
The streams application Users may want to specify which internal topics should be deleted. At present, the streams reset tool deletes all topic topics that start with "<application.id>-. This may lead to undesired behaviour when we have two similarly named application.id-s " and there are no options to control it.
The `--internal-topics` option is especially useful when there are prefix conflicts between applications, e.g. "app" and "app-v2". In this case, if we want to reset "app", the reset tool's default behaviour will delete both the internal topics of E.g. If people have two versions of the same application and name them "app" and "app-v2" (since both are prefixed by "app-", then resetting ). With the `--internal-topics` option, we can provide internal topic names for "app" would also and delete the internal topics of for "app" without deleting the internal topics for "app-v2".
Public Interfaces
Add an option to the streams reset tool.
internalTopicsOption = optionParser.accepts("
", "Comma
If specified, these are the only internal
that
the
tool will attempt to delete.")
.withRequiredArg()
.ofType(String.class)
.withValuesSeparatedBy(',')
.describedAs("list");
--internal-topics |
<String: list> Comma-separated list of internal topics |
Must be a subset of the |
marked |
for |
.withRequiredArg()
.ofType(String.class)
.withValuesSeparatedBy(',')
.describedAs("list");
deletion by |
Proposed Changes
Add an internal-topics option to streams reset tool.
The `--internal-topics` option will take a comma-separated list of internal topics to be deleted. These topics must be a subset of those flagged as internal during a `--dry-run` without the `internal-topics` option. If omitted, the tool will default to the original behaviour, i.e. delete all topics that start with "<application.id>-".
Usage
Usage of the "internal-topic" options will involve:
- Run `kafka-streams-application-reset.sh` with `--dry-run` to check which internal topics will be deleted.
- If everything looks okay, run the script again without `--dry-run`. Otherwise, run it with `--internal-topics` to specify the internal topics
...
- to be deleted.
Compatibility, Deprecation, and Migration Plan
...
If the new option is not specified, the stream resetter's behaviour will remain unchanged.
Rejected Alternatives
Disallow "dash" in the application.id
This fixes the example given above in the motivation, but does not allow users to specify the internal topics. Also, it will raise a lot of backward compatibility issues.
...
This may be more convenient than having to list out relevant internal topics which may be tedious. However, the internal-topics option is a lot more explicit and it's usage will be more familiar because of it's similarity to other options (e.g. input-topics and intermediate-topics). Usage of either "negative-prefix" or "internal-topic" options will involve doing a dry-run and then deciding whether to carry on or to limit the internal topics considered for deletion. Having Having displayed the internal-topics marked for deletion in the dry-run, it is more straightforward to simply specify the subset to be deleted instead of finding a prefix (or prefixes) to omit everything by.