Status

Current state: Under Discussion

Discussion thread: here 

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Summary

Add a new tool action, --cancel-all, to kafka-reassign-partitions that cancels all ongoing inter-broker partition reassignment in the cluster without requiring a reassignment JSON file.

The existing --cancel action remains unchanged and continues to support:

• JSON-scoped cancellation
• intra-broker log_dirs movement cancellation
• throttle cleanup behavior driven by the provided reassignment JSON

Motivation

Operators often need to stop all in-flight replica reassignments quickly—for example during an incident, a bad rollout, or when the original reassignment JSON is missing or out of date.

Today, the practical workflow is:

1. Call ListPartitionReassignments for the full cluster
2. Build an AlterPartitionReassignments request with replicas = null for each in-flight partition
3. Execute the request manually or through ad-hoc scripts

This workflow is:

• error-prone
• operationally slow under pressure
• difficult to execute safely when many topics are involved

The cluster already exposes the authoritative set of in-flight reassignments. The CLI should therefore provide a single, well-defined operation for the common case of cancelling all ongoing inter-broker partition reassignments.

Public Interfaces

CLI (kafka-reassign-partitions.sh)

--cancel-all becomes an additional top-level action alongside:

• --execute
• --verify
• --generate
• --cancel
• --list

Exactly one action may be specified per invocation, consistent with existing CLI behavior.

Proposed Changes

New Action: --cancel-all

The new action performs the following steps:

1. Fetch all ongoing partition reassignments using:
   • AdminClient.listPartitionReassignments()
2. Filter reassignments where:
   • addingReplicas or removingReplicas is non-empty
3. Submit cancellation requests using:
   • AdminClient.alterPartitionReassignments(... Optional.empty())

This provides a cluster-wide cancellation mechanism for inter-broker reassignments.

Behavior and Semantics

1. Scope: Cluster-wide inter-broker partition reassignments only (same notion as “ongoing” in ListPartitionReassignments when adding/removing is non-empty).
2. Log directories: --cancel-all does not cancel ongoing AlterReplicaLogDirs / log_dirs moves; operators continue to use --cancel with a JSON plan that includes log_dirs for those.
3. Throttles: If --preserve-throttles is not set, clear replication throttles only for brokers/topics implied by partitions that were actually cancelled (or document cluster-wide broker throttle behavior if implementation chooses otherwise—pick one and state it in the final KIP).
4. Partial failure: Per-partition errors from the controller (e.g. revert would require unclean leader election but topic disallows it) must be surfaced clearly; tool exits non-zero on any failed cancel.

Compatibility, Deprecation, and Migration Plan

• Wire protocol: None (reuses existing Admin APIs).
• CLI: Additive; no change to existing flags beyond new action.
• Security: Same ACL expectations as existing AlterPartitionReassignments / Describe paths; cluster-wide cancel is a high-privilege operation—call out in release notes / ops docs.

Why Not Just Document “List + Script”?

Listing and scripting is duplicated logic every organization reimplements, and it fails under stress. A first-class tool action matches how operators think (“cancel everything in flight”) and aligns with APIs the controller already supports.

Test Plan

• ReassignPartitionsCommandArgsTest: --cancel-all alone with --bootstrap-server parses; --reassignment-json-file rejected with cancel-all action; --cancel + --cancel-all rejected (single action); --preserve-throttles allowed; disallowed flags still rejected (e.g. --throttle, --reassignment-batch-size).
• ReassignPartitionsUnitTest (or equivalent): with MockAdminClient, start a partition reassignment, run cancelAllOngoingAssignments (or full command path), assert in-flight reassignment cleared and list state has no adding/removing for that partition.
• Error path: mock/controller response so one partition cancel fails → tool surfaces error and non-zero exit (if covered at unit level).

Rejected Alternatives

• Extend --cancel with a magic empty JSON / sentinel: Confusing and easy to misuse; a dedicated action is clearer.
• New RPC: Unnecessary; existing AlterPartitionReassignments already supports per-partition cancel.