...
In order to support submitting extra more reassignments while existing reassignments are still in-flight. An extra znode /admin/reassign_partitions_queue
which has the same JSON format as /admin/reassign_partitions
. Three more options --generate-queue --verify-queue --execute-queue
will be added to kafka-reassign-partitions.sh.
The controller will be informed of the extra queued reassignments via a ZooKeeper watch. It will get all topic/partitions from /admin/reassign_partitions_
queue queue
and add to /admin/reassign_partitions
, then trigger the reassignments onPartitionReassignment()
of the topic/partitions.
In case inside the /admin/reassign_partitions_queue
, there are topic/partitions which exist in /admin/reassign_partitions
(pending reassignments), the conflict resolution for those duplicate topic/partitions is to first cancel / rollback the reassignments of those topic/partitions in
/admin/reassign_partitions
, then submit new reassignments from /admin/reassign_partitions_queuequeue
to /admin/reassign_partitions
. This This approach will be simpler than the algorithm proposed by Tom previously to infer the final replicas assignments for those duplicate topic/partitions. After the topic/partition is put in /admin/reassign_partitions
, it will be removed from /admin/reassign_partitions_queue
, and when /admin/reassign_partitions_queue
is empty, the znode will be deleted.
...