THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Sometimes the restoration time of learner tasks are not equivalent. When assigned with 1+ tasks to replay, the stream worker could require immediate rebalance as a subset of learning tasks are finished in order to speed up the load balance and resource waste of double task processing, with the sacrifice of global efficiency with many more rebalances introduced. We could supply user with a config to decide whether they want to take eager approach or stable approach.
Example is like:
A stream instance S1 takes two learner tasks T1, T2, where restoring time time(T1) < time(T2). Under eager rebalance approach, the instance will call out rebalance immediately when T1 finishes replaying. While under stable rebalance, instance will rejoin the group until it finishes replaying of both T1 and T2.
...
Don’t forget the original purpose of standby task is to mitigate the issue during scaling down. When performing learner assignment, we shall prioritize instances which currently have standby tasks that match learner assignment. Therefore the group should rebalance pretty soon and let the leaving member shutdown themselves fairly quickly. tasks that match learner assignment. Therefore the group should rebalance pretty soon and let the leaving member shutdown themselves fairly quickly.
Scale Down Timeout
Sometimes end user wants to reach a sweet spot between ongoing task transfer and streaming resource free-up. So we want to take a similar approach as KIP-415, where we shall introduce a client config to make sure the scale down is time-bounded. If the time takes to migrate tasks outperforms this config, the leaving member will shut down itself immediately instead of waiting for the final confirmation. And we could simply transfer learner tasks to active because they are now the best shot to own new tasks.
Algorithm Trade-offs
We open a special section to discuss the trade-offs of the new algorithm, because it's important to understand the change motivation and make the proposal more robust.
...
We will be adding following new configs:
learn.partial.rebalance Default : true | If this config is set to true, new member will proactively trigger rebalance when it finishes restoring one task state each time, until it eventually finishes all the task replaying. Otherwise, new instance will batch the ready stage to ask for single round of rebalance. |
scale.down.timeout.ms Default: infinity | Timeout in milliseconds to force terminate the stream instance when informed to be scaled down. |
A public interface is any change to the following:
...