THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
To enable learner resource shuffling behavior, we need to have the following task status indicators to be provided:
| Tag Name | Task Type | Explanation |
|---|---|---|
| isStateful | both | Indicate whether given task has a state to restore. |
| isLearner | standby | Indicate whether standby task is a learner task. |
| beingLearned | active | Indicate whether active task is being learned by some other stream thread. |
| isReady | standby | Indicate whether standby task is ready to serve as active task. |
Optimizations
Stateful vs Stateless Tasks
...
Delivery goal: Task state labeling, eventual workload balance
Question here: we could deviate a bit from designing the ultimate goal, instead providing user a handy tool to do that.
The 4.0 and the final version will take application eventual load balance into consideration. If we define a balancing factor x, the total number of tasks each instance owns should be within the range of +-x% of the expected number of tasks (according to relative instance capacity), which buffers some capacity in order to avoid imbalance. A stream.imbalance.percentage will be provided for the user to configure. The smaller this number sets to, the more strict the assignment protocol will behave.
...
Also adding new configs for user to better apply and customize the scaling change.
| stream.rebalancing.mode Default: incremental Version 1.0 | The setting to help ensure no downtime upgrade of online application. Options : upgrading, incremental |
Default: infinity Version 2.0 | Time in milliseconds to force terminate the stream thread when informed to be scaled down. |
learner.partial.rebalance Default : true Version 3.0 | If this config is set to true, new member will proactively trigger rebalance when it finishes restoring one learner task state each time, until it eventually finishes all the replaying. Otherwise, new stream thread will batch the ready call to ask for a single round of rebalance. |
| stream.imbalance.percentage Default: 0.2 (20%) Version 4.0 | The tolerance of task imbalance factor between hosts to trigger rebalance. |
Implementation Plan
To make sure the delivery is smooth with fundamental changes of KStream internals, we build a separate Google Doc here that could be sharable to outline the step of changes. Feel free to give your feedback on this plan while reviewing the algorithm, because some of the algorithm requirements are highly coupled with internal architecture reasoning.
...