Page History

...

Delivery goal: eager rebalance analysis

We want to have a A detailed analysis and benchmark support need to be built before fully devoting into effort to this feature, because intuitively . Intuitively most applications should be able to tolerate minor discrepancy of task replaying time, however while the cost of extra rebalances and increased debugging complexity are definitely things we are not in favor of. 

The success of version 3.0 is upon version 1.0 success. This work could be done concurrently with version 2.0.

Version 4.0

Delivery goal: balancing factor

We will take ending task application eventual load balance into consideration. Specifically for each instance, the total number of tasks it owns should be within the range of 0.5 ~ 2 times of the expected number of tasks, which buffers some capacity in order to avoid imbalancimbalance.

We could even provide a stream.balancing.factor for the user to configure. The smaller this number sets to, the more strict the assignment will behave.  If the factor is set to r, the number of tasks a host could own is (w/total tasks)

...