Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Flink should rescale immediately only if last rescale was done more than scaling-interval.min ago.
  • Otherwise it should schedule a rescale at (now + scaling-interval.min) point in time. It is equivalent to resetting the cooldown period when new slots arrive during a cooldown period. Indeed, we decided to lower the scaling-interval.min default value to be more reactive (cf compatibility part), resetting the period allows to protect against too frequent rescales. 
  • The rescale is done like this: 
    • if minimum scaling requirements are met (AdaptiveScheduler#shouldRescale), the job is restarted with new parallelism (as before)
    • if minimum scaling requirements are not met but
      • if last rescale was done more than scaling-interval.max ago, a rescale is forced.
      • otherwise, schedule a forced rescale in scaling-interval.max

                     => except when scaling-interval.max is exceeded, AdaptiveScheduler#shouldRescale is always called upon a rescale When a rescale is forced, the rescale is done as long as the parallelism has changed. Otherwise, to avoid unnecessary restarts, the rescale is done when added resources are above the configured minimum. 

B. when slots are lost (most of the time after a  TaskManager failure), there will be no change compared to the current behavior:

...