Versions Compared

Old Version 39

changes.mady.by.user Till Rohrmann

Saved on

compared with

New Version 40

changes.mady.by.user Till Rohrmann

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

PlantUML
@startuml

package "Declarative Scheduler" {
  [SlotAllocator]
  [FailureHandler]
  [ScalingPolicyScalingController]
}

@enduml

SlotAllocator

The SlotAllocator is the component responsible for determining the resource requirements and mapping a JobGraph and its contained vertices to slots.

...

  1. Check whether the failure is recoverable. If not, then go to Failing state
  2. Ask the configured RestartBackoffTimeStrategy whether we can restart. If not, then go to Failing state
  3. Ask the configured RestartBackoffTimeStrategy for the backoff time between failure and restart

...

ScalingController

Whenever the scheduler is in the Executing state and receives new slots, the scheduler checks whether the job can be run with an increased parallelism. If this is the case, then the scheduler will ask the ScalingPolicy ScalingController given the old and cumulative parallelism of all operators whether it should scale up or not.

Code Block
/**
 * Simple policycontroler for controlling the scale up behavior of the {@link
 * org.apache.flink.runtime.scheduler.declarative.DeclarativeScheduler}.
 */
public interface ScalingPolicyScalingControler {

    /**
     * This method gets called whenever new resources are available to the scheduler to scale up.
     *
     * @param currentCumulativeParallelism Cumulative parallelism of the currently running job
     *     graph.
     * @param newCumulativeParallelism Potential new cumulative parallelism with the additional
     *     resources.
     * @return true if the policy decided to scale up based on the provided information.
     */
    boolean canScaleUp(int currentCumulativeParallelism, int newCumulativeParallelism);
}

...