...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

Now these two SlotManager have some duplicated codes(eg, the delay check of checkResourceRequirements ) and some similar component names(eg, TaskManagerTracker, TaskExecutorManager). This causes duplicate development of features (, such as Make ResourceActions declarative).

This FLIP aims to unify the implementation of SlotManager. Since FineGrainedSlotManager has almost all the capability of DeclarativeSlotManager, we will implement all the lacks of FineGrainedSlotManager and enable it by default.

...

For long term plans, the DeclarativeSlotManager will be completely removed in the next release after the default value is changed.

Proposed Changes

Overview of FineGrainedSlotManager and DeclarativeSlotManager

Functionality supports

sub-components

DeclarativeSlotManager

There are tree subcomponents in DeclarativeSlotManager

• ResourceTracker:  Track all resource requirements of multiple jobs
• SlotTracker: Track slot status(free/allocated). It's used for fulfill resource requirements with free slots.
• TaskExecutorManager: Manage the lifecycle of task managers(allocated/release/keep redundant).
  

The main process flow of DeclarativeSlotManager is:

• When new/updated resource requirements arrived, DeclarativeSlotManager will send it to ResourceTracker.
• When dealing with all resource requirements tracked by ResourceTracker. DeclarativeSlotManager get all free slots from SlotTracker to fulfill requirements.
• If there are also some unfulfilled requirements, DeclarativeSlotManager get all pending task manager slots from TaskExecutorManager  to fulfill it.
• And the DeclarativeSlotManager will try to allocate new task managers when pending task manager slot is not enough.

FineGrainedSlotManager

There are four subcomponents in FineGrainedSlotManager

• ResourceTracker: The same as DeclarativeSlotManager#ResourceTracker.
• TaskManagerTracker: Track task managers and their resources.
• SlotStatusSyncer: Sync the slot status with task manager, and communicate with task manager to allocate/free slots.
• ResourceAllocationStrategy: Try fulfill the resource requirements with available/pending resources.

The main process flow of FineGrainedSlotManager is:

• When new/updated resource requirements arrived, FineGrainedSlotManager will send it to ResourceTracker.
• When dealing with all resource requirements tracked by ResourceTracker. FineGrainedSlotManager invoke ResourceAllocationStrategy with TaskManagerTracker.
• The ResourceAllocationStrategy will try to fulfill the resource requirements by available/pending resources orderly. It will create new pending resources if not enough.
• Then FineGrainedSlotManager will allocate new task managers according to the result of ResourceAllocationStrategy.

Add the missing capability of FineGrainedSlotManager

...

The redundant task managers are used to speed up failover. The

FineGrainedSlotManager has reserved the interface of heterogeneous task managers, but there are only one implementation which will requests task managers in same resources currently. Therefore, the current redundant task managers will not consider heterogeneity. This could be considered in detail when we decide to support heterogeneous task managers.

 The logic in FineGrainedSlotManager should be:

• Introduce redundantTaskManagerNum to DefaultResourceAllocationStrategy

• Invoke tryFulFillRedundantResourceProfiles at the end of tryFulfillRequirements. it should use the remaining registeredResources and pendingFreeResources to fulfill the redundant slot requirements(defaultSlotResourceProfile * redundantTaskManagerNum * numSlotsPerWorker) and try to add new PendingTaskManagers to resultBuilder if the resource is not enough.

  Code Block
  language java linenumbers true
  tryFulFillRedundantResourceProfiles(       Collection<InternalResourceInfo> registeredResources,       List<InternalResourceInfo> pendingFreeResources,        ResourceAllocationResult.Builder resultBuilder){}

  
  

Split resource allocate/release related logic from FineGrainedSlotManager to TaskManagerTracker

Currently the FineGrainedSlotManager is response to slots allocations and resources request/release. This makes logical of FineGrainedSlotManager complicated, So we will move task manager related work from FineGrainedSlotManager to TaskManagerTracker, which already tracks task managers but not including request/release.

After this change, the TaskManagerTracker will manage all the behavior of TaskManager:

• request new task manager
• tracking pending task manager
• tracking registered task manager
• release idle task manager
• deal with max resources limitation

Try reclaim inactive slots when job terminated

...

The DeclarativeSlotManager and related configs will be completely removed in the next release after the default value is changed.

Test Plan

Our current CI pipeline includes a stage dedicated to fine-grained resource management. To avoid adding too much overhead, this stage only runs tests from the 'flink-runtime' and 'flink-test' modules.
Before we deprecated the declarative slot manager. The FineGrainedSlotManager should be the default SlotManager, and it should pass all the tests in ci.

...