Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

SlotPool should avoid allocating slots from blocked task managers. Blocked task managers include those directly blocked and those located on blocked nodes. To do that, our core idea is to keep the SlotPool in such a state: there is no slot in SlotPool that is free (no task assigned) and blocked. Details are as following:

  1. When SlotPool starts, BlockedTaskManagerChecker will be passed in to check whether a task manager is blocked.When receiving slot offers from blocked task managers (including task managers on blocked nodes), all offers should be rejected.
  2. When a task manager is newly blocked, BlocklistContext will perform the following on SlotPool:
    1. If the action is MARK_BLOCKED, release all free(no task assigned) slots on the blocked task manager(it needs to introduce a new method: SlotPoolService#releaseFreeSlotsOnTaskManager).
    2. If the action is MARK_BLOCKED_AND_EVACUATE_TASKS, release all slots on the blocked task manager(it needs to introduce a new method: SlotPoolService#releaseSlotsOnTaskManager). Note that we should not unregister the TM directly, because in the case of batch jobs, there may be unconsumed result partitions on it.
  3. When a slot state changes from reserved(task assigned) to free(no task assigned), it will check whether the corresponding task manager is blocked. If yes, release the slot.

...