THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
In JM, there will be a component (JobMasterBlocklistHandler) responsible for generating managing all blocked items according to the exceptions notified by Scheduler, and managing them. SlotPool will be aware of the blocklist information and filter out blocked resources when allocating slots.and performing them on SlotPool.
In RM, there will also be a component (ResourceManagerBlocklistHandler) responsible for managing the cluster-level blocklist. SlotManager will be aware of the blocklist information and filter out resources in the blocklist when allocating slots from registered task managers. Similarly, when the ResourceManagerDriver requests new resources from an external resource manager (Yarn or Kubernetes), it also needs to filter out the blocked nodes.
...
JobMasterBlocklistHandler is the component in JM responsible for generating and managing all blocked items and performing them on SlotPool. It consists of three two sub-components：BlocklistStrategy, BlocklistTracker components：BlocklistTracker and BlocklistContext. When receiving a new exception from Schedulerblocked items, the JobMasterBlocklistHandler will handle it as following:
...
- Add the new blocked items to the BlocklistTracker.
- Synchronize the new blocked items to RM.
- Perform block actions on the resources via the BlocklistContext.
BlocklistStrategy
BlocklistStrategy is the component responsible for generating blocked items according to the exceptions and their locations notified by Scheduler. We can introduce different BlocklistStrategy implementations to adapt to different scenarios. In the first version, we will introduce a no-op implementation as default implementation. In the future, we will introduce a configurable blocklist strategy and plugin mechanism to load user-defined blocklist strategy implementations, details will be described in Future improvements.
...
| title | BlocklistStrategy |
|---|
...
BlocklistTracker
BlocklistTracker is the component responsible for tracking blocked items. The tracker will regularly remove timeout blocked items.
...