Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In order to better work under constrained resources (e.g. Flink cannot obtain all the requested resources), the JobManager JobMaster cannot expect that all of its slot requests are getting fulfilled. Currently, JobManagers JobMasters ask for every slot individually and fail the job if it cannot be obtained. Instead of deciding on the required set of slots before asking the ResourceManager, it would be more flexible if the JobManager JobMaster first declared what it needs to run the job. Based on what the JobManager JobMaster actually gets assigned by the ResourceManager, it can then decide to run the job with an adjusted parallelism. That way the JobManager JobMaster is able to react if it should get fewer slots than it declared.

...

Resources/Slots are released by the JobManager JobMaster by calling TaskExecutorGateway.freeSlot() and by updating the required resources by calling ResourceManagerGateway.declareRequiredResources with the updated resource requirements.

...