Page History

...

• The approach proposed by this FLIP should only applies apply to jobs of DataStream API and SQL/Table API by the Blink planner (unbounded streaming and bounded batch jobs). It should not affect jobs of DataSet API.
  • For DataSet jobs, there are already some fraction based approach (in TaskConfig and ChainedDriver), and we do not make any change to the existing approach. 
• This FLIP assumes that for jobs with known operators' resource requirements, the requirements are already properly described by ResourceSpecs in PhysicalTransformations.
  • This FLIP does not discuss how to set operators' resource requirements for a job.
  • Current status (including plans for Flink 1.10) of how to set operators' resource requirements for jobs can be described as follows:
    • SQL/Table API - Blink optimizer can set operator resources for the users, according to their configurations (default: unknown)
    • DataStream API -  There are no method / interface to set operator resources at the moment. It can be added in the future.
    • DataSet API - There are existing user interfaces to set operator resources.

...