Page History

...

We might provide operator-level resource configuration interface interfaces for expert users who know the exact resource usage of each operator and also want to control the scheduling strategy in a fine-grained approachfiner grain.

Default operator resource configuration

In this FLIP, the SSGs with unknown resource requirements will be scheduled with the resource profile of the default slot. We might allow user to configure the default operator resource and schedule those SSGs according to the number of operators inside it.

Known limitations and constraints to be documented

When the fine-grained resource management feature is finally exposed provided to the user. We should document its limitation clearly.user, the following limitations and constraints should be well documented, with potential impacts and suggestions.

• Setting chain-able operators to different SSGs may break operator chaining, thus change the performance.
• Change of data exchange mode (pipelined vs. blocking) with in a SSG may affect the resource requirement of the group.
• Parallelism differences between operators in the same SSG may reduce the resource utilization
• The SSG-based resource requirement might affect the internal optimization logic of Flink, such as chaining.
• Explain the difference between different execution modes and provide some configure suggestions for user.
• The operators of the same SSGs should with the same parallelism, otherwise there might be resource waste.