Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The web UI does not show all the concurrent executions of each ExecutionVertex/subtask. It only shows the one with the fastest progress.

User defined functions must not be affected by its speculative instances

When a user defined function and its speculative instances run at the same time, they must not affect each other. For example,

  • access to the same exclusive resources
  • overriding the output to external services which happens as a side effect, i.e. not via Flink sinks
  • competition for data ingestion. Note that it includes cases that
    • user defined source function competition
    • data ingestion happens as a side effect, i.e. not via Flink sources.
  • ...

Once the concurrent instances can affect each other, it may result in task failures, or even worse, data inconsistency. So that speculative executions should not be enabled in this case.

Compatibility, Deprecation, and Migration Plan

...