Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Based on FLINK-10205, we could ensure ....


simply return the InputSplits to the InputSplitAssigner also implies transaction between task and jobManager (maybe multiple one), we need to make sure the inputSplits get return to the InputSplitAssigner exactly once. what happened if we have speculative execution, which means two task consume the same set of InputSplits and but not fail at same time, does every InputSplitAssigner need to keep a list to deduplicate? what happened if the TM died or has network issue and InputSplit cannot be return?

Furthermore, if there are two executions run at the same time (in batch scenario), this two executions should process same splits.

下面这两个pr里面的讨论,要说一下,

这个pr里面的讨论,要说一下,https://github.com/apache/flink/pull/6684

https://github.com/apache/flink/pull/8125


Manage middle ResultPartition 

...