...
Based on FLINK-10205, we could ensure ....
simply return the InputSplits to the InputSplitAssigner
also implies transaction between task and jobManager (maybe multiple one), we need to make sure the inputSplits
get return to the InputSplitAssigner
exactly once. what happened if we have speculative execution, which means two task consume the same set of InputSplits and but not fail at same time, does every InputSplitAssigner need to keep a list to deduplicate? what happened if the TM died or has network issue and InputSplit cannot be return?
Furthermore, if there are two executions run at the same time (in batch scenario), this two executions should process same splits.
下面这两个pr里面的讨论,要说一下,
这个pr里面的讨论,要说一下,https://github.com/apache/flink/pull/6684
https://github.com/apache/flink/pull/8125
Manage middle ResultPartition
...