Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In this example, we can see that according to the hash function used by the Flink, the key group of key_0 is 64. However, it is forwarded to the downstream operator, who only takes responsibility for key groups from 0 to 63, but not including 64. The downstream operator would then try to extract and update the state of KeyGroup 64, which it doesn't take responsibility for. Therefore, a runtime error would be triggered.


This FLIP would like to address this problem and introduce the reinterpretAsKeyedStream() to the DataStream formally.

Public Interfaces

Anchor
reinterpretAsKeyedStream
reinterpretAsKeyedStream

...

However, the mapping from a record or a key to the SplitID could be obscure to the user. For example, the data could be computed in Flink, then streaming to the Flink through an MQ, e.g., Kafka. Neither In this example, neither the logic of partitioning of the different system nor the splitID is a public API for the users. It . This could be common under many scenarios. Therefore, it is hard and possibly meaningless for users to provide the Flink such mapping"SplitSelector".


Option 2. Assigned KeyGroup by the SourceOperator (Prefer one)

...