Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

the option 'watermark.alignment.update-interval' is not required. the default value is 1s.

Note source connectors have to implement watermark alignment of source split in order to use the watermark alignment feature since 1.17 according flip-217 [2]. If source connector does not implement flip-217, the task will run with an error, user could set `pipeline.watermark-alignment.allow-unaligned-source-splits` to true to disable watermark alignment of source split, and watermark alignment will be working properly only when your number of splits equals to the parallelism of the source operator.


All the options described above :

...

If some of the features are not supported, such as the watermark alignment feature on Kinesis connector, it will behave as FLIP-182 [23] and FLIP-217 [32] designed.

Rejected Alternatives

...

This idea is like extending FLIP-66[14]. However, since we already have many options for watermark related features, this would make the DDL complex and lengthy.

...

[1] https://github.com/yuchengxin/flink/commits/yuankui/watermark_params

[2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-66%3A+Support+Time+Attribute+in+SQL+DDL217: Support watermark alignment of source splits

[3] https://cwiki.apache.org/confluence/display/FLINK/FLIP-182%3A+182: Support + watermark + alignment + of + FLIP-27 + Sources 

[4] https://cwiki.apache.org/confluence/display/FLINK/FLIP-217%3A+Support+watermark+alignment+of+source+splits 66: Support Time Attribute in SQL DDL