Page History

...

Please note that the update do not change the abstraction of the information passed by the `Watermark` class but rather provide a natural extension to its current capabilities. When class but rather provide a natural extension to its current capabilities. When `useProcessingTime` is false, the watermark indicates that the stream has progressed to a specific event time. When `useProcessingTime` is true, it indicates that the stream progresses in synchronization with the system time. In both case, the watermark indicates the time progression in the data stream.

...

• `AbstractStreamOperator` and `AbstractStreamOperatorV2` manage and trigger event time and processing time timers through manage and trigger event time and processing time timers through `InternalTimeServiceManager`.

• Only when the operator receives a watermark, it calls `InternalTimeServiceManager#advanceWatermark` to advance the event time and trigger event time timers to advance the event time and trigger event time timers.

Updated Behavior：

• When receiving a watermark with `useProcessingTime` set to false, the behavior remains the same as before set to false, the behavior remains the same as before.

• When receiving a watermark with `useProcessingTime` set to true, the operator starts the `ScheduledThreadPoolExecutor` that trigger the event time timers as the system time progresses.

...

`WatermarkToDataOutput` is responsible for sending the watermark from the `SourceReader`  downstream and ensuring that the watermark timestamps sent are monotonically increasingdownstream and ensuring that the watermark timestamps sent are monotonically increasing.

Original Behavior:

• When the `WatermarkToDataOutput` receive a watermark, it checks the timestamp of the watermark. It only send the watermark downstream if the timestamp of the watermark is greater than the timestamp of the most recently sent watermark.

...

• When the `WatermarkToDataOutput` receive a watermark, it checks the timestamp and `useProcessingTime` field of the watermark. It only send the watermark downstream if the timestamp of the watermark is greater than the timestamp of the most recently sent watermark or if the `useProcessingTime` field is set to true.

• If the watermark to be sent has `useProcessingTime` set to true, and the current system time is lesser than the timestamp of the most recently sent watermark, an exception is thrown.

• If a watermark with `useProcessingTime` set to true has been previously sent, and the watermark to be sent has `useProcessingTime` set to false, an exception is thrownset to false, an exception is thrown.

The updated `WatermarkToDataOutput` ensures that when `useProcessingTime` is false, the watermark's timestamp is monotonically increasing. It also ensures that information with `useProcessingTime` set to true is sent downstream and that the `useProcessingTime` will not set back to false once it is set to true.

...