Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

More specifically, This operator will sort the input before aggregation, and avoid invoking window actions, which is similar to '5)'.

Analysis

...

After the proposed changes, the following DataStream API could specify that the window will end only at the end of its inputs when EndOfStreamWindows is used.

DataStream:

  • DataStream#cogroup

WindowedStream:

  • WindowedStream#reduce

  • WindowedStream#aggregate

  • WindowedStream#apply

  • WindowedStream#process

  • WindowedStream#sum

  • WindowedStream#min

  • WindowedStream#max

  • WindowedStream#minBy

  • WindowedStream#maxBy

KeyedStream:

  • KeyedStream#process

  • KeyedStream#intervalJoin

  • KeyedStream#reduce

  • KeyedStream#sum

  • KeyedStream#min

  • KeyedStream#max

  • KeyedStream#minBy

  • KeyedStream#maxBy

CoGroupedStream:

  • CoGroupedStream#WithWindow#apply

ConnectedStreams:

  • ConnectedStreams#process

...

of APIs affected by this FLIP

The following DataStream API can benefit from using EndOfStreamWindows.

  • DataStream#windowAll
  • KeyedStream#window
  • CoGroupedStreams#Where#EqualTo#window
  • JoinedStreams#Where#EqualTo#window

Benchmark results

To demonstrate our optimization improvements,we run each benchmark in different execution modes and configurations. Each result is measured by taking the average execution time across 5 runs with the given configuration.

...