...
To avoid such issues, we propose defining a parameterless fullWindowPartition()
method on DataStream class to express the semantics of full window processing. This method is functionally equivalent to the aforementioned dataStream.window(EndOfStreamWindows.get())
. By adopting this approach, users can explicitly apply full window processing without any misunderstandings.
The DataStream class has six subclasses, which all have access to the fullWindowPartition()
method. However, KeyedStream and IterativeStream should not support non-keyed full window processing. The table below indicates whether these six subclasses should support the feature:
DataStream subclass | Support non-keyed full window processing |
---|---|
KeyedStream | ❌ |
IterativeStream | ❌ |
DataStreamSource | ✅ |
CachedDataStream | ✅ |
SideOutputDataStream | ✅ |
SingleOutputStreamOperator | ✅ |
KeyedStream's records are all keyed and KeyedStream should only support keyed full window processing. Therefore, we will override the fullWindowPartition()
method in the KeyedStream class and apply keyed full window processing in it.
IterativeStream has been marked with the @Deprecated annotation. Consequently, we will override the fullWindowPartition()
method in the IterativeStream class and throw an UnsupportedOperationException.
The return type of the fullWindowPartition()
method is PartitionWindowedStream, which provides multiple APIs for full window processing, including mapPartition
, sortPartition
, aggregate
and reduce
.
...