THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Table of Contents |
|---|
Status
| Page properties | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Right now, in Flink a WindowFunction does not get a lot of information when a window fires.
...
- New class ProcessWindowFunction with extensible interface
- New overload of WindowedStream.apply() that takes the new window function
Proposed Changes
New ProcessWindowFunction
We propose to add a new window function that has an extensible interface so that we can easily add more meta information in the future. The initial signature would be this:
...
The context object has the same information as WindowFunction but can be extended in the future.
Required Changes
Internally the WindowFunction that a user uses is already decoupled from the windowing internals. Internally an InternalWindowFunction is used. Normally, an InternalWindowFunction wraps a WindowFunction. We propose to change InternalWindowFunction to match the newly proposed interface. It can still be used to present the current interface to WindowFunction while we can now also support the new ProcessWindowFunction. This is necessary because we cannot break the current API around WindowFunction.
Future Changes based on this FLIP
Adding a Window-Firing Counter
This requires to extend the Context given to the ProcessWindowFunction like this:
...
that will have to be updated when firing windows and that will have to be garbage collected when the window state is being garbage collected.
Adding a Window-Firing "Reason"
This requires to extend the Context given to the ProcessWindowFunction like this:
...
- The new interface will not impact users of the existing WindowFunction
- We can deprecate the old WindowFunction in Flink 2.0, if we want
Test Plan
Describe in few sentences how the FLIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?
Rejected Alternatives
If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.