...
In KIP-450: Sliding Window Aggregations in the DSL, we introduce Sliding Window to do calculations for each distinct window, the sliding window implementation would be a more efficient way to do these types of aggregations. In the Sliding Window, we was planning to have windows with inclusive on both the start and end time, but currently, we implemented in TimeWindow which is a half-open time interval. We should introduce a new window type: SlidingWindow, to have "\[start, end]" time interval, for SlidingWindows aggregation.
Public Interfaces
Add org.apache.kafka.streams.kstream.internals.SlidingWindow
/** |
SlidingWindow} covers a |
closed time interval with its start timestamp |
and |
end |
|
timestamp as |
an inclusive boundary. |
inclusive). |
inclusive) |
*
{@code startMs} |
SlidingWindow(final long startMs, final long endMs) throws IllegalArgumentException |
super(startMs, endMs);
if (startMs ==
|
{ |
throw new IllegalArgumentException("Window endMs must be greater than
|
} |
/** |
if (getClass() != other.getClass()) {
throw new IllegalArgumentException("Cannot compare windows of different type. Other window has type "
+ other.getClass() + ".");
}
final TimeWindow otherWindow = (TimeWindow) other;
return startMs < otherWindow.endMs && otherWindow.startMs < endMs;
}
}
Proposed Changes
} |
Proposed Changes
This KIP will change the KStreamSlidingWindowAggregate implementation, to replace the original TimeWindow type, with the new SlidingWindow type. So that we can have correct aggregation calculation for the set of records within the window.
Ex: In KIP-450, we have a graph to describe what expected result we want:
But because of we used the TimeWindow to represent the window time interval, it actually has 1 ms difference for each window.
ex: originally, we expected have a window \[-7, 3], but we actually have a window \[-7, 2], or \[-7, 3)
After this KIP, this issue will be fixed and have a correct aggregation calculation resultsDescribe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
- If we are changing behavior how will we phase out the older behavior?
- If we need special migration tools, describe them here.
- When will we remove the existing behavior?
Rejected Alternatives
...
No migration plan is needed.