Status

Current state: Under Discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: KAFKA-5636

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently, Kafka Streams implements session windows, tumbling windows, and hopping windows as windowed aggregation methods. While hopping windows with a small advance time are similar to a sliding window, this implementation's performance is poor because it results in many overlapping and often redundant windows that require expensive calculations. As sliding windows do calculations for each distinct window, the sliding window implementation would be a more efficient way to do these types of aggregations. Additionally, sliding windows are inclusive on both the start and end time, creating a different set of windows than hopping windows, which are only inclusive on the start time.  

This KIP proposes adding sliding windows to give users an efficient way to do sliding aggregation. 

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

Add kafka.streams.kstream.SlidingWindows and # APIs

/**
 * The fixed-size sliding window specifications used for aggregations.
 * <p>
 * The semantics of sliding aggregation windows are: Compute the aggregate total for the unique current window, from
 * the current stream time to T1 (size) milliseconds before. Both the lower and upper time interval bounds are inclusive.
 *
 * Thus, the specified {@link TimeWindow} is not aligned to the epoch, but to record timestamps.
 * Aligned to the epoch means, that the first window starts at timestamp zero.
 * <p>
 * For time semantics, see {@link TimestampExtractor}.
 *
 * @see TimeWindows
 * @see SessionWindows
 * @see UnlimitedWindows
 * @see JoinWindows
 * @see KGroupedStream#windowedBy(Windows)
 * @see TimestampExtractor
 */

public final class SlidingWindows extends Windows<TimeWindow> {
    /** The size of the windows in milliseconds. */
    public final long sizeMs;

    /** The grace period in milliseconds. */
    private final long graceMs;

	public static SlidingWindows of(final Duration size) throws IllegalArgumentException{

		//allows the user to set the size of the window in milliseconds, must be larger than 0
    }

	public SlidingWindows grace(final Duration afterWindowEnd) throws IllegalArgumentException{

		//allows the user to set the "grace period" or retention time for the windows, cannot be negative

	}


}

Proposed Changes

Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.