Status

Current state: Under Discussion

Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

The DSL currently supports windowed aggregations for only two types of time-based window: hopping and tumbling. A third kind of window is defined, but only used in join operations: sliding windows. Users needing sliding window semantics can approximate them with hopping windows where the advance time is 1, but this workaround only artificially resembles the sliding window model; aggregates would be output for every defined hopping window, of which there will likely be a large number (specifically, the size of the window in milliseconds).

More importantly, the semantics differ slightly. Sliding windows are inclusive at both ends (ie start and end time bounds), whereas hopping/tumbling windows are inclusive/exclusive.

Also of interest is the potential for reuse as the underlying mechanism for a stream-self join. Currently this is not supported, or even allowed, in the Streams DSL. But with the addition of sliding window aggregations to the built-in operators, we could detect a stream-self join and convert it to a sliding window aggregation under the covers. It may be easier to reason about stream-self joins than aggregations for some users, or in some use cases, so there is value in adding both possibilities to the DSL. It's basically a "buy one, get one free" situation with respect to filling in gaps in the API.

Public Interfaces

Proposed Changes 

Compatibility, Deprecation, and Migration Plan

N/A

Rejected Alternatives