Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Technically speaking, if you have an element with timestamp t and windows of size duration (both in milliseconds) the start of the window that this element is in is given by t - (t % duration). The same applies to sliding time windows on timestamp time. 

 

Aligned Time Windows

 

General Windows

 

 

Accessing the Timestamp of an Element

Accessing the Timestamp of an Element

Still TBD, this is a first idea.

If you have to access the timestamp of an element or change it you must use the special flatMapWithTimestamp() operation. This expects a FlatMapWithTimestampFunction:

Code Block
languagejava
titleDataStream Example
linenumberstrue
public interface FlatMapWithTimestampFunction<T, O> {
    void flatMap(T value, long timestampInMillis, TimestampCollector<O> out);
}
 
public interface TimestampCollector<O> {
    public void collect(O value, long timestampInMillis);
}

The timestamp of the emitted element must not be lower than the timestamp of the input element. Otherwise the guarantees given by the watermarks would no longer hold. 

 

Implementing Windows on Ingress- and Event Time

 Still TBD, I have a proof-of-concept for time windows (aljoscha)


Time-windows

Pane based approach

...