Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

See below

Proposed Changes

 

(A) The proposal is to change the semantics of punctuate() to be purely "time driven", instead of "part time driven, and part data-driven". That is, the punctuate function triggering will no longer be dependent whether there are new data arriving, and hence not on the timestamps of the arriving data either. Instead it will be triggered only by system wall-clock time.

...

The above approach will have consequences for implementations built against the current semantics.

 

(B) An alternative could be to leave current semantics as the defaults for the punctuate method but allow a configuration switch between event and system time.

 

(C) Another alternative would be to leave current semantics as-is and add another callback method to the Processor interface that can be scheduled similarly to punctuate() but would always be called at fixed, wall clock based intervals

 

(D) Yet another A non-invasive alternative would be to leave current semantics as-is but allow users to provide a function determining the timestamp of the stream task. In a similar way to how the TimestampExtractor allows users to decide what the current timestamp is for a given message (event-time, system-time or other), this feature would allow users to decide what the current timestamp is for a given stream task irrespective of the arrival of messages to all of the input partitions. This approach is less preferable to the above due to the extra complexity, however allows for backward compatibility.

 

Yet another non-invasive alternative could be leave current semantics as the defaults for the puncutate method but allow configuration switch between event and system timebrings more flexibility at the expense of added complexity.

Compatibility, Deprecation, and Migration Plan

...

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.

scheduled similarly to punctuate() but was always called at fixed, wall
clock based intervals