Status

Current state: "Under Discussion"Abandoned

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-9-Trigger-DSL-td13065.html

...

1. There is no out-of-the-box way to combine two triggers. As an example, the user cannot specify a trigger that fires “after the watermark passes the end of the window and only if the number of elements is at least 5”. This could be useful for cases where data are anonymized using K-anonymity strategies.
2. In a previous document we described the notion of “allowed lateness”. Allowed lateness is a setting of the WindowOperator and a window for which maxTimestamp + allowedLateness <= currentWatermark is considered late. Lateness affects two things: 1) elements that belong to late windows are dropped, and 2) the state of a window is dropped as soon as the watermark passes the end of the window plus the allowed lateness (garbage collection). Although we allow elements to be late, the current triggers do not provide any possibility to customize the way a pipeline fires for late but not dropped elements. These firings are called “late firings”. In addition, there is no way to specify “early firings”, i.e. firings that happen before the watermark (or the processing time) passes the end of the window. To illustrate the above, the user cannot specify a policy that says: “fire in daily windows with allowed lateness of 12 hours, but give me early results every hour, and after the watermark passes the end of the window, fire every 10 elements”.

...

Proposed Changes

A summary of the triggers we propose to offer out-of-the-box are:

...

Existing users should not be affected by the proposed changes. All existing triggers and custom triggers will continue work without any problems.