Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • The end-to-end latency of intermediate results will be higher. This is because each operator might buffer the intermediate results for up to the configured interval before emitting the intermediate result. This will negatively affect the data freshness of the job's output.
  • State size will be larger. This is because the corresponding operators will need to buffer / merge output records by key before emitting those outputs periodically.
  • There is higher state backend access overhead. Each buffering operation would bring a read & write state access. Note that this overhead can be mitigated with the LRU cache introduced in FLIP-325369.

Built-in operators and functions that would be affected by this FLIP

...

2) Deprecate table.exec.mini-batch.* configurations after both FLIP-325 369 and FLIP-365 has been implemented.

table.exec.mini-batch brings two optimizations: reduced state backend access and reduced number of output records. The design proposed in this FLIP and FLIP-325 369 can cover these optimizations with the following additional advantages:

...