...
We started this discussion with Matthias J. Sax here: https://issues.apache.org/jira/browse/KAFKA-6953
Public Interfaces
[existing class] org.apache.kafka.streams.StreamBuilder
[added function] KTable<K, V> scheduledTable(String storeName, String scheduleExpression, boolean allInstances
Proposed Changes
Create a new DSL Source Type with the following characteristics:
...
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
- If we are changing behavior how will we phase out the older behavior?
- If we need special migration tools, describe them here.
- When will we remove the existing behavior?There won’t be any compatibility issues because this is a brand new feature/interface
Rejected Alternatives
1- Sink all instances’ local aggregation result to a Topic with a single partition so that we could have another Graph with a single instance that could aggregate all results
- In this case, if I had 500 instances processing 1000 messages/s each (with no bottlenecks), I would have a single partition topic with 500k messages/s for my single aggregating instance to process that much messages (great bottleneck)
...