Page History

...

So if we create KStream from the stream of business events and create a KTable(s) from all database tables that store business entity data (for example using Change Data Capture) we can assemble complete Business Entity using stream-table joins . and then aggregating on event key.

For the Order Completion example above the sequence of order extraction from Kafka topics is as follows:

• As the first step Business event stream could be joined with business entity parent table using stream-table join on PK. Using the order management example on the diagram below this will be Order Processing Events stream joined with Orders KTable on OrderId key resulting in a stream of completed orders. 
• As the second step the stream of completed orders could be left joined with Order Details table using the foreign key (OrderId) resulting in a stream of competed orders with corresponding order details.

...

• If order to order details relation 1:n then additional aggregation on OrderId will be required. 

This KIP proposes the new Kafka Streams feature with capability to join the KStream with KTable using left Foreign Key Join

...