DRAFT

Current state: Under Discussion

...

JIRA:

Motivation

Kafka Streams enforces a strict non-null-key policy in the DSL across all key-dependent operations (like aggregations and joins).
For left-joins, it might make makes sense to still accept a `null`, and add the left-hand record with an empty right-hand-side to the result.
Similarily, for outer-joins it makes sense to keep the record for the non null-key side.
The defined semantics for left/outer join are: keep the stream record if no matching join record was found.
Thus, Kstreams todays behavior is incosistent with the defined semantics.

Public Interfaces

This KIP won't change any public interfaces. However, It will change the behavior of several stream dsl operators:

• left KStream-tableKtable/globalTableGlobalTable
• left/outer streamKstream-streamKstream
• left-foreign-key tableKtable-table Ktable - allow foreign key to be null.

Proposed Changes

...

• left join Kstream-Kstream: no longer drop left records with null-key and call ValueJoiner with 'null' for right value.
• outer join Kstream-Kstream: no longer drop left/right records with null-key and call ValueJoiner with 'null' for right/left value.
• left join KStream-Ktable/GlobalTable: emits record even when right key is not found / null.
• left-foreign-key join Ktable-Ktable: no longer drop left records with null-foreign-key returned by the ForeignKeyExtractor and call ValueJoiner with 'null' for left value.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other wayUsers who want to keep the current behavior can prepend a .filter() operator to the aforementioned operators and filter accordingly.

Why not make this change opt in?
By default we would like the DSL be consistent with the defined semantics.