Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Warning

Prior to version 3.1.x Kafka Streams might emit so called "spurious" left/outer join result. In this section we only explain the different new behavior that avoids spurious left/outer stream-stream join results. See New Join Semantics below that describe all joins in more details, including spurious left/output join behavior in versions 0.10.2.x  to 3.0.x

...

ts

STREAM_1 (left)

STREAM_2 (right)

innerJoin (same as in older versions)

leftJoin

outerJoin

1

null





2


null




3

A





4


a

A - aA - a

A - a

5

B


B - a

B - a

B - a

6


b

A - b

B - b

A - b

B - b

A - b

B - b

7

null





8


null




9

C


C - a

C - b

C - a

C - b

C - a

C - b

10


c

A - c

B - c

C - c

A - c

B - c

C - c

A - c

B - c

C - c

11


null




12

null





13


null




14


d

A - d

B - d

C - d

A - d

B - d

C - d

A - d

B - d

C - d

15

D


D - a

D - b

D - c

D - d

D - a

D - b

D - c

D - d

D - a

D - b

D - c

D - d

...




40E



...




60F

E - nullE - null
...




80
f
F - nullF - null
...




100G


null - f

...

Warning

This section describes the new join semantics as of version 0.10.2.x. For old join semantics (version 0.10.0.x and 0.10.1.x) see Old Join Semantics below.

(See KIP-77: Improve Kafka Streams Join Semantics)

...

The table below marks so called "spurious" left/outer join results, that are in the result in version 0.10.2.x to 3.0.x, in bold face. Compare Improved left/outer stream-stream join above for version 3.1.x that avoids spurious results.

...

ts

left

right

innerJoin

leftJoin

outerJoin

1

null





2


null




3

A



A - null

A - null

4


a

A - a

A - a

A - a

5

B


B - a

B - a

B - a

6


b

B - b

B - b

B - b

7

null


null

null

null - b

8


null



null

9

C



C - null

C - null

10


c

C - c

C - c

C - c

11


null

null

C - null

C - null

12

null



null

null

13


null




14


d



null - d

15

D


D - d

D - d

D - d


16




17
dD - dD - dD - d

KTable-KTable Foreign-Key Join (v2.4.x and newer)

This is a symmetric non-window join. There are two streams involved in this join, the left stream and the right stream, each of which are usually keyed on different key types. The left stream is keyed on the primary key, whereas the right stream is keyed on the foreign key. Each element in the left stream has a foreign-key extractor function applied to it, which extracts the foreign key. The resultant left-event is then joined with the right-event keyed on the corresponding foreign-key. Updates made to the right-event will also trigger joins with the left-events containing that foreign-key. It can be helpful to think of the left-hand materialized stream as events containing a foreign key, and the right-hand materialized stream as entities keyed on the foreign key.

KTable lookups are done on the current KTable state, and thus, out-of-order records can yield non-deterministic result. Furthermore, in older versions of Kafka Streams there is no guarantee that all records will be processed in timestamp order (even if processing records in timestamp order is the goal, it is only best effort).

...