Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The recursively method applies input records to its op argument. The results are then both emitted as a result of recursively and also fed back in to the op KStream.The op KStream 

Restrictions:

  • op cannot be UnaryOperator.identity, or an equivalent function that simply returns its argument unmodified - this would produce an infinite recursive loop, since there's no opportunity refine the output to break out of the loop.
  • op MUST "terminate"; that is, it must have some condition which eventually prevents further recursion of a record. In our example here, the terminating condition is the join, since the root node of our graph will have no parent, so the join will produce no output for the root node.
    • We can attempt to detect "definitely non-terminating" arguments by failing to detect operations that can cause the stream to terminate (e.g. filter, join, flatMap, etc.) in the process graph produced by the function.
    • We cannot guarantee that a function that includes terminating operations (filter, join, flatMap, etc.) actually terminates.

Automatic Repartitioning

If the op argument applies a key-changing operation (as it does in our example above), a repartition  topic may be automatically created. The optional Produced argument can be provided to customize repartitioning behaviour. This argument is ignored if a repartition  topic is not necessary.

...

Repartitioning is required only when the op KStream requires repartitioning and the current KStream does not require repartitioning. The reason for this is that if both streams require repartitioning, then the current stream is guaranteed to automatically repartition records before any operation that requires repartitioning. In the above example, recursively would not include a repartition topic, because join already includes one.

Restrictions:

...

.

Implementation

In KStreamImpl, implementation is fairly simple:

...