...
JIRA:
Jira | ||||||
---|---|---|---|---|---|---|
|
Jira | ||||||||
---|---|---|---|---|---|---|---|---|
|
Jira | ||||||
---|---|---|---|---|---|---|
|
PR: https://github.com/apache/kafka/pull/7170
...
New KStream#repartition operations shall be introduced in order to give the user control over parallelism for sub-topologies:. Additionally, we deprecate KStream#through in favor of the new #repartition methods.
Code Block | ||||
---|---|---|---|---|
| ||||
public interface KStream<K, V> {
@Deprecated
KStream<K, V> through(final String topic);
@Deprecated
KStream<K, V> through(final String topic, final Produced<K, V> produced);
/**
* Materialize this stream to a auto-generated repartition topic and creates a new {@code KStream}
* from the auto-generated topic using default serializers, deserializers, producer's {@link DefaultPartitioner}.
* Number of partitions is inherited from the source topic.
*
* @return a {@code KStream} that contains the exact same (and potentially repartitioned) records as this {@code KStream}
* @see #repartition(Repartitioned)
* @see #repartition(KeyValueMapper, Repartitioned)
*/
KStream<K, V> repartition();
/**
* Materialize this stream to a auto-generated repartition topic and creates a new {@code KStream}
* from the auto-generated topic using {@link Serde key serde}, {@link Serde value serde}, {@link StreamPartitioner},
* number of partitions and topic name part as defined by {@link Repartitioned}.
*
* @param repartitioned the {@link Repartitioned} instance used to specify {@link org.apache.kafka.common.serialization.Serdes},
* {@link StreamPartitioner} which determines how records are distributed among partitions of the topic,
* part of the topic name and number of partitions for a repartition topic, if repartitioning is required.
* @return a {@code KStream} that contains the exact same (and potentially repartitioned) records as this {@code KStream}
* @see #repartition()
* @see #repartition(KeyValueMapper, Repartitioned)
*/
KStream<K, V> repartition(final Repartitioned<K, V> repartitioned);
} |
...
This is a pure KStream library change that shouldn't affect previously setup applications. Since we introduce new KStream#groupBy operations, existing ones shouldn't be affected by this change. Using using KStream#through can either switch to the new #repartition method (which should be the common use case) or rewrite their code to use #to() and StreamsBuilder#stream() (note that #through() is just syntactic sugar for those two calls anyway).
Rejected Alternatives
Repartition "hint" in groupBy operations
...