THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
The main motivation of this KIP is stated in the related JIRA: "Today the downstream sub-topology's parallelism (aka the number of tasks) are purely dependent on the upstream sub-topology's parallelism, which ultimately depends on the source topic's num.partitions. However this does not work perfectly with dynamic scaling scenarios". By delegating the stream topology power to create repartition topic with customized number of partitions gives user more flexibility. Also, for API like #to or #through, KStream has no access to sink topic creation, which means user has to create their own connecting Kafka topic for every single new application which is cumbersome. Thus we are proposing extending the capability of #Produced and #Grouped API to automatically create topics when the target topic has not created.
...
We shall expand the Produced API to contain topicName and numPartitions:
| Code Block | ||||
|---|---|---|---|---|
| ||||
public class Produced<K, V> {
protected Serde<K> keySerde;
protected Serde<V> valueSerde;
protected StreamPartitioner<? super K, ? super V> partitioner;
protected final String topicName; // new
protected final String Integer numPartitions; // new
public static <K, V> Produced<K, V> with(final Serde<K> keySerde,
final Serde<V> valueSerde,
final StreamPartitioner<? super K, ? super V> partitioner,
final topicName,
final Integer numPartitions);
} |
We also want to expand Grouped API with a numPartition configuration. numPartitions configuration：
| Code Block | ||||
|---|---|---|---|---|
| ||||
public class Grouped<K, V> {
protected final Serde<K> keySerde;
protected final Serde<V> valueSerde;
protected final String name;
protected final StringInteger numPartitions; // new
public static <K, V> Grouped<K, V> with(final String name,
final Serde<K> keySerde,
final Serde<V> valueSerde,
final Integer numPartitions);
}
|
...