Page History

...

1. Dynamic partition discovery is disabled by default, and users have to specify the interval of discovery in order to turn it on.
2. The strategy used for new partitions is same as the initial offset strategy. According to the semantics, if the startup strategy is latest, the consumed data should include all data from the moment of startup, which also includes all messages from new created partitions. However, the latest strategy currently maybe used for new partitions, leading to the loss of some data (thinking a new partition is created and might be discovered by Kafka source several minutes later, and the message produced into the partition within the gap might be dropped if we use for example "latest" as the initial offset strategy).if the data from all new partitions is not read, it does not meet the user's expectations. Other ploblems see User specifies OffsetsInitializer for new partition Section.
3. Current JavaDoc does not display the whole behavior of OffsetsInitializers. There will be a different offset strategy when the partition does not meet the condition.(See Public Interfaces Section).

...

1. Enable partition discovery by default.

2. Provide a different EARLIEST strategy for later discovered partitions, by which users can also customize strategies for new- discovered and initial partitions.

3. Organize the code logic of the current built-in OffsetsInitializer, then modify the JavaDoc to let users know.

...