Status
Current state: Under Discussions
Discussion thread: here
JIRA: here [Change the link from KAFKA-1 to your own ticket]
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Users currently need to type in a HashMap for connection properties on Producers and Consumers.
Example:
Map<String, Object> config = new HashMap<>();
config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9999");
config.put(ConsumerConfig.RECEIVE_BUFFER_CONFIG, -2);
KafkaConsumer consumer = new KafkaConsumer<>(config, new ByteArrayDeserializer(), new ByteArrayDeserializer());
The idea is that you should be able to connect to the server using a connection string instead:
KafkaConsumer consumer = new KafkaConsumer("localhost:9999;parameter1=value2;parameter2=value3", new ByteArrayDeserializer(), new ByteArrayDeserializer());
This is similar to what users are used when connecting to Databases (e.g. JDBC), other message systems or many other server solutions.
Public Interfaces
No public interfaces would be added, however a new contructor would be added to KafkaConsumer and KafkaProducer
public KafkaConsumer(String connectionString,
Deserializer<K> keyDeserializer,
Deserializer<V> valueDeserializer) {
}
Proposed Changes
- As mentioned above, there would be an addition to KafkaConsumer and KafkaProducer.
- An utility class will be added to map the string parameters to current configurations. That constructor would then delegate to the already existing code.
My proposed format for the connection string would be:
IP:host;parameterName=value1;parameterName2=value2
Invalid conversions would throw a proper Exception (TBD).
Invalid parameters would also throw a proper exception (TBD).
Compatibility, Deprecation, and Migration Plan
- There wouldn't be any compatibility issues. The new constructor would be selected by simple method override
- Previous client implementations will continue to work without any issues.
Rejected Alternatives
Create a Factory Method for KafkaConsumer or KafkaProducer providing the Connection String:
KafkaFactory:
buildConsumer(String connectionString, Serializer, Deserializer);
buildProducer(String connectionString, Serializer, Deserializer);