Status
Current state: Under Discussions
Discussion thread: KIP-209 Connection String Support
JIRA: a JIRA will be created once there is consensus about this KIP
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Users currently need to type in a HashMap for connection properties on Producers, Consumers and AdminClient
Example:
Map<String, Object> config = new HashMap<>();
config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9999");
config.put(ConsumerConfig.RECEIVE_BUFFER_CONFIG, -2);
KafkaConsumer consumer = new KafkaConsumer<>(config, new ByteArrayDeserializer(), new ByteArrayDeserializer());
The idea is that you should be able to connect to the server using a connection string instead:
KafkaConsumer consumer = new KafkaConsumer("localhost:9999,localhost2:99999;parameter1=value2;parameter2=value3", new ByteArrayDeserializer(), new ByteArrayDeserializer());
This is similar to what users are used when connecting to Databases (e.g. JDBC), other message systems or many other server solutions.
Public Interfaces
No public interfaces would be added, however a new contructor would be added to KafkaConsumer, KafkaProducer and KafkaAdminClient
public KafkaConsumer(String connectionString,
Deserializer<K> keyDeserializer,
Deserializer<V> valueDeserializer) {
}
Proposed Changes
- As mentioned above, there would be an addition to KafkaConsumer, KafkaProducer and KafkaAdminClient
- An utility class will be added to map the string parameters to current configurations. That constructor would then delegate to the already existing code.
My proposed format for the connection string would be:
IP1:host1,IP2:host2,...IPN:hostn;parameterName=value1;parameterName2=value2;... parameterNameN=valueN
Invalid conversions would throw InvalidArgumentException (with a description of the invalid conversion)
Invalid parameters would throw InvalidArgumentException (with the name of the invalid parameter).
Compatibility, Deprecation, and Migration Plan
- There wouldn't be any compatibility issues. The new constructor would be selected by simple method override
- Previous client implementations will continue to work without any issues.
Rejected Alternatives
Create a Factory Method for KafkaConsumer or KafkaProducer providing the Connection String:
KafkaFactory:
buildConsumer(String connectionString, Serializer, Deserializer);
buildProducer(String connectionString, Serializer, Deserializer);