Status

Current state: Under Discussions

Discussion thread: KIP-209 Connection String Support

JIRA: a JIRA will be created once there is consensus about this KIP

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Users currently need to type in a HashMap for connection properties on Producers and Consumers.

Example:

Map<String, Object> config = new HashMap<>();
config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9999");
config.put(ConsumerConfig.RECEIVE_BUFFER_CONFIG, -2);
KafkaConsumer consumer = new KafkaConsumer<>(config, new ByteArrayDeserializer(), new ByteArrayDeserializer());

The idea is that you should be able to connect to the server using a connection string instead:

KafkaConsumer consumer = new KafkaConsumer("localhost:9999;parameter1=value2;parameter2=value3", new ByteArrayDeserializer(), new ByteArrayDeserializer());

This is similar to what users are used when connecting to Databases (e.g. JDBC), other message systems or many other server solutions.

Public Interfaces

No public interfaces would be added, however a new contructor would be added to KafkaConsumer and KafkaProducer

public KafkaConsumer(String connectionString,

 Deserializer<K> keyDeserializer,
 Deserializer<V> valueDeserializer) {
}

A new Exception would be added: 

public InvalidParameterException extends APIException

Proposed Changes

• As mentioned above, there would be an addition to KafkaConsumer and KafkaProducer.
• An utility class will be added to map the string parameters to current configurations. That constructor would then delegate to the already existing code.

My proposed format for the connection string would be:

IP1:host1,IP2:host2,...IPN:hostn;parameterName=value1;parameterName2=value2;... parameterNameN=valueN

Invalid conversions would throw InvalidParameterException (without a description of the invalid conversion)

Invalid parameters would throw InvalidParameterException (with the name of the invalid parameter).

Compatibility, Deprecation, and Migration Plan

• There wouldn't be any compatibility issues. The new constructor would be selected by simple method override
• Previous client implementations will continue to work without any issues.

Rejected Alternatives

Create a Factory Method for KafkaConsumer or KafkaProducer providing the Connection String:

KafkaFactory:

   buildConsumer(String connectionString, Serializer, Deserializer);

   buildProducer(String connectionString, Serializer, Deserializer);