...

• Client Behaviors

  • Clients won’t attempt to resolve the bootstrap addresses upon initialization.

  • Clients won’t exit fatally if DNS resolution fails.will retry bootstraping until the bootstrap timer expires

  • KafkaConsumer: Users must poll to retry the lookup retry bootstraping via poll(), if it fails.  Each retry is bounded by the poll timer.

  • KafkaAdminClient: Users will need to resend the request if failing.Bootstrap exception is thrown when user tries to materialize the result future. The retry is bounded by the API timeout.

  • KafkaProducer: Bootstrap is done in the background thread.  If the client hasn't been bootstrapped when the user attempts to send  can wait up to max block timerKafkaProducer: The sender loop should already be polling continuously.

  Exception Handling

  • Failed DNS resolution will result in NetworkException

...

To help illustrate the proposed changes, we provide some examples of how clients might behave in different scenarios.

...

Case 1: Unable to connect to the bootstrap (For example: misconfiguration)

Suppose If the user instantiates a KafkaConsumer with an invalid bootstrap config. When the user invokes assign() and starts configuration and then initiates a poll(), the poll() method will continue to return empty ConsumerRecords and log a warning message.The user can continue to retry for the configured duration. After method will block until the poll timer or the bootstrap timeout expires. When the bootstrap timeout expires, the client will throw a BootstrapConnectionException.

Case 2: Transient Network Issue (For example: transient DNS failure)

Now, suppose the user instantiates a KafkaConsumer with a valid bootstrap config, but there is a transient network issue, such as slow DNS resolution. When the user starts poll(), if the poll() method will return an empty ConsumerRecord and log a warning message.The user can continue to retry, and the network issue will be successfully resolved after some time. The KafkaConsumer will then continue to function normally.transient error is resolved before the poll timer runs out, the client will behave normally.  Otherwise, 

KafkaProducer

Case 1: Unable to connect to the bootstrap (For example: misconfiguration)

Suppose If the user instantiates a KafkaProducer is instantiated with an invalid bootstrap config. As the produce is instantiated, the sender thread starts running.  A , a warning message is logged everytime every time the NetworkClient tries to poll().If the user tries to produce messages, the producer callback may be completed with a TimeoutException until the bootstrap timeout bootstrap. A BootstrapconnectionException will be thrown when the timer runs out.

Eventually, a BootstrapConnectionException will be thrown.

Case 2: Transient Network Issue (For example: transient DNS failure)

...

Case 1: Unable to connect to the bootstrap (For example: misconfiguration)

...

When the user instantiates a new admin client

...

and makes admin client API calls

...

1. If the API timeout before the bootstrap timeout expires, a TimeoutException will be thrown upon invoking .get()
2. If the bootstrap timeout expires, a BootstrapConnectionException will be thrown.
3. Any API calls will not be completed.

, the result will either timeout if the request times out first; otherwise, the request will be completed exceptionally with a BootstrapConnectionException. Note, a warning message will be logged every time the Network Client tries to bootstrap.

Case 2: Transient Network Issue (For example: transient DNS failure)

...

In case if there is a transient network issue, such as a transient DNS failure, the user won't be able to get the results back until the bootstrap issue is resolved.  Meanwhile, if the call time expires, the request will be completed exceptionally with a TimeoutException.

Test Plan

1. NetworkClient

   1. Test DNS resolution upon its initial poll

   2. Test if the right exception type is thrown

2. Existing clients (Consumer, Producer, AdminClient)

   1. Test successful bootstrapping upon retrying

...