Status
Current state: Under discussion.
Discussion thread: here (Not happening yet)
JIRA: here
Motivation
Currently, clients would fail if they cannot bootstrap upon starting, and one common reason is DNS lookup failure. The application owner will either need to implement retry logic or manually restart the application. This is inconvenient and hard to handle because:
Bootstrap failure throws a ConfigException, which does not indicate the actual problem (the message is fine, but just the exception type); unless the developer tries to parse and match the error message.
It can take minutes before the bootstrap server is registered to the DNS server, and it is reasonable to allow clients to continue to retry.
Public Interfaces
- Users will no longer get ConfigException upon failed DNS resolution.
- Several logging (warn) around failing DNS resolution will be removed.
Proposed Changes
Move the DNS resolution and metadata bootstrap logic to the NetworkClient, which means bootstrapping only happens upon invoking the poll.
Changes
Client Constructor: Only parse the bootstrap config and validate its format there
NetworkClient:
Bootstrapping should now occur in the poll method before attempting to update the metadata. This includes resolving the addresses and bootstrapping the metadata.
Throws and logs a NetworkException if DNS resolution fails.
Compatibility, Deprecation, and Migration Plan
Client Behaviors
Clients won’t attempt to resolve the bootstrap addresses upon initialization.
Clients won’t exit fatally if DNS resolution fails.
KafkaConsumer: Users will need to poll to retry the lookup if it failed.
KafkaAdminClient: Users will need to resend the request if failing.
KafkaProducer: The sender loop should already be polling continuously.
Exception Handling
Failed DNS resolution will result in NetworkException
Test Plan
NetworkClient
Test bootstrap upon its initial poll
Test if the right exception type is thrown
Existing clients (Consumer, Producer, AdminClient)
Test successful bootstrapping upon retrying
Rejected Alternatives
Allow the application owner to specify a retry period. The clients will fail after exceeding the timeout. The default set to 0s, which makes retry an opt-in config.
Pros: Allows users to have more control over how long to retry
Cons: Require a new config; client instantiation can block.
No retry. Let the application owner handle the DNS resolution exception. This means we would still throw a DNSLookupException upon failing.
Pros: No additional config is needed
Cons: This is a behavioral change, and the application owner might need to rewrite the exception handling, i.e. catching the DNS failure logic.