Status

Current state: Draft

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA:

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently, the SocketServer does not support invalid static configurations, as it will crash Kafka with an uncaught exception if the static configuration is invalid during server startup, even if previous dynamic reconfigurations make the configuration valid. It would be better to support invalid static SocketServer configurations so long as there were dynamically set changes that made them valid.

Public Interfaces

Based on the approach chosen below, enabling this functionality may require an additional file on disk.

Proposed Changes

There are two general approaches: 

1. Load dynamic configuration changes via the latest local metadata log snapshot and apply them before constructing the SocketServer (current preferred approach).
   1. Specifically, this will involve installing and then initializing the BrokerMetadataPublisher  (specifically the DynamicConfigPublisher ) first so that dynamic configuration changes are processed from the current metadata image before reading any configs in the SocketServer . 
   2. Need to wait until the scheduled event MetadataLoader#initializeNewPublishers  completes before reading configs in the SocketServer to avoid a race. This is because the publisher installation future is returned before publisher initialization is guaranteed to be finished since it's scheduled on the separate MetadataLoader thread.
2. Write SocketServer dynamic configuration changes to a file. Load this file and apply the changes before constructing the SocketServer.

Points of discussion on the snapshot approach:

• Does not have to deal with any cases where the file is not readable/writeable.
• Reuses state we already persist to disk.
• May not have the most up-to-date information depending on how old the snapshot is. One approach to remediate this is to have the broker take a snapshot while in controlled shutdown.
• Requires us to read the snapshot twice during "startup", first at the raft layer and then in BrokerServer; although the first read should bring the snapshot file blocks into the page cache. Since the snapshot's size is independent of the length of the log this read is always relatively fast and won't delay startup.
• When no snapshot exists or the latest snapshot does not contain dynamic configuration changes, use supplied static configurations.

Points of discussion on the file approach:

• Have to deal with cases where file is not readable/writeable. Should the dynamic configuration updates go through but potentially be lost if the broker crashes?

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.