Status

Current state: DraftClosed – Approach does not require any public facing changes.

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

...

Currently, the SocketServer does not support invalid static configurations, as it will crash Kafka with an uncaught exception if the static configuration is invalid during server startup, even if previous dynamic reconfigurations make the configuration valid. It would be better to support invalid static SocketServer configurations so long as there were dynamically set changes that made them valid.

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• Binary log format

• The network protocol and api behavior

• Any class in the public packages under clientsConfiguration, especially client configuration

  • org/apache/kafka/common/serialization

  • org/apache/kafka/common

  • org/apache/kafka/common/errors

  • org/apache/kafka/clients/producer

  • org/apache/kafka/clients/consumer (eventually, once stable)

• Monitoring

• Command line tools and arguments

• Anything else that will likely break existing users in some way when they upgrade

Proposed Changes

...

Based on the approach chosen below, enabling this functionality may require an additional file on disk.

Proposed Changes

There are two general approaches: 

1. Load dynamic configuration changes via the latest local metadata log snapshot and apply them before constructing the SocketServer (current preferred approach). Specifically, circumvent KRaft and metadata layers entirely and directly read the snapshot file records from disk. 
   
   1. Circumventing KRaft layer: Read in the local snapshot file from disk without going through KRaft layer. In this approach, there are two cases: one where the broker requires a new snapshot from the controller during metadata catch up because its fetch offset is less than the controller's log start, and one where the broker does not require a new snapshot. When defaulting to loading in dynamic configurations from snapshot, the former case means two different snapshot reads occur during startup, and the latter means the first read will hopefully bring file blocks into the page cache so it's not as bad as the first (not really sure on this yet). If this approach is taken, this read should only occur when static configurations are invalid and reading them would crash Kafka.

Points of discussion on the snapshot approach:

• Does not have to deal with any cases where the file is not readable/writeable.
• Reuses state we already persist to disk.
• The server may not have the most up-to-date information depending on how old the snapshot is. One approach to remediate this is to have the broker take a snapshot while in controlled shutdown.
• When no snapshot exists or the latest snapshot does not contain dynamic configuration changes, use supplied static configurations.

Compatibility, Deprecation, and Migration Plan

...

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

...

• Use initial metadata state retrieved from quorum: This approach is more straightforward and less hacky, but may run into some issues. For BrokerServer , if its SocketServer  configurations are invalid in such a way which it is unable to connect to the controller quorum, then this approach cannot work.
• Write SocketServer dynamic configuration changes to a file. Load this file and apply the changes before constructing the SocketServer.
  • Some issues with this approach: 
  • Have to deal with cases where file is not readable/writeable. Should the dynamic configuration updates go through but potentially be lost if the broker crashes?