Versions Compared

Old Version 15

changes.mady.by.user Stanislav Kozlovski

Saved on

compared with

New Version Current

changes.mady.by.user Kamal Chandraprakash

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Ideally, Kafka would support dynamically changing log levels and address all of the aforementioned concerns out of the box.

We propose extending the IncrementalAlterConfig/DescribeConfig Admin API with functionality for dynamically altering the a single broker's log level.

This approach would also pave the way for even finer-grained logging logic (e.g log DEBUG level only for a certain topic) and would allow us to leverage the existing AlterConfigPolicy for custom user-defined validation of log-level changes.
These log-level changes will be temporary and reverted on broker restart - we will not persist them anywhere.

...

SET: Set the log level to the desired value
REMOVE: Sets Unsets the log level to the same as of the logger. This effectively means it is set to the root logger's log level, as the logging library goes up the chain of configured loggers until it finds one. By default - the next logger in the hierarchy is root.

IncrementalAlterConfigsOp => INT8
0: SET
1: REMOVE # sets log level to the root logger's level
2: APPEND # NOT SUPPORTED
3: SUBTRACT # NOT SUPPORTED
  
IncrementalAlterConfigsRequest (Version: 0) => [resources] validate_only
 validate_only => BOOLEAN
 resources => resource_type resource_name [configs]
    resource_type => INT8 # BROKER_LOGGER (8)
    resource_name => STRING # ID of broker
    configs => config_name config_op config_value
        config_name => STRING
        config_op => INT8 # support SET and APPEND only
        config_value => NULLABLE_STRING
 
IncrementalAlterConfigsResponse (Version: 0) => [responses] 
  responses => resource_type resource_name error_code error_message
  resource_type => INT8 # BROKER_LOGGER (8)
  resource_name => STRING # ID of broker
  error_code => INT16
  error_message => NULLABLE_STRING

...

In the case of an invalid config_value or an invalid/non-existent logger name, the broker will return an INVALID_CONFIG (40) error for that config resource (BROKER_LOGGER).
INVALID_REQUEST will be returned when attempting to reset the unset the ROOT logger's log level

Tools Changes

...

bin/kafka-configs.sh --bootstrap-server localhost:9092 --describe --entity-type broker-logger loggers --entity-name 0  // show all the log levels for broker 0
 
bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config "kafka.server.ReplicaManager=WARN,kafka.server.KafkaApis=DEBUG" --entity-type broker-logger loggers --entity-name 0  // set some log levels for broker 0
 
bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --delete-config kafka.server.ReplicaManager --entity-type broker-logger loggers --entity-name 0 // will set the log level to the ROOT logger level

...