Current state: Under Discussion.
Discussion thread: https://www.mail-archive.com/dev@kafka.apache.org/msg99656.html
Vote thread:
JIRA:
Released:
Pull request:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Kafka Connect does not provide an out-of-the-box facility to change log levels. When debugging connectors or the Connect framework, users have to update the log4j.properties
file and restart the worker to see new logs. This is cumbersome in most cases, and restarting the worker sometimes hides bugs by resetting the internal state.
This change will introduce an /admin/loggers
endpoint to the Connect worker. In addition, a new configuration property admin.listeners
will be provided to specify which listener to use for the /admin
endpoint. The /admin/loggers
endpoint will support the following operations:
$ curl -Ss http://localhost:8083/admin/loggers | jq [ ... "org.apache.kafka.connect.runtime.WorkerSinkTask", "org.apache.kafka.connect.runtime.WorkerSourceTask", ... ] |
$ curl -Ss http://localhost:8083/admin/loggers/org.apache.kafka.connect.runtime.WorkerSinkTask | jq { "level": "INFO" } |
$ curl -Ss -X PUT '{"level": "TRACE"}' http://localhost:8083/admin/loggers/org.apache.kafka.connect.runtime.WorkerSinkTask // no output |
An additional configuration property will be used to control how the admin endpoint is made available.
Configuration Name | Description | Default Behavior | Domain |
---|---|---|---|
admin.listeners | Control where the admin endpoint will be made available. | the endpoint will be added to the rest of Connect's existing endpoints | List of comma-separated URIs the REST API will listen on. |
admin.listeners.https. | Prefix for SSL settings when using HTTPS | Look at Connect REST docs for a list of supported configs that with this prefix. | Look at Connect REST docs for a list of supported configs that can go with this prefix. |
Set a blank string to the admin.listeners property to disable the entire /admin endpoint.
admin.listeners= |
Bring up the admin endpoint on a separate listener:
admin.listeners=http://localhost:9093 |
Set up SSL properties in a separate https listener:
admin.listeners=https://localhost:9093 admin.listeners.https.ssl.truststore.location=/path/to/truststore.jks admin.listeners.https.ssl.truststore.password=tpass admin.listeners.https.ssl.keystore.location=/path/to/keystore.jks admin.listeners.https.ssl.keystore.password=kpass |
/admin
endpoint that can be configured to be attached to a different listener than the existing Connect endpoints. /admin/loggers
endpoint that be used to modify log levels.WorkerConfig
to enable/disable the /admin
endpoint along with optionally configuring any HTTPS listeners if needed.Log level changes are not persisted across worker restarts. These changes also only apply to the worker that receives the PUT request described above.
This is a new feature. Existing features will not be changed.
It must be noted that the changes proposed only work when log4j 1.x is used to log messages and is available on classpath. This is the current default in the AK distribution as well. Other loggers (logback, log4j2 etc) will not be supported and this feature (and others mentioned below in related work) will have to be reworked to support these frameworks.