...

Modification times will be tracked in-memory and determined by when they are applied by the worker, as opposed to when they are requested by the user or persisted to the config topic (details below). If no modifications to the namespace have been made since the worker was started, they finished startup, the timestamp will be null.

The GET /admin/loggers  endpoint will have this new response format, where ${last_modified} is the last modified timestamp:

...

• Ensure that records produced to the config topic have the expected format
• Ensure that updates to a logging level are reported with the correct last modified timestamp
• Ensure that logging levels that have not been updated have a null last modified timestamp
• Ensure that distributed workers that have completed startup correctly handle logging adjustment config topic records
• Ensure that distributed workers that have not completed startup ignore logging adjustment config topic records
• Ensure that requests to the existing PUT /admin/loggers/{logger} endpoint with no scope query parameter, and with scope=worker result in the same herder-level behavior as before (mostly likely accomplished by verifying that no interactions with the Herder object have taken place)
• Ensure that requests to the existing PUT /admin/loggers/{logger} endpoint with an unrecognized value for the scope query parameter result in the same herder-level behavior as before, but also cause a warning log message to be emitted
• Ensure that cluster-scoped requests with invalid logging levels are rejected with a 404 response
• Ensure that repeated requests to set the same logging level for a namespace do not cause its last modified timestamp to be updated

...

• Higher maintenance burden: we would have to be able to serve requests that expect both kinds of response format
• Setting an expensive precedent for the Kafka Connect REST API: unless absolutely necessary, we should encourage consumers of the API to tolerate unknown fields in order to permit flexibility in future changes we may opt to make that would only involve adding new fields

Persistent logging level updates

Both the new cluster-wide API proposed in this KIP and the existing worker-local API added in KIP-495 only support ephemeral updates: any dynamic logging level changes will be discarded if a worker restarts, and the worker will revert to the levels specified in its Log4j configuration.

The rationale for keeping these updates ephemeral is to continue to give priority to workers' Log4j configuration files, with the underlying philosophy that this endpoint is still only intended for debugging purposes, as opposed to cluster-wide configuration. Permanent changes can already be made by tweaking the Log4j file for a worker and then restarting it. If a restart is too expensive for a permanent change, then the change can be applied immediately via the REST API, and staged via the Log4j configuration file (which will then be used the next time the worker is restarted, whenever that happens).

Future work

More scope types

...