Page History

...

This KIP proposes to add a new query parameter "timeoutrequest header "Request-Timeout"  (integer value in milliseconds; for instance `POST /connectors?timeout=120000` for "Request-Timeout: 120000" for a timeout of 120000 milliseconds / 2 minutes) to the following Kafka Connect REST API endpoints:

...

The POST /connectors  and PUT /connectors/{connector}/config  endpoints internally do a config validation first (and only proceed to connector creation / updation if the validation passes), which is why the "timeout"  parameter is Request-Timeout" header is relevant for these endpoints too.

A new Kafka Connect worker configuration - rest.api.max.request.timeout.ms  will be added to configure an upper bound for the timeout  query parameter on "Request-Timeout" header on the above 3 REST API endpoints. The default value for this config will be 600000 (10 minutes) and it will be marked as a low importance config. 

...

The timeout here will be updated to use the value from the timeout query parameter "Request-Timeout" header if specified (else fallback to the current default of 90 seconds) for the aforementioned endpoints. If the value for the "timeout" parameter is Request-Timeout" header is invalid (<= 0 or > rest.api.max.request.timeout.ms), a 400 Bad Request response will be returned.

...

This KIP also proposes to change the behavior of the POST /connectors  and the PUT /connectors/{connector}/config  endpoints on request timeouts - currently, even if the connector config validation takes too long and causes a timeout response to be returned to the user, the connector create/update request is still made if the config validation completes successfully eventually. This can be pretty confusing to users and is generally a poor user experience because a 500 Internal Server Error  response should mean that the request couldn't be fulfilled. This behavior will be changed to be more intuitive - if the config validation takes too long and exceeds the timeout (either configured via the proposed new timeout query parameter or "Request-Timeout" header or the default 90 seconds), the request will be aborted and the connector won't be created / updated.

...

The proposed changes are fully backward compatible since we're just introducing a new optional query parameter to request header to 3 REST API endpoints along with a new worker configuration that has a default value.

...

A simple integration test will be added to ensure that a validate REST API request for a connector that takes longer than the default REST API request timeout (90 seconds) doesn't fail if the query parameter timeout is "Request-Timeout" header is set to a higher value. Unit tests will be added wherever applicable.

...

Rejected because: This doesn't allow for per request timeout configuration and also requires a worker restart if changes are requested. Configuring the timeout via a request query parameter allows header allows for much more fine-grained control.

...