Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

A new optional field "state"  will be added to the request body format for the POST /connectors endpoint (the existing format can be seen in Kafka Connect's current OpenAPI specification). The new optional field "state" can have a value of either "RUNNING", "PAUSED"  or "STOPPED" (case-insensitive). If the value of the "state"  field is invalid, a 400 Bad Request response will be returned. If the field is omitted in the request body, the connector will be created in the RUNNING state by default (preserving the existing behavior). An example request body would look like:

...

When a connector is created in a PAUSED or STOPPED state, the connector will be assigned to a worker post group rebalance and the connector thread will be created but the actual Connector instance won't be started. No tasks will be created for such connectors until they're resumed. The only difference between creating a connector in the PAUSED state vs versus the STOPPED state is that offsets modification will only be possible in the STOPPED state. There isn't any scenario where creating a connector in the PAUSED state is advantageous over creating it in the STOPPED state - however, since it's still a valid connector state and has been present for a lot longer than the STOPPED state, we allow it ( PAUSED was introduced in AK 0.10.0.0 and STOPPED was introduced in AK 3.5.0).


The changes in standalone mode will be similar - the connector's target state will be updated prior to the connector being started if a valid "state" is specified in the connector creation request.

Compatibility, Deprecation, and Migration Plan

...

Summary: The current target states that can be written to the config topic are STARTED (when a connector is resumed), PAUSED and STOPPED and we could use these as valid values for the new "state"  field in the connector creation request body.

Rejected because: These target states aren't currently directly exposed to the users and the states that are actually exposed to the users via the status REST API endpoints GET /connectors/{connector}/status  endpoints are: UNASSIGNED , RUNNING , PAUSED , FAILED , RESTARTING  and STOPPED. Using RUNNING over STARTED makes more sense since that's the state that users are already familiar with (the PAUSED and STOPPED states are common across and aren't an issue).

...

Rejected because: It isn't particularly useful to allow modifying the state of a connector along with updating its configuration since both those operations can already be done separately (whereas with the connector creation case, it isn't currently possible to modify the state of a non-existent connector) and the POST /connectors endpoint already covers the connector creation case. Furthermore, the PUT /connectors/{connector}/config endpoint currently accepts a map of connector key-value  configurations as its request body (OpenAPI specification for reference), and adding a new "state"  field would require significantly modifying the request body format and supporting two different request body formats for the endpoint. We could potentially use a request parameter or request header to specify the state, but it would be fairly unusual to have two completely different ways of specifying the connector state across two endpoints (and it seems more appropriate to include the state in the request body anyway since it's a part of the request itself rather than metadata or a modifier).