Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Status

Current state: "Under DiscussionAccepted"

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

Vote thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

  • GET /connectors/{connector}/tasks-config -  this endpoint will be deprecated in 3.7 and removed in 4.0. A note will be added to the public documentation indicating that the endpoint is deprecated, Connect's OpenAPI spec will be updated to mark the endpoint as deprecated, and a warning log will also be emitted every time the endpoint is invoked.

...

The GET /connectors/{connector}/tasks-config endpoint will be deprecated in 3.7 and removed in 4.0. A note will be added to the public documentation indicating that the endpoint is deprecated, Connect's OpenAPI spec will be updated to mark the endpoint as deprecated, and a warning log will also be emitted every time the endpoint is invoked.

...

  • The GET /connectors/{connector}/tasks-config endpoint will be deprecated in 3.7 and removed in 4.0
  • Connect's embedded cluster for integration testing will be updated to use the GET /connectors/{connector}/tasks endpoint here.
  • Any users of the endpoint will need to migrate their usage to the GET /connectors/{connector}/tasks endpoint starting from 4.0. The response structures are slightly different (although the content is functionally identical) and that will be called out in the deprecation documentation and warning. Connect's embedded cluster for integration testing will be updated to use the GET /connectors/{connector}/tasks endpoint hereExample responses for each of the two endpoints -


Code Block
languagejs
titleGET /connectors/file-sink-test/tasks-config
linenumberstrue
{
  "file-sink-test-0": {
    "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
    "file": "test.txt",
    "task.class": "org.apache.kafka.connect.file.FileStreamSinkTask",
    "topics": "test-topic",
    "tasks.max": "2",
    "name": "file-sink-test"
  },
  "file-sink-test-1": {
    "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
    "file": "test.txt",
    "task.class": "org.apache.kafka.connect.file.FileStreamSinkTask",
    "topics": "test-topic",
    "tasks.max": "2",
    "name": "file-sink-test"
  }
}


Code Block
languagejs
titleGET /connectors/file-sink-test/tasks
linenumberstrue
[
  {
    "id": {
      "connector": "file-sink-test",
      "task": 0
    },
    "config": {
      "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
      "file": "test.txt",
      "task.class": "org.apache.kafka.connect.file.FileStreamSinkTask",
      "topics": "test-topic",
      "tasks.max": "2",
      "name": "file-sink-test"
    }
  },
  {
    "id": {
      "connector": "file-sink-test",
      "task": 1
    },
    "config": {
      "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
      "file": "test.txt",
      "task.class": "org.apache.kafka.connect.file.FileStreamSinkTask",
      "topics": "test-topic",
      "tasks.max": "2",
      "name": "file-sink-test"
    }
  }
]


Rejected Alternatives

  1. Deprecate and remove the GET /connectors/{connector}/tasks endpoint instead. Rejected because - 
    • This endpoint is much older and has existed ever since Kafka Connect (or Copycat 🐱 as it was known back then) was released (in 0.9.0.0) which means it likely has much wider adoption.
  2. Continue to support both the endpoints. Rejected because - 
    • As evidenced in
      Jira
      serverASF JIRA
      serverId5aa69414-a9e9-3523-82ec-879b028fb15b
      keyKAFKA-15377
      , it becomes a bit of a maintenance burden to have two different endpoints for the same purpose. All contributors and maintainers may not even be aware about the existence of both the endpoints and it'd be difficult to diligently ensure that any fixes or changes are applied across both the endpoints.

...