Status

Current state: Under Discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: TBD

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Connect's API provides a list method which returns all running connector names as well as info methods parameterized by name. This leads to a situation where users of this API need to make N api calls to get information about all their running connectors.

Public Interfaces

The current `/connectors` endpoint will gain a new query param `?expand=(true|false)` which will return a map {string->ConnectorStateInfo}.

w:kafka norwood$ curl -s http://localhost:8083/connectors | jq
[
  "blah"
]

w:kafka norwood$ curl -s http://localhost:8083/connectors?expand=true | jq
{
  "blah": {
    "name": "blah",
    "config": {
      "connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector",
      "file": "/tmp/lol",
      "tasks.max": "10",
      "name": "blah",
      "topic": "test-topic"
    },
    "tasks": [
      {
        "connector": "blah",
        "task": 0
      }
    ],
    "type": "source"
  }
}

Proposed Changes

The changes are ~entirely in ConnectorsResource and the Herders. (will add link once pr is open.)

Compatibility, Deprecation, and Migration Plan

This change is backwards compatible.

Rejected Alternatives

• Adding an entirely new endpoint.
  • The way that ConnectorsResource is setup right now is not conducive to adding another endpoint that is not namespaced under /connectors/{connectorName}
  • We could conceivably rework more code in ConnectorsResource to allow us to have a /connectorsExpanded endpoint and then retain the other /connectors/{connectorName} endpoints, but this introduces even more questions, e.g. what would be under /connectorsExpanded/{connector}? 
• Add a connect rest extension.
  • This seems like overkill for basic functionality. We have all the data available in the herder now, so requiring additional install steps to get basic info seems like a bit much.