Status

Current state: Under DiscussionAccepted

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
https://www.mail-archive.com/dev@kafka.apache.org/msg97405.html

JIRA: 

Release: AK 2.3.0JIRA: TBD

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

Connect's API provides a list method which returns all running connector names (/connectors) as well as state info methods parameterized by name (/connectors/{connectorName}/status). This leads to a situation where users of this API need to make N api calls to get information about all their running connectors.

...

The current `/connectors` endpoint will gain a new multi query param `?expand=(truestatus|falseinfo)` which will return a map {stringconnectorName->expansion->(ConnectorStateInfo|ConnectorInfo)}.

the following expansions are allowed

w:kafka norwood$ curl -s http://localhost:8083/connectors | jq
[
  "blah"
]

...

w:kafka norwood$ curl -s 'http://localhost:8083/connectors?expand=truestatus&expand=info' | jq
{
  "blah": {
    "info": {
      "name": "blah",
      "config": {
        "connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector",
        "file": "/tmp/lol",
        "tasks.max": "10",
        "name": "blah",
        "topic": "test-topic"
      },
      "tasks": [
        {
          "connector": "blah",
          "task": 0
        }
      ],
      "type": "source"
    },
    "status": {
      "name": "blah",
      "connector": {
        "state": "RUNNING",
        "worker_id": "10.200.25.241:8083"
      },
      "tasks": [
        {
          "id": 0,
          "state": "RUNNING",
          "worker_id": "10.200.25.241:8083"
        }
      ],
      "type": "source"
    }
  }
}

Proposed Changes

The changes are ~entirely in ConnectorsResource and the Herders. (will add link once pr is open.)PR: https://github.com/apache/kafka/pull/6658

Compatibility, Deprecation, and Migration Plan

...

• Adding an entirely new endpoint.
  • The way that ConnectorsResource is setup right now is not conducive to adding another endpoint that is not namespaced under /connectors/{connectorName}
  • We could conceivably rework more code in ConnectorsResource to allow us to have a /connectorsExpanded endpoint and then retain the other /connectors/{connectorName}/status endpoints, but this introduces even more questions, e.g. what would be under /connectorsExpanded/{connector}/status? 
• Add a connect rest extension.
  • This seems like overkill for basic functionality. We have all the data available in the herder now, so requiring additional install steps to get basic info seems like a bit much.