Status
Current state: Under Discussion
Discussion thread:https://www.mail-archive.com/dev@kafka.apache.org/msg97405.html
JIRA:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Connect's API provides a list method which returns all running connector names as well as info methods parameterized by name. This leads to a situation where users of this API need to make N api calls to get information about all their running connectors.
Public Interfaces
The current `/connectors` endpoint will gain a new query param `?expand=(true|false)` which will return a map {string->ConnectorStateInfo}.
w:kafka norwood$ curl -s http://localhost:8083/connectors | jq [ "blah" ]
w:kafka norwood$ curl -s http://localhost:8083/connectors?expand=true | jq { "blah": { "name": "blah", "connector": { "state": "RUNNING", "worker_id": "10.200.7.135:8083" }, "tasks": [ { "id": 0, "state": "RUNNING", "worker_id": "10.200.7.135:8083" } ], "type": "source" } }
Proposed Changes
The changes are ~entirely in ConnectorsResource and the Herders. PR: https://github.com/apache/kafka/pull/6658
Compatibility, Deprecation, and Migration Plan
This change is backwards compatible.
Rejected Alternatives
- Adding an entirely new endpoint.
- The way that ConnectorsResource is setup right now is not conducive to adding another endpoint that is not namespaced under /connectors/{connectorName}
- We could conceivably rework more code in ConnectorsResource to allow us to have a /connectorsExpanded endpoint and then retain the other /connectors/{connectorName}/status endpoints, but this introduces even more questions, e.g. what would be under /connectorsExpanded/{connector}/status?
- Add a connect rest extension.
- This seems like overkill for basic functionality. We have all the data available in the herder now, so requiring additional install steps to get basic info seems like a bit much.