Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Connect has a REST API where clients can easily ask for which connectors are loaded by the Worker. SMTs are popular feature, and it would be great to have a programmatic way to ask the worker which SMTs it has loaded.

Currently the best way to do this is by parsing logs.verify which SMTs the worker has loaded is to visually parse logs. This sometimes impossible for a long-running worker, and tedious otherwise. It also prevents programmatic querying and verification that SMTs loaded.

Existing endpoint to get Connectors is actually called "GET /connector-pluings", implying it should be used for all plugins. In reality, Connectors are just one type of plugin: Other types include Simple Message Transformations (SMTs), Converters, HeaderConverters, ConfigProviders, and RestExtensions (loaded by the "DelegatingClassLoader").

...

The new endpoints are specified below in bold in OpenApi format.

Code Block
languageyml
titlespec.yaml
# This already exists in production
/connector-plugins

...


get:

...


summary: Lists available Connector plugins

...


...

...


...

...




# Newly Proposed Endpoints
/plugins

...


  get:

...


    summary: Returns plugin descriptions of all

...

....

 types
    responses:
      '200':
        description: A list of plugins
        content:
          application/json:
            schema:
              # Consistent with existing "/connector-plugins" endpoint
              type: array
              items:
                type: object
                properties:
                  class:
                    type: string
                    description: Plugin class name (e.g. "io.confluent.connect.hdfs.HdfsSinkConnector")

/plugins/{plugin-type}:

...


  get:

...


    summary: Returns plugin descriptions of the given

...

    parameters:

...

 type
    parameters:
      -in:

...

 path
      name: plugin-type

...

        schema:

          type: string

        required: true

        description: plugin-type can be one of connector, converter, header-converter, transformation, config-provider, connect-rest-extension

....

...


      schema:
        type: string
        required: true
        description: plugin-type can be one of connector, converter, header-converter, transformation, config-provider, connect-rest-extension
    responses:
      '200':
        description: A list of plugins
        content:
          application/json:
            schema:
              # Consistent with existing "/connector-plugins" endpoint
              type: array
              items:
                type: object
                properties:
                  class:
                    type: string
                    description: Plugin class name (e.g. "io.confluent.connect.hdfs.HdfsSinkConnector")
      '400':
        description: Invalid request, such as unknown plugin type, as constructed with existing ConnectRestException
        content:
          application/json:
            schema:
              # Consistent with https://docs.confluent.io/platform/current/connect/references/restapi.html#status-and-errors
              type: object
              properties:
                error_code:
                  type: integer
                  description: HTTP Error code (e.g. 400).
                message:
                  type: string
                  description: Human readable message to suggest action to caller.

Proposed Changes

The new endpoints are described in the API Spec above.

...