Versions Compared

Old Version 17

changes.mady.by.user Ewen Cheslack-Postava

Saved on

compared with

New Version 18

changes.mady.by.user Zhihong Yu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 

Code Block
HTTP/1.1 200 OK
Content-Type: application/json

{
    "name": "hdfs-sink-connector",
    "config": {
        "connector.class": "io.confluent.connect.hdfs.HdfsSinkConnector",
        "tasks.max": "10",
        "topics": "test-topic",
        "hdfs.url": "hdfs://fakehost:9000",
        "hadoop.conf.dir": "/opt/hadoop/conf",
        "hadoop.home": "/opt/hadoop",
        "flush.size": "100",
        "rotate.interval.ms": "1000"
    },
    "type": "sink",
    "tasks": [
        { "connector": "hdfs-sink-connector", "task": 1 },
        { "connector": "hdfs-sink-connector", "task": 2 },
        { "connector": "hdfs-sink-connector", "task": 3 }
    ]
}

For the following REST API endpoint:

Code Block
GET /connectors/(string:name)/status

Proposed Changes

The aforementioned endpoint currently returns the following structure:

Code Block
HTTP/1.1 200 OK
{
    "name": "hdfs-sink-connector",
    "connector": {
        "state": "RUNNING",
        "worker_id": "fakehost:8083"
    },
    "tasks":
    [
        {
            "id": 0,
            "state": "RUNNING",
            "worker_id": "fakehost:8083"
        },
        {
            "id": 1,
            "state": "FAILED",
            "worker_id": "fakehost:8083",
            "trace": "org.apache.kafka.common.errors.RecordTooLargeException\n"
        }
    ]
}

 

We will add a `type` field to the response to indicate whether the given Connector is a Source or SInk. e.g.:
Code Block
HTTP/1.1 200 OK
{
    "name": "hdfs-sink-connector",
    "connector": {
        "type": "sink",
        "state": "RUNNING",
        "worker_id": "fakehost:8083"
    },
    "tasks":
    [
        {
            "id": 0,
            "state": "RUNNING",
            "worker_id": "fakehost:8083"
        },
        {
            "id": 1,
            "state": "FAILED",
            "worker_id": "fakehost:8083",
            "trace": "org.apache.kafka.common.errors.RecordTooLargeException\n"
        }
    ]
}

 

 

 

Compatibility, Deprecation, and Migration Plan

This propsal only adds a field to existing JSON response messages from existing REST API endpoints, and does not otherwise change the structure of the responses. Many applications will be able to handle the response documents and ignore fields they don't know about. Therefore, this should be backwards compatible.

KIP-151 brought in the connector type enum generating lowercase. This is kept in this KIP.

Though consumers of the API should not make assumptions about the capitalization of connector type.

Rejected Alternatives