Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

We propose to add two new interfaces SourceConnectorContext and SinkConnectorContext (similar to SourceTaskContext and SinkTaskContext). Those methods could be used then to expose specifics methods to SourceConnector and SinkConnector. 

Code Block
languagejava
/**
 * SinkConnectorContext is provided to SinkConnector to allow them to interact with the underlying
 * runtime.
 *
 * This interface can be used to add specifics methods to a SinkConnector.
 */
public interface SinkConnectorContext extends ConnectorContext {
}

...

The SourceConnectorContext class will provide access to an OffsetStorageReader.

In additionTo expose these context objects, we propose to add supplement the existing protected field with a new method context() to the Connector class. 

Code Block
languagejava
public ConnectorContext context() {
    return context;
}

 

This method can be overrided is then overridden by the SourceConnector and SinkConnector classes in order to change the return type to be a subtypeSourceConnectorContext and SinkConnectorContext, respectively.


Code Block
languagejava
/**
 * SinkConnectors implement the Connector interface to send Kafka data to another system.
 */
public abstract class SinkConnector extends Connector {

    /**
     * <p>
     * Configuration key for the list of input topics for this connector.
     * </p>
     * <p>
     * Usually this setting is only relevant to the Kafka Connect framework, but is provided here for
     * the convenience of Connector developers if they also need to know the set of topics.
     * </p>
     */
    public static final String TOPICS_CONFIG = "topics";

    public SinkConnectorContext context() {
        return (SinkConnectorContext) context;
    }

}

...

Code Block
languagejava
/**
 * SourceConnectors implement the connector interface to pull data from another system and send
 * it to Kafka.
 */
public abstract class SourceConnector extends Connector {

    @Override
    public SourceConnectorContext context() {
        return (SourceConnectorContext) context;
    }
}


Proposed Changes

A straightforward first pass is The changes described above are implemented as GitHub PR 4794.

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?
  • If we are changing behavior how will we phase out the older behavior?
  • If we need special migration tools, describe them here.
  • When will we remove the existing behavior?

Rejected Alternatives

This proposal is backward compatible and binary compatible with earlier versions. The current design requires connector implementations to access the existing ConnectorContext object through a protected field in the Connector base class. There are no plans to change this access, so existing implementations will continue to work as before without changes.

This proposal adds new subinterfaces of ConnectorContext called  SinkConnectorContext and SourceConnectorContext and new methods to the SinkConnector and SourceConnector abstract classes that will allow subclasses to access the specific connector's context without having to cast. Connectors that require this version of the API will be able to optionally use these new methods or even check and cast the context field.

Note that the new SinkConnectorContext and SourceConnectorContext interfaces added here will make it easier in the future to provide new methods to access runtime components.

Rejected Alternatives

This proposal is simple and straightforward, and reflects the patterns used in the API in other areas. Another similar approach was considered but evolved to this design following a discussion. Another possibility of moving the context field from Connector to SourceConnector and SinkConnector (with the appropriate type) was ruled out as it would not be as binary-compatible as the current approach.

 If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.