Status

Current state: Under Discussion

Discussion thread: here 

JIRA: Unable to render Jira issues macro, execution error.  

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Connect Framework offers REST API that is used to mange the lifecycle of the connector. Its imperative in most enterprises to secure the API and also add authorization to the end points. We could add the ability for authentication and authorization in the framework. But the security requirements are so broad that it's not practical to support all of them in the framework.  Hence we must provide ability for users to plug resources that help achieve the required capabilities.

 While security is prime use cases for this extension. Its not limited to that. Some of the common use cases are

• Build a custom Authentication filter
• Build a custom Authorization filter
• Complex extensions can even provide filters that rewrite/validate the connector requests to enforce additional constraints on the connector configurations

Public Interfaces

Developers would be required to implement only the ConnectRestExtension interface to provide an extension.  ConnectRestExtension provides an implementation of ConnectRestExtensionContext whose configurable() provides an implementation of javax.ws.rs.core.Configurable. Using this developers can register new JAX-RS resources like filter, new end points, etc

interface ConnectRestExtension extends Closeable{
    void register(ConnectRestExtensionContext restPluginContext);
}

interface ConnectRestExtensionContext{
    Configurable configurable();
    WorkerConfig workerConfig();
    ConnectClusterState clusterState();
}

interface ConnectClusterState{
    /**
     * Get a list of connectors currently running in this cluster. This is a full list of connectors in the cluster gathered
     * from the current configuration.
     */
    Collection<String> connectors();
 
    /**
     * Lookup the current status of a connector.
     * @param connName name of the connector
     */
    ConnectorStateInfo connectorStatus(String connName);
 
}

Proposed Changes

Plugin Interface

Users will be able to create a plugin by implementing the interface ConnectRestExtension. The  key method to implement is the register method which takes ConnectRestExtensionContext . This allows us to change the interface easily in future to add new parameters. Connect runtime would also provide a default implementation for the interface ConnectRestExtensionContext.  One or more of the implementation can be configured via the configuration rest.plugins as a comma separated list of class names.

interface ConnectRestExtension extends Closeable{
    void register(ConnectRestExtensionContext restPluginContext);
}
 
interface ConnectRestExtensionContext{
    Configurable configurable();
    WorkerConfig workerConfig();
    ConnectClusterState clusterState();
}
 
class ConnectRestExtensionContextImpl implements ConnectRestExtensionContext{
    private final Configurable configurable;
    private final WorkerConfig workerConfig;
    private final ConnectClusterState clusterState;
 
    ConnectRestExtensionContext(Configurable configurable, WorkerConfig workerConfig, ConnectClusterState clusterState){
        this.configurable = configurable;
        this.workerConfig = workerConfig;
        this.clusterState = clusterState;
    }
 
    public ResourceConfig configurable(){
        return this.configurable;
    }
 
    public WorkerConfig workerConfig(){
        return this.workerConfig;
    }
 
    public ConnectClusterState clusterState(){
        return this.clusterState;
    }
}

interface ConnectClusterState{
    /**
     * Get a list of connectors currently running in this cluster. This is a full list of connectors in the cluster gathered
     * from the current configuration.
     */
    Collection<String> connectors();
 
    /**
     * Lookup the current status of a connector.
     * @param connName name of the connector
     */
    ConnectorStateInfo connectorStatus(String connName);
 
}
 
class ConnectClusterStateImpl implements ConnectClusterState{
    private final Herder herder;
 
    public ConnectClusterStateImpl(Herder herder){
        this.herder = herder;
    }
 
    @Override
    Collection<String> connectors(){
        //delegate to herder
    }
 
    @Override
    ConnectorStateInfo connectorStatus(String connName);{
        //delegate to herder
    }
}

class  ExampleConnectPlugin implements ConnectRestExtension{
    public void register(ConnectRestExtensionContext restPluginContext){
        resourceConfig.register(new AuthenticationFilter(restPluginContext.workerConfig()));
        resourceConfig.register(new HealthCheckResource(restPluginContext.workerConfig(), restPluginContext.clusterState()));
    }
 
    public void close() {
        //close any resources
    }
}
 
class AuthenticationFilter implements ContainerRequestFilter {
 
  public AuthenticationFilter(WorkerConfig workerConfig){
    //set up filter
  }
     
  @Override
  public void filter(ContainerRequestContext requestContext) {
        //authentication logic
  }
 
}
 
@Path("/connect")
class HealthCheckResource {
 
    public HealthCheckResource(WorkerConfig workerConfig, ConnectClusterState clusterState){
        //initialize resource
    }
 
    @path("/health")
    public void healthcheck(){
        //check herder health
    }
}

 // configure plugin
rest.plugins=com.example.ExampleConnectPlugin

Compatibility, Deprecation, and Migration Plan

• This is entirely new functionality so there are no compatibility, deprecation, or migration concerns.

Rejected Alternatives

1. Creating configs specific to the plugin and just passing them to the plugin based on a prefix. It was considered much easier to make the complete WorkerConfig. Also, in many cases the plugins would need to know just more than their configs to implement their actions. 
2. Passing the Herder to the plugin was considered but it was rejected since the Herder API is not public and we don't want to expose the complete Herder capabilities to the plugin.
3. Providing ability to just add Filters insteda of any kind of Jersey resource was considered but it was rejected because it was too limiting in its capability that one canot add new resource end points or add a jersey provider.
4. Registering the plugin after the default Connector resources are registered is rejected because of the fact that we don't want to allow users to remove resources at the moment.
5. Having the Connect REST plugin as part of class path is rejected for the same reason why we have custom interfaces like Converters and Connectors as plugin and loaded via plugin path.