Versions Compared

Old Version 6

changes.mady.by.user Chris Egerton

Saved on

compared with

New Version 7

changes.mady.by.user Chris Egerton

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Clarify targeted version, improve ConnectClusterState snippet to be less ambiguous, add paragraph on how details will be retrieved from herder

...

Code Block
languagejava
titleConnectClusterState .javainterface - additional methods
public interface ConnectClusterState {

    /**
     * Lookup the current configuration of a connector. This provides the current snapshot of configuration by querying the underlying
     * herder. A connector returned by previous invocation of {@link #connectors()} may no longer be available and could result in {@link
     * org.apache.kafka.connect.errors.NotFoundException}.
     *
     * @param connName name of the connector
     * @return the configuration of the connector for the connector name
     * @throws org.apache.kafka.connect.errors.NotFoundException if the requested connector can't be found
     */
	Map<String, String> connectorConfig(String connName);

	/**
	 * Lookup the current task configurations of a connector. This provides the current snapshot of configuration by querying the underlying
	 * herder. A connector returned by previous invocation of {@link #connectors()} may no longer be available and could result in {@link
	 * org.apache.kafka.connect.errors.NotFoundException}.
	 *
	 * @param connName name of the connector
	 * @return the configuration for each task ID
	 * @throws org.apache.kafka.connect.errors.NotFoundException if the requested connector can't be found
	 **/
	Map<Integer, Map<String, String>> taskConfigs(String connName);

  	/**
	 * Get the cluster ID of the Kafka cluster backing this Connect cluster.
	 * @return the cluster ID of the Kafka cluster backing this connect cluster
	 **/
	String kafkaClusterId();
}

Proposed Changes

The basic idea here is to add as much information to the ConnectClusterState interface as is available via the Connect REST API; this includes all currently-available read-only methods of the Herder interface.

For some of these methods, such as connectorConfig(...) and taskConfigs(...), communication with the underlying Herder will be necessary. The return values will reflect the most up-to-date information that the worker has available locally; the herder will not forward requests to the leader of the worker group. An exception will be thrown if the herder is expecting an impending rebalance. This aligns with the behavior of the current ConnectClusterStateImpl class.

Right now, it is possible for request filters added via a REST extension to "intercept" new connector configurations that are submitted via the REST API (through a PUT request to /connectors/<name>/config, or a POST request to /connectors, for example) and validate and/or modify them.

...

Not applicable; all proposed changes are backwards compatible.

Since this is a feature addition and not a bug fix, the targeted version is the upcoming 2.3 release.

Rejected Alternatives

Query the Connect REST API from within the extension itself

...