You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »


Status

Current state: Under Discussion

Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Developers who are implementing a Kafka Connector do not have a convenient way to access the Task ID and Connector Name within their task implementation. The Task ID and Connector Name are important pieces of information that can be used for custom metrics, debugging, and logging. 

KIP-449 added the Task ID and Connector Name to the SLF4J MDC. It was accepted, and released as part of AK 2.3.0. That added the task context to the logs. This KIP proposes taking it a step further and making the Task ID and Connector Name available for developers to use within their task implementation however they see fit.

Public Interfaces

  1. The SourceTaskContext and SinkTaskContext interfaces in the connect api would get two new methods:
    1. int taskId() which would return the Task ID
    2. String connectorName()  which would return the Connector Name 

Proposed Changes

As mentioned above, there would be two changes to the Connect API: Add a taskId method and a connectorName method to the SourceTaskContext and SinkTaskContext.

Then, in the Connect Runtime, there would need to be a few changes to support these API interface changes:

  • Update WorkerSourceTaskContext to implement the taskId and connectorName methods (would just need to return this.task.id.task() and this.task.id.connector() respectively)
  • Update WorkerSinkTaskContext to implement the taskId  and connectorName methods (would just need to return this.sinkTask.id.task() and this.sinkTask.id.connector() respectively)

Compatibility, Deprecation, and Migration Plan

As this change is purely additive, it should not pose any compatibility problems to current users. The new methods will be documented in the JavaDoc.

Rejected Alternatives

Technically, developers can access the Task ID and Connector Name via the SLF4J MDC or via reflection. Neither of these seem like good practices to encourage.


  • No labels