This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state:  [One of "Under Discussion", "Accepted", "Rejected"] Under discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here [Change the link from KAFKA-1 to your own ticket]

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Describe the problems you are trying to solve.

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• Binary log format

• The network protocol and api behavior

• Any class in the public packages under clientsConfiguration, especially client configuration

  • org/apache/kafka/common/serialization

  • org/apache/kafka/common

  • org/apache/kafka/common/errors

  • org/apache/kafka/clients/producer

  • org/apache/kafka/clients/consumer (eventually, once stable)

• Monitoring

• Command line tools and arguments

• Anything else that will likely break existing users in some way when they upgrade

Proposed Changes

Kafka changes message format from release to release. When upgrading Kafka version, typically the message format version recommended to set to old version before all clients upgrade to the latest release. Otherwise broker has to pay significant performance penalty to down convert message. Only when most clients have upgraded can the broker change to the latest message format. 

In this process, there is no visibility from broker side to see what portion of the clients have been upgraded. Therefore changing the message format version becomes a difficult decision. 

Proposed Changes

We want to amend RequestsPerSec metric to have "version" tag so that we have the insight of versions that clients are using. This can be done in RequestChannel where it already parses an keeps the client API version in memory and updates the RequestsPerSec metric. With this change, an additional hash lookup is needed when updating the metric to locate the metric corresponding to a specific version.

The metrics for all versions will be cleaned up at broker shutdown.

Details of the change can be viewed from the pull requestDescribe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Rejected Alternatives

• This may break the user who build monitoring tools which does not expect additional tag for RequestsPerSec metric for a specific request. To see the total count, they need to aggregate over all versions.
  
  

Rejected Alternatives

• We can create a new metric for it. However, it will incur more more storage cost for any metric system since we will be duplicating some request information in the new metric. It also makes it less user friendly for metric systems that can already "slice and dice" on different dimensions (tags) of metrics. In this case, it is easy to "drill down" to a tag without having to looking at a completely different metric.

 If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.