This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.KIP is meant to follow up the discussion and decision around the old Scala consumer deprecation that started by Ismael Juma Juma in this thread. 

Status

Current state:  [One of "Under Discussion", " Accepted", "Rejected"]

Discussion thread: here

JIRA: here

Released: <Kafka Version> (per discussions on the discussion thread and also the voting thread, this KIP will be included in the release following 0.10.2.0)

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Describe the problems you are trying to solve.

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• Binary log format

• The network protocol and api behavior

• Any class in the public packages under clientsConfiguration, especially client configuration

  • org/apache/kafka/common/serialization

  • org/apache/kafka/common

  • org/apache/kafka/common/errors

  • org/apache/kafka/clients/producer

  • org/apache/kafka/clients/consumer (eventually, once stable)

• Monitoring

• Command line tools and arguments

• Anything else that will likely break existing users in some way when they upgrade

Proposed Changes

Quoting Ismael Juma Juma on the original thread (Oct 25, 2016):

In 0.10.1.0, we removed the beta label from the new Java consumer
documentation and updated the various tools so that they can use the new
consumer without having to pass the `--new-consumer` flag (more
specifically the new consumer is used if `bootstrap-server` is set). More
details of the reasoning can be found in the original discuss thread:
http://search-hadoop.com/m/Kafka/uyzND1e4bUP1Rjq721

The old consumers don't have security or `offsetsForTimestamp` (KIP-79)
support and the plan is to only add features to the new Java consumer. Even
so, the old consumers are a significant maintenance burden as they
duplicate protocol request/response classes (the SimpleConsumer exposes
them in the public API sadly). I experienced this first hand most recently
while working on KIP-74.

Given the above, I propose we deprecate the old consumers in trunk to nudge
users in the right direction. Users will have the 0.10.1.0 cycle to start
migrating to the new Java consumer with no build warnings. Once they
upgrade to the next release (i.e. 0.10.2.0), users who are still using the
old consumers will get warnings at build time encouraging them to move to
the new consumer, but everything will still work fine.

In a future major release, the old consumers (along with the old producers)
will be removed. We will have a separate discuss/vote thread for that to
make sure the time is right.

Another enticing feature of the Java consumer, that's missing in the old consumer, is its compatibility with older brokers (0.10.0 and later).

Public Interfaces

No new public interfaces are proposed, or existing interfaces are modified, or removed.

Proposed Changes

As described by Ismael Juma Juma the intention at this point is to just mark the old consumer as deprecated and issue build warnings to users and let them know that the old consumer will be removed in a following major release (potentially 0.11.0.0).

 Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

There is no impact to existing users other than seeing the deprecation warnings at build time. Since the old consumer will be removed in a next major release a migration plan will be introduced later (some options are already suggested in the original discussion thread), if necessary, to lower the impact (downtime, etc.) to existing users of the old consumer.

Test Plan

None as no behavior changes is proposed at this time.

Rejected Alternatives

 If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.