You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: Under Discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: https://issues.apache.org/jira/browse/KAFKA-8200

PR: https://github.com/apache/kafka/pull/6556

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

When using the TopologyTestDriver, one examines the output on a topic with the readOutput method. This method returns one record at a time, until no more records can be found, at which point in returns null.

Many times, the usage pattern around readOutput will involve writing a loop to extract a number of records from the topic, building up a list of records, until it returns null.

It would be helpful to offer an iterable signature of readOutput, which returns either an iterator or list over the records that are currently available in the topic. This would effectively remove the loop that a user needs to write for him/herself each time.

Public Interfaces

This KIP would propose to add methods which return an Iterable of the output records as byte arrays and with key/value deserializers:

public Iterable<ProducerRecord<byte[], byte[]>> iterateOutput(final String topic)

public <K, V> Iterable<ProducerRecord<K, V>> iterateOutput(final String topic,
                                                               final Deserializer<K> keyDeserializer,
                                                               final Deserializer<V> valueDeserializer)

Proposed Changes

As above.

Compatibility, Deprecation, and Migration Plan

  • As these are new methods there should be no impact on existing users

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.

  • No labels