Status

Current state: Under Discussion

Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

When using the TopologyTestDriver to assert topologies are working as expected, it can be useful to know the full set of topics the topology output to during a test run.  This information allows all the outputs of a test run to be captured without prior knowledge of the output topics. The set of output topics and the output records can then be captured and used to ensure future runs output the same records to the same topics.  This can be useful to ensure compatibility after changes to the topology or after changes to the Kafka Streams version.

Public Interfaces

Addition of a new method to TopologyTestDriver:

/**
 * Get all the names of all the topics to which records have been output.
 * <p>
 * Call this method after piping the input into the test driver to retrieve the full set of topics the topology
 * produced records to.
 * <p>
 * The returned set of topic names includes changelog, repartition and sink topic names.
 *
 * @return the set of output topic names.
 */
public final Set<String> getOutputTopicNames(){...}

Proposed Changes

Phase 1:

public final Set<String> getOutputTopicNames() {
    return Collections.unmodifiableSet(outputRecordsByTopic.keySet());
}

Phase 2:

The implementation will be changed such that the key set of outputRecordsByTopic is initialized with all known output, i.e. internal and sink, topic names upon construction of the TopologyTestDriver from the TopologyDescription.  This second phase will mean getOutputTopicNames() will return most topic names even before any input is piped through the topology. Topics that use a non-static TopicNameExtractor will not  be populated on construction.

Compatibility, Deprecation, and Migration Plan

None

Rejected Alternatives

None