Status

Current state: Draft

Discussion thread:

JIRA: Unable to render Jira issues macro, execution error.

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Users of Kafka in the JVM often write integration tests involving a Kafka cluster that is launched in the same JVM process. It would be helpful for such users if Kafka were to provide a public Java API as a separate library that would make it simple to start/stop a Kafka cluster in integration tests. Additionally, it should be easy to launch the cluster with TLS and/or SASL enabled.

Public Interfaces

We will introduce a small number of classes in the org.apache.kafka.testkit package, which will be considered public API. That is, we will only make incompatible changes in major releases after a suitable deprecation period.

package org.apache.kafka.testkit;

/**
 * Runs an in-memory, "embedded" instance of a Kafka broker using a randomly generated port by default.
 * <p>
 * Requires a running ZooKeeper instance to connect to.
 */
public class EmbeddedBroker {

    public EmbeddedBroker(final Map<String, String> configs) throws IOException { ... }

    public EmbeddedBroker(final Map<String, String> configs, Time time) throws IOException { ... }


    /** Return the broker id */
    public int id() { ... }

    /**
     * The ZooKeeper connection string aka `zookeeper.connect`.
     */
    public String zooKeeperConnect() { ... }

    /** Return the broker's log directory */
    public Path logDir() { ... }

    /**
     * Stop the broker.
     */
    public void stop() { ... }
}

Proposed Changes

We will introduce a new artifact called kafka-testkit that will contain the Java classes for the test library. This module will depend on the clients and core kafka artifacts, but it won't depend on any of the Kafka test artifacts. As such, users won't have our tests on their classpath.

Compatibility, Deprecation, and Migration Plan

Since this is a new library, there is no compatibility issue. Going forward, we will have to evolve the library carefully so that compatibility is maintained.

Rejected Alternatives

1. Continue with the status quo where users who want to write integration tests involving a Kafka cluster have to rely on internal APIs.