Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Native binaries operate independently and do not require specific packages to run. Consequently, opting for the most minimal base images, such as Alpine and Distroless, will images will enable us to produce compact Docker images.
We propose to make use of alpine image as the base image.

...

Adhering to the outlined constraints, image tagging can follow this format
<image-name>:<kafka-version>-<optional-suffix>

  • [Preferred Approach]kafka-localnative:3.5.1

    • local native indicates that this image is intended only for the local development use.3.5.1 is the sample kafka versionthe image consists of the native binary.

    • For many users, native might not make much sense.

    • kafka-localnative:3.5.1-1 In case of docker dedicated release, added a suffix -1

OR

  • kafka-native:3.5.1

    • native indicates that the image consists of the native binary.

    • For many users, native might not make much sense.

    • kafka-native:3.5.1-1 In case of docker dedicated release, added a suffix -1

Directory Structure

A new directory named docker will be added to the repository. This directory will contain all the Docker related code.
Directory Structure:

Directory Structure

A new directory named docker will be added to the repository. This directory will contain all the Docker related code.
Directory Structure:

kafka/
    - docker/

        - native-image/
            - Dockerfile          #Dockerfile for the GraalVM native-image based Apache Kafka Docker image.
        - jvm/
            - Dockerfile          #Dockerfile for the JVM-based Apache Kafka Docker image.
        - resources/              #Contains resources needed to create the Docker image.
        - test/                   #Contains sanity tests for the Docker image.
        -

kafka/
    - docker/

        - native-image/
            - Dockerfile          #Dockerfile for the GraalVM native-image based Apache Kafka Docker image.
        - jvm/
            - Dockerfile          #Dockerfile for the JVM-based Apache Kafka Docker image.
        - resources/              #Contains resources needed to create the Docker image.
        - test/                   #Contains sanity tests for the Docker image.
        -

docker_build_test.py    #Python script for building and testing the Docker image.
        - docker_release.py       #Python script for building the Docker image and pushing it to Docker Hub.

NOTE: This structure is designed with the anticipation of introducing another Docker image based on the native Apache Kafka Broker (as per KIP-975). Both images will share the same resources for image building. 

Configuring Properties

We offer two methods for passing the above properties to the container:

  1. File Mounting: Users can mount a properties file to a specific path within the container (we will clearly document this path). This file will then be utilized to start up Kafka.

  2. Using Environment Variables: Alternatively, users have the option to provide configurations via environment variables. Here's how to structure these variables:

    • Replace . with _
    • Replace _ with __(double underscore)
    • Replace - with ___(triple underscore)
    • Prefix the result with KAFKA_

    Examples:

    • For abc.def, use KAFKA_ABC_DEF
    • For abc-def, use KAFKA_ABC___DEF
    • For abc_def, use KAFKA_ABC__DEF

This way, you have flexibility in how you pass configurations to the container, making it more adaptable to various user preferences and requirements.
NOTE:

  1. Secrets will be provided to the container using folder mount.
  2. If a property is provided both in the mounted file and as an environment variable, the value from the environment variable will take precedence.

Compatibility, Deprecation, and Migration Plan

...

  • Suggestion: The docker image release should be owned by the Release Manager.

  • As per the current release process, only PMC members should be allowed to push to apache/kafka docker image.

  • If the RM is not a PMC member, they’ll need to take help from a PMC member to release the image.

Rejected Alternatives

...

Docker Base Image

The Distroless image was also evaluated as a potential base image. It required additional packages to be installed for our application to function.. Ultimately, we found that Alpine offered more flexibility, although there's no strong preference..

Image Name

Another option considered for the image was kafka-local:3.5.1

  • local indicates that this image is intended only for the local development use. 3.5.1 is the sample kafka version
  • This image may be used in the production in the future. In that case, the name "kafka-local" will be counter-intuitive.