Status

Current state: "Under Discussion"

Discussion thread: None yet. [DISCUSS] KIP-1149 Helm Chart for Apache Kafka

JIRA: KAFKA-6416

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Deploying an Apache Kafka cluster to Kubernetes is a complex process and a Helm chart can greatly simplify this, making it possible to have a cluster up and running in a few minutes. There are third party charts available for example

...

The Helm chart will only support deploying Kafka 4.1.0 and later. I did have it working with 3.9.0, but changes to logging configuration would make this complex to support moving forward.

Public Interfaces

This change will not add or change any interfaces the application has. The Helm chart is separate to the Java application.

Proposed Changes

A draft PR with proposed changes is at PR #19249

...

The CD pipeline would need to be changed to build and publish the Helm chart with Kafka. An example of an Apache project that publishes it's own Helm chart is Apache Airflow. The chart repository index is in the file airflow.apache.org/index.yaml and the charts are downloaded from downloads.apache.org like any other binary.

Test Plan

Deploy Kubernetes to a few different Kubernetes distributions (and Cloud services) and run any existing system validation tests.

CI could be changed to run the Helm lint command and the Helm template command is useful to validate the templating works.

Rejected Alternatives

Use of existing third party deploying solutions such as Confluent or Bitnami, as listed above.