...

Current state: Under Discussion

Discussion thread: here [Not started]

JIRA: here

Motivation

This KIP aims to introduce Docker Official Image (DOI) for Apache Kafka, ensuring the Kafka Docker images adhere to Docker's quality standards.

...

Proposed Workflow for Release Process

1. Initial Setup:

   • Establish a new directory, docker/docker_official_images, within trunk to keep Docker Official Image assets.

   • Ensure the Dockerfile is compliant as per Docker Official Images guidelines.

2. Release Candidate (RC) Preparation:

   • The RM would have generated and pushed Apache Kafka's Release Candidate tarball to apache sftp server hosted in home.apache.org by release.py script.

   • For all subsequent steps - the above Release Candidate tarball URL will be used.

3. Docker Image Preparation via GitHub Actions:

   • Implement an automated GitHub Actions workflow to:

     • Generate a version-specific folder, following the major_version.minor_version.patch_version-rc naming scheme, under the docker/docker_official_images directory, for housing static Dockerfile and scripts. For example 3.8.0-rc.

     • With each new RC, this folder will be overwritten.

     • This folder will contain all static files necessary for creating the docker image.

     • The github actions workflow generates this folder which can be downloaded and verified.

   • The RM will check if there is already a folder with the same name in trunk. If yes, the RM deletes this, and puts this folder under the docker/docker_official_images directory in their fork of the kafka repo, and merges this change into the official kafka repo by raising a PR. For example the RM replaces any existing folder named 3.8.0-rc with the latest artifact named 3.8.0-rc, downloaded from the Github actions workflow. To make this step more automated, we will do the same via a script that takes in the local path of the downloaded artifact as an input.

4. Review and Integration:

   • A PR will be raised by the RM which will contain the all the RC docker official image assets.

   • Upon PR approval and merge, the RM will cherry-pick the changes into the designated branch.

5. Docker Build and Release Testing:

   • Perform Docker build tests to ensure image integrity. The dockerfiles for the same will be used from the specific folder in the docker/docker_official_images directory.

   • Start the Voting for RC, which will include the Docker image, docker sanity tests report and docker official image source folder.

   • In case any docker image specific issue is detected, that will be evaluated by the community, if it’s a release blocker or not.

   • Once the vote passes, the RM will run the Docker release pipeline to publish RC images to the Docker Hub open-source software (OSS) sponsored images. The image will be pushed to OSS sponsored apache/kafka with the version as tag. For example: apache/kafka:3.8.0-rc1. Cop

6. Final Release Promotion:

   • Upon final release approval, the RM will run the Docker promote pipeline from the RC branch to the OSS Docker Hub. The latest major_version.minor_version.patch_version image will now be available on the OSS Docker Hub. For example: apache/kafka:3.8.0.

7. Post-Release Maintenance:

   • Return to the trunk to perform maintenance tasks:

     • Remove unsupported version folders from the docker_official_images directory to maintain repository cleanliness.

     • Remove the -rc identifier from the major_version.minor_version.patch_version-rc folder. This serves as the new source of truth for the DOI. Ensure that the folder of the version being released does not have the -rc tag at the end. For example rename 3.8.0-rc to 3.8.0.

     • The RM will cherry pick the updated folder name commit to release branch.

8. Docker Hub Submission:

   • Run a script from trunk to automate the creation of a file which contains all the necessary information for submission to the Docker Hub official images library.

   • The RM will manually raise the final PR to Docker Hub’s official images repository using the contents of the generated file. Following PR approval, it is now Docker’s responsibility to build and publish the Docker Official Images. For example kafka:3.8.0.

9. Post Release Process - if Dockerhub folks suggest changes to the Dockerfiles:

   • Make the same changes in the Dockerfile present in docker/jvm/Dockerfile, so that the changes are propagated to newer rc branches and new releases as well. This ensures that the source of truth is kept updated. Commit these changes.

   • The RM will cherry pick these changes into the branch of that particular release.

   • Follow steps 3 and 4, and then run the docker_build_and_test workflow to test the images with the latest changes.

   • If the image builds successfully, and passes all tests , switch to the trunk, and follow step 8 again to raise a new PR in the docker official images repo.

Directory Structure

A new directory named docker_official_images will be added to the /docker repository. This directory will contain all the Official Docker Images related code.

...