...

1. Initial Setup:

   • Establish a new directory, docker/docker_official_images, within trunk to keep Docker Official Image assets.

   • Ensure dockerfile is compliant as per Docker Official Images guidelines.

2. Release Candidate (RC) Preparation:

   • RM would have generated and pushed Apache Kafka's Release Candidate tarball to apache sftp server hosted in home.apache.org by release.py script.

   • For all subsequent steps - the above Release Candidate tarball URL will be used.

3. Docker Image Preparation via GitHub Actions:

   • Implement an automated GitHub Actions workflow to:

     • Generate a version-specific folder, following the major_version.minor_version.patch_version-rc naming scheme, under the docker/docker_official_images directory, for housing static Dockerfile and scripts.

     • With each new RC, this folder will be overwritten.

     • This folder will contain all static files necessary for creating the docker image.

     • The github actions workflow generates this folder which can be downloaded and verified.

     • The RM checks if there is already a folder with the same name in trunk. If yes, he deletes this, and put this folder under the docker/docker_official_images directory in his fork of the kafka repo, and merges this change into the official kafka repo by raising a PR. To make this step more automated, we will do the same via a script that takes in the local path of the downloaded artifact as an input.

4. Review and Integration:

   • A PR will be raised by RM which will contain the RC docker official image.

   • Upon PR approval and merge, RM will cherry-pick the changes into the designated branch.

5. Docker Build and Release Testing:

   • Perform Docker build tests to ensure image integrity. The dockerfiles for the same will be used from the specific folder in the docker/docker_official_images directory.

   • Start the Voting for RC, which will include the Docker image, docker sanity tests report and docker official image source folder.

   • In case any docker image specific issue is detected, that will be evaluated by the community, if it’s a release blocker or not.

   • Once the vote passes, run the Docker release pipeline to publish RC images to the Docker Hub open-source software (OSS) sponsored images. The image will be pushed to OSS sponsored apache/kafka with the version as tag.

6. Iterative RC Updates:

   • Repeat steps 2 through 5 for subsequent RCs, ensuring the repository contains the latest RC.

7. Final Release Promotion:

   • Upon final release approval, run the Docker promote pipeline from the RC branch to the OSS Docker Hub. The latest major_version.minor_version.patch_version image will now be available on the OSS Docker Hub.

8. Post-Release Maintenance:

   • Return to the trunk to perform maintenance tasks:

     • Remove unsupported version folders from the docker_official_images directory to maintain repository cleanliness.

     • Remove the -rc identifier from the major_version.minor_version.patch_version-rc folder. This serves as the new source of truth for the DOI. Ensure that the folder of the version being released does not have the -rc tag at the end.

     • Cherry pick the updated folder name commit to release branch.

9. Docker Hub Submission:

   • Run a script in trunk to automate the creation of a PR template for the Docker Hub official images library submission.automatically create the file needed to create a PR on Docker Hub's official images repository. 

   • Submit the final PR to Docker Hub’s official images repository. Following PR approval, the Kafka Docker image becomes officially available.

10. Post Release Process - if Dockerhub folks suggest changes to the Dockerfiles:

    • Make the same changes in the Dockerfile present in docker/jvm/Dockerfile, so that the changes are propagated to newer rc branches and new releases as well. This ensures that the source of truth is kept updated. Commit these changes.

    • Cherry pick these changes into the branch of that particular release.

    • Follow steps 3 and 4, and then run the docker_build_and_test workflow to test the images with the latest changes.

    • If the image builds successfully, and passes all tests , switch to the trunk, and follow step 8 again to raise a new PR in the docker official images repo.

Directory Structure

A new directory named docker_official_images will be added to the /docker repository. This directory will contain all the Official Docker Images related code.

kafka/
        - docker/
        - docker_official_images/ #Contains resources needed to create the official docker images.
            - 3.8.2-rc/           #Contains resources needed to create the docker images for the latest RC tag for the 3.8.2 version. 
              - jvm/              #Contains Dockerfile for the JVM-based Apache Kafka Docker image.
                - Dockerfile + necessary scripts 
            - 3.8.1/              #Contains resources needed to create the official docker images for the 3.8.1 version.
              - jvm/              #Contains Dockerfile for the JVM-based Apache Kafka Official Docker image.
                - Dockerfile + necessary scripts
            - 3.9.0/
              - jvm/
                - Dockerfile + necessary scripts
                - jvm/
            - Dockerfile                     - Dockerfile          #Dockerfile for the JVM-based Apache Kafka Docker image.
                - resources/                           #Contains resources needed to create the Docker image.
                - test/                                     #Contains sanity tests for the Docker image.
                - docker_build_test.py   py    #Python script for building and testing the Docker image.
                - docker_release.py      py       #Python script for building the Docker image and pushing it to Docker Hub.
        - docker_build_latest_artifact.py #Python script for building the artifact for the current RC release.
        - generate-stackbrew-library-docker.sh #Shell script to automate PR template for Docker Official Images.

Dockerhub Repo PR

RM will run a script provided in the docker folder to generate a file which can be used to create a PR on https://github.com/docker-library/official-images/tree/master
The final file will look like the following:-

Maintainers: The Apache Kafka Project <dev@kafka.apache.org> (@ApacheKafka)
GitRepo: https://github.com/apache/kafka.git

Tags: 3.8.1
Architectures: amd64,arm64v8
GitCommit: 9o7q27wmmu5kzh0erwr2rmvr5h7pdjh8a56a77u8
Directory: ./docker/docker_official_images/3.8.1/jvm

Tags: 3.9.0, latest
Architectures: amd64,arm64v8
GitCommit: 2w6tit74aog1o6w2m0jvfk7xtyd6qdep9opp00fd
Directory: ./docker/docker_official_images/3.9.0/jvm

Test Plan

Existing tests for docker image should pass.

...