...

1. Secrets will be provided to the container using folder mount.
2. If a property is provided both in the mounted file and as an environment variable, the value from the environment variable will take precedence.

Class Data Sharing(CDS)

We did a small POC with class data sharing (CDS) feature that can help reduce the startup time and memory footprints for Java applications.

System details: M2 Mac, 32GB memory.

JSA file created for bringing up Kafka (default Kraft configs): ~45MB
Local Kafka startup time (without JSA): 1.592 secs
Local Kafka startup time (with JSA): 1.016 secs
Local Kafka startup memory usage (without JSA): 440MB
Local Kafka startup memory usage (with JSA): 380MB

NOTE: The jsa files were created only by covering the Kafka Startup code path and resulted in ~45MB of size.

We can see a ~30% reduction in the startup time.

Following are the points to consider:

1. Docker Image Size: Adding the jsa files will increase the size of the Docker image. Classes stored in the CDS are a few times (e.g., 2 – 5x) larger than classes stored in JAR files
2. Choosing Application Usage for JSA: For our POC, we focused on Kafka startup using default configurations. It's crucial to decide how broadly the application should be used to generate jsa files.
3. Startup Memory Footprint: While we did observe a reduction in startup memory footprint, we believe this difference may not significantly scale with production load.
4. CDS Archive Exclusions: The CDS archive doesn't include pre-JDK 6 classes, which are present in our codebase.
5. Limitations of CDS:
   1. The OpenJDK version used for building the jsa files and during runtime must be the same. If they are not, application wont fail but the expected optimisation won't be achieved.
   2. If a custom JAR is provided by the user, it's currently prepended to the existing classpath. However, CDS requires that the classpath used during jsa file generation should either be the same or a prefix of the classpath used during runtime. Otherwise, it will be disabled, and the expected optimisation won't be achieved.

Given the notable reduction in the startup time, we've made the decision to incorporate the CDS in the Docker image.
The jsa file will be generated dynamically as a docker layer using the following workflow in the kraft mode:

1. Start Apache Kafka
2. Create a topic, produce messages, and consume messages
3. Stop Kafka

Compatibility, Deprecation, and Migration Plan

...