...

The testcontainers utilities can be used to create multiple Kafka clusters and the file based implementation of Kafka metadata service would be essential in testing metadata changes.

Future Improvements

KafkaMetadataService can also coordinate the configurations of the source such as Kafka properties and offset initialization strategies. For now, the proposal only includes making 1 common configuration defined by the user. The Kafka properties is essential for cases such as enforcing the max message size configuration from the Kafka server and security configurations.

Rejected Alternatives

None

...