Page History

...

We propose adding a mode to the specof the FlinkDeployment CRD to allow both standalone and native clusters to be deployed. This would allow 2 new types of Flink clusters to be created: standalone-application, standalone-session. This will default to native to maintain compatibility.

Also replicas will be added to the taskManager spec to specify the number of TaskManager pods to spin up, this will only be used for standalone session clusters. For application clusters the number of replicas to spin up can be calculated using the job parallelism and the taskmanager.numberOfTaskSlots configuration

Standalone mode

All interactions with the Flink cluster is currently done via the FlinkService which is integrated with the Kubernetes native nature of the cluster. This will be forked into a FlinkNativeService and FlinkStandaloneService to enable communication with both cluster types.

Version support

With standalone mode being supported the operator can also support deploying Flink clusters older than 1.14 (as far back as 1.2). Doing this we can increase the potential user-base of the operator and provide those users.

Supported Flink images are available on the docker repo from version 1.11 [2] so these can be supported by the connector in standalone mode. Previous Flink versions could also be used by the standalone mode, but not fully supported.

✅: Fully supported
?: Compatible but not supported
?: Not supported

...