Page History

...

• Custom Resource to express Flink application in Kubernetes native way (for details see CR example section below)
  • External jar artifact fetcher support (s3, https etc.) via init container
  • similar to https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#pod-template
  • creates an empty session cluster, no application/job management
  • the session cluster can be used to control jobs externally (like submission via REST API)
  • Supports all Flink configuration properties
  • Docker image
  • Upgrade policy (savepoint, stateless)
  • Restore policy (savepoint, latest externalized checkpoint, stateless)
  • Pod template for jobmanager and taskmanager
    
    • full control over k8s pod template (no mapping/whitelisting)
    • layering/merging of pod templates (operator itself could also apply cluster wide defaults)
  • Support explicit session cluster (no job management) and application mode
• Create & deploy new Flink application
  • Empty state
  • From savepoint
• Upgrade Flink application with or w/o savepoint on any CR change, including:
  • Flink configuration change
  • Job jar change
  • Docker image change
• Pause/Resume Flink application
  • the job will not continue its data processing
  • the job will not be deleted from the cluster
  • the job will release its resources back to the cluster (can be used by other jobs)
  • Stops job with savepoint, tracks savepoint/last checkpoint in CR status for resume.
• Delete Flink application
• Integrate with Flink Kubernetes HA module [4]
  • When selected, operator can obtain latest checkpoint from config map and does not depend on a potentially unavailable Flink job REST API
  • This should the default, but not a hard dependency
• Support Flink UI ingress
• CI/CD with operator Docker image artifact, publish image in dockerhub

...