airflowLocalSettings

airflow_local_settings file as a string (templated). You can bake an airflow_local_settings.py into your image instead. In that case, set this value to null.

See values.yaml

keep

airflowVersion

Airflow version (Used to make some decisions based on Airflow Version being deployed).

3.1.7

keep

apiSecretAnnotations

Annotations to add to the Api secret.

{}

Description remains from v2 terms. 

apiSecretKey

The Flask secret key for Airflow Api to encrypt browser session.

~

Description remains from v2 terms. 

apiServer.apiServerConfig

This string (templated) will be mounted into the Airflow API Server as a custom webserver_config.py. You can bake a webserver_config.py in to your image instead or specify a configmap containing the webserver_config.py.

~

Examples:

apiServerConfig: |-
  from airflow import configuration as conf

  # The SQLAlchemy connection string.
  SQLALCHEMY_DATABASE_URI = conf.get('database', 'SQL_ALCHEMY_CONN')

  # Flask-WTF flag for CSRF
  CSRF_ENABLED = True

This is still relevant if the user is using providers-fab and adding more fine-grained authentication integration through fab. That also generalised to BaseAuthManager and to providers that implement 3rd-party libraries and tools. I think the description should be updated.

webserver_config.py migration check mentioned above.

apiServer.apiServerConfigConfigMapName

The configmap name containing the webserver_config.py.

~

Examples:

apiServerConfigConfigMapName: my-api-server-configmap

keep

config

Settings to go into the mounted airflow.cfg

See values.yaml

keep

createUserJob.defaultUser.enabled

Enable default user creation.

True

keep

defaultAirflowDigest

Default airflow digest to deploy. Overrides tag.

~

defaultAirflowRepository

Default airflow repository. Overrides all the specific images below.

apache/airflow

keep

defaultAirflowTag

Default airflow tag to deploy.

3.1.7

keep

executor

Airflow executor.

CeleryExecutor

fernetKey

The Fernet key used to encrypt passwords (can only be set during install, not upgrade).

~

keep

fernetKeySecretAnnotations

Annotations to add to the Fernet Key secret.

{}

keep

fullnameOverride

Provide a name to substitute for the full names of resources

""

jwtSecret

Secret key used to encode and decode JWTs to authenticate to public and private APIs (can only be set during install, not upgrade).

~

keep

jwtSecretAnnotations

Annotations to add to the JWT secret.

{}

keep

nameOverride

Override the name of the chart

""

pgbouncer.enabled

Enable PgBouncer.

False

keep

pgbouncer.mountConfigSecret

Whether to mount the config secret files under /etc/pgbouncer/ by default.

True

keep

schedulerName

Specify kube scheduler name for Pods.

~

useStandardNaming

Use standard naming for all resources using airflow.fullname template

False

webserver.webserverConfig

This string (templated) will be mounted into the Airflow webserver as a custom webserver_config.py. You can bake a webserver_config.py in to your image instead or specify a configmap containing the webserver_config.py.

~

Examples:

webserverConfig: |-
  from airflow import configuration as conf

  # The SQLAlchemy connection string.
  SQLALCHEMY_DATABASE_URI = conf.get('database', 'SQL_ALCHEMY_CONN')

  # Flask-WTF flag for CSRF
  CSRF_ENABLED = True

webserver.webserverConfigConfigMapName

The configmap name containing the webserver_config.py.

~

Examples:

webserverConfigConfigMapName: my-webserver-configmap

webserverSecretAnnotations

Annotations to add to the webserver secret.

{}

webserverSecretKey

The Flask secret key for Airflow Webserver to encrypt browser session.

~

workers.schedulerName

Specify kube scheduler name for Airflow Celery workers objects and pods created with pod-template-file.

~

workers.affinity

Specify scheduling constraints for Airflow Celery worker pods and pods created with pod-template-file.

See values.yaml

workers.annotations

Annotations to add to the Airflow Celery worker deployment.

{}

workers.args

Args to use when running Airflow Celery workers (templated) (deprecated, use workers.celery.args instead).

['bash', '-c', 'exec \\\nairflow {{ semverCompare ">=2.0.0" .Values.airflowVersion | ternary "celery worker" "worker" }}\n{{- if and .Values.workers.queue (ne .Values.workers.queue "default") }}\n{{- " -q " }}{{ .Values.workers.queue }}\n{{- end }}']

workers.celery.args

Args to use when running Airflow Celery workers (templated).

~

workers.celery.command

Command to use when running Airflow Celery workers (templated).

~

workers.celery.enableDefault

Enable the default worker defined by the workers and workers.celery configurations.

True

workers.celery.kerberosInitContainer.enabled

Enable Kerberos init container.

~

workers.celery.kerberosInitContainer.resources

Resources on kerberos init container.

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

workers.celery.livenessProbe.command

Command for LivenessProbe

~

workers.celery.livenessProbe.enabled

Enable liveness probe for Airflow Celery workers.

~

workers.celery.livenessProbe.failureThreshold

Minimum consecutive failures for the probe to be considered failed after having succeeded. Minimum value is 1.

~

workers.celery.livenessProbe.initialDelaySeconds

Number of seconds after the container has started before liveness probes are initiated.

~

workers.celery.livenessProbe.periodSeconds

How often (in seconds) to perform the probe. Minimum value is 1.

~

workers.celery.livenessProbe.timeoutSeconds

Number of seconds after which the probe times out. Minimum value is 1 seconds.

~

workers.celery.persistence.annotations

Annotations to add to Airflow Celery worker volumes.

{}

workers.celery.persistence.enabled

Enable persistent volumes.

~

workers.celery.persistence.fixPermissions

Execute init container to chown log directory. This is currently only needed in kind, due to usage of local-path provisioner.

~

workers.celery.persistence.size

Volume size for Airflow Celery worker StatefulSet.

~

workers.celery.persistence.storageClassName

If using a custom StorageClass, pass name ref to all StatefulSets here (templated).

~

workers.celery.podManagementPolicy

Specifies the policy for managing pods within the Airflow Celery worker. Only applicable to StatefulSet.

~

workers.celery.queue

Queue name for the worker.

default

workers.celery.replicas

Number of Airflow Celery workers.

~

workers.celery.sets

List of worker sets. Each item can overwrite values from the parent workers and workers.celery sections.

[]

workers.celery.strategy

Specifies the strategy used to replace old Airflow Celery worker pods by new ones when deployed as a Deployment.

~

workers.celery.updateStrategy

Specifies the strategy used to replace old Airflow Celery worker pods by new ones when deployed as a StatefulSet.

~

workers.command

Command to use when running Airflow Celery workers and using pod-template-file (templated). Use workers.celery.command and/or workers.kubernetes.command to separate value between Celery workers and pod-template-file.

~

workers.env

Add additional env vars to the Airflow Celery workers and pods created with pod-template-file.

[]

workers.extraContainers

Launch additional containers into Airflow Celery workers and pods created with pod-template-file (templated). Note, if used with KubernetesExecutor, you are responsible for signaling sidecars to exit when the main container finishes so Airflow can continue the worker shutdown process!

[]

workers.extraInitContainers

Add additional init containers into Airflow Celery workers and pods created with pod-template-file (templated).

[]

workers.extraPorts

Expose additional ports of Airflow Celery worker container.

[]

workers.extraVolumeMounts

Additional volume mounts attached to the Airflow Celery workers and pods created with pod-template-file.

[]

workers.extraVolumes

Additional volumes attached to the Airflow Celery workers and pods created with pod-template-file.

[]

workers.hostAliases

Specify HostAliases for Airflow Celery worker pods and pods created with pod-template-file.

[]

Examples:

hostAliases:
- hostnames:
  - test.hostname.one
  ip: 127.0.0.2
hostAliases:
- hostnames:
  - test.hostname.two
  ip: 127.0.0.3

workers.hpa.behavior

HorizontalPodAutoscalerBehavior configures the scaling behavior of the target.

{}

workers.hpa.enabled

Allow HPA autoscaling (KEDA must be disabled).

False

workers.hpa.maxReplicaCount

Maximum number of Airflow Celery workers created by HPA.

5

workers.hpa.metrics

Specifications for which to use to calculate the desired replica count.

[{'type': 'Resource', 'resource': {'name': 'cpu', 'target': {'type': 'Utilization', 'averageUtilization': 80}}}]

workers.hpa.minReplicaCount

Minimum number of Airflow Celery workers created by HPA.

0

workers.keda.advanced

Advanced KEDA configuration.

{}

workers.keda.advanced.horizontalPodAutoscalerConfig

HorizontalPodAutoscalerConfig specifies horizontal scale config.

{}

workers.keda.advanced.horizontalPodAutoscalerConfig.behavior

HorizontalPodAutoscalerBehavior configures the scaling behavior of the target.

{}

workers.keda.cooldownPeriod

How many seconds KEDA will wait before scaling to zero.

30

workers.keda.enabled

Allow KEDA autoscaling.

False

workers.keda.maxReplicaCount

Maximum number of Airflow Celery workers created by KEDA.

10

workers.keda.minReplicaCount

Minimum number of Airflow Celery workers created by KEDA.

0

workers.keda.namespaceLabels

Labels used in matchLabels for namespace in the PgBouncer NetworkPolicy.

{}

workers.keda.pollingInterval

How often KEDA polls the airflow DB to report new scale requests to the HPA.

5

workers.keda.query

Query to use for KEDA autoscaling. Must return a single integer.

SELECT ceil(COUNT(*)::decimal / {{ .Values.config.celery.worker_concurrency }}) FROM task_instance WHERE (state='running' OR state='queued') AND queue IN ( {{- range $i, $q := splitList "," .Values.workers.queue -}} {{- if $i }},{{ end }}'{{ $q | trim }}' {{- end -}} ) {{- if contains "CeleryKubernetesExecutor" .Values.executor }} AND queue != '{{ .Values.config.celery_kubernetes_executor.kubernetes_queue }}' {{- else if contains "KubernetesExecutor" .Values.executor }} AND executor IS DISTINCT FROM 'KubernetesExecutor' {{- else if contains "airflow.providers.edge3.executors.EdgeExecutor" .Values.executor }} AND executor IS DISTINCT FROM 'EdgeExecutor' {{- end }}

workers.keda.usePgbouncer

Weather to use PGBouncer to connect to the database or not when it is enabled. This configuration will be ignored if PGBouncer is not enabled.

True

workers.kerberosInitContainer.enabled

Enable Kerberos init container.

False

workers.kerberosInitContainer.resources

Resources on kerberos init container.

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

workers.kerberosSidecar.enabled

Enable Kerberos sidecar.

False

workers.kerberosSidecar.resources

Resources on kerberos sidecar.

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

workers.kubernetes.command

Command to use in pod-template-file (templated).

~

workers.kubernetes.kerberosInitContainer.enabled

Enable kerberos init container.

~

workers.kubernetes.kerberosInitContainer.resources

Resources on kerberos init container.

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

workers.labels

Labels to add to the Airflow Celery workers objects and pods created with pod-template-file.

{}

workers.livenessProbe.command

Command for LivenessProbe (deprecated, use workers.celery.livenessProbe.command instead)

~

workers.livenessProbe.enabled

Enable liveness probe for Airflow Celery workers (deprecated, use workers.celery.livenessProbe.enabled instead).

True

workers.livenessProbe.failureThreshold

Minimum consecutive failures for the probe to be considered failed after having succeeded. Minimum value is 1 (deprecated, use workers.celery.livenessProbe.failureThreshold instead).

5

workers.livenessProbe.initialDelaySeconds

Number of seconds after the container has started before liveness probes are initiated (deprecated, use workers.celery.livenessProbe.initialDelaySeconds instead).

10

workers.livenessProbe.periodSeconds

How often (in seconds) to perform the probe. Minimum value is 1 (deprecated, use workers.celery.livenessProbe.periodSeconds instead).

60

workers.livenessProbe.timeoutSeconds

Number of seconds after which the probe times out. Minimum value is 1 seconds (deprecated, use workers.celery.livenessProbe.timeoutSeconds instead).

20

workers.nodeSelector

Select certain nodes for Airflow Celery worker pods and pods created with pod-template-file.

{}

workers.persistence.annotations

Annotations to add to Airflow Celery worker volumes (deprecated, use workers.celery.persistence.annotations instead).

{}

workers.persistence.enabled

Enable persistent volumes (deprecated, use workers.celery.persistence.enabled instead).

True

workers.persistence.fixPermissions

Execute init container to chown log directory. This is currently only needed in kind, due to usage of local-path provisioner (deprecated, use workers.celery.persistence.fixPermissions instead).

False

workers.persistence.size

Volume size for Airflow Celery worker StatefulSet (deprecated, use workers.celery.persistence.size instead).

100Gi

workers.persistence.storageClassName

If using a custom StorageClass, pass name ref to all StatefulSets here (templated) (deprecated, use workers.celery.persistence.storageClassName instead).

~

workers.podAnnotations

Annotations to add to the Airflow Celery workers and pods created with pod-template-file.

{}

workers.podDisruptionBudget.config.maxUnavailable

Max unavailable pods for worker.

1

workers.podDisruptionBudget.config.minAvailable

Min available pods for worker.

1

workers.podDisruptionBudget.enabled

Enable pod disruption budget.

False

workers.podManagementPolicy

Specifies the policy for managing pods within the Airflow Celery worker (deprecated, use workers.celery.podManagementPolicy instead). Only applicable to StatefulSet.

~

workers.priorityClassName

Specify priority for Airflow Celery worker pods and pods created with pod-template-file.

~

workers.replicas

Number of Airflow Celery workers (deprecated, use workers.celery.replicas instead).

1

workers.resources

Resource configuration for Airflow Celery workers and pods created with pod-template-file.

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

workers.runtimeClassName

Specify runtime for Airflow Celery worker pods and pods created with pod-template-file.

~

workers.safeToEvict

This setting tells Kubernetes that it’s ok to evict when it wants to scale a node down. It is used by Airflow Celery workers and pod-template-file.

False

workers.securityContext

Security context for the Airflow Celery worker pods and pods created with pod-template-file (deprecated, use securityContexts instead). If not set, the values from securityContext will be used.

{}

Examples:

securityContext:
  fsGroup: 0
  runAsGroup: 0
  runAsUser: 50000

workers.serviceAccount.annotations

Annotations to add to the worker Kubernetes ServiceAccount.

{}

workers.serviceAccount.automountServiceAccountToken

Specifies if ServiceAccount’s API credentials should be mounted onto Pods

True

workers.serviceAccount.create

Specifies whether a ServiceAccount should be created.

True

workers.serviceAccount.name

The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name.

~

workers.strategy

Specifies the strategy used to replace old Airflow Celery worker pods by new ones when deployed as a Deployment (deprecated, use workers.celery.strategy instead).

{'rollingUpdate': {'maxSurge': '100%', 'maxUnavailable': '50%'}}

workers.terminationGracePeriodSeconds

Grace period for tasks to finish after SIGTERM is sent from Kubernetes. It is used by Airflow Celery workers and pod-template-file.

600

workers.tolerations

Specify Tolerations for Airflow Celery worker pods and pods created with pod-template-file.

[]

workers.topologySpreadConstraints

Specify topology spread constraints for Airflow Celery worker pods and pods created with pod-template-file.

[]

workers.updateStrategy

Specifies the strategy used to replace old Airflow Celery worker pods by new ones when deployed as a StatefulSet (deprecated, use workers.celery.updateStrategy instead).

~

workers.volumeClaimTemplates

Specify additional volume claim template for Airflow Celery workers.

[]

Examples:

volumeClaimTemplates:
- accessModes:
  - ReadWriteOnce
  name: data-volume-1
  resources:
    requests:
      storage: 10Gi
  storageClassName: storage-class-1
volumeClaimTemplates:
- accessModes:
  - ReadWriteOnce
  name: data-volume-2
  resources:
    requests:
      storage: 20Gi
  storageClassName: storage-class-2

workers.waitForMigrations.enabled

Enable wait-for-airflow-migrations init container.

True

workers.waitForMigrations.env

Add additional env vars to wait-for-airflow-migrations init container.

[]