airflowLocalSettings

airflow_local_settings file as a string (templated). You can bake an airflow_local_settings.py into your image instead. In that case, set this value to null.

See values.yaml

keep

Keep

airflowVersion

Airflow version (Used to make some decisions based on Airflow Version being deployed).

3.1.7

keep

Keep

apiSecretAnnotations

Annotations to add to the Api secret.

{}

Description remains from v2 terms. 

apiSecretKey

The Flask secret key for Airflow Api to encrypt browser session.

~

Description remains from v2 terms. 

apiServer.apiServerConfig

This string (templated) will be mounted into the Airflow API Server as a custom webserver_config.py. You can bake a webserver_config.py in to your image instead or specify a configmap containing the webserver_config.py.

~

Examples:

apiServerConfig: |-
  from airflow import configuration as conf

  # The SQLAlchemy connection string.
  SQLALCHEMY_DATABASE_URI = conf.get('database', 'SQL_ALCHEMY_CONN')

  # Flask-WTF flag for CSRF
  CSRF_ENABLED = True

This is still relevant if the user is using providers-fab and adding more fine-grained authentication integration through fab. That also generalised to BaseAuthManager and to providers that implement 3rd-party libraries and tools. I think the description should be updated.

webserver_config.py migration check mentioned above.

apiServer.apiServerConfigConfigMapName

The configmap name containing the webserver_config.py.

~

Examples:

apiServerConfigConfigMapName: my-api-server-configmap

keep

keep

config

Settings to go into the mounted airflow.cfg

See values.yaml

keep

keep

createUserJob.defaultUser.enabled

Enable default user creation.

True

keep

defaultAirflowDigest

Default airflow digest to deploy. Overrides tag.

~

defaultAirflowRepository

Default airflow repository. Overrides all the specific images below.

apache/airflow

keep

keep

defaultAirflowTag

Default airflow tag to deploy.

3.1.7

keep

keep

executor

Airflow executor.

CeleryExecutor

keep

fernetKey

The Fernet key used to encrypt passwords (can only be set during install, not upgrade).

~

keep

fernetKeySecretAnnotations

Annotations to add to the Fernet Key secret.

{}

keep

jwtSecret

Secret key used to encode and decode JWTs to authenticate to public and private APIs (can only be set during install, not upgrade).

~

keep

jwtSecretAnnotations

Annotations to add to the JWT secret.

{}

keep

pgbouncer.enabled

Enable PgBouncer.

False

keep

pgbouncer.mountConfigSecret

Whether to mount the config secret files under /etc/pgbouncer/ by default.

True

keep

schedulerName

Specify kube scheduler name for Pods.

~

webserver.webserverConfig

This string (templated) will be mounted into the Airflow webserver as a custom webserver_config.py. You can bake a webserver_config.py in to your image instead or specify a configmap containing the webserver_config.py.

~

Examples:

webserverConfig: |-
  from airflow import configuration as conf

  # The SQLAlchemy connection string.
  SQLALCHEMY_DATABASE_URI = conf.get('database', 'SQL_ALCHEMY_CONN')

  # Flask-WTF flag for CSRF
  CSRF_ENABLED = True

webserver.webserverConfigConfigMapName

The configmap name containing the webserver_config.py.

~

Examples:

webserverConfigConfigMapName: my-webserver-configmap

webserverSecretAnnotations

Annotations to add to the webserver secret.

{}

webserverSecretKey

The Flask secret key for Airflow Webserver to encrypt browser session.

~

workers.schedulerName

Specify kube scheduler name for Airflow Celery workers objects and pods created with pod-template-file.

~

airflowHome

Airflow home directory. Used for mount paths.

/opt/airflow

keep

keep

allowJobLaunching

Whether various Airflow components launch jobs.

False

It can be automated indeed. We can map to both a feature and a deprecation along with it.

allowPodLaunching

Whether various Airflow components launch pods.

True

It can be automated indeed. We can map to both a feature and a deprecation along with it.

apiSecretKeySecretName

The Secret name containing Flask secret_key for the Api.

~

Description remains from v2 terms. 

dags.gitSync.branch

Git branch

v2-2-stable

It makes sense. There are multiple ways in various systems of mounting the dags into the components. Making it bundled can help with easier additions on top of it for mounting operations.

dags.gitSync.containerName

Git sync container name.

git-sync

dags.gitSync.credentialsSecret

Name of a Secret containing the repo GIT_SYNC_USERNAME and GIT_SYNC_PASSWORD.

~

dags.gitSync.depth

Repository depth.

1

dags.gitSync.emptyDirConfig

Configuration for dags empty dir volume.

~

dags.gitSync.enabled

Enable Git sync.

False

dags.gitSync.env

Environment variables for git sync container.

[]

Examples:

env:
- name: GIT_SYNC_TIMEOUT
  value: '60'
env:
- name: GIT_SYNC_USERNAME
  valueFrom:
    secretKeyRef:
      key: username
      name: git-secret

dags.gitSync.envFrom

Extra envFrom ‘items’ that will be added to the definition of Airflow gitSync containers; a string or array are expected (templated).

~

Examples:

envFrom: |-
  - secretRef:
      name: 'proxy-config
envFrom: |-
  - configMapRef:
      name: 'proxy-config

dags.gitSync.extraVolumeMounts

Mount additional volumes into git sync container.

[]

dags.gitSync.knownHosts

When using a ssh private key, the contents of your known_hosts file.

~

Examples:

knownHosts: |-
  <host1>,<ip1> <key1>
  <host2>,<ip2> <key2>
knownHosts: <host1>,<ip1> <key1>

dags.gitSync.maxFailures

The number of consecutive failures allowed before aborting.

0

dags.gitSync.period

Interval between git sync attempts in Go-style duration string. High values are more likely to cause DAGs to become out of sync between different components. Low values cause more traffic to the remote git repository.

5s

dags.gitSync.ref

Git revision branch, tag, or hash.

v2-2-stable

dags.gitSync.repo

Git repository.

https://github.com/apache/airflow.git

dags.gitSync.resources

Resources on workers git-sync sidecar

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

dags.gitSync.rev

Git revision.

HEAD

dags.gitSync.securityContext

Security context for the gitSync container (deprecated, use securityContexts instead). If not set, the values from securityContext will be used.

{}

Examples:

securityContext:
  runAsGroup: 0
  runAsUser: 50000

dags.gitSync.sshKey

SSH private key

~

dags.gitSync.sshKeySecret

Name of a Secret containing the repo sshKeySecret.

~

dags.gitSync.subPath

Subpath within the repo where dags are located.

tests/dags

dags.gitSync.uid

Git sync container run as user parameter.

65533

dags.gitSync.wait

Interval between git sync attempts in seconds. High values are more likely to cause DAGs to become out of sync between different components. Low values cause more traffic to the remote git repository.

~

dags.mountPath

Where dags volume will be mounted. Works for both persistence and gitSync. If not specified, dags mount path will be set to $AIRFLOW_HOME/dags

~

dags.persistence.accessMode

Access mode of the persistent volume.

ReadWriteOnce

dags.persistence.annotations

Annotations for the dag PVC

{}

dags.persistence.enabled

Enable persistent volume for storing dags.

False

dags.persistence.existingClaim

The name of an existing PVC to use.

~

dags.persistence.size

Volume size for dags.

1Gi

dags.persistence.storageClassName

If using a custom StorageClass, pass name here (templated).

~

dags.persistence.subPath

Subpath within the PVC where dags are located.

~

elasticsearch.connection

Elasticsearch connection configuration.

{}

Examples:

connection:
  host: '...'
  pass: '...'
  port: '...'
  scheme: https
  user: '...'

elasticsearch.connection.host

Host

""

elasticsearch.connection.pass

Password

""

elasticsearch.connection.port

Port

80

elasticsearch.connection.scheme

Scheme

http

elasticsearch.connection.user

Username

""

elasticsearch.enabled

Enable Elasticsearch task logging.

False

elasticsearch.secretAnnotations

Extra annotations to apply to the elasticsearch secret.

{}

elasticsearch.secretName

A secret containing the connection string.

~

enableBuiltInSecretEnvVars.AIRFLOW_CONN_AIRFLOW_DB

Enable AIRFLOW_CONN_AIRFLOW_DB variable to be read from the Metadata Secret

True

SQLAlchemy connection string from K8s secret. Keeping with a better description and maybe renaming to reflect better.

enableBuiltInSecretEnvVars.AIRFLOW__API_AUTH__JWT_SECRET

Enable AIRFLOW__API_AUTH__JWT_SECRET variable to be read from the JWT Secret

True

Enable using K8S secret in env var to pass jwt_secret config

enableBuiltInSecretEnvVars.AIRFLOW__API__SECRET_KEY

Enable AIRFLOW__API__SECRET_KEY variable to be read from the Api Secret Key Secret

True

Similar to above

secret_key

enableBuiltInSecretEnvVars.AIRFLOW__CELERY__BROKER_URL

Enable AIRFLOW__CELERY__BROKER_URL variable to be read from the Celery Broker URL Secret

True

All these are the same as above, as setting some config that enables to read secret to pass to the env var. Maybe move them to Kustomize?

enableBuiltInSecretEnvVars.AIRFLOW__CELERY__CELERY_RESULT_BACKEND

Enable AIRFLOW__CELERY__CELERY_RESULT_BACKEND variable to be read from the Celery Result Backend Secret - Airflow 1.10.* variant

True

enableBuiltInSecretEnvVars.AIRFLOW__CELERY__RESULT_BACKEND

Enable AIRFLOW__CELERY__RESULT_BACKEND variable to be read from the Celery Result Backend Secret

True

enableBuiltInSecretEnvVars.AIRFLOW__CORE__FERNET_KEY

Enable AIRFLOW__CORE__FERNET_KEY variable to be read from the Fernet key Secret

True

enableBuiltInSecretEnvVars.AIRFLOW__CORE__SQL_ALCHEMY_CONN

Enable AIRFLOW__CORE__SQL_ALCHEMY_CONN variable to be read from the Metadata Secret

True

enableBuiltInSecretEnvVars.AIRFLOW__DATABASE__SQL_ALCHEMY_CONN

Enable AIRFLOW__DATABASE__SQL_ALCHEMY_CONN variable to be read from the Metadata Secret

True

enableBuiltInSecretEnvVars.AIRFLOW__ELASTICSEARCH__ELASTICSEARCH_HOST

Enable AIRFLOW__ELASTICSEARCH__ELASTICSEARCH_HOST variable to be read from the Elasticsearch Host Secret - Airflow <1.10.4 variant

True

enableBuiltInSecretEnvVars.AIRFLOW__ELASTICSEARCH__HOST

Enable AIRFLOW__ELASTICSEARCH__HOST variable to be read from the Elasticsearch Host Secret

True

enableBuiltInSecretEnvVars.AIRFLOW__OPENSEARCH__HOST

Enable AIRFLOW__OPENSEARCH__HOST variable to be read from the OpenSearch Host Secret

True

enableBuiltInSecretEnvVars.AIRFLOW__WEBSERVER__SECRET_KEY

Enable AIRFLOW__WEBSERVER__SECRET_KEY variable to be read from the Webserver Secret Key Secret

True

env

Environment variables for all Airflow containers.

[]

Examples:

env:
- name: MYENVVAR
  value: something_fun

keep

extraEnv

Extra env ‘items’ that will be added to the definition of Airflow containers; a string is expected (templated).

~

Examples:

extraEnv: |-
  - name: AIRFLOW__CORE__LOAD_EXAMPLES
     value: True

keep

extraEnvFrom

Extra envFrom ‘items’ that will be added to the definition of Airflow containers; a string is expected (templated).

~

Examples:

extraEnvFrom: |-
  - secretRef:
      name: '{{ .Release.Name }}-airflow-connections'
extraEnvFrom: |-
  - configMapRef:
      name: '{{ .Release.Name }}-airflow-variables'

keep

fernetKeySecretName

The Fernet key secret name.

~

gid

Group of airflow user.

0

jwtSecretName

The JWT secret name.

~

logs.emptyDirConfig

Configuration for logs empty dir volume.

~

logs.persistence.annotations

Annotations to add to logs PVC

{}

logs.persistence.enabled

Enable persistent volume for storing logs.

False

logs.persistence.existingClaim

The name of an existing PVC to use.

~

logs.persistence.size

Volume size for logs.

100Gi

logs.persistence.storageClassName

If using a custom StorageClass, pass name here (templated).

~

logs.persistence.subPath

The subpath of the existing PVC to use.

~

multiNamespaceMode

Whether Airflow can launch workers and/or pods in multiple namespaces. If true, it creates ClusterRole/ClusterRolebinding (with access to entire cluster)

False

opensearch.connection

OpenSearch connection configuration.

{}

Examples:

connection:
  host: '...'
  pass: '...'
  port: '...'
  scheme: https
  user: '...'

opensearch.connection.host

Host

""

opensearch.connection.pass

Password

""

opensearch.connection.port

Port

80

opensearch.connection.scheme

Scheme

http

opensearch.connection.user

Username

""

opensearch.enabled

Enable OpenSearch task logging.

False

opensearch.secretName

A secret containing the connection string.

~

podTemplate

The content of pod_template_file.yaml used for KubernetesExecutor workers (templated). The default (see files/pod-template-file.kubernetes-helm-yaml) already takes into account normal workers configuration parameters (e.g. workers.resources), so you normally won’t need to override this directly.

~

Examples:

podTemplate: |-
  apiVersion: v1
  kind: Pod
  metadata:
    name: placeholder-name
    labels:
      tier: airflow
      component: worker
      release: {{ .Release.Name }}
  spec:
    priorityClassName: high-priority
    containers:
      - name: base
      ...

secret

Secrets for all Airflow containers.

[]

Examples:

secret:
- envName: SecretEnvVar
  secretKey: somekey
  secretName: somesecret

keep

uid

User of airflow user.

50000

volumeMounts

VolumeMounts for all Airflow containers.

[]

keep

volumes

Volumes for all Airflow containers.

[]

keep

webserverSecretKeySecretName

The Secret name containing Flask secret_key for the Webserver.

~

data.brokerUrlSecretAnnotations

Annotations to add to the broker url secret.

{}

data.metadataConnection.db

The name of the database.

postgres

data.metadataConnection.host

The database host.

~

data.metadataConnection.pass

The user’s password.

postgres

data.metadataConnection.port

The database port.

5432

data.metadataConnection.protocol

The database protocol.

postgresql

data.metadataConnection.secretAnnotations

Annotations to add to the metadata connection secret.

{}

data.metadataConnection.sslmode

The database SSL parameter.

disable

data.metadataConnection.user

The database user.

postgres

data.metadataSecretName

Metadata connection string secret.

~

data.resultBackendConnection

Result backend connection configuration.

~

data.resultBackendConnection.db

The name of the database.

~

data.resultBackendConnection.host

The database host.

~

data.resultBackendConnection.pass

The database password.

~

data.resultBackendConnection.port

The database port.

~

data.resultBackendConnection.protocol

The database protocol.

~

data.resultBackendConnection.sslmode

The database SSL parameter.

~

data.resultBackendConnection.user

The database user.

~

data.resultBackendConnectionSecretAnnotations

Annotations to add to the result backend connection secret.

{}

data.resultBackendSecretName

Result backend connection string secret.

~

postgresql.auth.enablePostgresUser

Assign a password to the ‘postgres’ admin user. Otherwise, remote access will be blocked for this user

True

postgresql.auth.password

Password for the custom user to create.

""

postgresql.auth.postgresPassword

Password for the ‘postgres’ admin user.

postgres

postgresql.auth.username

Name for a custom user to create

""

postgresql.enabled

Enable PostgreSQL subchart.

True

postgresql.image.repository

The PostgreSQL image repository.

bitnamilegacy/postgresql

postgresql.image.tag

The PostgreSQL image tag.

16.1.0-debian-11-r15

pgbouncer.affinity

Specify scheduling constraints for PgBouncer pods.

{}

pgbouncer.annotations

Annotations to add to the PgBouncer deployment

{}

pgbouncer.args

Args to use for PgBouncer (templated).

~

pgbouncer.auth_file

The name of the file to load user names and passwords from

/etc/pgbouncer/users.txt

pgbouncer.auth_type

Method of authenticating users

scram-sha-256

pgbouncer.certificatesSecretAnnotations

Annotations to add to the PgBouncer certificates secret.

{}

pgbouncer.ciphers

The allowed ciphers, might be ‘fast’, ‘normal’ or list ciphers separated with ‘:’.

normal

pgbouncer.command

Command to use for PgBouncer (templated).

['pgbouncer', '-u', 'nobody', '/etc/pgbouncer/pgbouncer.ini']

pgbouncer.configSecretAnnotations

Annotations to add to the PgBouncer config secret.

{}

pgbouncer.configSecretName

The PgBouncer config Secret name.

~

pgbouncer.env

Add additional env vars to pgbouncer container.

[]

pgbouncer.extraContainers

Launch additional containers into pgbouncer.

[]

pgbouncer.extraIni

Add extra general PgBouncer ini configuration: https://www.pgbouncer.org/config.html

~

pgbouncer.extraIniMetadata

Add extra metadata database specific PgBouncer ini configuration: https://www.pgbouncer.org/config.html#section-databases

~

pgbouncer.extraIniResultBackend

Add extra result backend database specific PgBouncer ini configuration: https://www.pgbouncer.org/config.html#section-databases

~

pgbouncer.extraNetworkPolicies

Additional NetworkPolicies as needed.

[]

pgbouncer.extraVolumeMounts

Mount additional volumes into PgBouncer.

[]

pgbouncer.extraVolumes

Mount additional volumes into PgBouncer.

[]

pgbouncer.labels

Labels to add to the PgBouncer objects and pods.

{}

pgbouncer.logConnections

Log successful logins.

0

pgbouncer.logDisconnections

Log disconnections with reasons.

0

pgbouncer.maxClientConn

Maximum clients that can connect to PgBouncer (higher = more file descriptors).

100

pgbouncer.metadataPoolSize

Metadata pool size.

10

pgbouncer.metricsExporterSidecar.extraVolumeMounts

Mount additional volumes into PgBouncer Metrics Exporter.

[]

pgbouncer.metricsExporterSidecar.livenessProbe.initialDelaySeconds

Metrics Exporter liveness probe initial delay

10

pgbouncer.metricsExporterSidecar.livenessProbe.periodSeconds

Metrics Exporter liveness probe frequency

10

pgbouncer.metricsExporterSidecar.livenessProbe.timeoutSeconds

Metrics Exporter liveness probe command timeout

1

pgbouncer.metricsExporterSidecar.readinessProbe.initialDelaySeconds

Metrics Exporter readiness probe initial delay

10

pgbouncer.metricsExporterSidecar.readinessProbe.periodSeconds

Metrics Exporter readiness probe frequency

10

pgbouncer.metricsExporterSidecar.readinessProbe.timeoutSeconds

Metrics Exporter readiness probe command timeout

1

pgbouncer.metricsExporterSidecar.resources

Resources for the PgBouncer metric exporter.

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

pgbouncer.metricsExporterSidecar.sslmode

SSL mode for metricsExporterSidecar

disable

pgbouncer.metricsExporterSidecar.statsSecretAnnotations

Annotations to add to the PgBouncer stats secret.

{}

pgbouncer.metricsExporterSidecar.statsSecretKey

Key referencing the PGBouncer Metrics connection URI within an existing Secrets object. Defaults to connection if left null.

~

pgbouncer.metricsExporterSidecar.statsSecretName

Name of an existing Secrets object containing PgBouncer Metrics secrets.

~

pgbouncer.nodeSelector

Select certain nodes for PgBouncer pods.

{}

pgbouncer.podAnnotations

Add annotations for the PgBouncer Pod.

{}

pgbouncer.podDisruptionBudget.config.maxUnavailable

Max unavailable pods for PgBouncer.

1

pgbouncer.podDisruptionBudget.config.minAvailable

Min available pods for PgBouncer.

1

pgbouncer.podDisruptionBudget.enabled

Enabled PodDistributionBudget.

False

pgbouncer.priorityClassName

Specify priority for PgBouncer pods.

~

pgbouncer.replicas

Number of PgBouncer replicas to run in Deployment.

1

pgbouncer.resources

Resources for the PgBouncer pods.

{}

Examples:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

pgbouncer.resultBackendPoolSize

Result backend pool size.

5

pgbouncer.service.clusterIp

Specific ClusterIP for the PgBouncer Service.

~

pgbouncer.service.extraAnnotations

Extra annotations for the PgBouncer Service.

{}

pgbouncer.serviceAccount.annotations

Annotations to add to the worker Kubernetes ServiceAccount.

{}

pgbouncer.serviceAccount.automountServiceAccountToken

Specifies if ServiceAccount’s API credentials should be mounted onto Pods.

True

pgbouncer.serviceAccount.create

Specifies whether a ServiceAccount should be created.

True

pgbouncer.serviceAccount.name

The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name.

~

pgbouncer.ssl.ca

Certificate Authority for server side

~

pgbouncer.ssl.cert

Server Certificate for server side

~

pgbouncer.ssl.key

Private key used to authenticate with the server

~

pgbouncer.sslmode

SSL mode for PgBouncer.

prefer

pgbouncer.tolerations

Specify Tolerations for PgBouncer pods.

[]

pgbouncer.topologySpreadConstraints

Specify topology spread constraints for PgBouncer pods.

[]

pgbouncer.uid

PgBouncer run as user parameter.

65534

pgbouncer.verbose

Increase PgBouncer verbosity.

0

apiServer.affinity

Specify scheduling constraints for API server pods.

See values.yaml

apiServer.allowPodLogReading

Allow API server to read k8s pod logs. Useful when you don’t have an external log store.

True

apiServer.annotations

Annotations to add to the API server deployment

{}

apiServer.args

Args to use when running the Airflow API server (templated). When running behind a reverse proxy, add –proxy-headers to enable Uvicorn to respect X-Forwarded-Proto, X-Forwarded-For, and X-Forwarded-Port headers.

['bash', '-c', 'exec airflow api-server']

Examples:

args:
- bash
- -c
- exec airflow api-server --proxy-headers

apiServer.command

Command to use when running the Airflow API server (templated).

~

apiServer.configMapAnnotations

Extra annotations to apply to the API server configmap.

{}

apiServer.enabled

Enable Airflow API server deployment.

True

apiServer.env

Add additional env vars to API server. When running behind a reverse proxy, set FORWARDED_ALLOW_IPS to specify which IPs are trusted to send X-Forwarded-* headers. Use “*” for trusted environments, or specify proxy IP ranges for production.

[]

Examples:

env:
- - name: FORWARDED_ALLOW_IPS
    value: '*'

apiServer.extraContainers

Launch additional containers into API server.

[]

apiServer.extraInitContainers

Add additional init containers into API server.

[]

apiServer.extraVolumeMounts

Mount additional volumes into API server.

[]

apiServer.extraVolumes

Mount additional volumes into API server.

[]

apiServer.hostAliases

HostAliases for the API server pod.

[]

Examples:

hostAliases:
- hostnames:
  - foo.local
  ip: 127.0.0.1
hostAliases:
- hostnames:
  - foo.remote
  ip: 10.1.2.3

apiServer.hpa.behavior

HorizontalPodAutoscalerBehavior configures the scaling behavior of the target.

{}

apiServer.hpa.enabled

Enable HPA autoscaling for API server

False

apiServer.hpa.maxReplicaCount

Maximum number of API server replicas created by HPA if HPA is enabled.

5

apiServer.hpa.metrics

Specifications for which to use to calculate the desired replica count.

[{'type': 'Resource', 'resource': {'name': 'cpu', 'target': {'type': 'Utilization', 'averageUtilization': 50}}}]

apiServer.hpa.minReplicaCount

Minimum number of API server replicas created by HPA if HPA is enabled.

1

apiServer.labels

Labels to add to the API server objects and pods.

{}

apiServer.livenessProbe.failureThreshold

API server Liveness probe failure threshold.

5

apiServer.livenessProbe.initialDelaySeconds

API server Liveness probe initial delay.

15

apiServer.livenessProbe.periodSeconds

API server Liveness probe period seconds.

10

apiServer.livenessProbe.scheme

API server Liveness probe scheme.

HTTP

apiServer.livenessProbe.timeoutSeconds

API server Liveness probe timeout seconds.

5

apiServer.networkPolicy.ingress.from