Page History

...

• Privacy by design - Collect only minimal, non-personal data with differential privacy techniques
• Transparency - Users can see exactly what data is collected and sent
• User control - Easy opt-out with clear visibility into telemetry status
• Community governance - Changes to collected data require community approval
• Trust rebuilding - Demonstrate Airflow's commitment to user privacy and autonomy
• Statistical usefulness - Collect enough data to make meaningful decisions (targeting >20% adoption)
• No-impact on performance - Ensure that any data collection does not affect the performance of Airflow

Non-Goals

• Collecting personally identifiable information (PII)
• Collecting DAG names, task names, or other deployment-specific identifiers
• Collecting connection details, credentials, or sensitive configuration
• Real-time monitoring or performance profiling of individual deployments
• Commercial use of telemetry data
• Tracking individual users or installations over time

...

• Active providers: List of installed provider packages including version (e.g., apache-airflow-providers-amazon==8.x.y)
• Executor type: Configured executor(s) (e.g., LocalExecutor, CeleryExecutor, KubernetesExecutor - only executors that are known to 'Official' Airflow otherwise "unspecified")
• Database backend: Database type only (e.g., postgres, mysql, sqlite)
• Enabled features: Boolean flags for optional features (e.g., has_webserver_auth, has_dag_serialization)

...

Operator Usage (aggregated with noise)

• Top 10 operator types onlyoperator only: Count of usage with ±5 noise added to each count
• Percentages only: Instead of raw counts for privacy (e.g., "PythonOperator: 45% of tasks")
• No parameters or task names

...

When Airflow is started for the first time (or after upgrade to a version with this AIP):

CLI Installation: A 1030-second interactive prompt appears when a TTY is detected:

╔══════════════════════════════════════════════════════════════════╗
║              Apache Airflow Anonymous Telemetry                  ║
╠══════════════════════════════════════════════════════════════════╣
║                                                                  ║
║  Help improve Airflow by sharing anonymous usage data.           ║
║                                                                  ║
║  We collect: Feature usage, performance metrics, errors          ║
║  We DON'T collect: Personal data, file contents, IPs             ║
║                                                                  ║
║  Privacy protections:                                            ║
║  • Differential privacy (noise added to all counts)              ║
║  • Daily rotating session IDs (no long-term tracking)            ║
║  • Open source implementation you can inspect                    ║
║                                                                  ║
║  View details: airflow.apache.org/docs/telemetry                 ║
║  Debug mode: airflow telemetry --debug                           ║
║                                                                  ║
║  Enable telemetry? [y/N] (auto-decline in 10 seconds)            ║
╚══════════════════════════════════════════════════════════════════╝

...

Any changes to the data collected (additions or removals) require:

1. AIP or GitHub Discussion on Dev List: Proposal explaining the change and justification
2. Dev List Vote: Lazy consensus vote on dev@airflow.apache.org (72-hour voting period)
3. Documentation Update: Update telemetry documentation with exact fields
4. Release Notes: Prominent mention in release notes under "Telemetry Changes" section
5. In-App Notification: Users with telemetry enabled see a one-time notification in Web UI about data collection changes with option to review and opt-out
6. Cli Notification: Users with telemetry enabled see a one-time cli notification with an option to review and opt-out

...

• No Authentication Required: Telemetry endpoint is unauthenticated (prevents tracking via auth tokens) - access is through greylisting if possible (check with ASF Infra)
• Rate Limiting: Backend needs to implement rate limiting per daily session ID to prevent abuse (dependent on ASF)
• Schema Validation: All payloads are validated against JSON schema before processing
• Data Retention: Raw telemetry data retained for 90 days, then deleted; aggregates retained indefinitely
• ASF Infrastructure: Hosted on ASF infrastructure with ASF security policies
• HTTPS Only: All transmissions over TLS 1.2+
• No External Dependencies: Telemetry collection uses only Python standard library (except HTTP client)
• Silent Failures: Network errors, timeouts, and transmission failures never raise exceptions or log above DEBUG level

...