Test cases AIP-72 Task Execution Interface aka Task SDK

TC-001

Verify task execution through the Task SDK interface

1. Implement a sample DAG using Task SDK.
2. Trigger the DAG run.

Task should execute successfully using the SDK without direct database access.

Integration Test

TC-002

Ensure compatibility of Task SDK with existing Airflow hooks and operators

1. Implement a DAG with existing Airflow operators and hooks.
2. Run the DAG and monitor execution.

Task should execute successfully.

Regression Test

TC-003

Measure the performance of tasks executed via Task SDK

1. Create a DAG with multiple tasks.
2. Execute the DAG multiple times.
3. Measure and record the execution time of each run.

Execution times should be within acceptable limits, with minimal overhead introduced by the SDK interface.

Performance Test

TC-004

Verify error handling and task retries within the Task SDK

1. Create a task that is designed to fail (e.g., a division by zero error).
2. Enable retries for the task.
3. Execute the DAG run.

Task should fail and automatically retry according to configured retry logic, with proper error messages logged by the Task SDK.

Functional/Negative Test

TC-005

Verify Task SDK can run "the same" DAGs as on v2.x (*with minimal to no DAG file-level changes). <br> Ensure all the important features like: <br> - Dynamic Mapping <br> - Datasets <br> - Setup & Teardown <br> - Async Operators <br> - Timetables <br> - XCom Validation <br> - Variable Usage <br> - Using Context Variables(

https://airflow.apache.org/docs/apache-airflow/stable/templates-ref.html

). <br> Note: Run this test case for all executors (Local, Celery & K8s).

1. Use the same regression DAGs of 2.x.
2. Execute DAGs.

DAGs should be executed without any issues.

Compatibility Test

TC-006

Validate task metadata and event tracking through the Task SDK

1. Create a sample DAG with tasks using the Task SDK.
2. Trigger a DAG run.
3. Verify metadata and events captured by the SDK during execution.

Task metadata and events (e.g., start time, end time, status) should be accurately captured and accessible through the Task SDK interface.

Functional Test

TC-007

Direct DB access from workers will not be allowed at all. <br> Note: Any custom operators that access the database directly will need rewriting.

1. Use any existing V2.x task which used to connect with the database.

Task will not be permitted to connect to the Airflow metadata database.

Negative Test

TC-008

Pre-declare task variables, connections, and only allow those in one mode.

1. Create a task which can pre-declare variables and connections to use.

Only declared variables and connections should be allowed to be accessed.

Positive Test

TC-009

Verify Task logs are visible correctly.

1. Create a DAG which prints huge task logs.

Task log should be visible properly.

Positive Test

TC-010

Verify running CLI commands.

1. Execute CLI commands.

All the CLI commands should execute successfully.

Positive Test

TC-011

Verify executing API endpoint.

1. Execute all API endpoints.

All the API endpoints should execute successfully.

Positive Test

TC-012

Verify DAG-parsing performance.

1. Import a large number of DAGs.

DAG-parsing performance and functionality should work as expected.

Positive Test

TC-013

Validate language-agnostic task execution.

1. Install the Golang Task SDK.
2. Create a DAG and a task using Golang.

Tasks should execute successfully, confirming language-agnostic support in the Task SDK.

Positive Test