Versions Compared

Old Version 1

changes.mady.by.user Rahul Vats

Saved on

compared with

New Version 2

changes.mady.by.user Rahul Vats

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Test Case ID

Test Objective

Test Steps

Expected Outcome

Test Type

Execution Status

AIP78_TC_001

Verify user

is able to use new API endpoints that allow CRUD operations for backfill runs

can create a backfill using /public/backfill endpoint

1. Send a POST request to create a backfill job for a specific DAG using API with valid parameters.

2. Send a GET request to fetch the created backfill job details and verify its presence.

{
  "dag_id": "string",
  "from_date": "2025-02-20T08:55:22.239Z",
  "to_date": "2025-02-20T08:55:22.239Z",
  "run_backwards": false,
  "dag_run_conf": {},
  "reprocess_behavior": "none",
  "max_active_runs": 10
} 

  1. Verify backfill response and check backfill has been created
  2. Verify in database in backfill and backill dag_run_table

Positive

Pass

All CRUD operations are successful without errors, and changes are reflected correctly.

Positive

AIP78_TC_002

Verify user is able to create backfills for different time ranges

1. Create multiple backfill jobs for the same DAG using different combinations of start_date and end_date via API.
2. Fetch each created backfill job to verify the specified time ranges.

Backfill jobs are created successfully with the correct time ranges.

Positive

Pass

AIP78_TC_003

Verify user is still able to run backfills using CLI

1. Run the backfill command using CLI for a specific DAG with valid parameters.
2. Verify that parameters mentioned here are not available.

CLI backfill runs successfully with only the supported parameters.

Positive

Pass

AIP78_TC_004

Verify

backfills work with CLI new config parameters (from_date and to_date)

user is able to perform a dry run using backfill dry_run endpoint

  1. Send a POST request to dry_run endpoint with below request

    {
      "dag_id": "string",
      "from_date": "2025-02-20T08:55:22.239Z",
      "to_date": "2025-02-20T08:55:22.239Z",
      "run_backwards": false,
      "dag_run_conf": {},
      "reprocess_behavior": "none",
      "max_active_runs": 10
    } 

  1. The response does list the backfill date details which will be created

Positive

Pass

1. Run backfill command using CLI with newly introduced config parameters.
2. Verify the backfill execution and its parameters from the job status and logs.

Backfill runs successfully with new configuration parameters.

Positive

AIP78_TC_005

Verify user can create, monitor, clear, and manage backfill jobs from UI

1. Create a backfill job from the API or CLI.
2. Access the web UI and monitor the backfill job status.
3. Clear or manage the backfill job using available UI options.

Users can create, monitor, and manage backfill jobs from the UI without issues.

Positive

Not Implemented


AIP78_TC_006

Verify attributes depends_on_past, start_date

1. Create a backfill job for a DAG.

depends_on_past and start_date should work as expected

Positive

AIP78_TC_007

Verify BackfillRunDagRun table correctly maps Backfill job with DAG run

1. Create a backfill job and verify entries in the BackfillRunDagRun table.
2. Verify that each entry correctly associates a backfill job with the corresponding DAG run.

BackfillRunDagRun table correctly maps each backfill job with the appropriate DAG runs.

Positive

Pass

AIP78_TC_

008

007

Verify backfill DAG runs respect concurrency limitations

1. Create a backfill job with multiple DAG runs.
2. Verify that the scheduler respects defined concurrency limitations when creating and scheduling DAG runs.

Backfill DAG runs are created and managed within defined concurrency limitations.

Positive

AIP78_TC_009

Verify user cannot use LocalExecutor for backfills

1. Run a backfill command with --local parameter in CLI.
2. Verify that the command fails with an appropriate error message indicating that local execution is not supported.

CLI command fails with an error, and backfills are not executed using LocalExecutor.

Negative

Pass

AIP78_TC_

010

008

Perform scheduler benchmarking (task throughput tests)

1. Run backfill jobs under different scenarios (e.g., large backfill range, high number of tasks).
2. Measure scheduler throughput and observe any performance degradation.

Scheduler maintains acceptable performance under increased load due to backfill jobs.

Performance

Pass

AIP78_TC_

011

009

Verify backfills for different types of DAGs (using dynamic mapping, datasets, setup & teardown, etc.)

1. Create backfill jobs for different types of DAGs (e.g., dynamic mapped tasks, setup & teardown tasks).
2. Monitor and verify the execution and outcome of each backfill.

Backfill jobs work correctly with various DAG types without errors.

Positive

Pass

AIP78_TC_010

Verify user can list backfill using List backfill endpoint.

  1. Provide DAG ID in List Endpoint request.

All the backfill details listed in response from the backfill table

Positive

Pass

AIP78_TC_011

Verify user can get details of specific backfill ID

  1. Make a get call to backill endpoint with backfill id 

Details of a specific backfill id are listed in the backfill table

Positive

Pass

AIP78_TC_012

Verify user can pause running backfill

  1. Execute a backfill with many dates
  2. Make a call to pause endpoint with backfill id

Running backfills should be paused successfully

Positive

Pass

AIP78_TC_013

Verify user can unpause running backfill


  1. Make a call to unpause endpoint with any of paused backfill id

Paused backfills should be resumed successfully

Positive

Pass

AIP78_TC_014

Verify user can cancel running backfill

  1. Make a call to cancek endpoint with any of running backfill id

Backfill should be cancelled successfullt

Positive

Pass

AIP78_TC_015

Verify user can create backfill with `run_backwards` as True

1. Send a POST request to create a backfill job for a specific DAG using API with valid parameters.

{
  "dag_id": "string",
  "from_date": "2025-02-20T08:55:22.239Z",
  "to_date": "2025-02-20T08:55:22.239Z",
  "run_backwards": True,
  "dag_run_conf": {},
  "reprocess_behavior": "none",
  "max_active_runs": 10
} 

  1. Verify backfill response and check backfill has been created
  2. Verify in database in backfill and backill dag_run_table
  3. Verify backfill should be executed in backwards way

Positive

Pass

AIP78_TC_016Verify user cannot create backfill with reprocess behavior as None when there are already success DAG runs on same date
  1. Send a POST call to backfill create endpoint with reprocess behavior as none. Also ensure for specific dates run should be in success state
  1. Backfills should not be created. Verify in backfill dag run table there should be new record with exception as `already exits`
NegativePass

AIP78_TC_017

Verify user can create backfill with reprocess behavior as Completed.

  1. Ensure there are runs of specific dates with success state.
  2. Try creating backfill for same dates with reprocess behavior as completed
Backfill should be created(internally we will clear the DAG run for same dates)

Positive

Pass

AIP78_TC_018

Verify user can create backfill with reprocess behavior as Failed.

  1. Ensure there are runs of specific dates with failed state.
  2. Try creating backfill for same dates with reprocess behavior as failed
Backfill should be created(internally we will clear the DAG run for same dates)

Positive

Pass

AIP78_TC_019 

Verify user perform dry run using CLI

  1. execute CLI command with dry-run command option
User should get list of dates for which backfill will be created

Positibe

Fail