...

1. ExecutionJobVertexInitializedEvent: This event is responsible for recording the initialization information of ExecutionJobVertex,   its content contains the decided parallelism of this job vertex, and its its content contains the decided parallelism of this job vertex, and its input information. This event will be triggered and written out when an execution job vertex is initialized.

2. ExecutionVertexFinishedEvent: This event is responsible for recording the information of finished task. Our goal is that all finished tasks don’t need to re-run, so the simple idea is to trigger an event when a task is finished.  The content of this event contains:

   1. The state of the finished task/ExecutionVertex, including IO metrics, accumulators, etc. These contents can be easily obtained from ExecutionGraph.

   2. If the job vertex which this task belongs to has operator coordinators, the states of the operator coordinators also need to be recorded.

...

In FLIP-283, adaptive batch scheduler has been the default scheduler of Flink batch jobs, so we intend to only support working with the adaptive batch scheduler.

When using ApplicationMode, only support single-execute jobWhen using ApplicationMode, only support single-execute job

As mentioned in flink docs , HA in Application Mode is only supported for single-execute applications. Job Recovery HA in Application Mode is only supported for single-execute applications. Job Recovery relies on HA,so when using ApplicationMode, the Job Recovery can only support single-execution applicationsso when using ApplicationMode, the Job Recovery can only support single-execution applications.

Test Plan

The proposed changes will be tested for correctness and stability in a real clusterThe proposed changes will be tested for correctness and stability in a real cluster.