Page History

...

Scheduling of Speculative Executions

Because of the introduction of speculative execution, the ExecutionVertex can't default to only one execution is running at the same time. So change the currentExecution in ExecutionVertex to an arrayList named executionList(The purpose of using list is to increase the code extensibility, in case that multiple speculative executions will run at the same time in the future), which means that there can be multiple executions in an ExecutionVertex at the same time.

The scheduler should scheduling an execution(original or speculative execution) according to the index of this execution in executionList instead of default to scheduling the currentExecution in current code.

In each ExecutionJobVertex there is a speculative execution thread used for detect the long tail execution in this ExecutionJobVertex periodically based on above three rules.

Limitations

Configuration

Compatibility, Deprecation, and Migration Plan

...