Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For each ExecutionJobVertex, I use six metrics to measure and evaluate the efficiency of speculative execution that can be summarized in the job status and web page.

...

(1)minFinishedForSpeculationThresholdMetrics is defined as the minimal number of the finished ExecutionVertexs before scheduling speculative executions.

...

(2)finishedExecutionCountMetrics is defined as the number of finished ExecutionVertexs.

...

(3)speculativeExecutionCountMetrics is defined as the number of speculative executions that are scheduled by scheduler.

...

(4)speculativeExecutionFastestFinishedCountMetrics is defined as the number of ExecutionVertex's speculative execution that reach FINISHED state faster than the original execution.

(5)speculativeThresholdOfTime is defined as the threshold time of speculative

...

execution.

...

(6)executionIndex2RunningTimespan is defined as the running time of the original execution in each ExecutionVertex.

...

Image Added

Web UI

If we don't modify the code of web UI, when the speculative execution runs faster than the original execution, the web UI will show that this task has been cancelled. But the result of the batch job is correct.

More discussion is needed to decide whether the web UI needs to be modified.

      Image Added                       Image Added

Limitations

(1)JobType must be batch job.

...