Versions Compared

Old Version 30

changes.mady.by.user Yun Gao

Saved on

compared with

New Version 31

changes.mady.by.user Yun Gao

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Compare to the first option, this option introduces a new RPC and asks CheckpointCoordinator to maintain a set of finished tasks. CheckpointCoordinator also need to remove the tasks from the set if it restarted. However, the maintained set could be discarded on JobMaster Failover, which decrease the complexity. Besides, this option avoid the cascade finish problem. 

For the first version we would go with the option 1 to avoid complex implementation involving new RPC and duplicate finished state.

Extend the task Barrier Alignment

...