Page History

...

Component	Who is responsible?	Why?
`JobResultStore`	`Dispatcher`	It needs to work without a `JobMaster`.
`JobGraphStore`	`Dispatcher`	We need to be able to fetch a `JobGraph` for recovery before starting `JobMaster`.
`HighAvailableServices` JM LeaderElection RM LeaderElection	`Dispatcher`	We can clean this up after there won’t be any more contenders.
`BlobStore` `BlobServer` (for a given job)	`Dispatcher`User JARs. This is needed for recovery.
`CheckpointCounter`/`CheckpointStore`	`JobMaster`	This is tied to a single job execution (JobMasterLifecycle).
`JobMaster`	`Dispatcher`	`Dispatcher` is responsible for `JobMaster` management.

The plan is to unite the cleanup logic in a single component that is, i.e. the Dispatcher, as it is also the component being in charge of accessing the JobResultStoreto provide a common interface that enables all components to trigger the cleanup based on the JobID. This cleanup component can encapsulate the retry mechanism. The actual cleanup can then be triggered either in the JobMaster or the Dispatcher depending on whether a JobGraph is still available or not.

Implementation

Interface

draw.io Diagram

border	true

diagramName	FLIP-140 - Interfaces
simpleViewer	false
width
links	auto
tbstyle	top
lbox	true
diagramWidth	561
revision	2

...

Page tree

Versions Compared

Old Version 5

New Version 6

Key

Implementation

Interface