Status
Current state: Accepted
Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-10-Unify-Savepoints-and-Checkpoints-tt13149.html
...
Page properties | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
With this FLIP, I propose to allow to unify checkpoints and savepoints by allowing savepoints to be triggered automatically.
Proposed Changes
Operation | Manual Savepoint Flink 1.1 | Automatic with Checkpoints FLIP-10 | Periodic Savepoints FLIP-10 |
---|---|---|---|
Trigger Savepoint | Manual | Automatic | Automatic |
Dispose Savepoint | Manual | Automatic | Manual |
Automatic with Checkpoints
Persistent Checkpoints
The checkpoint coordinator has a fixed-size FIFO queue of completed checkpoints that are retained (current default size is 1). Checkpoints are discarded when they are removed from this queue. I propose to store allow persisting these checkpoints as like savepoints. This means that if a job fails permanently the user will have a savepoint checkpoint available to restore from. When a newer checkpoint completes, old savepoints will be automatically discardedThese can be subsumed like regular checkpoints..
As an example think of the following scenario: a job runs smoothly until it hits a bad record that it cannot handle. The current behaviour will be that the job will try to recover, but it will hit the bad record again and keep on failing. With the proposed change, some recent checkpoint is stored as a savepoint and the user can update the program to handle bad records and restore from the savepoint.
...
Allow the user to configure periodic triggering of savepoints. The behaviour should be similar to the periodic checkpoints with an extra configuration value for the maximum number of retained periodic savepoints. These savepoints are only discarded when the queue is full.manually triggering savepoints periodically. Furthermore, we bound the number of available periodic savepoints. On shut down, they are never removed.
What's the Difference?
Although persistent checkpoints and periodic savepoints look similar, persistent checkpoints have a major difference: they can only be recovered with the same job and Flink version whereas savepoints allow to modify both the job and Flink version.
Miscellaneous
Always Create CheckpointCoordinator
Currently, the checkpoint coordinator is only created if checkpointing is enabled. This means that it is not possible to trigger a savepoint for stateful jobs that don't have periodic checkpointing enabled.
...
Furthermore, there is the option to specify the directory ad-hoc per savepoint (via CheckpointConfig
or the CLI, see below). This ad-hoc value has precedence over the default value. If no savepoint path is configured (no default and no ad-hoc), the savepoint will fail.
...
Default Directory
Configured?
...
...
Extend CheckpointConfig
Code Block |
---|
// Stores completed checkpoints as savepoints @PublicEvolving void enablePersistentCheckpoints(String path); @PublicEvolving void disablePersistentCheckpoints(); // Enables periodic savepoints @PublicEvolving void enablePeriodicSavepoints(long interval, TimeUnit unit, String savepointPathpath): @PublicEvolving void disablePeriodicSavepoints(); // Sets the maximum number of retained periodic savepoints @PublicEvolving void setMaximumNumberOfRetainedPeriodicSavepoints(int max); |
Furthermore we can extend the StreamExecutionEnvironment
with shortcuts:
Code Block |
---|
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); env.enableSavepoints(); env.enableSavepointsenablePersistentCheckpoints(String savepointPathpath); env.disableSavepointsdisablePersistentCheckpoints(); |
Add Optional Savepoint Path Argument to CLI
...
Code Block |
---|
bin/flink savepoint <JobID> [<SavepointPath<Path>] |
Add REST API and Logging
...