THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Info |
---|
We need to wait for a checkpoint to complete, that started after the finish() method. However, we support concurrent checkpoints. Moreover there is no guarantee the notifyCheckpointComplete arrives or the order in which they will arrive. It should be enough though to wait for notification for any checkpoint that started after finish(). |
Skip Waiting for the Final Checkpoint If possible
For the final checkpoint case, there is one possibility that users in fact do not use the 2pc, thus for the source finished case the following task do not need to wait for the final checkpoint. As an optimization, to allow users specify such logic, we could enhance the StreamOperator and CheckpointListener interface with the following changes:
Code Block | ||
---|---|---|
| ||
interface StreamOperator {
default boolean requiresFinalCheckpoint() {
return true;
}
}
interface CheckpointListener {
default boolean requiresFinalCheckpoint() {
return true;
}
}
class AbstractUdfStreamOperator {
@Override
boolean requiresFinalCheckpoint() {
return userFunction instanceof CheckpointListener &&
((CheckpointListener) userFunction).requiresFinalCheckpoint();
}
} |
Rejected Alternatives
CheckpointCoordinator Maintains the Finished Status of Tasks Separately
...