Page History

...

1. The task (StreamTask, StandbyTask) registers its state stores. State stores load offset metadata from the checkpoint file (link). That step aims to establish a mapping between data in the state store and the offset of the changelog topic.
   1. In case of crash failure, if the state store has data, but the checkpoint file does not exist, ProcessorStateManager throws an exception for EOS tasks. This is an indicator to throw away local data and replay the changelog topic (link).
2. The task processes data and writes its state locally.
3. The task commits EOS transaction. TaskExecutor#commitOffsetsOrTransaction calls StreamsProducer#commitTransaction that sends new offsets and commits the transaction.
4. The task runs a postCommit method (StreamTask, StandbyTask) that:
   1. flushes the state stores and
   2. updates the checkpoint file (link) for non-EOS tasks (link).
5. The Go to step 2 until task shuts down. It During shutdown, the task stops processing data, then writes its current offset to the checkpoint file and halts.

...