You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Status

Current state: Under Discussion

Discussion thread: TBD

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently the ERROR state in KStreams means that there are no threads running. However, in KIP-663 streamThreads can be added dynamically so this definition no longer is useful. In that kip, the automatic transition to ERROR state upon the death of all threads is removed. Also, ERROR state should be terminal. This KIP should clarify the use of ERROR state as well as bring it into alignment with the other states

Public Interfaces

KafkaStreams will have added State PENDING_ERROR.

The following transitions will be added:

  • RUNNING → PENDING_ERROR
  • REBALANCING → PENDING_ERROR
  • PENDING_ERROR → ERROR

The following transitions will be removed:

  • ERROR → PENDING_SHUTDOWN

The SHUTDOWN_CLIENT should leave the client state in ERROR instead of NOT_RUNNING

Proposed Changes

ERROR will be redefined to mean that the streams client is in an unrecoverable state and should not be restarted until the problem has been investigated. Streams will only reach ERROR state in the event of an exceptional failure in which the `StreamsUncaughtExceptionHandler` chose to either shutdown the application or the client. It is not recommended to automatically restart from ERROR state.

As ERROR will now be a terminal state, PENDING_ERROR will be added. This will mirror PENDING_SHUTDOWN and mean that resources are closing before the client transitions to ERROR. Currently, the client goes to ERROR before it closes the resources and does not signal when done.

In order to be consistent, SHUTDOWN_CLIENT will leave the client state in ERROR instead of NOT_RUNNING. ERROR should be the state that exceptional failures leave the application in, not NOT_RUNNING.

Close() called on ERROR will be idempotent and not throw an exception.

Compatibility, Deprecation, and Migration Plan

  • These changes will affect the state machine. However this has already changed this release. 
  • The old behavior is already deprecated

Rejected Alternatives

  • not having PENDING_ERROR
  • No labels