Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Currently, Kafka Streams does not support ProcessingExceptionHandler for GlobalKTable processors. When a processing exception occurs during GlobalKTable record processing, the GlobalStreamThread fails and terminates, causing the entire Kafka Streams application to shut - down. This behavior behaviour is inconsistent with regular KStream/KTable processing, where ProcessingExceptionHandler allows applications to handle exceptions gracefully and continue processing.

Public Interfaces

`default.processing.exception.handler.invoke.for.global` (Type: boolean, Default: false)
    - Controls whether the ProcessingExceptionHandler is invoked for GlobalKTable processing exceptions
    - When false (default), maintains backwards-compatible behaviour where global exceptions terminate the application
    - When true, enables the ProcessingExceptionHandler for GlobalKTable exceptions
    - **Deprecated immediately upon introduction** - This config will be removed in Kafka Streams 5.0, where global exception handling will be enabled by defaultNo changes in public interfaces. We will be reusing existing interfaces. 

Proposed Changes

This KIP extends the applicability of the existing ProcessingExceptionHandler to GlobalKTable processors. Currently, this exception handler only applies to regular stream processing (KStream/KTable). After this KIP, the same handler will also handle also handle processing exceptions in GlobalKTable. The feature is gated behind a configuration flag to prevent unexpected behaviour in existing handler implementations that may not be designed to handle GlobalKTable exceptions, which could cause crashes or undesired side effects. This provides a safe migration path for users to test and adapt their exception
  handling logic.

Exception Handling Scope

After this KIP, the ProcessingExceptionHandler configured via `default.processing.exception.handler` will be invoked for:

...

Add ProcessingExceptionHandler as a constructor parameter to GlobalStateUpdateTask and pass it to processor initializationinitialisation, mirroring the existing DeserializationExceptionHandler implementation.

...

Compatibility, Deprecation, and Migration Plan

This change is fully backward compatible:
  - Applications without a configured handler experience no behavior change
  - Applications with a configured handler automatically get GlobalKTable support
  - The handler interface is not modified - The handleError() method (introduced in KIP-1034) will be called for GlobalKTable exceptions. 
  - No new configuration properties are introduced- Fully backwards compatible by default - existing behaviour is preserved unless config is explicitly enabled
  - New configuration `default.processing.exception.handler.invoke.for.global` defaults to `false`
  - Config is deprecated immediately upon introduction
  - In Kafka Streams 5.0, this config will be removed, and the handler will always be invoked for GlobalKTable exceptions
  - Migration path: Users should test with the config enabled before upgrading to 5.0

Test Plan

Since this is a fairly small change, unit tests and integration test tests should be sufficient.

Rejected Alternatives

  • Create a new ProcesseExceptionalHandler ProcessExceptionalHandler interface for the global thread which require introducing new configuration at the stream config level. This would create unnecessary operational overload for the end users as it require to implement multiple interfaces.

...