Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

log.message.timestamp.after.max.ms: This configuration defines that the message timestamp must always can not be later than or equal to the broker's timestamp, with a maximum allowable difference determined by the value of log.message.timestamp.after.max.ms.

...

Code Block
languagepy
titleNew Validation Logic
if message.timestamp <= broker.timestamp:
    time_difference = broker.timestamp - message.timestamp
    if time_difference <= log.message.timestamp.before.max.ms:
        # Validation passed for timestamp before
        //# Proceed with further logic
    else:
        # Validation failed for timestamp before
        return Error code 32 (INVALID_TIMESTAMP)
else if message.timestamp > broker.timestamp:
    time_difference = message.timestamp - broker.timestamp
    if time_difference <= log.message.timestamp.after.max.ms:
        # Validation passed for timestamp after
        //# Proceed with further logic
    else:
        # Validation failed for timestamp after
        return Error code 32 (INVALID_TIMESTAMP)

...

Compatibility, Deprecation, and Migration Plan

  • The configuration log.message.timestamp.difference.max.ms will be deprecated.
  • The semantics of the configuration regarding time differences in the past and future were not explicitly defined with the configuration log.message.timestamp.difference.max.ms. With this KIP, message timestamps with past and future values are now validated using different default values, defined by the configurations log.message.timestamp.before.max.ms and log.message.timestamp.after.max.ms respectively.
  • After this KIP messages with future timestamps are validated using the new configuration log.message.timestamp.after.max.ms. This will result in for some messages to be rejected that were previously accepted. We consider this an acceptable breaking change.


There are no other changes to public interfaces that will impact clients.

...

  • Lack of Flexibility: Users may have use cases that require extending the future timestamp validation beyond the default one hour, and the hardcoded constant value restricts this flexibility.
  • Compatibility Issues: The hardcoded constant value of one hour can create compatibility issues when upgrading existing clusters or integrating with existing producers.

...