Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Property pattern

Type

Default

Description

Excluded properties
config.storage.<topic-specific-setting>
severalbroker value

Additional topic-specific settings used when creating the internal Kafka topic where Connect stores connector configurations. Here "<topic-specific-setting>" corresponds to any of the must be any valid Kafka topic-level configurations for the version of the Kafka broker where the topic should be created; the Connect worker will fail upon startup if the "<topic-specific-setting>" is not known to the broker.

  • partitions (always set to 1)
  • cleanup.policy (always set to `compact`)
offset.storage.<topic-specific-setting>
severalbroker valueAdditional topic-specific settings used when creating the internal Kafka topic where Connect stores source offsets for source connectors. Here  Here "<topic-specific-setting>" corresponds to any of the must be any valid Kafka topic-level configurations for the version of the Kafka broker where the topic should be created; the Connect worker will fail upon startup if the "<topic-specific-setting>" is not known to the broker.
  • cleanup.policy (always set to `compact`)
status.storage.<topic-specific-setting>
severalbroker valueAdditional topic-specific settings used when creating the internal Kafka topic where Connect stores connector and task statuses. Here  Here "<topic-specific-setting>" corresponds to any of the must be any valid Kafka topic-level configurations for the version of the Kafka broker where the topic should be created; the Connect worker will fail upon startup if the "<topic-specific-setting>" is not known to the broker.
  • cleanup.policy (always set to `compact`)

...

Note that some topic-specific properties are excluded because the distributed worker always sets specific values. Therefore, if a distributed worker configuration does set any of these excluded properties, the distributed worker will issue a warning that such properties should not be set and will be ignored.

The Connect worker will fail upon startup if any of the topic-settings specified in the above configurations are not known to the Kafka broker.

This proposal does not otherwise change any other behavior of how or when the Connect worker creates internal topics. This proposal also does not affect Connect standalone behavior.

...

These changes are backward compatible, and existing Connect distributed worker configurations will continue to work with no change in behavior, unless those configurations define unknown topic settings using any of the patterns defined above, in which case the Connect worker or MirrorMaker2 process will fail upon startup. All new properties are prefixed with one of three prefixes (e.g., `config.storage.`, `offset.storage.`, and `status.storage.`) already used for other worker-level properties, and thus are not expected to clash with any properties used for REST Extensions or config providers.

...