DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
| Table of Contents |
|---|
Status
Current state: Under Discussion Accepted
Discussion thread: None https://lists.apache.org/thread/6zsr70lpzqp96g4jf6kngyms4kkz1tp1
Vote thread: https://lists.apache.org/thread/dtc48g4zjdj4s4n0hm99hm0xoho5sp1n
JIRA:
| Jira | ||||||||
|---|---|---|---|---|---|---|---|---|
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
Motivation
Since
...
Apache
...
Kafka
...
4.0,
...
ZooKeeper
...
has
...
been
...
removed
...
and
...
KRaft
...
is
...
the
...
only
...
supported
...
mode
...
(
...
...
).
...
In
...
KRaft
...
mode,
...
a
...
single
...
process
...
can
...
serve
...
as
...
a
...
broker,
...
a
...
controller,
...
or
...
both.
...
...
therefore
...
introduced
...
node.id
...
to
...
replace
...
broker.id
...
as
...
the
...
canonical
...
node
...
identifier.
However,
...
the
...
legacy
...
broker.id
...
configuration
...
still
...
exists
...
and
...
is
...
treated
...
as
...
a
...
synonym
...
of
...
node.id
...
.
...
If
...
only
...
one
...
is
...
set,
...
the
...
other
...
is
...
automatically
...
populated
...
with
...
the
...
same
...
value.
...
If
...
both
...
are
...
explicitly
...
set,
...
they
...
must
...
be
...
equal;
...
otherwise,
...
a
...
ConfigException
...
is
...
thrown.
Having
...
two
...
configuration
...
properties
...
that
...
must
...
always
...
hold
...
the
...
same
...
value
...
causes
...
unnecessary
...
confusion and contradicts the documentation, which describes node.id
...
as
...
required
...
in
...
KRaft
...
mode
...
.
...
Related
...
ZooKeeper-era
...
configurations
...
such
...
as
...
broker.id.generation.enable
...
and
...
reserved.broker.max.id
...
were
...
already
...
removed
...
in
...
Kafka
...
4.0,
...
making
...
broker.id
...
the
...
only
...
remaining
...
legacy
...
identifier
...
config.
...
We
...
should
...
deprecate
...
it
...
to
...
simplify
...
the
...
configuration
...
surface
...
and
...
guide
...
all
...
users
...
toward
...
the
...
single,
...
canonical
...
configuration:
...
node.id
...
.
...
Public
...
Interfaces
This
...
KIP
...
proposes
...
to
...
deprecate
...
the
...
broker.id
...
server
...
configuration
...
property
...
in
...
Apache
...
Kafka
...
4.
...
4 and
...
remove
...
it
...
in
...
Apache
...
Kafka
...
5.0.
...
Config | Default | 4.4 | 5.0 |
|---|
...
...
|
...
-1 |
...
Deprecated | Removed | |
|
...
(required) |
...
No |
...
change |
...
No |
...
change |
...
...
Notes:
- The default value of
-1forbroker.idis effectively unused in KRaft mode. Ifnode.idis set, the synonym mechanism overwritesbroker.idwith the value ofnode.id. If neither is set,node.idvalidation fails with aConfigExceptionbefore the default is ever used. - The documentation describes
node.idas required in KRaft mode, but it can actually be omitted currently ifbroker.idis set, because the synonym mechanism automatically populatesnode.idfrombroker.id. After the removal in 5.0,node.idmust be explicitly set.
The behavior is the same for all process.roles (broker, controller, or combined). No changes to public APIs, network protocols, metrics, or command-line tools.
Proposed Changes
Phase 1: Deprecation (Apache Kafka 4.4)
- Log deprecation warning: When
broker.idis explicitly set in the server configuration, log a WARN-level message at startup:Code Block The 'broker.id' configuration is deprecated and will be removed in Apache Kafka 5.0. Please use 'node.id' instead.
...
- Preserve backward compatibility: The existing synonym mechanism will continue to function. Users who only set
broker.idwill still have it automatically copied tonode.id. The validation requiring both to be equal (if both are explicitly set) also remains unchanged.
Phase 2: Removal (Apache Kafka 5.0)
- Remove
broker.idconfiguration: TheBROKER_ID_CONFIGwill be removed fromServerConfigs. Setting it will be silently ignored. - Remove synonym logic: Remove the
broker.id/node.idsynonym handling fromAbstractKafkaConfig.populateSynonyms(). - Retain internal
brokerId()accessor: ThebrokerId()method will continue to exist for internal use, delegating tonodeId().
Behavior Matrix
The following table summarizes the expected behavior for each configuration scenario across version ranges:
Configuration | Current (4.0 ~ 4.2) | Phase 1: Deprecated (4.4 ~ 4.x) | Phase 2: Removed (5.0) |
|---|---|---|---|
Only | Valid | Valid | Valid |
Only | Valid; | Valid + |
|
Both set, same value | Valid | Valid + | Valid; |
Both set, different values |
|
| Valid; |
Neither set |
|
|
|
Note: In the 5.0 column, "ignored" means broker.id is treated as an unrecognized configuration and silently ignored. The Kafka server does not log warnings for unrecognized configurations.
Compatibility, Deprecation, and Migration Plan
Migration Steps
Users should replace broker.id with node.id in their server configuration. If both are configured, simply remove broker.id.
Downgrade Considerations
Downgrading from 5.0 to 4.x is safe as long as the same configuration file is used — any config that was valid on 4.x before the upgrade remains valid after the rollback. The only edge case is if broker.id and node.id are set to different values on a 5.0 binary (where broker.id is ignored) and then downgraded to 4.x, which would result in a ConfigException. In practice, this cannot happen if the config was originally used on 4.x.
Test Plan
Phase 1: Deprecation (Apache Kafka 4.4)
Unit tests to verify:
- A deprecation warning is logged when
broker.idis explicitly set (with or withoutnode.id). - Backward compatibility: setting only
broker.idstill works (existing tests inKafkaConfigTestalready create configs with onlybroker.idset).
Phase 2: Removal (Apache Kafka 5.0)
Unit tests to verify:
broker.idis no longer recognized and is silently ignored.- Setting only
broker.idwithoutnode.idresults in aConfigException.
Rejected Alternatives
Throw ConfigException when broker.id is set in 5.0
Instead of silently ignoring broker.id in 5.0, we could explicitly throw a ConfigException. However, this would force users to update their configuration files before upgrading, making the upgrade process more disruptive. Silently ignoring unrecognized configurations is consistent with how other removed configurations are handled in Kafka.
Log a customized warning when broker.id is set in 5.0
We could add special-case logic to detect and warn users that broker.id has been removed. However, the Kafka server does not log warnings for unrecognized configurations, and adding a one-off check just for broker.id would be inconsistent with how other removed configurations are handled. The deprecation warning in Phase 1 already provides sufficient notice for users to migrate.