DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
| Format | Example | What is does |
|---|---|---|
| Exact IPv4 | 192.168.1.100 | Matches one address (already works) |
| Exact IPv6 | 2001:db8::1 | Matches one address (already works) |
| Wildcard | * | Matches everything (already works) |
| IPv4 CIDR | 192.168.0.0/24 | Matches a subnet (new) |
| IPv6 CIDR | 2001:db8::/32 | Matches a subnet (new) |
Admin Client API
For programmatic use via the Admin Client, nothing changes in the API signaturesNo changes to the Admin interface or CreateAclsResult. CIDR notation goes where a host would normally go:
...
| language | java |
|---|
...
in the host field of AccessControlEntry exactly as an exact IP would.
Metadata Version Gating
Moreover, we propose adding We add a new metadata version to gate this feature. This is important because older brokers will not understand CIDR patterns, and we need to prevent users from creating ACLs that would break things during a rolling upgrade.
...
| language | java |
|---|
...
.
...
Attempting to create a CIDR ACL before the cluster is ready results in a clear error:
| Code Block | ||
|---|---|---|
| ||
org.apache.kafka.common.errors.UnsupportedVersionException: CIDR-based ACL host patterns require metadata version IBP_4_X_IVZ or higher. Current cluster metadata version: IBP_4_0_IV0 |
Note that didMetadataChange is set to true for IBP_4_4_IV0. This means that downgrading below this version is automatically classified as an unsafe metadata downgrade.
Proposed Changes
1. Host Matching Logic
Today, StandardAuthorizerData.java does simple string comparison :
| Code Block | ||
|---|---|---|
| ||
// Current code
if (!acl.host().equals(WILDCARD) && !acl.host().equals(host)) {
return null;
} |
We extend this to handle CIDR:
| Code Block | ||
|---|---|---|
| ||
// Check if the host matches. If it doesn't, return no result (null).
if (!acl.host().equals(WILDCARD) && !acl.host().equals(host) && !hostMatchesCidr(host, acl.host())) {
return null;
} |
The hostMatchesCidr method handles CIDR matching for both IPv4 and IPv6:
...
| language | java |
|---|
...
and we would extend this to handle CIDR.
2. Validation Rules
When creating ACLs, we validate CIDR patterns in AclControlManager.java:
| Code Block | ||
|---|---|---|
| ||
/**
* Validates the host pattern of an ACL entry.
*
* Accepts:
* - Wildcard "*" (matches any host)
* - Valid IPv4 address (e.g., "192.168.1.1")
* - Valid IPv6 address (e.g., "2001:db8::1")
* - Valid IPv4 CIDR notation (e.g., "192.168.0.0/24"), which requires cidrSupported=true
* - Valid IPv6 CIDR notation (e.g., "2001:db8::/32"), which requires cidrSupported=true
*
* @param host The host pattern to validate
* @param cidrSupported Whether CIDR notation is supported by the current metadata version
* @throws InvalidRequestException if the host pattern is invalid
* @throws UnsupportedVersionException if CIDR notation is used but not supported
*/
static void validateHostPattern(String host, boolean cidrSupported) {
if (host == null || host.isEmpty()) {
throw new InvalidRequestException("Host pattern cannot be null or empty");
}
if ("*".equals(host)) {
return;
}
if (host.contains("/")) {
if (!cidrSupported) {
throw new UnsupportedVersionException(
"CIDR-based ACL host patterns require metadata version " +
MetadataVersion.IBP_4_X_IVZ + " or higher.");
}
validateCidrNotation(host);
}
}
/**
* Validates a CIDR notation pattern.
* Supports both IPv4 (e.g., "192.168.0.0/24") and IPv6 (e.g., "2001:db8::/32") CIDR patterns.
*
* @param cidrPattern The CIDR pattern to validate
* @throws InvalidRequestException if the CIDR pattern is invalid
*/
static void validateCidrNotation(String cidrPattern) {
try {
if (cidrPattern.contains(":")) {
new SubnetUtils6(cidrPattern);
} else {
new SubnetUtils(cidrPattern);
}
} catch (IllegalArgumentException e) {
throw new InvalidRequestException("Invalid CIDR notation '" + cidrPattern + "': " + e.getMessage());
}
} |
...
the MetadataVersion is passed into AclControlManager.createAcls() (following the same pattern as DelegationTokenControlManager) and threaded through to validateNewAcl(), which calls validateHostPattern():
This ensures that:
- (i.) IPv4 prefix length is 0-32 (enforced
...
- by
SubnetUtils); - (ii.) IPv6 prefix length is 0-128 (enforced
...
- by
SubnetUtils6) and - (iii.) invalid patterns are rejected with clear error message.
Apache Commons Net and its SubnetUtils class handles IPv4 CIDR matching nicely as well as SubnetUtils6 as SubnetUtils6 for IPv6. Moreover, we need to add the dependency (i.e., commons-net) into Kafka.
QuorumController.createAcls() passes featureControl.metadataVersionOrThrow() to AclControlManager.createAcls(), consistent with how other managers such as DelegationTokenControlManager receive the metadata version.
DENY priority for overlapping CIDR ACLs (IPv4 and IPv6)
The existing Kafka ACL evaluation semantics (i.e., where DENY always takes precedence over ALLOW), regardless of specificity apply unchanged to CIDR-based host patterns for both IPv4 and IPv6. If a client IP matches both an ALLOW CIDR and a DENY CIDR, the request is denied, regardless of prefix length. For example, an ALLOW on 2001:db8::/32 combined with a DENY on 2001:db8:abcd::/48 will deny any client within the /48 range, even though it also falls within the broader /32 ALLOW. So to be even more concrete:
- Client at 2001:db8:abcd::5 matches both ALLOW /32 and DENY /48 results in DENIED (i.e., DENY always wins)
- Client at 2001:db8:ffff::1 matches only ALLOW /32, not the DENY /48 results in ALLOWED
This is consistent with how Kafka already handles overlapping exact-IP ALLOW and DENY entries, and no new priority rules are introduced.
Compatibility
Existing ACLs work exactly as before (i.e., no changes to exact IP or wildcard matching). The CIDR matching logic is additive and only activates when an ACL host pattern contains a '/' character.
...
CIDR ACLs cannot be created until all brokers supported support them. This is enforced at ACL creation time in AclControlManager.validateHostPattern(). via MetadataVersion. Any attempt to create a CIDR-based ACL on an older metadata version results in a an UnsupportedVersionException(where an user will be warned with the specific version , which is required). This guarantees that CIDR host patterns never appear in AccessControlEntryRecord entries the metadata log on clusters where brokers may not understand them.
...
Since CIDR host patterns are persisted in AccessControlEntryRecord metadata records, allowing metadata version downgrade while such records exist would leave the cluster in an inconsistent state (i.e., the ACLs would be still stored and replayed, but older brokers would not regnonize the CIDR format and would fail to match correctly.
So to prevent this we introduce pre-downgrade validation step. Before any metadata version downgrade is applied, FeatureControlManager invokes a validator registered by QuorumController that checks whether CIDR ACLs currently exists. If the target metadata version does not support CIDR and any ACL host pattern contains CIDR notation, the downgrade is rejected with the error:
...
| language | java |
|---|
...
metadata records, IBP_4_4_IV0 is defined with didMetadataChange = true. This means any downgrade below IBP_4_4_IV0 is automatically treated as an unsafe metadata downgrade. No additional pre-downgrade validation hook is needed.
IPv4-mapped IPv6 address
Javas Java’s networking stack automatically resolves IPv4-mapped IPv6 addresses (e.g., ::ffff:192.168.0.5) to their native IPv4 form (192.168.0.5). As a result, the host address seen by the authorizer is always the plain IPv4 address. Administrators should use IPv4 CIDR notation (e.g., 192.168.0.0/24) for IPv4 subnets and native IPv6 CIDR notation (e.g., 2001:db8::/32) for IPv6 subnets.
Creating ACLs using IPv4-mapped IPv6 CIDR patterns such as is not supported:
- (i.) ::ffff:192.168.0.0/
...
- 24 (RFC 4291 IPv4-mapped form): the JVM normalizes ::ffff:x.x.x.x to a plain IPv4 address, but the CIDR string still contains the ::ffff: prefix which is not valid IPv4 notation. This is rejected at ACL creation time with an InvalidRequestException.
- (ii.) ::192.168.0.0/24 (deprecated RFC 4291 §2.5.5.1 IPv4-compatible form): this is interpreted as a native IPv6 address (0:0:0:0:0:0:c0a8:0) and passes validation as a regular IPv6 CIDR. However, a /24 on a 128-bit IPv6 address masks the top 24 bits, covering 0:: through 0:ff:ffff:ffff:ffff:ffff:ffff:ffff, which is an enormous and almost certainly unintended range. It would match IPv6 clients within that range (e.g., ::1) but would not match plain IPv4 clients (e.g., 192.168.1.5) since IPv4 clients are always evaluated against IPv4 CIDR rules, not IPv6.
Test plan
All related stuff within subnet handling is tested via commons-net library via RFC examples for IPv6 and IPv4 (i.e., https://datatracker.ietf.org/doc/html/rfc5952, https://datatracker.ietf.org/doc/html/rfc1519). In AclControlManagerTest, we will cover validation of host patterns (i.e., valid/invalid IPv4 or IPv6 CIDR, null/empty inputs and even malformed prefixes) and metadata version gating (i.e., CIDR rejected on older version, accepted on IBP_x_x_IVx, end-to-end ACL creation with CIDR hosts and backwards compatibility with exact IPs and wildcards on older versions. Moreover, in FeatureControlManagerTest we will cover pre-downgrade validation mechanism i.e, block metadata version downgreade when validator returns an error, allows when downgrade pass. Also in StandardAuthorizerTest, CIDR host matching for IPv4 and IPv6 (boundary addresses, range membership, invalid/null patterns), full authorizer flow iterating all addresses in a /24 and /120 range, and overlaping CIDR ACLs semantics which would confirm DENY always takes precedence regardless of prefix length. The downgrade lifecycle will be tested in AclControlManagerTest (i.e., creating CIDR ACL (on supported version), attempting and failing to downgrade (to un-supported version), removing ACL, then succesfully downgrading).
...
Custom Implementation from Scratch We considered writing all the IP parsing and matching code ourselves, but there's no reason to reinvent the wheel when Apache Commons Net already handles IPv4 well. We just need to add IPv6 support.
Pre-downgrade validator hook We initially implemented a preDowngradeValidator callback in FeatureControlManager that QuorumController would register to check for existing CIDR ACLs before allowing a metadata version downgrade. This was replaced by setting didMetadataChange = true on IBP_4_4_IV0, which gives the same protection through the existing unsafe-downgrade mechanism without introducing bespoke validation infrastructure.