Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Status

Current State: Under Discussion

...

JIRA:

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-20088

Motivation

Managing Kafka ACLs in large environments is painful: Kafka only allows specifying exact IP addresses or a wildcard (*) for host-based access control. With 50 clients in a subnet, this means 50 separate ACL entries. Not fun. This becomes especially frustrating when the goal is simply to say "allow anything from 10.0.0.0/8" like every other system does.

...

  1. Proper metadata version gating for safe cluster-wide rollout
  2. Full IPv4 and IPv6 CIDR support because it is 2026, and we shoul not ignore IPv6
  3. A practical implementation path using Apache Commons Net with IPv6 support (currently not implemented but proposed changes to extend Apache Commons Net with IPv6 support.)

Public Interfaces

What Changes for Users

The --allow-host and --deny-host options in kafka-acls.sh will accept CIDR notation alongside the existing formats:

FormatExampleWhat is does
Exact IPv4192.168.1.100Matches one address (already works)
Exact IPv62001:db8::1Matches one address (already works)
Wildcard*Matches everything (already works)
IPv4 CIDR192.168.0.0/24Matches a subnet (new)
IPv6 CIDR2001:db8::/32Matches a subnet (new)

Admin Client API

For programmatic use via the Admin Client, nothing changes in the API signatures. CIDR notation goes where a host would normally go:

Code Block
languagejava
AclBinding binding = new AclBinding(
    new ResourcePattern(ResourceType.TOPIC, "events", PatternType.LITERAL),
    new AccessControlEntry("User:app", "10.0.0.0/8", AclOperation.READ, AclPermissionType.ALLOW)
);
adminClient.createAcls(Collections.singleton(binding));

Metadata Version Gating

Moreover, we propose adding a new metadata version to gate this feature. This is important because older brokers will not understand CIDR patterns, and we need to prevent users from creating ACLs that would break things during a rolling upgrade.

...

Code Block
languagejava
org.apache.kafka.common.errors.UnsupportedVersionException:
CIDR-based ACL host patterns require metadata version IBP_4_X_IVZ or higher.
Current cluster metadata version: IBP_4_0_IV0

Proposed Changes

1. Host Matching Logic

Today, StandardAuthorizerData.java does simple string comparison:

...

Code Block
languagejava
static boolean hostMatchesCidr(String host, String cidrPattern) {
    if (cidrPattern == null || !cidrPattern.contains("/")) {
        return false;
    }

    try {
        if (cidrPattern.contains(":")) {
            SubnetUtils6 subnet = new SubnetUtils6(cidrPattern);
            return subnet.getInfo().isInRange(host);
        } else {
            SubnetUtils subnet = new SubnetUtils(cidrPattern);
            subnet.setInclusiveHostCount(true);
            return subnet.getInfo().isInRange(host);
        }
    } catch (IllegalArgumentException e) {
        return false;
    }
}

2. Validation Rules

When creating ACLs, we validate CIDR patterns in AclControlManager.java:

...

Apache Commons Net and its SubnetUtils class handles IPv4 CIDR matching nicely. But unfortunately, it does not support IPv6 yet as previously mentioned above. Therefore, our approach is to wait for proposed SubnetUtils6 addition to Apache Commons Net and then use it directly when the dependency is available.

Compatibility

Backward Compatibility

  • Existing ACLs work exactly as before - no changes to exact IP or wildcard matching
  • Mixed-version clusters are safe - CIDR ACLs cannot be created until all brokers support them (enforced by metadata version)
  • Downgrades require cleanup - CIDR ACLs must be removed before downgrading to an older version

Rejected Alternatives

IP Range Notation (e.g., 10.0.0.1-10.0.0.100) KIP-252 proposed supporting arbitrary IP ranges. We are not doing this because CIDR covers the vast majority of real-world use cases, and adding range support would complicate the implementation without much benefit.

...