You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next » 

{toc}

h2. Status

||Item||Value||
|Current State|Under Discussion|
|Discussion Thread|[TBD|https://lists.apache.org/list.html?dev@kafka.apache.org]|
|JIRA|[TBD|https://issues.apache.org/jira/browse/KAFKA-XXXXX]|

h2. Motivation

Managing Kafka ACLs in large environments is painful: Kafka only allows specifying exact IP addresses or a wildcard ({{*}}) for host-based access control. With 50 clients in a subnet, this means 50 separate ACL entries. Not fun. This becomes especially frustrating when the goal is simply to say "allow anything from {{10.0.0.0/8}}" like every other system does.

Two prior KIPs tried to solve this:

* [KIP-252|https://cwiki.apache.org/confluence/display/KAFKA/KIP-252+-+Extend+ACLs+to+allow+filtering+based+on+ip+ranges+and+subnets] proposed CIDR and IP range support but went stale without addressing metadata version gating.
* [KIP-753|https://cwiki.apache.org/confluence/display/KAFKA/KIP-753:++ACL+authentication,+Host+field+support+IP+network+segment] focused on CLI changes but didn't fully specify IPv6 or version gating.

This KIP picks up where they left off with:

# *Proper metadata version gating* for safe cluster-wide rollout
# *Full IPv4 and IPv6 CIDR support* because it is 2026, and we should not ignore IPv6
# *A practical implementation path* using Apache Commons Net with IPv6 support (currently not implemented but [proposed changes to extend Apache Commons Net with IPv6 support|https://github.com/apache/commons-net/pull/391])

h2. Public Interfaces

h3. What Changes for Users

The {{--allow-host}} and {{--deny-host}} options in {{kafka-acls.sh}} will accept CIDR notation alongside the existing formats:

||Format||Example||What it does||
|Exact IPv4|{{192.168.1.100}}|Matches one address (already works)|
|Exact IPv6|{{2001:db8::1}}|Matches one address (already works)|
|Wildcard|{{*}}|Matches everything (already works)|
|*IPv4 CIDR*|{{192.168.0.0/24}}|Matches a subnet (new)|
|*IPv6 CIDR*|{{2001:db8::/32}}|Matches a subnet (new)|

h3. Admin Client API

For programmatic use via the Admin Client, nothing changes in the API signatures. CIDR notation goes where a host would normally go:

{code:java}
AclBinding binding = new AclBinding(
    new ResourcePattern(ResourceType.TOPIC, "events", PatternType.LITERAL),
    new AccessControlEntry("User:app", "10.0.0.0/8", AclOperation.READ, AclPermissionType.ALLOW)
);
adminClient.createAcls(Collections.singleton(binding));
{code}

h3. Metadata Version Gating

Moreover, we propose adding a new metadata version to gate this feature. This is important because older brokers will not understand CIDR patterns, and we need to prevent users from creating ACLs that would break things during a rolling upgrade.

{code:java}
// In MetadataVersion.java
IBP_4_X_IVZ(XX, "4.X", "IVZ", true),

public boolean isCidrAclSupported() {
    return this.isAtLeast(IBP_4_X_IVZ);
}
{code}

Attempting to create a CIDR ACL before the cluster is ready results in a clear error:

{code}
org.apache.kafka.common.errors.UnsupportedVersionException:
CIDR-based ACL host patterns require metadata version IBP_4_X_IVZ or higher.
Current cluster metadata version: IBP_4_0_IV0
{code}

h2. Proposed Changes

h3. 1. Host Matching Logic

Today, {{StandardAuthorizerData.java}} does simple string comparison:

{code:java}
// Current code
if (!acl.host().equals(WILDCARD) && !acl.host().equals(host)) {
    return null;
}
{code}

We extend this to handle CIDR:

{code:java}
// Check if the host matches. If it doesn't, return no result (null).
if (!acl.host().equals(WILDCARD) && !acl.host().equals(host) && !hostMatchesCidr(host, acl.host())) {
    return null;
}
{code}

The {{hostMatchesCidr}} method handles CIDR matching for both IPv4 and IPv6:

{code:java}
static boolean hostMatchesCidr(String host, String cidrPattern) {
    if (cidrPattern == null || !cidrPattern.contains("/")) {
        return false;
    }

    try {
        if (cidrPattern.contains(":")) {
            SubnetUtils6 subnet = new SubnetUtils6(cidrPattern);
            return subnet.getInfo().isInRange(host);
        } else {
            SubnetUtils subnet = new SubnetUtils(cidrPattern);
            subnet.setInclusiveHostCount(true);
            return subnet.getInfo().isInRange(host);
        }
    } catch (IllegalArgumentException e) {
        return false;
    }
}
{code}

h3. 2. Validation Rules

When creating ACLs, we validate CIDR patterns in {{AclControlManager.java}}:

{code:java}
/**
 * Validates the host pattern of an ACL entry.
 *
 * Accepts:
 * - Wildcard "*" (matches any host)
 * - Valid IPv4 address (e.g., "192.168.1.1")
 * - Valid IPv6 address (e.g., "2001:db8::1")
 * - Valid IPv4 CIDR notation (e.g., "192.168.0.0/24"), which requires cidrSupported=true
 * - Valid IPv6 CIDR notation (e.g., "2001:db8::/32"), which requires cidrSupported=true
 *
 * @param host The host pattern to validate
 * @param cidrSupported Whether CIDR notation is supported by the current metadata version
 * @throws InvalidRequestException if the host pattern is invalid
 * @throws UnsupportedVersionException if CIDR notation is used but not supported
 */
static void validateHostPattern(String host, boolean cidrSupported) {
    if (host == null || host.isEmpty()) {
        throw new InvalidRequestException("Host pattern cannot be null or empty");
    }

    if ("*".equals(host)) {
        return;
    }

    if (host.contains("/")) {
        if (!cidrSupported) {
            throw new UnsupportedVersionException(
                "CIDR-based ACL host patterns require metadata version " +
                MetadataVersion.IBP_4_X_IVZ + " or higher.");
        }
        validateCidrNotation(host);
    }
}

/**
 * Validates a CIDR notation pattern.
 * Supports both IPv4 (e.g., "192.168.0.0/24") and IPv6 (e.g., "2001:db8::/32") CIDR patterns.
 *
 * @param cidrPattern The CIDR pattern to validate
 * @throws InvalidRequestException if the CIDR pattern is invalid
 */
static void validateCidrNotation(String cidrPattern) {
    try {
        if (cidrPattern.contains(":")) {
            new SubnetUtils6(cidrPattern);
        } else {
            new SubnetUtils(cidrPattern);
        }
    } catch (IllegalArgumentException e) {
        throw new InvalidRequestException("Invalid CIDR notation '" + cidrPattern + "': " + e.getMessage());
    }
}
{code}

This ensures that:
* IPv4 prefix length is 0-32 (enforced by {{SubnetUtils}})
* IPv6 prefix length is 0-128 (enforced by {{SubnetUtils6}})
* Invalid patterns are rejected with clear error message

Apache Commons Net and its {{SubnetUtils}} class handles IPv4 CIDR matching nicely. But unfortunately, it does not support IPv6 yet as previously mentioned above. Therefore, our approach is to wait for [proposed SubnetUtils6 addition|https://github.com/apache/commons-net/pull/391] to Apache Commons Net and then use it directly when the dependency is available.

h2. Compatibility

h3. Backward Compatibility

* *Existing ACLs work exactly as before* - no changes to exact IP or wildcard matching
* *Mixed-version clusters are safe* - CIDR ACLs cannot be created until all brokers support them (enforced by metadata version)
* *Downgrades require cleanup* - CIDR ACLs must be removed before downgrading to an older version

h2. Rejected Alternatives

h3. IP Range Notation (e.g., {{10.0.0.1-10.0.0.100}})

KIP-252 proposed supporting arbitrary IP ranges. We are not doing this because CIDR covers the vast majority of real-world use cases, and adding range support would complicate the implementation without much benefit.

h3. Custom Implementation from Scratch

We considered writing all the IP parsing and matching code ourselves, but there's no reason to reinvent the wheel when Apache Commons Net already handles IPv4 well. We just need to add IPv6 support.

h3. Using an External Library (e.g., IPAddress)

Libraries like [IPAddress|https://github.com/seancfoley/IPAddress] offer comprehensive IP handling, but adding a new dependency for something this focused feels like overkill.


  • No labels