Current State: Under Discussion
Discussion Thread: https://lists.apache.org/thread/4o5lcl2nbzt0sf28rzx90rnyyf50xfx2
JIRA:
Managing Kafka ACLs in large environments is painful: Kafka only allows specifying exact IP addresses or a wildcard (*) for host-based access control. With 50 clients in a subnet, this means 50 separate ACL entries. Not fun. This becomes especially frustrating when the goal is simply to say "allow anything from 10.0.0.0/8" like every other system does.
Two prior KIPs tried to solve this:
This KIP picks up where they left off with:
The --allow-host and --deny-host options in kafka-acls.sh will accept CIDR notation alongside the existing formats:
| Format | Example | What is does |
|---|---|---|
| Exact IPv4 | 192.168.1.100 | Matches one address (already works) |
| Exact IPv6 | 2001:db8::1 | Matches one address (already works) |
| Wildcard | * | Matches everything (already works) |
| IPv4 CIDR | 192.168.0.0/24 | Matches a subnet (new) |
| IPv6 CIDR | 2001:db8::/32 | Matches a subnet (new) |
No changes to the Admin interface or CreateAclsResult. CIDR notation goes in the host field of AccessControlEntry exactly as an exact IP would.
We add a new metadata version to gate this feature. This is important because older brokers will not understand CIDR patterns, and we need to prevent users from creating ACLs that would break things during a rolling upgrade.
Attempting to create a CIDR ACL before the cluster is ready results in a clear error:
org.apache.kafka.common.errors.UnsupportedVersionException: CIDR-based ACL host patterns require metadata version IBP_4_X_IVZ or higher. Current cluster metadata version: IBP_4_0_IV0 |
Note that didMetadataChange is set to true for IBP_4_4_IV0. This means that downgrading below this version is automatically classified as an unsafe metadata downgrade.
Today, StandardAuthorizerData.java does simple string comparison and we would extend this to handle CIDR.
When creating ACLs, the MetadataVersion is passed into AclControlManager.createAcls() (following the same pattern as DelegationTokenControlManager) and threaded through to validateNewAcl(), which calls validateHostPattern():
This ensures that:
SubnetUtils);SubnetUtils6) andApache Commons Net and its SubnetUtils class handles IPv4 CIDR matching nicely as well as SubnetUtils6 for IPv6. Moreover, we need to add the dependency (i.e., commons-net) into Kafka.
QuorumController.createAcls() passes featureControl.metadataVersionOrThrow() to AclControlManager.createAcls(), consistent with how other managers such as DelegationTokenControlManager receive the metadata version.
The existing Kafka ACL evaluation semantics (i.e., where DENY always takes precedence over ALLOW), regardless of specificity apply unchanged to CIDR-based host patterns for both IPv4 and IPv6. If a client IP matches both an ALLOW CIDR and a DENY CIDR, the request is denied, regardless of prefix length. For example, an ALLOW on 2001:db8::/32 combined with a DENY on 2001:db8:abcd::/48 will deny any client within the /48 range, even though it also falls within the broader /32 ALLOW. So to be even more concrete:
This is consistent with how Kafka already handles overlapping exact-IP ALLOW and DENY entries, and no new priority rules are introduced.
Existing ACLs work exactly as before (i.e., no changes to exact IP or wildcard matching). The CIDR matching logic is additive and only activates when an ACL host pattern contains a '/' character.
CIDR ACLs cannot be created until all brokers support them. This is enforced at ACL creation time via MetadataVersion. Any attempt to create a CIDR-based ACL on an older metadata version results in an UnsupportedVersionException with the specific version required. This guarantees that CIDR host patterns never appear in the metadata log on clusters where brokers may not understand them.
Since CIDR host patterns are persisted in metadata records, IBP_4_4_IV0 is defined with didMetadataChange = true. This means any downgrade below IBP_4_4_IV0 is automatically treated as an unsafe metadata downgrade. No additional pre-downgrade validation hook is needed.
Java’s networking stack automatically resolves IPv4-mapped IPv6 addresses (e.g., ::ffff:192.168.0.5) to their native IPv4 form (192.168.0.5). As a result, the host address seen by the authorizer is always the plain IPv4 address. Administrators should use IPv4 CIDR notation (e.g., 192.168.0.0/24) for IPv4 subnets and native IPv6 CIDR notation (e.g., 2001:db8::/32) for IPv6 subnets.
Creating ACLs using IPv4-mapped CIDR patterns is not supported:
All related stuff within subnet handling is tested via commons-net library via RFC examples for IPv6 and IPv4 (i.e., https://datatracker.ietf.org/doc/html/rfc5952, https://datatracker.ietf.org/doc/html/rfc1519). In AclControlManagerTest, we will cover validation of host patterns (i.e., valid/invalid IPv4 or IPv6 CIDR, null/empty inputs and even malformed prefixes) and metadata version gating (i.e., CIDR rejected on older version, accepted on IBP_x_x_IVx, end-to-end ACL creation with CIDR hosts and backwards compatibility with exact IPs and wildcards on older versions. Moreover, in FeatureControlManagerTest we will cover pre-downgrade validation mechanism i.e, block metadata version downgreade when validator returns an error, allows when downgrade pass. Also in StandardAuthorizerTest, CIDR host matching for IPv4 and IPv6 (boundary addresses, range membership, invalid/null patterns), full authorizer flow iterating all addresses in a /24 and /120 range, and overlaping CIDR ACLs semantics which would confirm DENY always takes precedence regardless of prefix length. The downgrade lifecycle will be tested in AclControlManagerTest (i.e., creating CIDR ACL (on supported version), attempting and failing to downgrade (to un-supported version), removing ACL, then succesfully downgrading).
IP Range Notation (e.g., 10.0.0.1-10.0.0.100) KIP-252 proposed supporting arbitrary IP ranges. We are not doing this because CIDR covers the vast majority of real-world use cases, and adding range support would complicate the implementation without much benefit.
Custom Implementation from Scratch We considered writing all the IP parsing and matching code ourselves, but there's no reason to reinvent the wheel when Apache Commons Net already handles IPv4 well. We just need to add IPv6 support.
Pre-downgrade validator hook We initially implemented a preDowngradeValidator callback in FeatureControlManager that QuorumController would register to check for existing CIDR ACLs before allowing a metadata version downgrade. This was replaced by setting didMetadataChange = true on IBP_4_4_IV0, which gives the same protection through the existing unsafe-downgrade mechanism without introducing bespoke validation infrastructure.