Status
Current state: Under Discussion
Discussion thread: <TBD> and Kafka Security Proposal
JIRA: KAFKA-1688
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
As more enterprises have started using Kafka, there is a increasing demand for authorization for who can publish or consume from the topics. Authorization can be based on different available session attributes or context, like user, IP, common name in certificate, etc. Having an extendable authorization interface will help us to implement the core requirements in the initial phase and make it enterprise ready. Having a pluggable interface will enable other security focused products to provide more advanced and enterprise grade implementations.
Public Interfaces
Binary log format
No
The network protocol and api behavior
No
Any class in the public packages under clientsConfiguration, especially client configuration
<TDB>
Monitoring
No
Command line tools and arguments
No
- Anything else that will likely break existing users in some way when they upgrade
- No
Proposed Changes
The high level proposal in the requirement calls for new interface similar to the one given below which can be implemented by the authentication service provider.
PermissionManager.isPermitted(Subject subject, InetAddress ip, Permissions permission, String resource)
New Kafka Classes
Session
- Represents the life of a (TCP) connection
- Container to session specific data. E.g. one or more of the below
- Username
- Client IP
- Certificate
- Mode of authentication
SessionManager (Singleton)
- This is a singleton object
- This contains the mapping between connections and Session objects
- Facilitates authentication for new connections
- Contains utility methods to retrieve Session object corresponding to the connection
PermissionProvider (Interface)
- Contains the interface methods that needs to be implemented by the authorization provider
- It is assumed that there only one active PermissionProvider. Cascading providers can be supported in the future
DefaultPermissionProvider
- Out of the box implementation of the PermissionProvider
- Stores policies in the properties file or ZooKeeper
- Self contained and no dependencies with any other vendor or providers
PermissionManager (Singleton)
- Helper methods to instantiate the implementation of PermissionProvider
- Proxies all calls to PermissionProvider. E.g. provides helper method for checkPermission
- Enriches context sent to PermissionProvider. E.g. adds session context
- Scalable approach to support multiple providers in the future
Permissions
- Contains the list of permission supported
Subject
- Holds username or other principal
Data Flows
Session Initialization
This flow depicts how the session first created or initialized. Once the session is created, there is a map between the session object and network connection. On subsequent client interaction (socket read/write), the session for the connection is pulled from the map and assigned to the request thread. Then through out the thread, the session is available for any access control.
For client IP authorization, this flow might also call the PermissionProvider to check whether connection from the IP address is allow.
Authorization Flow
This flow is executed during publish/consume or any other request from the client. When the read/write request is received from the client, first it is mapped to the Session object for the connection. This design assumes the session is put in the ThreadLocal or DynamicVariable and is available throughout the scope of the request. When the flow reaches the request handler, e.g. ProducerRequest, the request handler will call the PermissionManager with the topic name (where applicable) and access type (e.g. publish, consume, etc.). The PermissionManager will delegate the check to the provider. Before passing on to the delegate, it will access the session context (user,group, IP, etc) from the session object and pass it on to the PermissionProvider. If the PermissionProvider approves the request, then the flow will continue as usual, however if it is denied, then appropriate error handlers will be called.
Initialize Provider
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
- This shouldn't affect any existing users
- If we are changing behavior how will we phase out the older behavior?
- No. The default implementation would maintain all existing usability behavior
- If we need special migration tools, describe them here.
- No
- When will we remove the existing behavior?
- No
Rejected Alternatives
If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.