THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Delegation tokens will help processing frameworks to distribute workload to available workers in a secure environment without the added cost of distributing keytabs or TGT. i.e. In case of Storm, Storm’s master (nimbus) is the only node that needs a keytab. Using this keytab Nimbus will authenticate with kafka broker and acquire a delegation token. Nimbus can then distribute this delegation token to all of its worker hosts and all workers will be able to authenticate to kafka using tokens and will have all the access that nimbus keytab principal has.
Public Interfaces
...
APIs and request/response classes
...
getDelegationToken(request: DelegationTokenRequest): DelegationTokenResponse
class DelegationTokenRequest(renewer: Set[KafkaPrincipal] = Set.empty, maxLifeTime: long = -1)
...
class ExpireTokenRequest(hmac: byte[], expireAt: long = Systemtime.currentTimeMillis)
Protocol changes
DelegationTokenRequest
| Code Block |
|---|
DelegationTokenRequest => [Renewer] MaxLifeTime
Renewer => string
MaxLifeTime => INT64 |
Field | Description |
|---|---|
Renewer | Renewer is an user who is allowed to renew this token before the max lifetime expires. If Renewer list is empty, then Renewer will default to the owner. |
MaxLifeTime | Max lifetime for token in milli seconds to future date. if value is -1, then MaxLifeTime will default to a server side config value |
DelegationTokenResponse
| Code Block |
|---|
DelegationTokenResponse => ErrorCode TokenDetails
ErrorCode => INT16
TokenDetails => Owner IssueTimeStamp ExpiryTime MaxLifeTime TokenId HMAC [Renewer]
Owner => String
IssueTimeStamp => INT64
ExpiryTime => INT64
MaxLifeTime => INT64
TokenId => String
HMAC => bytes
Renewer => String
|
Field | Description |
|---|---|
Owner | Kakfa Principal which requested the delegation token |
IssueTimeStamp | Token Issue timestamp. Unit is milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
ExpiryTime | Token expiry time in mills. ExpiryTimeStamp = IssueTimeStamp + ExpiryTime |
MaxLifeTime | Token max life time in mills. MaxLifeTimeStamp = MaxLifeTimeStamp + MaxLifeTime |
TokenId | Token Id. |
HMAC | Keyed-hash message authentication code |
Renewer | Renewers list |
Possible Error Codes
* AuthorizationException ()RenewDelegationTokenRequest
| Code Block |
|---|
RenewDelegationTokenRequest => HMAC ExpiryTime
HMAC => bytes
ExpiryTime => INT64 |
Field | Description |
|---|---|
HMAC | HMAC of the delegation token to be renewed |
ExpiryTime | Token Expiry time in milli seconds to future date. |
RenewDelegationTokenResponse
| Code Block |
|---|
RenewDelegationTokenResponse => ErrorCode
ErrorCode => INT32
|
Possible Error Codes
* AuthorizationException ()
* TokenExpiredException ()* TokenRenewerMismatchException ()
* TokenNotFoundException () ExpireTokenRequest
| Code Block |
|---|
ExpireTokenRequest => HMAC
HMAC => bytes
|
ExpireTokenResponse
| Code Block |
|---|
ExpireTokenResponse => ErrorCode
ErroCode => INT32 |
Possible Error Codes
* AuthorizationException ()
Configuration options
The following options will be added to KafkaConfig.java and can be configured as properties for Kafka server:
delegation.token.max.life.time : Default max life time for delegation tokens.- delegation.token.expiry.time :
- delegation.token.secret
Proposed Changes
Token acquisition
...
The client authenticates using Kerberos or any other available authentication scheme. A token can not be renewed if the initial authentication is done through delegation token, client must use a different auth scheme.
Client sends a request to renew a token with an optional renew life time which must be < max life time of token.
Broker looks up the token, if token is expired or if the renewer’s identity does not match with the token’s renewers, or if token renewal is beyond the Max life time of token, broker disallows the operation by throwing an appropriate exception.
If none of the above conditions are matched, broker updates token’s expiry. Note that the HMAC-SASM is unchanged so the token on client side is unchanged. Broker updates the expiration in its local cache and on zookeeper so other brokers also get notified and their cache statuses are updated as well.
Token expiration and cancellation
...
The current approach requires a deployment when the secret needs to be rotated. If we want to make it automated and more frequent we can offload the responsibility of secret generation to controller.
Details
Wire Protocol Extensions
...
SCRAM Extensions
...
SCRAM messages have an optional extensions field which is a comma-separated list of key=value pairs.
After KIP-84 implementation , an extension will be added to the first client SCRAM message to indicate
that authentication is being requested for a delegation token. This will enable Kafka broker to obtain
credentials and principal using a different code path for delegation tokens.
...
KIP-85 allows dynamic JAAS configuration for Kafka clients. After this we can easily configure the
delegation token for SCRAM-SHA-256 authentication.
Example Usage
Alternatives
Originally we considered to not have any shared Secret at config level. This required us to chose one of the 3 options:
...