Page History

Status

Current state: ["DISCUSSION"]. Accepted

Discussion thread: here

JIRA: KAFKA-1696 

...

getDelegationToken(request: DelegationTokenRequestCreateDelegationTokenRequest):  CreateDelegationTokenResponse

class DelegationTokenRequestCreateDelegationTokenRequest(renewer: Set[KafkaPrincipal] = Set.empty, maxLifeTime: long = -1)

class DelegationTokenResponseCreateDelegationTokenResponse(expiryTimeMillisissueTimeMillis: long, renewer: Set[KafkaPrincipal], maxLifeTimeexpiryTimeMillis: long, maxLifeTime: long, tokenId: String, hmac: byte[])  

renewDelegationToken(request: RenewDelegationTokenRequest): RenewRenewDelegationTokenResponse

class RenewDelegationTokenRequest(hmac: byte[], expiryTimeMillis: long) 

expireToken(request: ExpireTokenRequestExpireDelegationTokenRequest): ExpireTokenResponseExpireDelegationTokenResponse

class ExpireTokenRequestExpireDelegationTokenRequest(hmac: byte[], expireAt: long  = Systemtime.currentTimeMillis) 

describeToken(request: DescribeTokenRequestDescribeDelegationTokenRequest): DescribeTokenRespopnseDescribeDelegationTokenResponse

class DescribeDescribeDelegationTokenRequest(owner: Set[KafkaPrincipal] ) 

Protocol changes

DelegationTokenRequest

CreateDelegationTokenRequest

DelegationTokenRequestCreateDelegationTokenRequest => [Renewer] MaxDateMs
  Renewer => string
  MaxDateMs => INT64

Renewer

MaxDateMs

DelegationTokenResponse

CreateDelegationTokenResponse

DelegationTokenResponseCreateDelegationTokenResponse => ErrorCode TokenDetails
  ErrorCode => INT16
  TokenDetails =>  IssueDateMs ExpiryDateMs MaxDateMs TokenId HMAC 
    IssueDateMs  => INT64
    ExpiryDateMs => INT64
    TokenIdMaxDateMs => String INT64	
    HMACTokenId => bytesString 
    HMAC => bytes

IssueDateMs

MaxDateMs

ExpiryDateMs

TokenId

HMAC

...

* TokenNotFoundException 

ExpireTokenRequest

ExpireDelegationTokenRequest

ExpireTokenRequestExpireDelegationTokenRequest => HMAC expiryDateMs
  HMAC => bytes
  ExpiryDateMs => INT64

ExpiryDateMs

...

ExpireDelegationTokenResponse

ExpireTokenResponseExpireDelegationTokenResponse => ErrorCode
  ErroCode => INT32
  ExpiryDateMs => INT64

...

* TokenNotFoundException 

DescribeTokenRequest

DescribeDelegationTokenRequest

DescribeTokenRequestDescribeDelegationTokenRequest => [Owner]
    Owner => String

ErrorCode

Owner

...

DescribeDelegationTokenResponse

DescribeTokenResponseDescribeDelegationTokenResponse => ErrorCode [TokenDetails]
  ErrorCode => INT16
  TokenDetails => Owner IssueDateMs ExpiryDateMs TokenId HMAC [Renewer]
    Owner => String
    IssueDateMs  => INT64
    ExpiryDateMs => INT64
    MaxDateMs => INT64	
    TokenId => String 
    HMAC => bytes
    Renewer => String

Owner

IssueDateMs

MaxDateMs

TokenId

ExpiryDateMs

TokenId

HMAC

HMAC

Renewer

...

...

The MasterKey/secret is used to generate and verify delegation tokens. This is supplied using config option. Same secret key must be configured across all the brokers.  If the secret is not set or set to empty string, brokers will disable the delegation token support. The current proposal does not support rotation of secret.

Procedure to manually rotate the secret:

We require a re-deployment when the secret needs to be rotated. During this process, already connected clients will continue to work. But any new connection requests and renew/expire requests with old tokens can fail.

• expire expire all existing tokens
•  rotate the secret by rolling upgrade, and
• generate new tokens

Token acquisition

...

Token is stored in Zookeeper as properties in the path /tokenauthdelegation_token/tokens/<tokenUID>. S  During server startup and token creation, scram credentials are generated during token creation. We will use password=HMAC, iterations=4096 for scram credential generation.and stored in memory (TokenCache).  

//Delegation Token Details for tokenID token123: Zookeeper persistence path /tokenauth/tokens/token123
{
   "version":1,
   "owner" : "owner",
   "renewer" : "renewer",
   "issueDateissueTimestamp" : "issueDateissueTimestamp",
   "tokenID" : "sequence-number",
   //Store SCRAM credentials as per KIP-84
    "credentials": {
          "SCRAM-SHA-512" : "salt=djR5dXdtZGNqamVpeml6NGhiZmMwY3hrbg==,stored_key=sb5jkqStV9RwPVTGxG1ZJHxF89bqjsD1jT4S...==,server_key=...,iterations=4096",
          "SCRAM-SHA-256" : "salt=10ibs0z7xzlu6w5ns0n188sis5,stored_key=+Acl/wi1vLZ95Uqj8rRHVcSp6qrdfQIwZbaZBwM0yvo=,server_key=nN+fZauE6vG0hmFAEj/49+2yk0803y67WSXMYkgh77k=,iterations=4096"
    }maxTimestamp" : "maxTimestamp",
   "expiryTimestamp" : "expiryTimestamp",
   "tokenID" : "UUID",
};

SCRAM Extensions

SCRAM messages have an optional extensions field which is a comma-separated list of key=value pairs.
After KIP-84 implementation , an extension will be added to the first client SCRAM message to indicate
that authentication is being requested for a delegation token. This will enable Kafka broker to obtain
credentials and principal using a different code path for delegation tokens.

...

Below diagram shows the steps required to use the delegation tokens.

ACL

Currently, we only allow a user to create delegation token for that user only. Renew and expire requests should come from owner or renewers of the token. So we dont don't need ACLs for create/renew/expire requests.  

For describe, Owners and the renewers can always describe their own tokens. To describe others tokens, we can add DESCRIBE operation on Token Resource.  In future, when we extend the support to allow a user to acquire delegation tokens for other users, then we can enable CREATE/DELETE operations.

...