THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Please read http://carfield.com.hk:8080/document/distributed/hadoop-security-design.pdf HDFS section for more detailed explanation of all the disadvatages disadvantages above. To address the problems listed above we propose to add support for delegation tokens to secure Kafka. Delegation tokens are shared secret between kafka brokers and clients so authentication can be done without having to go through KDC.
Delegation tokens will help processing frameworks to distribute the workload to available workers in a secure environment without the added cost of distributing keytabs or TGT. i.e. In case of Storm, Storm’s master (nimbus) is the only node that needs a keytab. Using this keytab Nimbus will authenticate with kafka broker and acquire a delegation token. Nimbus can then distribute this delegation token to all of its worker hosts and all workers will be able to authenticate to kafka using tokens and will have all the access that nimbus keytab principal has.
...
Field | Description |
|---|---|
Renewer | Renewer is an Kafka PrincipalType+name string, who is allowed to renew this token before the max lifetime expires. If Renewer list is empty, then Renewer will default to the owner (Principal which requested this token). |
MaxDateMs | Max lifetime for the token in milli secondsmilliseconds. if the value is -1, then MaxLifeTime will default to a server side config value. |
...
Field | Description |
|---|---|
IssueDateMs | timestamp (in msec) when this token was generated. Unit is milliseconds since the beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
ExpiryDateMs | timestamp (in msec) at which this token expires. Unit is milliseconds since the beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
TokenId | Sequence number to ensure uniqueness |
HMAC | Keyed-hash message authentication code |
...
Field | Description |
|---|---|
ErrorCode | |
ExpiryDateMs | timestamp (in msec) at which this token expires.Unit is milliseconds sincebeginning of the epoch (midnight Jan 1, 1970 (UTC)). |
Possible Error Codes
Field | Description |
|---|---|
ErrorCode | |
ExpiryDateMs | timestamp (in msec) at which this token expires. Unit is milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
Possible Error Codes
* DelegationTokenDisabledException...
| Code Block |
|---|
ExpireTokenRequest => HMAC expiryDateMs HMAC => bytes expiryDateMsExpiryDateMs => INT64 |
Field | Description |
|---|---|
HMAC | HMAC of the delegation token to be renewed |
expiryDateMsExpiryDateMs | Token expiry timestamptimestamp (in msec) at which this token expires. Unit is milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
ExpireTokenResponse
| Code Block |
|---|
ExpireTokenResponse => ErrorCode ErroCode => INT32 ExpiryDateMs => INT64 |
Field | Description |
|---|---|
ErrorCode | |
ExpiryDateMs | timestamp (in msec) at which this token expires.Unit is milliseconds sincebeginning of the epoch (midnight Jan 1, 1970 (UTC)). |
Field | Description |
ErrorCode | |
ExpiryDateMs | timestamp (in msec) at which this token expires.Unit is milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
Possible Error Codes
* DelegationTokenDisabledException...
Field | Description |
|---|---|
ErrorCode | |
Owner | Kakfa Principal which requested the delegation token |
Field | Description |
|---|---|
ErrorCode | |
Owner | Kakfa Principal which requested the delegation token. If the Owner list is null (i.e., length is -1), the response contains all tokens from all owners |
DescribeTokenResponse
| Code Block |
|---|
DescribeTokenResponse => ErrorCode [TokenDetails]
ErrorCode => INT16
TokenDetails => Owner IssueDateMs ExpiryDateMs TokenId HMAC [Renewer]
Owner => String
IssueDateMs => INT64
ExpiryDateMs => INT64
TokenId => String
HMAC => bytes
Renewer => String
|
Field | Description |
|---|---|
Owner | Kakfa Principal which requested the delegation token |
IssueDateMs | timestamp (in msec) when this token was generated. Unit is milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
ExpiryDateMs | timestamp (in msec) at which this token expires. Unit is milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
TokenId | Sequence number to ensure uniqueness |
HMAC | Keyed-hash message authentication code |
Renewer | Renewers list |
Possible Error Codes
* DelegationTokenDisabledException...
SCRAM is a suitable mechanism for authentication using delegation tokens. KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients can authenticate using SCRAM-SHA-256, providing the delegation token HMAC as password.
Server will look up the token from its token cache, if it finds a match and token is not expired it will authenticate the client and the identity will be established as the owner of the delegation token.
If the token is not matched or token is expired, broker throws appropriate exception back and does not allow the client to continue.
Token renewal
The (Admin/Delegation Token) client authenticates using Kerberos or any other available authentication scheme. A token can not be renewed if the initial authentication is done through delegation token, client must use a different auth scheme.
Client sends a request to renew a token with an optional renew life time which must be < max life time of token.
Broker looks up the token, if token is expired or if the renewer’s identity does not match with the token’s renewers, or if token renewal is beyond the Max life time of token, broker disallows the operation by throwing an appropriate exception.
If none of the above conditions are matched, broker updates token’s expiry. Note that the HMAC-SASM is unchanged so the token on client side is unchanged. Broker updates the expiration in its local cache and on zookeeper so other brokers also get notified and their cache statuses are updated as well.
Token expiration and cancellation
...