...

There are several configuration options for this callback handler:

...

. Sensitive configuration options and SASL extensions appear under the JAAS configuration (sasl.jaas.config) while the rest are top-level configuration.

The JAAS configuration options are:

• clientId: supports OAuth clientcredentials grant type

• clientSecret: supports OAuth's clientcredentials grant

• scope: optional scope to reference in the call to the OAuth server

The top-level configuration options for the client login callback handler are:

• sasl.login.token.endpoint.uri: OAuth issuer token endpoint URI
• sasl.login.scope.claim.name: optional override name of the scope claimscopeClaimName: optional override name of the scope claim; defaults to scope
• subClaimNamesasl.login.sub.claim.name: optional override name of the sub claim; defaults to sub
• loginConnectTimeoutMssasl.login.connect.timeout.ms: optional value in milliseconds for HTTPS connect timeout; defaults to 10000
• loginReadTimeoutMssasl.login.read.timeout.ms: optional value in milliseconds for HTTPS read timeout; defaults to 10000
• loginAttemptssasl.login.attempts: optional number of attempts to make to connect to the OAuth/OIDC identity provider; defaults to 3
• loginRetryWaitMssasl.login.retry.wait.ms: optional value in milliseconds for the amount of time to wait between HTTPS call attempts; defaults to 250
• loginRetryMaxWaitMssasl.login.retry.max.wait.ms: optional value in milliseconds for the maximum wait between HTTPS call attempts (as described above); defaults to 10000

...

sasl.login.callback.handler.class=...OAuthBearerLoginCallbackHandler

sasl.jaaslogin.config=token...OAuthBearerLoginModule required \
tokenEndpointUri="endpoint.uri=https://myidp.example.com/oauth2/default/v1/token"

sasl.jaas.config=...OAuthBearerLoginModule required \
clientId="abc123" \
clientSecret="S3cr3t!" \
scope="sales-pipeline" \
extension_supportFeatureX="true" \
extension_organizationId="sales-emea" ;

In the above example, only the OAuth provider’s tokenEndpointUri to retrieve an access token provider’s sasl.login.token.endpoint.uri has been specified. The values for clientId and clientSecret as provided by the OAuth provider for an “API” or “machine-to-machine” account are required. The optional scope value will allow the inclusion of a scope parameter when requesting the token.

...

Once the login has occurred for this client, the returned access token can be reused by other connections from this client. While these additional connections will not issue the token retrieval HTTP call on the client, the broker will still validate the token repeatedly once for each time it is receivedsent by each client connection.

Per KIP-368, the OAuth token re-authentication logic from the existing implementation is automatically "inherited" by this implementation, so no additional work is needed to support that feature.

...

It may be that the names of the claims used by the OAuth provider differ from what is expected. For example, the security principal for which the token is valid is usually contained in the sub (subject) JWT claim. There may be cases where the value of that claim may not be valid or usable, and instead the value will need to be extracted from, for example, the email claim.

There are a handful of several configuration options for this callback handler:. Since there are no sensitive configuration options, they are all in the top-level configuration:

• listener.name.<listener name>.oauthbearer.sasl.jwks.endpoint.urijwksEndpointUri: OAuth issuer's JWK Set endpoint URI from which to retrieve the set of JWKs managed managed by the provider; mutually exclusive with jwksFile

• jwksEndpointRefreshIntervalMs: optional value in milliseconds for how often to refresh the JWKS from the URL pointed to by jwksEndpointUri. Only used when using jwksEndpointUri. Defaults to 3600000 (1 hour)

• jwksFile: specifies a locally-accessible file name that holds a file

  this can be a file://-based URL that points to a broker file system-accessible file-based copy of the JWKS data. This allows the JWKS data to be updated on the file system and refreshed on the broker when the file is updated, thus avoiding any HTTP communication with the OAuth/

  OIDC provider; mutually exclusive with jwksEndpointUri

  OIDC provider

• listener.name.<listener name>.oauthbearer.sasl.jwks.refresh.interval.ms: optional value in milliseconds for how often to refresh the JWKS from the URL pointed to by listener.name.<listener name>.oauthbearer.sasl.jwks.endpoint.uri. Only used when using an HTTP or HTTPS-based URI for listener.name.<listener name>.oauthbearer.sasl.jwks.endpoint.uri. Defaults to 3600000 (1 hour)

• listener.name.<listener name>.oauthbearer.sasl.sub.claim.namesubClaimName: name of the scope from which to extract the subject claim from the JWT; defaults to sub

• scopeClaimNamelistener.name.<listener name>.oauthbearer.sasl.scope.claim.name: name of the scope from which to extract the scope claim from the JWT; defaults to scopescope

• listener.name.<listener name>.oauthbearer.sasl.clock.skew.secondsclockSkew: optional value in seconds for the clock skew between the OAuththe OAuth/OIDC provider and the broker. Only used when using jwksEndpointUrian HTTP or HTTPS-based URI for listener.name.<listener name>.oauthbearer.sasl.jwks.endpoint.uri. Defaults to 30
• expectedAudiencelistener.name.<listener name>.oauthbearer.sasl.expected.audience: The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences. The JWT will be inspected for the standard OAuth aud claim and if this configuration option is set, the broker will match the value from JWT's aud claim to see if there is an exact match. If there is no match, the broker will reject the JWT and authentication will fail.
• expectedIssuerlistener.name.<listener name>.oauthbearer.sasl.expected.issuer: Optional setting for the broker to use to verify that the JWT was created by the expected issuer. The JWT will be inspected for the standard OAuth iss claim and if this configuration option is set, the broker will match the value from JWT's iss claim to see if there is an exact match. If there is no match, the broker will reject the JWT and authentication will fail.

Here's an example of the configuration as a part of a Java properties file:

listener.name.<listener name>.oauthbearer.sasl.login.callback.handler.class=o.a.k...OAuthBearerValidatorCallbackHandler
sasl
listener.jaas.config=o.a.k...OAuthBearerLoginModule required \
jwksEndpointUri="name.<listener name>.oauthbearer.sasl.jwks.endpoint.uri=https://myidp.example.com/oauth2/default/v1/keys"

listener.name.<listener name>.oauthbearer.sasl.jaas.config=o.a.k...OAuthBearerLoginModule required \
scopeClaimName="scp" ;

In the above configuration the broker points to the appropriate OAuth provider jwksEndpointUriprovider listener.name.<listener name>.oauthbearer.sasl.jwks.endpoint.uri to retrieve a the set of JWKs for validation. In this example, a non-default value for scopeClaimName has been provided because the provider uses scp for the name of the scope claim in the JWT it produces.

...

1. Providing a JWKS URL. In this mode, the JWKS data will be retrieved from the OAuth provider via the configured URL on broker startup. All then-current keys will be cached on the broker (per the ‘max age’; the jose4j library has a means to keep these-up-to-date when they age out) for incoming requests. If an authentication request is received for a JWT that includes a kid that isn’t yet in the cache, the JWKS endpoint will be queried again on demand. However, we prefer polling via a background thread to hopefully pre-populate the cache with any forthcoming keys before any JWT requests that include them are received.

2. Providing a JWKS file. On startup, the broker will load the JWKS file from a the configured location and will watch the file for updates, allowing for dynamic configuration updates. Any updates to the file will require a restart of the broker. The means by which the JWKS file is updated is left to the cluster administrator. In the event that an that an unknown JWT key is encountered, this implementation will simply issue an error and validation will fail.

...

The use of OAuth credentials for broker-to-broker communication will continue to be supported. As with the existing implementation, users can specify the protocols and implementations to use for broker-based communication. This would require providing the appropriate configuration for both client login and broker validation.

Testing

In addition to unit and integration tests, there will be a standalone tool in the tools directory/module: org.apache.kafka.tools.OAuthCompatibilityTest. This test can be run via the existing bin/kafka-run-class.sh script thusly:

...