You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Note that this KIP proposal is derived from a proposal by Grant Henke that was part of KIP-4 - Command line and centralized administrative operations.

Status

Current state: Draft

Discussion thread: TBD

JIRA:  Unable to render Jira issues macro, execution error.

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

KIP-4 - Command line and centralized administrative operations outlines the motivation for exposing admin operations via the protocol:

  • Allows clients in any language to administrate Kafka
    • Wire protocol is supported by any language
  • Provides public client for performing admin operations
    • Ensures integration test code in other projects and clients maintains compatibility
    • Prevents users from needing to use the Command classes and work around standard output and system exits
  • Removing the need for admin scripts (kafka-configs.sh, kafka-topics.sh, etc) to talk directly to Zookeeper. 
    • Allows ZNodes to be completely locked down via ACLs
    • Further hides the Zookeeper details of Kafka

A couple of specific use cases are worth pointing out:

  1. The Metadata request exposes topic metadata, but it does not expose topic configs. DescribeConfigs will make that information available to any client of the Kafka protocol (including the upcoming Java AdminClient).
  2. AlterConfigs would make it possible to update topic configs, but also client and replication quotas via the protocol.

Public Interfaces (WIP)

Describe Configs

 

DescribeConfigs Request (Version: 0) => [entities]   
  entities => entity_type entity_name 
    entity_type => INT8
    entity_name => STRING

Request semantics:
  1. Can be sent to any broker
  2. If there are multiple instructions for the same entity in one request the extra request will be ingnored
    • This is because the list of entities is modeled server side as a set
    • Multiple describes results in the same end goal, so handling this error for the user should be okay
    • This is similar to how delete topics handles requests
  3. Entity types are "Topic" (existing), "Client" (existing), and "Broker" (new). 
    1. Broker type is read only
  4. Below are the authorization requirements for each type:
    • Broker: Must be authorized to the "Describe" Operation on the "Cluster" resource
    • Topic: Must be authorized to the "Describe" Operation on the "Topic" resource
    • Client: Must be authorized to the "Describe" Operation on the "Client" resource
      • This is a new resource needed
      • TODO: best way to handle this...
  5. Arbitrary configurations are allowed
    1. This provides flexibility for custom clients, and allows all "plugin" or extra configs to be shown
    2. The user can validate the configs after the describe command in their client to heck for errors, but the wire protocol should relay all information.

 

DescribeConfigs Response (Version: 0) => [responses]   
  responses => entity config error_code error_message 
    entity => entity_type entity_name 
      entity_type => INT8
      entity_name => STRING
    config => config_key config_value 
      config_key => STRING
      config_value => STRING
    error_code => INT16

Response semantics:

Alter Configs

TBD

Proposed Changes

TBD

Compatibility, Deprecation, and Migration Plan

TBD

Rejected Alternatives

TBD

  • No labels