...

• The following wire protocol APIs will change. We are bumping the protocol version to support backward compatibility:
  • UpdateMetadataRequest - will contain multiple host/port pairs, not just one. The new protocol is:
    
    [ controllerId  controllerEpoch partitionStateInfoCount partitionStateInfoCount [ Topic Partition controllerEpoch leader leaderEpoch isr.size [ isr ] zkVersion ] numAliveBrokers [ brokerId numEndpoints [ host port securityProtocol ] ] ]
    The bold part is new. it used to be [ brokerId host port ]. 
    host => String 
    port => Int32
    securityProtocol => Int16 (corresponding to SecurityProtocol ENUM)

• The following client libraries will change:

  • We are adding org.apache.kafka.common.protocol.SecurityProtocol - an ENUM describing the possible security protocols the broker may support and the client may try to connect to.
    
    

• Quite a few changes from user configuration perspective:
  • Producers and consumers need to specify security.protocol - which protocol they'll use to connect to the server. Current valid values are PLAINTEXT and TRACE (for testing). Future values should include TLS and SASL.
  • Brokers that wish to support multiple protocols and multiple host/port pairs will need to specify a new configuration parameter: "listeners" and/or "advertised.listeners" - replacing host/port and advertised.host/advertised.port. Listener format is a comma-separated list of "protocol://host:port
  • Brokers need to specify replication.security.protocol - which protocol / port will be used to replicate data between brokers? this must be identical in all brokers.
  • Following an upgrade, brokers need to specify use.new.protocol = true, to start using the new wire protocol and to support new security protocols.
    
    

• Broker registration in Zookeeper will change to include all host/port pairs the broker supports. The new json is:

  {"version":2,
    "jmx_port":9999,
    "timestamp":"2233345666",
    "endpoints": [
      {"host":"myhost",
       "port":9092,
       "protocolType":"plain"},
     {"host":"myhost",
      "port":9093,
      "protocolType":"ssl"}]}

Proposed Changes

See detailed description here: Multiple Listeners for Kafka Brokers

...