...

After this process, all servers will be aware of the latest set of internal topics and can cache internal topics in MedatadaCache. Thus, that KafkaApi can construct the metadata response with the information of all clients created internal topics by referring MetadataCache. 

Internal topic behaviors

Let's denote

• topics `__consumer_offsets` and `__transaction_state` as system-defined internal topics,
• topics whose config contain `internal=true` as user-defined internal topics.

ACL: 

Below will be the Kafka default allowed operations for internal topics. However, Cluster admins admin might want to control the operations on internal topics using ACLs as they might be dangerous.

1. Internal topic creation will be allowed.
2. Internal topic deletion will be allowed except for` __consumer_offsets` and `__transaction_state`.
3. Producing to internal topic partitions other than `__consumer_offsets` and `__transaction_state` will be allowed.
4. Adding internal topics to transactions will be allowed.

Post ZK world

...

add restrictions using ACLs on user-defined internal topics depending on the actual user application logic.

Metadata:

For the metadata operation, user-defined internal topics will be treated in the same way as system-defined internal topics. For example, in the metadata request (`ApiKeys.METADATA`), broker will mark both user-defined internal topics and system-defined internal topics as `internal`.

Client-side:

To get the internal topic information, instead of using the static internal topic testing or implementing their own logic, clients can utilize KafkaAdminClients and make a MetadataRequest (ApiKey.METADATA). 

Post ZK world

KIP-500 proposed metadata quorum. Since the changes proposed in this KIP interact directly with KafkaApi instead of Zookeeper and modifies the cached metadata, it should be easily migrated in Post ZK world.

Compatibility, Deprecation, and Migration Plan

...